Cosyne: Safe Control Systems with Neural Networks
Control systems are ubiquitous in our daily life. For example, ponds are installed all over the country to give a regular supply of water throughout the year, with fluctuating weather and demand, while avoiding flooding. Water in Denmark is controlled in a static way based on human-engineered controllers. While robust, the controllers' static nature does not make use of all the data we have at our disposal today. In recent years, artificial intelligence has made tremendous progress with the emergence of neural networks, which outperform humans at predictions from vast amounts of data. However, neural networks are large black boxes that cannot be reasoned about like traditional controllers. My project lays the grounds for automatic analysis and certification of neural networks so that we can trust this new generation of intelligent controllers when employed in critical applications.
Neural networks are arguably the most exciting topic in computer science today. On the one hand, they are a break-through technology and already heavily used in many disciplines. On the other hand, they also pose a grand challenge to computer science because we do not fully understand their limitations and are hesitant to employ them in applications where safety is critical. I generally enjoy developing algorithms to analyze complex systems in my work. In my project, I will have plenty of opportunities to do that and will also enter new grounds to challenge myself. The strong potential for practical impact of my project to exploit neural networks in safety-critical applications is an extra motivation for me.
Neural networks are not designed by humans but learned from data, and while they often exhibit impressive performance, by their nature, they can also make mistakes. In some applications, these mistakes can be critical. A major challenge is to actually distinguish between correct behavior and mistakes in complex, real-world systems - the world is not black and white! This makes it very hard to reason about neural networks - a little bit like humans. However, in the context of control systems, we have formal specifications of the intended behavior. My project exploits this fact and develops techniques to algorithmically analyze neural-network controllers.
Today we are not exploiting the full potential of controllers, simply because we still rely on static control policies. Currently, we do not dare to employ artificial intelligence to help us learn from the available data in safety-critical applications because we are concerned about the potential risk. My hope is that, with the development of certification procedures, we can build the necessary trust in this new generation of intelligent controllers and unleash their high performance. This will be a process over many years, but we have to start now to keep up with technology.
The prestigious Sapere Aude: DFF Starting Grant allows me to establish a strong and internationally recognized research group, focus my work on a challenging and exciting problem, and consolidate my research agenda. With the resources from this grant, I will be able to study the problem in depth and from different angles. At the same time, I will have enough flexibility to react to insights I hope to make. This Sapere Aude grant will thus be a cornerstone in building a successful career as a research leader.
In my spare time I enjoy programming. In my work, I develop the open-source library JuliaReach; sometimes after work I add some "quality-of-life" features that I do not have time to add as part of my work time. There are now quite a number of users of this library, and I like engaging with them. On the physical side, I cycle to work every day, which helps me stay healthy.
Aalborg University
Computer Science
Aalborg
Heinrich-Suso-Gymnasium Konstanz, Germany