Raghavendra Selvan

Project title

CITADL: Complexity Informed Theory for Accelerated Deep Learning

What is your project about?

Recent advanced artificial intelligence (AI) models require large-scale computations, which in turn leads to enormous energy consumption, carbon emissions, water consumption and other environmental consequences. There are many algorithmic improvements that can reduce the resource consumption of AI methods. This can be done, for example, by reducing the amount of training data, the number of bits used to store the AI ​​models, or by removing unnecessary computations. These strategies make AI models more efficient, but we lack a common theory to explain why we can make them more efficient. In the CITADL project, I want to build on ideas from theoretical computer science from the 1960s, where researchers developed strong mathematical frameworks to understand randomness, information and complexity in data, to explain how AI models learn and how we can build them with fewer resources.

How did you become interested in your particular field of research?

During my PhD, I started building AI models for medical image analysis, and I found them not only exciting but also extremely useful. However, in recent years, we have been moving towards a trend where we are building ever larger AI models that require massive computational resources, which entails large environmental costs. I have become particularly interested in how these costs not only affect the climate, but also issues of social equity and access to developing AI models. My recent research has focused on improving the sustainability of AI. Along the way, I also realized that there was no coherent explanation for why we need to build large models in order to then make them more efficient. This has become my current research focus, which will be pursued in this project.

What are the scientific challenges and perspectives in your project?

Consider the two sequences, A = [101 001 110] and B = [100 100 100]. If we look closely at them, we can observe that one sequence is more complex than the other. While sequence B has a repeating pattern of 100, there is no clear pattern in sequence A; therefore, we consider sequence A to be more complex than B. This emphasizes a fundamental idea in algorithmic information theory: sequences with patterns can be compressed, whereas random sequences cannot. We will use mathematical frameworks developed around algorithmic complexity to explain how deep learning models learn and how we can compress them to make them more efficient. There are many challenges that we aim to solve in this project. The most important of these is to bridge the gap between algorithmic complexity and the needs of modern AI methods.

What is your estimate of the impact, which your project may have to society in the long term?

Globally, the electricity consumption and environmental costs of AI are growing rapidly. This is related to the enormous amounts of resources required to build them. The aim of the CITADL project is to create a theoretical understanding of learning and compression in AI models and to develop new classes of efficient AI methods. I expect that the results of this project will help to address the growing resource costs of modern AI methods. This could have a huge impact on improving the sustainability of AI by reducing their climate impact and improving access to these methods globally.

Which impact do you expect the Sapere Aude programme will have on your career as a researcher?

The recognition from Sapere Aude: The DFF Research Leader program will help me pursue the ideas about sustainable AI with the necessary resources and ambitions. This will also help me consolidate my research group so that we can address the technical challenges of the project. I also expect that the Sapere Aude grant will help to elevate the research work carried out in my group at the University of Copenhagen – and in Danish research in general – further to an international level.