Niels van Berkel

Project title

Safeguarding Users’ Cognitive Autonomy in Human-AI Interaction

What is your project about?

This project explores how digital systems can be designed to protect users’ independent thinking when interacting with AI systems in both everyday and professional life. We aim to unobtrusively detect when users switch into 'cognitive autopilot’, where they, for example, directly copy text suggestions into their writing, or they accept AI-generated content without verifying its correctness. Based on this, we will dynamically adjust system behaviour to reduce uncritical reliance on AI suggestions, safeguarding people’s cognitive autonomy while still embracing the possibilities AI offers.

How did you become interested in your particular field of research?

Digital technology now mediates a large share of our daily lives. The tools we use shape how we work, connect with others, collaborate, reflect, and make decisions. Designing systems that support human needs and values requires the insights and perspectives from a variety of disciplines. To me, this interdisciplinarity makes my field of research highly interesting and ensures there is always more to learn. As digital technology touches on so many aspects of our lives, working collaboratively towards a more human-centred digital future can be highly impactful and provide a glimpse of what lies ahead.

What are the scientific challenges and perspectives in your project?

The project addresses three core scientific challenges. First, we seek to develop privacy-preserving methods to identify fluctuations in users’ cognitive state unobtrusively and continuously based solely on interaction data. Second, we must design intervention techniques to support cognitive autonomy that are not experienced as frustrating or disrupting, as otherwise they will simply not be used. Finally, the project aims to validate these detection and intervention methods in real-world settings. While this introduces new scientific challenges, it is critical to assess whether findings generalise beyond controlled laboratory settings.

What is your estimate of the impact, which your project may have to society in the long term?

As AI's role in society grows, it is highly timely to develop a better understanding of its impact on our individual and collective reasoning. Our aim is to develop AI tools that augment rather than offload human reasoning by detecting when people cognitively disengage and subsequently re-engage their critical thinking. In the long term, the societal implications of this span across a wide range of activities, from students learning how to interpret and apply new information to high-stakes clinical decision-making under time pressure. Concretely, the work seeks to make an impact through both the technical design of systems and the development of policies that guide the responsible deployment of AI.

Which impact do you expect the Sapere Aude programme will have on your career as a researcher?

The Sapere Aude programme provides the scientific freedom to pursue a novel research direction. It enables me to form a team with the diverse skills required to gain new insights into human-AI interaction. This strengthens our ongoing work and fosters further synergies in method and tool development. Finally, the grant enables me to both build new connections and strengthen existing ones with leading researchers worldwide, thereby supporting the continued development of our work through cross-disciplinary exchange of ideas.