Exploring the future: Assessing the impact of emerging AI technologies on individuals
2024 marks the 20th anniversary of the EDPS and the year the European Union adopts the Artificial Intelligence Act. Over the past two decades, the EDPS has played an important role in personal data protection at European level. We will continue to ensure that EU institutions, bodies, offices and agencies (EUIs) respect data protection rules when processing personal data. However, the role of the EDPS has recently expanded with the entry into force of the AI Act on 2 August 2024: we will act as the competent authority for AI systems provided and deployed by EUIs, ensuring a high level of protection against the potentially harmful effects of AI systems.
Many AI systems process personal data during their lifecycle (e.g. training, development or deployment). Consequently, artificial intelligence related technology trends are relevant for our two roles, as the AI Act's competent authority and as a data protection authority. With this in mind, we have decided to dedicate this issue of TechSonar entirely to AI technology trends, focusing on how these trends could impact the rights and freedoms of individuals.
This year's TechSonar report includes six trends: Retrieval-augmented generation (RAG), a technique that allows AI systems to generate more relevant output by retrieving and combining relevant information from multiple knowledge bases. On-device AI, a system architecture designed to place data processing at the edge of the network, reducing latency and increasing control over the data processed by AI systems. Machine unlearning, a technique that enables trained AI systems to forget specific data or remove its influence upon request. Multimodal AI, which deals with the integration of multiple types of data (e.g., text images or audio), offering richer insights. Scalable oversight, focusing on the ability to use AI systems to effectively monitor other AI systems as they grow in complexity and scale, ensuring that AI applications remain transparent, accountable, and aligned with ethical standards. And finally, neuro-symbolic AI, which combines neural networks with symbolic reasoning to enhance accuracy and decision-making processes.
Each of these trends starts with a fictional scenario illustrating a potential application of the technology in our daily lives. Following the scenario, there is a description of the technology trend and its current development status. Then you will find our assessment of how the trend could affect individuals. To conclude each trend, we have compiled a list of recommended reading material for those wishing to gain a deeper understanding on the subject.
Without revealing too much about the trend reports ahead, I would like to share a few common elements and interesting patterns.
First, independently of the trend we consider, I observe that most use cases of impactful AI applications process personal data. It can easily be concluded that the deployment of AI systems in our daily lives will significantly rely on the processing of personal data. During the AI development and training phases, vast amounts of personal data, including text, images, audio, and video - often containing sensitive information such as biometric and behavioural data - are collected, posing significant risks such as potential data breaches, misuse or the incorporation of biased or unrepresentative data into AI models. Once trained, AI models may also memorise parts of their training datasets and be subject to data extraction attacks. Additionally, during the AI system deployment phase, user interactions with the models may involve further processing of personal data, raising privacy concerns, especially when biometric data is involved.
Second, some of these TechSonar trends address challenges created by the way AI systems are currently developed. For example, machine unlearning is linked to the problem of AI systems trained on poorly curated datasets, while retrieval-augmented generation contributes to solving the well-known problem of large language model hallucinations and scalable oversight relates to the cost of ensuring that increasingly complex AI systems are aligned with human values. Any technology that helps mitigate risks to human rights is welcome, but I wonder if we should first focus on avoiding the creation of these risks.
Third, while leading AI companies are introducing new AI models and trends at breath-taking speed, we see a growing pressure on organizations to rapidly adopt and develop new AI systems quickly. Given the new capabilities these new models and technology trends bring, there is growing fear among managers and employees within organizations of missing out on opportunities. These fears are understandable, but it is still paramount not to be dragged down by them as they could lead to poor risk management for individuals.
While some of the technologies in this TechSonar edition may contribute to mitigating the risks to individuals’ fundamental rights, some, while promising substantial economic benefits, may also pose significant risks to individuals if not properly managed. The rapid progress in this area, combined with the potential for high returns on investment, is fuelling an AI race that is likely to make AI systems increasingly pervasive in our daily lives. This ubiquity of AI systems, combined with advanced capabilities that can profoundly influence our integrity and autonomy as persons, such as emotion detection, persuasive promotion of ideas, and content creation (including fake content), calls for certain limitations. Additional controls and safeguards are essential to ensure that the benefits of these advancements do not come at the expense of individual rights. Rigorous risk analysis and the implementation of robust safeguards are crucial. Providers and deployers of AI systems must conduct thorough impact assessments to identify potential risks and establish measures to mitigate them. This includes ensuring data privacy, preventing bias and discrimination, and maintaining transparency in AI operations. By prioritizing these safeguards, we can grab the full potential of AI technologies while preserving the fundamental rights of individuals.
I hope this TechSonar can help disseminate knowledge about the key trends we see driving the field of AI for the coming years, and contribute to the ongoing debate by shedding light on the potential impacts—both positive and negative—of the AI trends presented. As with any foresight exercise, only time will reveal whether the technologies discussed here will evolve into major trends.
After all, reality is often more surprising than fiction!
