Author: Vítor Bernardo
Extended reality (XR) is an emerging umbrella term for all immersive technologies, including virtual reality, augmented reality, and mixed reality.
Virtual reality (VR) is a technology that creates a digitally simulated immersive environment or experience for users. It typically involves the use of specialised hardware and software to create a computer-generated 3D spatial, and possibly multi-sensorial, environment that can be interacted with in a seemingly real or physical way. VR allows users to experience and interact with a digital environment as though they were real.
Augmented reality (AR) on the other hand is a technology that overlays digital information, such as text; images; sound; videos; or 3D models, onto the real-world environment. AR enhances the real world by adding computer-generated elements to it.
Mixed reality (MR) systems are immersive technologies that bring physical objects into digital environments or digital objects into physical reality. One type of MR is Cinematic Reality, offering immersive 360 degrees viewing with live camera footage.
XR technologies typically rely on smartphones, tablets, smart glasses, or other wearable devices to deliver the augmented and/or virtual experiences. The wearables are also required to collect certain basic information provided by the user as a starting point, and then a continuous stream of new feedback data generated as the user interacts with their virtual environments to create the illusion of interaction with the virtual elements.
Whether combined or alone, these technologies can have many applications. Professional training, entertainment, education, and architecture are some of the fields that are expected to be changed profoundly as VR evolves. AR is expected to provide users with valuable context-related information on the real world, enhancing their understanding of the environment.
These technologies can have many benefits in several different fields. They can provide contextual information during surgery in healthcare, information about sights or museums and historical augmented environments in tourism, and directions and/or warnings in navigation.
Positive impacts foreseen on data protection:
- Providing information to data subjects with AR
Augmented reality's ability to provide contextual information (i.e. available in a specific area or in the presence of a specific object) can also be used to provide more information about the processing of personal data. For example, individuals entering a CCTV-covered area with AR-enabled devices could be presented with information about the data controller, the purpose of the data processing, and possible ways to exercise their rights.
It should be borne in mind that, in such situations, AR would be an additional channel to provide information that should not replace the mechanisms already in place. In addition, the use of these mechanisms for the provision of information should not lead to further processing of the data for other purposes.
Negative impacts foreseen on data protection:
- Intensive collection of personal data from users and user profiling
VR systems might capture user’s behaviours, such as head orientation, and position. Some systems can track other body part movements to increase immersion (e.g. hand, feet, chest, elbow or knee). XR can also incorporate gaze tracking, respiration, heart rate or even brain-computer interface (BCI) neural signal interpretation.
User movement data can be collected at frequencies of up to 1000Hz, meaning that systems can take 1000 user measurements per second.
This can lead to intensive data collection of multiple characteristics from users that can allow the definition of a detailed description of characteristics and behaviours. Not only can these data collections contain multiple types of personal and possibly sensitive information, but XR devices can also combine this information to reveal or infer additional details about individual users (distance from the floor, for instance, can be used to infer the user's height). According to some authors, head position and movement can be used to infer neurological conditions such as attention deficit hyperactivity disorder, autism or dementia.
Finally, most VR services currently on the market require users to log in to the device, further increasing the risk of user profiling across different devices.
Such a high volume of data processing is difficult to reconcile with the principles of data minimising and purpose limitation.
- Unintentional disclosure of personal data
In VR, an avatar is a digital representation of a user or player within the virtual environment. Avatars can be customised to varying degrees, depending on the VR system or platform. Users have the ability to choose different appearances, clothing, and even gestures or expressions to personalise their virtual identities.
However, studies have shown that when configuring avatars for social VR, people tend to construct avatars that match their physical selves, reflecting their aesthetics, gender, race and age/maturity, increasing the risk of identification. There is also research indicating the possibility to correlate the movements of an avatar with data of users’ movement recorded whilst performing a set of movements in real life.
Users can become emotionally immersed in these virtual spaces, which they may be able to access through different devices using a single virtual identity. This may make them more likely to (unintentionally) disclose personal information in the immersive environment that they would not otherwise.
Additionally, VR environments are evolving into complete virtual worlds with the persistence of the user's online actions over time (i.e. the state of the user's actions is preserved in the VR environment to replicate the physics of the real world). In such a scenario, there would be an increased risk of revealing personal data from the user’s online activities (e.g. activities reflecting political views or personal interests).
- Personal data collection from non-users
AR systems are designed to interact with the user's real environment. When in use, they can continuously collect video and sound from the user's surroundings, which may include other people who are not users and who are unaware of the data processing. This can lead to unauthorised collection of data from others.
Unauthorised data collection from the real environment may also occur in the case of VR, especially since some of the largest social media providers have shown interest in creating virtual reality environments that duplicate the real world we live in (i.e. an environment that replicates the physical objects and beings of the real world). Although still a concept, such an endeavour would require a massive collection of unauthorised personal data.
Suggestions for further reading:
- Dick, E. (2021). Balancing user privacy and innovation in augmented and virtual reality. Information Technology and Innovation Foundation
- Miller, M. R., Herrera, F., Jun, H., Landay, J. A., & Bailenson, J. N. (2020). Personal identifiability of user tracking data during observation of 360-degree VR video. Scientific Reports, 10(1), 17404.
- Roesner, F., Kohno, T., & Molnar, D. (2014). Security and privacy for augmented reality systems. Communications of the ACM, 57(4), 88-96.