What is extended reality?

How does extended reality work?

Because extended reality is an umbrella term, how it works differs depending on the type of XR and its implementation. For example, augmented reality will work differently on a smartphone app versus on a headset or a car's HUD.

What goes into making XR work is the hardware medium used, the software that runs on it and the data that needs to be collected. These commonly include the following components:

Hardware

XR works on mediums like headsets, smart glasses, smartphones or HUDs. The technologies in these mediums typically include the following:

• Sensors. These collect data on a user's positioning and surrounding environment, including accelerometers, gyroscopes, cameras and other depth and tracking systems.
• Displays. This presents digital content, including glass or digital displays.
• Input devices. These let users interact with digital elements, including controllers or speakers for voice commands.
• Processing units. These process sensor data, render graphics and manage audio.

Data and processing

• Cloud and edge computing. Cloud computing can provide remote computation and storage, while edge computing is any data processing that occurs on or near the device.
• AI. AI can be used to analyze data in real-time to recognize objects and to interpret a user's surroundings.

Software

• Operating systems. This is the underlying software that runs XR applications.
• Development platforms. These are the engines used to create XR content, like Unity and Unreal Engine.
• Tracking software. Tracking software like gesture, face and eye tracking enables the device to read user movements.
• Spatial mapping software. This software builds a map of the user's environment, enabling the correct placement and orientation of digital elements.

Benefits and challenges of extended reality

XR offers the following benefits:

• Design experimentation. XR enables designers or engineers to experiment and visualize prototypes in a three-dimensional (3D) space.
• Collaboration. XR enables remote collaboration in a shared virtual space.
• Immersive training. XR can replicate physical environments and situations that enable employees to train for complex or high-risk tasks.
• Customer engagement. XR can enable prospective buyers to interact with a product before purchasing it.

These technologies aren't without their downsides, however. Some challenges include the following:

• Mainstream adoption. User comfort, user interface, cost and general reluctance to adopt the technologies over traditional practices all impede a broader adoption of XR.
• Amount of data needed. High-quality XR experiences require the collection of large amounts of data, which also requires an equal number of resources.
• Hardware. Current headset hardware might be considered too heavy for prolonged use, and face limitations in battery life, sensor quality and graphical quality.
• Cost. Higher-quality XR experiences are more costly, making it a barrier for enterprise-level adoption.
• Eye strain and motion sickness. Some users might experience eye strain or motion sickness.

Key use cases for extended reality

Extended reality has numerous potential use cases across different industries, including the following:

• Product design. XR enables teams to visualize and make changes to 3D prototypes of a product.
• Entertainment. XR products can be used for entertainment like video games, movies or surfing the web.
• Retail. E-commerce platforms enable users to try on or interact with products using AR before a purchase.
• Training. XR enables an organization to simulate employee jobs for training purposes. This is especially useful in high-risk situations like simulating surgical procedures.
• Marketing. Organizations can use XR to create an interactive marketing campaign
• Remote work. Remote workers can connect to their peers in a single digital space.
• Architecture. XR can be used to simulate walkthroughs of a space. This could be useful in real estate or for simulating the walkthrough of museums or other public spaces.
• Navigation. AR is used to overlay contextual information like directions over a car's HUD, through a map on a smartphone or through smart glasses.

Future of XR technology

The future of XR will be shaped by current advances in hardware, software and connectivity that will improve the overall XR user experience, while also enabling the technology to be more immersive and accessible.

First, the technologies involved are likely to continue improving. For example, hardware improvements like lighter and more comfortable headsets are likely to be focused on, while software improvements like more accurate spatial mapping will make usage smoother. Also, newer versions of headsets often offer improved displays and processing speeds over their previous versions. Trends in enhancements like these are likely to continue, contributing to an overall improved user experience.

XR is also likely to see increased adoption. AR, for example, started gaining traction in 2016 and 2017 after the release of Pokémon Go and the AR view feature in both the IKEA and Amazon apps. As XR technology continues to mature, costs are likely to decrease, lowering the barrier of entry to more users.

According to Fortune Business Insights, the global XR market is also likely to continue growing. In 2024, it was valued at $183.96 billion and is projected to grow to $1,625.48 billion by 2032.

XR is likely to continue improving in the future, partially due to its diverse definition of the technology. Learn more about XR and its different mediums, such as AR, VR and MR.