spatial computing

What is spatial computing?

Spatial computing broadly characterizes the processes and tools used to capture, process and interact with 3D data. Components of spatial computing can include IoT, digital twins, ambient computing, augmented reality, virtual reality, AI and physical controls.

The term spatial computing was coined by Simon Greenwold, who described the concept in his 2003 master's thesis at MIT as follows: "Spatial computing is human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces. It is an essential component for making our machines fuller partners in our work and play."

Spatial computing in action could be as simple as controlling the lights when you walk into a room or as complex as using a network of 3D cameras to model a factory process. Spatial computing concepts also play a role in orchestrating autonomous computing applications in warehouse automation, self-driving cars and supply chain automation.

Users interact with spatial computing applications through screens embedded on physical devices, VR headsets that mirror the physical world, or mixed reality devices that overlay data onto a view of the physical world.

Spatial computing also connects the dots between virtual worlds in the metaverse and so-called digital twins, which represent physical space, objects, processes or people, thus providing a bridge between the digital and the real world, said Brannin McBee, co-founder and CSO of CoreWeave, a cloud GPU provider.

How does spatial computing work?

Spatial computing mirrors how we interact with objects, people, animals and goals in the real world. Humans translate the 2D images from our eyes into a 3D model of the world, make sense of objects in the world, and then direct our hands to act. For example, when we pour a cup of tea, we watch the cup as we pour, determine when the cup is full, and then stop when the cup is full. Spatial computing does the same, but with sensors, computers and actuators.

Spatial computing involves multiple steps.

1. First, techniques such as photogrammetry, lidar and radar capture a 3D model of the world. Lidar or radar capture a 3D model by measuring the reflection of a laser or radio signal off objects around a scanner to automatically capture a point cloud representing the distance to each point. Photogrammetry, described as the art and science of creating 3D models from photographs, combines imagery from multiple images or cameras. Newer AI techniques like neural radiance fields (NeRF) can capture a richer representation using a handful of images.
   
   
2. Second, techniques like machine vision analyze this data to make sense of the imagery. AI techniques help identify individual objects in a scene, look for defects, understand gait patterns or analyze how different workers perform a process. In construction, for example, these techniques can help monitor the progress on a job, pinpoint problems like a door not closing properly and identify snafus such as a contractor forgetting to install power outlets.
   
   
3. The third aspect of spatial computing concerns taking action. For example: A self-driving car detects a pedestrian in front and decides to stop the vehicle in real time. A building control system decides to adjust the heat or the light when someone walks into a room in response to their preferences stored in a database. A construction management system schedules a team to install the missing plug. The actions in these examples are possible because of the analysis on the digital 3D image that was captured from the physical processes.

Key features and benefits of spatial computing

Spatial computing can improve enterprise processes in many ways, including by doing the following:

• aligns computer programming with how humans think of the world
• enables new physical workflows
• combines data from multiple types of sensors to streamline user experience
• automates the process of creating digital twins
• connects the dots between robotic process automation and physical automation
• enables new ways of interacting between people, robots and products in physical space
• helps companies measure the performance of physical process variations
• enables the orchestration of multiple physical processes
• improves the design of physical facilities and processes

Industry use cases for spatial computing

Spatial computing is transforming every aspect of industries that involve manufacturing and moving physical products. It also plays a role in improving the use and management of office spaces and facilities. In addition, spatial computing is being used to enhance the operation and user experience of physical products.

• In a manufacturing facility, spatial computing can help monitor the production line at every step of the process. This can help identify the different steps involved in making a product. It can also determine when and why different teams might take different approaches during their shift and the impact that has on time and quality.
• Spatial computing can also combine data about the physical location of products in a large warehouse with the movement of robots and humans that pack these goods. This can help guide people and robots to the products or suggest better routes. The capability can also be used to simulate different layouts to improve overall efficiency and reduce worker burnout.
• There are many ways spatial computing could play a role in offices. Property management firms could use spatial computing to build a model of an office overlayed with different layouts to optimize the use of space. Automated lighting and environmental controls could be programmed to adjust lights and heat to worker preferences. In a hospital, location tags could help teams procure special equipment in an emergency.
• Physical products are being transformed by spatial computing, as seen in the crucial role spatial computing plays in guiding autonomous things like cars. Other uses are limited only by our imagination: Digitized furniture (e.g., beds, kitchen tables, TVs) could automatically move depending on the time of day and the occupant's schedule. Settings could be changed at the wave of a hand or a specific gesture.

Examples of spatial computing

The following are examples of spatial computing:

• a mixed reality headset overlays a repair manual to guide the technician
• a network of cameras automatically models a car production process
• a spatial model of the production process lets managers simulate variations to optimize the process
• teams improve product quality by connecting production defects with specific steps in a process and changing those steps
• offices dynamically tailor office lighting and environmental controls to individual workers
• ergonomic analytics programs coach employees on how to reduce harmful movements
• occupancy analytics programs automate elderly safety checks for relatives and caregivers