Enhancing XR Experiences with LiDAR
Apple’s latest technological development has gone largely unnoticed even among many tech enthusiasts. While this new development lacks the flair of high-profile additions like iOS 14, 5G, or a 4K camera, this new hardware feature has already started to transform the up incoming AR and VR industry. We’re talking about the new LiDAR sensor available on higher-end iOS devices.
What is LiDAR?
While it has been around for a while, only recently has LiDAR (Light Detecting and Ranging) become small enough and cheap enough for consumer uses. It essentially allows you to scan and recreate a real-life three-dimensional landscape on your phone or device. It works by shooting many non-visible laser beam pulses from a scanner that travel forward until they bounce off an object or wall. Once a pulse hits an object or wall, it’s detected by the scanner, which then determines the distance between each of these pulse collisions and the original coordinates of the scanner. As this process repeats many times per second, different data points and distances are accumulated which allows the LiDAR device and scanner to create a 3D representation of the world around it. With additional processing, it can also track the movement, distance, orientation, and shape of all objects and walls in the area.
As you can probably guess, the main components of LiDAR are its laser, scanner, and processor. While this process is intensive and utilizes lots of traditionally expensive technology, Apple has implemented its own “terrestrial LiDAR” technology on its iPhone 12 Pro and iPad Pro – making it easier than ever to use LiDAR technology for your personal and business use. In this article, we will take a closer look at how LiDAR has grown, developed, and integrated itself in the AR and VR industry as well as the different AR and VR apps that use LiDAR to improve user experience.
LiDAR in AR/VR
Firstly, LiDAR differs from other tracking and mapping and tracking services such as RADAR and SONAR as LiDAR utilizes lasers signals whereas RADAR and SONAR use radio and sound waves respectively. Since LiDAR uses light signals, it can create a more detailed and accurate landscape due to its shorter wavelengths as the longer wavelengths from RADAR and SONAR tend to contribute to more small objects being missed and less detailed landscapes.
The ability to quickly create thorough and precise landscapes through LiDAR allows AR and VR companies to overlay their data and platform over an accurate three-dimensional map to create the most immersive user experience and enhance the existing 3D modeling and sharing workflows. Instead of a traditional two-dimensional picture or video meeting, users can scan, share, and experience a three-dimensional workplace that is more realistic and immersive than ever before. Essentially, this new technology has allowed AR and VR apps to save time and money when recreating and developing a high-resolution environment, which in turn allows remote work communication to be as seamless and instantaneous as possible. Furthermore, each of these precise environments has allowed remote workers to become more integrated with their surroundings and workers allowing for better communication, morale, and efficiency.
Apps with LiDAR
Spatial, MeetinVR, and Glue VR.
The goal for each of these apps is to provide an immersive office-like meeting setting for remote employees which allows for better communication and connection while these workers cannot be in the same location. For most of these apps, you put on your VR headset and are virtually transported to a meeting in which you are standing and working alongside your fellow employees. In each of these meetings, you can directly interact with coworkers, quickly discuss or write down your work and ideas, and screen share or upload any work material. The addition of LiDAR makes it possible to share 3D models from your environment in the real world with other users in virtual reality.
RoomScan & Magicplan
The goal for these apps is to transform the design and home décor industry by allowing users to quickly scan and create an accurate three-dimensional landscape of their home or room. Then, these apps utilize an intuitive interface that allows users to create an exact floorplan or layout (from this LiDAR scan) by dragging in potential furniture and objects that can be visually seen and compared. Moreover, each of these objects that are dragged into the landscape will be precise in terms of measurement and size, as well as space available.
Complete Anatomy & Airway Ex
The goal of these apps is to create a three-dimensional healthcare and surgery experience for aspiring and professional surgeons to practice surgery in the most accurate and realistic situations. With these apps, learners are placed in a realistic virtual world in which they can “investigate the minute detail of the human anatomy in incredible 3D.” Moreover, these immersive and descriptive environments allow health professionals to most accurately practice their skills in a controlled and safe scenario.
Case Studies
Spatial:
With the use of LiDAR technology within iOS devices, Spatial has been able to create an engaging digital meeting environment that increases connection, retention, and immersion among users.
For example, Spatial’s LiDAR feature has been used in:
- Barbie and Hot Wheels: A team of designers, engineers, marketers, and manufacturers that were scattered across the world joined in one business meeting which allowed each of the teams to reduce costs of traveling and increase communication and efficiency.
- BNP Paribas (A retail banking company): Allowed employees across different offices to review their 3D real estate plans in essentially “one room,” decreasing costs of travel and allowing for instant communication amongst employees of different offices.
- Vox writer, Adam Clark Estes, was able to sit down and join the co-founders of Spatial in a digital meeting. In this meeting, Adam described his time as “more real than a Zoom call” and “found [himself] marveling at the simple pleasure of looking around the room at the different people in it.” What Spatial and LiDAR technology has done is it has allowed the user to feel more connected and integrated with their fellow users which will hopefully boost confidence, morale, and communication across remote employees.
Complete Anatomy:
With the use of LiDAR technology on iOS devices, Complete Anatomy has been able to create accurate and immersive experiences and scenarios for aspiring and professional surgeons to learn how to perform different kinds of surgeries and strengthen their weaknesses.
Complete Anatomy has been able to teach:
- Basic anatomy and surgery processes to students at 350+ universities. Specifically, the University of Buckingham has used the platform to explore exact replicas of healthcare settings and patient interactions to best teach their learners.
- With Tidelands Health having opened a new multi-discipline outpatient clinic, many staff members need access to software that allows them to train and adapt to new situations. Complete anatomy provided three-dimensional anatomy for 100 staff members which has allowed the staff members within the health center to better train and learn new material as well as treat illness among the many new patients.
The future of LiDAR
In this article, we covered how LiDAR technology creates three-dimensional landscapes by emitting laser signals and using a scanner to ensure accuracy and precision. Furthermore, we covered the many AR and VR companies that have used this new technology to build more detailed and immersive user experiences in order to allow employees to better communicate and connect. Lastly, we discussed how LiDAR has transformed and benefited the many VR business meeting companies such as Spatial, MeetinVR, and Glue VR, the home décor industry with RoomScan and Magicplan, and the health industry with Complete Anatomy and Airway Ex.
While Apple has fully integrated LiDAR into its newest iPhones and iPads, there is still plenty of room for improvement within the AR and VR industry. In the future, look for increased speed of LiDAR processing, a higher maximum distance of LiDAR signals (and subsequent better recognition in cluttered rooms), and better overall resolution of scanned objects. Each of these limitations is currently or will soon be improved by Apple and other hardware developers, which will lead to the full-time expansion of LiDAR technology into the AR and VR workplace, classroom, hospital, and home.
