The year is 2020, and the ongoing quest for improvement has led to a blast from the past: LiDAR. Just when it felt like augmented reality hit a wall with the detection and recognition of scenes and objects, LiDAR provides a breath of fresh air to advance AR to the next level.
Today, Apple revealed the iPhone 12 Pro which packs its very own LiDAR-capable sensor. We previously discussed this feature in the iPad Pro, so seeing it on the iPhone is no surprise. For the uninitiated, the name “LiDAR” may give a clue to how it works. By using “light” the way “radar” uses radio waves, a LiDAR (Light Detection And Ranging) sensor fires an array of lasers and then measures how quickly the beams are reflected back. The result is a hyperrealistic topographic map of the environment and is the very same method being used by NASA to map the surface of Mars.
Imagine you’re playing with a flashlight in a dark room. When you shine it on something, the light bounces back to your eyes. LiDAR does something similar but with lasers. It sends out quick laser pulses and times how long they take to come back. By doing this lots of times really fast, it can build a detailed map of the area. It’s like how a bat uses sound to navigate in the dark, but way cooler because it uses light! It’s like a super-accurate measuring tape, but with lasers — and who doesn’t love lasers?
Why is this such a breakthrough for augmented reality? Because previous mobile AR experiences were only as good as the device’s camera allowed them to be. Limited by a single viewing angle, features like depth were easily thrown off by bad lighting and can’t always be counted on in the real world. By generating an accurate depth map with LiDAR, the iPhone 12 Pro will be able to place AR models accurately within its environment instead of relying on a shaky camera snapshot.
DOES MY PHONE HAVE LIDAR?
As of 2024, LiDAR technology has primarily been integrated into select models of Apple’s iPhone lineup. Specifically, iPhones such as the iPhone 12 Pro, iPhone 12 Pro Max, iPhone 13 Pro, and iPhone 13 Pro Max feature LiDAR sensors. Besides Apple’s iPhones, some other smartphones also incorporate LiDAR technology. For instance, certain models from manufacturers like Samsung and Google have begun integrating LiDAR sensors into their flagship devices. Samsung’s Galaxy Note series and Galaxy S series, as well as Google’s Pixel lineup, have introduced LiDAR capabilities in select models. However, the availability of LiDAR on these devices may vary depending on the specific model and release year. Users interested in utilizing LiDAR on smartphones other than iPhones should check the specifications of their respective devices to determine if this feature is included.
RAISING THE BAR
With a sensor that can create accurate environment maps, what’s on the table that wasn’t before? With depth on our side, developers have access to obscenely accurate measurements that play nice in any condition. Say your project needs a pinpoint accurate map of the room. Instead of detecting the room piecemeal, one wall or floor at a time, a LiDAR sensor creates a full 3D wireframe all at once, delivering both speed and accuracy. Gone are the days of digital content jittering around with the camera or hovering in space where they don’t belong. From mapping outer space to self-driving cars, LiDAR’s uses are numerous and have upped the standards in the AR game for good. What other applications use LiDAR?
- Gaming and Entertainment: Create immersive experiences and realistic virtual worlds. By accurately scanning rooms and environments, LiDAR enables augmented reality games and apps to interact with real-world surroundings, bringing characters and objects to life in a player’s own space.
- Home Improvement and Interior Design: Allows users to visualize furniture and decor in their homes with precision. By scanning rooms and generating accurate 3D models, LiDAR simplifies the process of planning renovations, arranging furniture, and exploring design ideas in virtual space.
- Fitness and Health Tracking: Measure body dimensions, track movement, and analyze posture. By capturing detailed 3D scans of the body, LiDAR enables personalized fitness routines, posture correction, and virtual fitting experiences, enhancing user engagement and promoting healthy lifestyles.
- Home Security and Monitoring: Create applications in home security and monitoring systems to detect motion, identify intruders, and enhance surveillance capabilities. By scanning rooms and analyzing spatial data, LiDAR enables smart security cameras to distinguish between humans and pets, reduce false alarms, and provide homeowners with peace of mind.
Between ARKit and AR-friendly hardware, Apple continues to serve as a powerful enabler of augmented reality. As developers, each new conference feels like Christmas, and we look forward to utilizing LiDAR to build better projects and continue smashing expectations.
