Welcome to the Augmented Reality Universe

Just imagine this. You land at the Denver International Airport in the US with barely 20 minutes to catch your connecting flight from a different terminal. So, you need to walk, take a train, and walk some more. Today, you have one option—run—with no idea if you will make it in time or end up missing your flight. But not in the augmented reality (AR) world of the future: as you step out of your aircraft, your smart lenses would scan the nearest gate number, search the airline database for the connecting flight’s gate number and, using the airport’s indoor maps, project directions on the lens itself and also display your estimated time of arrival basis your walking speed. If you have time, you could canter along. Glancing through the shops on the way would overlay the smart lenses with the in-store collections, which you could simply add to your cart and pay for with a voice command, and collect within minutes—all with the assistance of your smart lenses. Cool, right? This is a sample of what AR can power, not in the distant future, but barely within a decade from now. (And yeah, in case you’re wondering, if you’re really late, you still have to run!)

Pegged to be a promising technology over the past few years, AR uses technology to enhance reality with digital components. Think of it as a virtual layer added to the real world. And this layer, which is mostly visual with auditory or haptic, creates a composite view that can be used to convey additional information and create an immersive experience for the end user. It is, essentially, a real-time augmentation of the physical world.

AR was invented in 1957 by Morton Heilig, followed by US scientist Ivan Sutherland, who developed the first head-mounted display system in 1968, although the term ‘augmented reality’ was coined in the 1990s. Since then, the technology has come a long way, touching upon almost every industry—from NASA leveraging AR to assist in providing better navigation during test flights, to Volkswagen debuting the MARTA app (Mobile Augmented Reality Technical Assistance) for providing technicians with step-by-step repair instructions within the service manual, to Google developing AR glasses for immersive experiences, AR applications are being used in manufacturing, training and education, maintenance, simulation of complicated procedures, etc. It has consumer use cases, too—filters on social media platforms giving motion to animate objects; students viewing an information layer replete with visuals and audio by placing a mobile device on a three-dimensional toy; checking out how furniture will look in real life using the Flipkart app, among many others, are some everyday use cases.

In 2020, the AR market was valued at $14.7 billion globally, and is projected to reach $88.4 billion by 2026, according to Research and Markets, which incidentally has valued the Indian augmented and virtual reality market collectively at $1.83 billion in FY20 and forecasted to grow at a CAGR of 38 per cent until FY27. “The report is bullish on the share of APAC in the next five years, and that definitely augurs well for India, as we are in a good position to leapfrog and become market leaders here,” says Sachin Arora, Partner & Head-Digital Lighthouse (Analytics, AI and Data), KPMG in India.

Impressive tech potential. Impressive numbers. But that’s still scratching the surface. What’s holding it back? Multiple things, ranging from hardware to software and apps, deployment to use cases. With users expecting instant and highly customised solutions, the existing deployment simply isn’t measuring up. The majority of the AR hardware (including headsets and glasses) available today are bulky and many need to be connected to a computing device such as a PC, tablet or smartphone. Google Glass, introduced in 2013, was way ahead of its time, but suffered from short battery life and overheating. In the current age, Epson’s Moverio smart glasses continue to be deployed across industries but even its new-age Moverio BT-40 Smart Glass, which was launched in 2021, requires external hardware support.

Another shortcoming of these AR glasses has been the sensors. Industry experts explain that a sensor such as an accelerometer is easily disturbed by electric interference. So, what’s the solution? “AR, powered by LiDAR, will enable AR creations to appear more lifelike and avoid clunky experiences,” says Ranga Jagannath, Director-Growth, Agora, a real-time engagement platform based in Santa Clara, California. (LiDAR stands for light detection and ranging, a method of using light pulses to measure distances of objects on the Earth’s surface.) And as companies such as Mojo Vision experiment with hardware, AR will evolve from just smart glasses to smart contact lenses.

Everyone’s excited, from start-ups to the biggest technology firms. Apple, Microsoft, Google, Facebook, NVIDIA are fuelling investments in the development of AR software and hardware. Apple is working on its own AR hardware, and Qualcomm Technologies and Microsoft collaborated to develop custom AR chips to enable a new range of power-efficient, lightweight AR glasses to deliver rich and immersive experiences. Greg Sullivan, Director, Mixed Reality at Microsoft, says: “Today’s AR devices, like HoloLens 2, are targeted at the enterprise, but we are absolutely focussed on continued innovation and are building devices for the future that will serve both enterprises and consumers.”

Other hardware firms, too, have gone back to the drawing board to address the existing challenges. They are experimenting with real-time offloading of AR computations to a high-performance server using edge computing in a wireless network. If that sounds too nerdy to you, here’s what it really means. Edge computing-embedded AR headgear will not need to be connected to external hardware such as a smartphone or a PC. Instead, they will connect to an edge server, which will remotely provide access to apps through CPU- and GPU-intensive AR algorithms. All this would need high-speed wireless networks such as 5G (and 6G in the future). Why? You don’t want your immersive AR experience to buffer and stutter, right? “Latency, connectivity, and infrastructure are some key challenges that the AR industry will need to address,” says Sachin Kalantri, Senior Director, Product Marketing at Qualcomm India.

Thirty-two per cent of XR or ‘extended reality’ experts in the US surveyed by Statista in early 2020 cited user experience (bulky hardware, technical glitches) as the biggest obstacle to mass adoption of AR. Lack of quality content and the amount of AR content available came in second, with 18 per cent highlighting this. These two were followed by a reluctance to embrace AR by consumers and businesses, regulation and legal risks, financing and investing, and cost to consumers. So, even as the tech is being sharpened and perfected, cybersecurity and regulatory risks will also come in at some point. Right now, nobody knows much about them yet.

“While an exciting field, there are still several challenges regarding AR implementation on a wide scale. The cost would be the primary consideration at this stage of the journey, with regards to the infrastructure set-up and hardware requirements,” says Arora of KPMG. “Building a custom app especially will require greater financial investment than instances where an AR layer to an existing app will suffice. Since this is a relatively nascent sector, availability of skilled developers and designers in the market may also pose challenges.” Enterprises across the world are working towards making emerging technologies such as AR more widespread. “We’re working to make emerging technologies like AR more mainstream, which is why we’ve extended our partnership with CBSE to provide an AR curriculum to over 10 million students and 1 million educators,” a Meta (erstwhile Facebook) spokesperson told Business Today. Meta has also teamed up with Reskilll in India to teach students, developers, and tech enthusiasts about Spark AR Studio i.e., a platform that allows creating AR effects for mobile cameras.

While the challenges are being worked on and will be overcome as the technology matures, there are immense opportunities in the developer space to design innovative applications addressing a wide range of use cases. Let’s start with banking. Despite consumers migrating in droves to digital payments, branch banking remains at the core.

While banks would initially leverage AR to guide customers to the nearest ATM or sell new banking products, a decade from now, physical banks could increasingly be replaced with virtual ones. And as all services would be provided virtually, customers will continue to be the centre of everything. Using AR, customers can visit the virtual bank, interact with their relationship manager, open an account, even carry out a transaction with biometric authentication while sitting at home.

Similarly, shopping in the market, swiping or tapping cards would become redundant. AR-enabled devices would identify the commodity, its price and with the users’ biometric authentication, would process the payment. Insurance firms, too, will utilise this technology for investigating damages, explaining insurance policies and selling them, too.

Then, AR can transform urban planning. Building prototypes will be passé as AR tech efficiently designs a new smart city and redevelops existing ones. Be it an overlay on barren land, placing a bridge on a traffic intersection or planning an underground tunnel underneath a monument—a 360° life-like view could be overlaid without disturbing traffic movement or incurring any additional cost. When deployed with high-performance computing, architects can identify challenges posed by wind speeds, soil quality or a natural calamity the area might be prone to. “Imagine a landscape architect who is trying to explain his/her vision of a new backyard to a client. Using AR-powered smart glasses, the client will be able to see all the details clearly, as if in real life,” says Kalantri of Qualcomm India. Driven by data, AR experiences will be accurate, and eliminate cost of constructions and prototypes, resulting in a much more efficient deployment.

In healthcare, AR will make training of doctors easier, but can also help in surgeries by putting information into the eyesight of the surgeon. Using real-time data and scans, AR can locate tumours and blood vessels, resulting in successful surgical procedures. For neuropsychological conditions, specialised applications will use facial expressions, voice tone and behaviour to identify depression, phobia, anxiety and other mental health illnesses. The addition of AI/ML tech could truly personalise these AR experiences and help in treating symptoms. Grand View Research expects AR-VR in the healthcare segment to be a $9.5-billion market by 2028, expanding at a CAGR of 27.2 per cent from 2021 to 2028.

While AR and VR have been adopted by teams to enhance sportsmen and athletes’ methods to train or view statistics on their performance, or to increase fan engagement, the ultimate deployment would be playing sport in real-time with the player in a different country. In the future, AR will have the calibre to power global sporting events such as the Olympics virtually.

AR has the potential to further bridge the real and virtual worlds. Industry expects AR to replace VR as the metaverse matures. “In the metaverse, VR is one part today, primarily because it is immersive and cheap. [But] AR will take over from VR because everyone will stick to their smartphones (commonly used by consumers for AR experiences) over VR headsets,” says Hemanth Satyanarayana, Founder & CEO, Imaginate, a home-grown VR, AR and MR tech company.

From enhancing photos in low light to smart agriculture to assisting in counter defence, the possibilities of using AR are endless. But none of this will be possible without advanced tech solutions. And tech such as edge computing, 5G, AI and ML will revolutionise the AR experience over the next decade. Augmented reality, as a technology, has the calibre to become as ubiquitous in the workplace as PCs and mobile phones are today.

Wrap your head around that.

@nidhisingal