How AI-Powered Augmented Reality Transforms Digital Experiences
With the emergence of spatial computing, technologies like augmented reality (AR) — fueled by artificial intelligence (AI) — are moving the boundaries of the digital world beyond screens and into the free-flowing, multisensory 3D world. Spatial computing uses the space that surrounds us in the physical world as the medium for digital experiences.
This bridges the physical and digital worlds, opening up a new realm for creators and marketers to explore and experiment. Add to that the ongoing proliferation of voice interfaces, and it is clear to see that consumers are demanding more experiences that provide information in clever ways and stir emotions. Voice and immersive media can seamlessly permeate the environment around us and fulfill that demand with fresh, personal, and meaningful experiences.
“We think AR, combined with AI, can be a killer combination of experiences unlike anything we’ve seen before,” said Abhay Parasnis, EVP and CTO at Adobe.
AR promises retail environments where people can interact digitally with physical objects. For example, companies are already experimenting with using AR to give consumers X-ray vision to see internal construction or details of products, such as size and colors available in stock. AR also promises to impact the future of work, with more remote meetings in which on-site staff can collaborate creatively with virtual colleagues, as if they were working in the same room.
However, to realize the full promise of AR, the tools that let artists express their creative vision must evolve as well. Spatial design requires moving beyond current physical and digital tools to a multisensory approach that uses natural inputs, like voice, sight, touch, and gestures. As the means become more intuitive, the mechanics will fade into the background. Then, artists, designers — everyone — can explore the new dimension and craft experiences that know no limits.
Enabling AR With AI
For designers, the mental shift from 2D to 3D — and spatial design — requires new tools and skills to work and thrive in a different dimension. AI is enabling the shift to more realistic and engaging experiences and more powerful personalization.
Creating AR experiences that look and feel convincing calls for rendering, lighting, materials, and objects that behave according to real-world rules. Using AI and machine learning, 3D design tools can handle time-consuming tasks and create photorealistic results that previously required costly photo shoots or hours of creative work.
“AI is enabling creators to develop immersive experiences much faster and with greater fidelity than ever before, and promises to empower individuals to interact with those experiences in real time,” said Tatiana Mejia, product marketing and strategy at Adobe.
Indeed, AI provides invaluable capabilities beyond accelerating content creation. For example, personalizing experiences and providing contextually relevant information in real time requires prodigious processing power. Learning from users’ reactions and predicting what they’ll want next demands instantaneous decision-making. This is where AI shines. AI can help designers use data on customer preferences to show products in 3D instantly, with favorite logos, colors, and sizes.
On the flip side, AI and machine learning can help experience businesses learn from every customer interaction so they continually become better at delivering the right offer. And using computer vision and neural networks, AR can lower barriers to online purchases, letting customers virtually try on clothes and experiment with make-up colors.
AI-powered tools for spatial design will make it easier to bring stories to life around us. Creatives working on the forefront of AR are already offering a hint of the limitless possibilities.
Bridging The Physical And Digital Worlds
“We’re used to art that lives in the physical world,” said Stefano Corazza, head of augmented reality at Adobe. “And we’re familiar with interactive experiences that live on a screen. AR is bridging these two worlds. It’s providing new flexibility and fluidity to the physical world.”
With AI-powered AR, object recognition and tracking, gestural input, eye tracking, and voice commands can combine to let users engage with 2D and 3D objects in virtual space with hands, eyes, and words. Visual artists gain a new medium for expression, brands and consumers gain a new way to interact, and colleagues gain a new way to communicate.
While many of the applications of AR have yet to be discovered, below we look at four ways artists, designers, and brands are using AR to make experiences more expressive, informative, interactive, and collaborative.
Example 1: Animation Anywhere
AR can transform everyday experiences into the extraordinary. And for some, AR provides a new way to express feelings that can’t be captured in any other way.
Artists Zach Lieberman and Molmol Kuo incorporated multisensory design capabilities in the iOS AR app Weird Type, which lets users write and animate anything, anywhere, in 3D space. That led to people using the app to tell a story about location, build sculptures, and describe sensations difficult to put into words, like the feeling of the wind.
With AR, companies can connect to customers through all of their senses. Combining 2D and 3D in AR can also help sell a vision, such as a new product design, to stakeholders and clients. Designers and developers can work together to build engaging experiences that marketers distribute via a brand’s app. And in addition to retail, AR offers endless opportunities in other realms, including medical training, tourism, advertising, and entertainment.
Example 2: Augmented Retail Experiences
A demonstration at the 2019 Adobe EMEA Summit, as reported by CNBC, showed how customers in a Foot Locker store could rotate and “explode” an AR shoe to learn about its construction, see a model in various colors, virtually discard rejected options, and check the stock for shoes in the appropriate size. All the information a customer could want appears as if by magic with the power of AI — and without ever asking a clerk for help.
Example 3: Immersive Art
A recent San Francisco Museum of Modern Art exhibit featured interactive artwork inspired by René Magritte. As reported by NVIDIA, visitors could insert themselves in paintings via an AR experience, mixing their images with the art. The relationship between creator and audience transforms, as AR allows consumers to “add” to a design. Interacting with Magritte’s work created a strong connection between visitors and the art, deepening their appreciation of it.
Example 4: Close Collaboration From Afar
Spatial computing lets colleagues collaborate from anywhere in AR, making them more productive and creative — and helping everyone feel more connected. Users can share their reality with transported 3D copies of colleagues, point at objects of interest, and write on virtual post-it notes they can even stick on the wall.
“AR challenges us to design for human perception, not pixels, when we integrate sight, speech, movement, haptics, and context,” said Silka Miesnieks, head of emerging design at Adobe.
Overcoming The Boundaries
In the era of AR and AI, artists and designers, marketers and developers — and eventually everyone — will make the shift to the multidimensional, multisensory, immersive world. We must rethink design principles, the creator’s role, and reimagine experiences to center on what humans, not devices, can do and with deep understanding of what moves them.
As immersive technologies transform creativity, creative tools should set designers free to work and play in new worlds and to think in disruptive new ways, no longer constrained by the boundaries of the past. AI can help speed creation and personalize the experiences. But for mass adoption, creative tools must become more intuitive and contextually appropriate.
With further advances in machine learning, the tools professionals already know will become more intuitive, assisting humans to explore spatial design with more integrated, seamless workflows. Voice recognition using natural language commands can simplify user interfaces and streamline processes.
Spatial computing promises solutions to previously unsolved problems, such as better city planning to address climate change, medical breakthroughs, and education that caters to a greater diversity of abilities using our sense of spatial memory through AR. It will challenge designers to create more immersive experiences while being transparent about how experiences are created, enhanced, and delivered.
“I believe AR could be bigger than the web,” said Scott Belsky, chief product officer of Adobe Creative Cloud. “But we have to unlock artists’ full potential in this new medium. And we need rules and standards for where, when, and how AR appears. It can be a lifesaver, but it can also be dangerously distracting. We’ll find the boundaries through responsible exploration.”
Learn more about AR, AI, and spatial computing from the following resources:
- Adobe Blog: Technology (AR, AI, and virtual reality)
- The AR Show Podcast
- Coursera: Getting Started With AR
- Spatial Computing Design Resources