Bringing reefs to life on a mobile device.
The Smithsonian, The Hydrous and Adobe collaborate on the award-winning AR Reef project.
AR Reef project won multiple awards and was experienced by 10K+ people
Objectives
Raise awareness and offer solutions for climate change, coral bleaching and other issues affecting our oceans
Build compassion by exposing more people to inaccessible areas in underwater environments
Create educational content using innovative technologies and story-driven experiences
Results
Won multiple awards, including a Webby and multiple Anthem Awards and was experienced by more than 10K people
Optimised 3D objects to achieve strong AR performance on mobile devices
Used sophisticated triggers to create an immersive, interactive experience
The ocean has always fascinated Dr Erika Woolsey. Growing up in the San Francisco Bay Area, she spent many days in the water — swimming, diving and viewing tide pools. As an adult, this passion for the ocean combined with her love of biology, inspired her to pursue a PhD in marine biology with a focus on coral reef ecology.
Today, Dr Woolsey is spending more time on land as Chief Scientist and CEO of The Hydrous, a non-profit on a mission to create open access oceans. “Our oceans feed billions of people and coral reefs in particular support food sources and livelihoods around the world,” says Dr Woolsey. “Healthy oceans are important for everyone, but marine environments and the issues they face can seem very abstract and removed for most people. I wanted to find ways to bridge that disconnect, through technology, education and immersive experiences that bring the wonder of the oceans to people who can’t see it firsthand.”
The Hydrous teamed up with The Smithsonian Institution and Adobe to create an innovative and immersive augmented reality (AR) experience that brings coral reefs to life using Adobe 3D tools.
The project, AR Reef, won the 2022 Webby Award for Best Virtual Partner Experience while being nominated for the 2022 Virtual & Remote Best Installation or Experience. It also won two bronze awards in the Sustainability, Environment, & Climate category of the 2022 Anthem Awards. The Partnership or Collaboration Awareness award recognised how the project drove awareness of the oceans, while the Innovation Product, Innovation or Service award recognised how the project introduced a new technique for helping the non-profit community connect with the public.
“It's been such a privilege to be part of this wonderful example of art, science and technology coming together,” says Dr Woolsey. “I'm so pleased that with Adobe 3D tools, we can truly bring an experience of the ocean to people wherever they are.”
Creating an award-winning experience
The Smithsonian has more than 157 million specimens and objects in its collection. But due to physical space restrictions or concerns about preservation, fewer than 1% of those items are available for public display. To research, education and exhibits for wider audiences, the Smithsonian is undertaking a historic project to digitise its collection using 3D scanning technology.
Vincent Rossi, 3D Programme Supervisor at the Smithsonian Digitisation Programme Office, works with the 22 museums and 9 research centres across the Smithsonian organisation to unlock some of their most unique and priceless artefacts and specimens. All 2,500 scans completed so far are available to the public through the Smithsonian 3D Viewer. People are free to interact with them online, download them, 3D print them or use them as part of projects and stories to share Smithsonian artefacts and specimens with wider audiences.
Onsite photogrammetry data QC and processing.
Image credits: The Smithsonian Digitisation Programme Office
“The ultimate goal is to make the Smithsonian collection more accessible to researchers and educators,” says Rossi. “It’s why we pushed the Open Access initiative. Our Open Access items carry a CC0 designation, which means that it can be used by anyone, for any purpose. We just want to see content used across as many platforms as possible to help it to get seen by as many people as possible.”
During the COVID-19 pandemic, with museums and schools closed, Adobe and the Hydrous partnered with the Smithsonian to combine science, technology and art to create an immersive and educational experience that would bring the ocean into peoples’ homes.
“I’ve worked with augmented reality for art, prototyping and more,” says Vladimir Petkovic, Creative Director, Substance 3D team at Adobe. “But the AR Reef project gave me the unique opportunity to help guide our planet to a better future.”
The Smithsonian 3D team scanned more than 90 type specimens of coral — specimens that convey the defining features of each species of coral. By bringing these coral to life again in 3D, the Smithsonian can share these one-of-a-kind specimens that are typically never accessible to the public.
Screenshot of photogrammetry data processing.
Image credits: The Smithsonian Digitisation Programme Office
“Starting with the coral, we saw an opportunity to tell a unique story about climate change and share the beauty of the oceans with people who might never see it in person,” says Petkovic.
Adobe worked closely with marine biologists at The Hydrous including Dr Woolsey and Celine de Jong, to create an engaging, science-based story. The experience first introduces the concept of coral bleaching, a phenomenon where corals will turn white in warm water. Then, the story shifts to something more hopeful, explaining the coral lifecycle, showing how coral can recover and sharing the beauty of coral reefs. Finally, the experience suggests actions that we can all take to help coral recover.
Adobe brought its 3D and storytelling expertise to the project. Artists created an interactive diorama that combines the 3D scanned corals with rocks, plants and marine life. With Adobe Substance 3D Sampler, artists could transform real-world pictures into realistic 3D materials. Artists then used Substance Painter to apply photorealistic and scientifically accurate textures. This way, they were able to bring the life back into the original coral samples, some collected more than 170 years ago.
“Adobe Substance 3D Painter makes it possible to use textures to emulate high fidelity surface details for fish or coral on an optimised, mobile-ready 3D mesh,” says Petkovic.
Render depicting (from left to right) a final coral 3D model w/ colour, a coral 3D model showing only vertexes, a 3D model showing on surface normals.
Image credits: The Smithsonian Digitisation Programme Office
Delivering the experience with AR, Petkovic added interactive elements such as sound, narration, buttons or pop-up infographics that provide many ways to engage viewers. Some effects start up based on proximity, such as sounds that play when a viewer focuses on an object. Viewers can click buttons to display more information, including details about specific coral examples from the Smithsonian collection or poke at the diorama to find hidden inhabitants of the reef.
Petkovic layered multiple triggers to achieve certain effects. For example, the main AR Reef experience starts when the viewer clicks a button on their device where they then see digital assets overlaid on real physical surroundings. The viewer can move around to interact and view multiple perspectives of the 3D coral reef. Sounds start playing and as the animated infographic appears, narration from ocean activist and science communicator Danni Washington guides viewers through the experience. These interactions allow viewers to become more active participants who become immersed in the experience.
The collection corals at the National Museum of Natural History.
Image credits: The Smithsonian Digitisation Programme Office
“AR Reef has the power to guide people through an immersive and interactive ocean experience and the project has been a source of inspiration for my students and colleagues.”
Dr Erika Woolsey
Chief Scientist and CEO, The Hydrous
Inspiring the future of scientific engagement
AR Reef, which has been launched by more than ten thousand people so far, serves as an example of the ways that 3D can be used as a tool for teaching and engagement. AR can improve accessibility to places and experiences that many people cannot see firsthand.
“The entire Smithsonian digitisation programme is built around the idea of access. We’re only beginning to figure out how we can leverage AR to support our goals,” says Rossi. “It’s visual and compelling while providing a level of interaction that is unlike anything else out there.”
“AR Reef has the power to guide people through an immersive and interactive ocean experience and the project has been a source of inspiration for my students and colleagues. Augmented Reality is a promising tool for ocean science education and I hope it becomes a focus of research in the future,” says Dr Woolsey.
Recommended for you
https://main--bacom--adobecom.aem.live/fragments/resources/cards/thank-you-collections/substance
