8th Wall introduces Generative AI tools to its WebAR platform

In Augmented Reality News

TL;DR: 8th Wall has announced new GenAI modules that allow developers to integrate Inworld and OpenAI into their WebAR experiences. The new modules simplify the integration of GenAI into WebAR projects, allowing developers to create more dynamic, interactive and engaging experiences. Users will now be able to actively shape experiences, as opposed to just passively consuming content, according to 8th Wall.

August 30, 2023 –  8th Wall, a subsidiary of Niantic and a provider of a web-based augmented reality (WebAR) development platform, has announced the introduction of Generative AI (GenAI) Modules and sample projects to its platform for Inworld and OpenAI.

The introduction of GenAI combined with the immersive reality of 8th Wall’s WebAR will now offer developers, brands, and agencies a new way to tap into seemingly limitless creative resources and give audiences greater control of immersive experiences.

8th Wall noted that its platform already allows developers to integrate with GenAI tools to accelerate development and build WebAR experiences. This technology includes DALL·E and Stable Diffusion to add text-to-image input, Blockade Labs to add 360-degree images to the Sky, Inworld AI to incorporate human-like interactive characters, Hugging Face and ControlNet to add texture to the face, or Move.ai and Wonder Dynamics to add realistic movement.

With this latest announcement it will now be even easier for developers of all skill levels to build WebAR experiences with GenAI thanks to newly integrated GenAI Modules and sample projects for Inworld and OpenAI. According to 8th Wall, these new Modules help to remove the development overheads that typically stand in the way of adding GenAI technology to WebAR experiences, providing developers with a hassle-free way to bring the tech into WebAR experiences. Developers only need to add the Module to their project to enable GenAI-powered capabilities within an experience.

AI-powered filters on social apps such as TikTok have consistently gone viral in recent months, with one of the big reasons likely being down to the sometimes unpredictable nature of what the AI ends up generating. By combining WebAR with GenAI, developers are no longer creating single-use content. Instead, the two technologies combine to create dynamic, evolving experiences that respond to end-user interaction, which helps to keep users more engaged and wanting to re-visit filters and experiences in order to create new outcomes.

Plus, with the 8th Wall platform, developers are able to build once and deploy everywhere, in order to enable a consistent experience across smartphones, computers, tablets, and virtual and mixed reality devices.

Inworld’s Character Engine

Inworld is a powerful GenAI technology that enables developers to infuse their WebAR experiences with interactive and lifelike AI-driven characters. These characters, avatars, and brand ambassadors can learn, adapt, and engage with users in a more personal way.

8th Wall stated that it collaborated with Inworld to create the Inworld AI Integration Module as well as a sample project. The module allows developers to combine Inworld’s avatars with 8th Wall’s tools to offer new interactive experiences. For example, developers can combine Inworld with Image Targets or World Effects to create an experience where audiences see familiar character avatars on product packaging and initiate a natural conversation with them. With Inworld’s ‘Character Brain,’ avatars can mimic the entire range of human expression, which means conversations differ every time and are contextual, according to 8th Wall.

Developers can now incorporate such characters in their 8th Wall experiences without the complexities of server provisioning or token authentication work. Once added to a project, the Module takes care of the backend work, meaning developers only have to enter an API key to get started. Using the sample project, developers can build their first Inworld and 8th Wall experience, which uses Image Targets to initiate a brand encounter with a fictitious brand mascot, Rainbow Ray.

8th wall stated in a blog post on the announcement: “As the future of mixed reality takes shape, AI-powered characters like those powered by Inworld will be a crucial component of building virtual worlds, providing users with endless possibilities for interaction across digital spaces.”

New 8th Wall Integration Module for OpenAI 

As part of this latest release, 8th Wall has also introduced new developer tools that harness the power of OpenAI’s DALL·E and ChatGPT for image generation and AI-powered text chat.

On its own, OpenAI’s DALL·E is a powerful GenAI model that creates highly detailed and creative images based on text-based prompts from a user. When DALL·E is combined with WebAR, developers can use these text-to-image prompts to create experiences that put creativity in the control of the users. Users can then enter unique prompts that generate images to augment the sky, conform to faces, reimagine real-world objects, or even as new textures within the WebAR experience, according to 8th Wall.

Similarly, ChatGPT enables developers to create personalized and interactive WebAR experiences by understanding and responding to user inputs in real-time. 8th Wall stated that developers can now use this type of interaction to develop text-based conversations with a WebAR character, collaborate with ChatGPT to place objects in a scene, or even create experiences that combine DALL·E and ChatGPT to make real-time changes.

Like the Inworld AI Integration Module, a new Integration Module for OpenAI helps to save time for developers and sets up a project code to call DALL·E and ChatGPT. Once developers add the Module to an 8th Wall project, they simply need to add an OpenAI API key, and can then start building.

Again, 8th Wall has created a sample project that developers can use to test incorporating AI into WebAR experiences and discover how to use DALL·E to generate a 3D texture via text and apply it to a virtual object in real time. Additionally, developers can customize the ChatGPT prompt box to give users another way to control the elements of the experience.

Commenting on the tools, Ian Curtis, XR Developer at 8th Wall said: “What excites me most about integrating Generative AI into Augmented Reality is that it empowers every user to become an architect of their digital world. They’re no longer passively consuming content but actively creating it, shaping their experiences with the power of their imagination.”

Developers can now harness the potential of GenAI to transform WebAR experiences with 8th Wall’s newly announced tools, and can merge GenAI with many of the platform’s features, including but not limited to Image Targets, Face Effects, World Effects, Sky Effects, Lightship VPS, and Shared AR tools.

For more information on 8th Wall and its tools for developing and creating WebAR experiences, please visit the company’s website.

Image / video credit: 8th Wall / YouTube

About the author

Sam Sprigg

Sam is the Founder and Managing Editor of Auganix. With a background in research and report writing, he has been covering XR industry news for the past seven years.