Advances in artificial intelligence are dominating the mainstream conversation. AI’s applications seem limitless – and it’s no surprise that it will transform our world and the way we interact with it. In addition to many other industries, this applies in particular to augmented reality.
AI models are already being used to create immersive AR experiences, particularly in mobile device apps. The fields of augmented reality and artificial intelligence work together to create intuitive, unique experiences that better blend the real and digital worlds. The next frontier in AI and AR development is the implementation of advanced AI capabilities in consumer AR glasses to expand the possibilities of immersive interaction.
Generative AI will accelerate the creation of 3D worlds, creating unlimited possibilities for wearers to create their own realities
Generative AI is characterized by advances in algorithms, language models, and the increased computing power available to perform the calculations necessary to map and interact with the physical world.
So far there are limitations with 3D models in AR glasses as they are essentially manual. However, generative AI would create these very quickly and autonomously. Generative AI in AR glasses will help create 3D models faster and unlock the full potential of AR. This creation of the digital world to overlay the physical world will be faster, more complete and more immersive without requiring intensive manual work.
Generative AI will also transform the user experience and the way we interact with physical space. With generative AI-enabled AR glasses, wearers can literally transfer their imaginations into the real world. Without the need for coding, wearers can use speech recognition to “speak” their images and 3D objects through their AR glasses exactly the way they want to. They could say, “Imagine there’s a dolphin swimming across the room” – and it would appear in front of them. The possibilities for this immersion are virtually limitless for entertainment, work and beyond.
AR glasses that use generative AI will also transform the gaming world to create a far more immersive and personalized gaming experience. For example, with ChatGPT it will be easier to create more realistic characters and add new quests or game worlds. It could also be used to improve the gaming experience by analyzing player behavior and making the game easier or more challenging for the player in real-time by adjusting the experience autonomously.
AI-enabled translation of speech + written text reduces language barriers
Automatic Speech Recognition (ASR) uses audio-visual speech recognition by a neural network (an algorithm that relies on image processing to extract text). This can translate written text – like on a menu in a foreign country – into your native language in real time.
When applied in AR glasses, it can provide real-time subtitles in your native language while someone speaks in another language – all within the frames of your AR glasses. This removes the frustration of tourists and business people trying to communicate in the local language, promoting a more communicative and collaborative world.
Text recognition and translation combine AI Optical Character Recognition (OCR) techniques with text-to-text translation engines like DeepL. AI engines like Stable Diffusion can also augment communication with animations or other visual aids that can help convey complex or detailed concepts. This deepens user engagement: AR glasses using this AI can display a corresponding real-time image or video relevant to what the user is saying at a panel or presentation in front of them. Google recently teased the development of AR glasses with this function.
It also enables the deaf community to engage in everyday conversations without having to read lips or make eye contact by instantly converting audio to closed captions that are displayed in front of the wearer’s eyes.
AI is making an impact in healthcare, education and many other areas. Soon we will have intelligent AR glass that takes people like popular sci-fi into augmented or virtual reality environments where the AI quickly maps the space and speaker’s position to make virtual communication seamless and less awkward shape, regardless of where either party is located.
AI helps AR glasses easily recognize and label objects in the real world – deepening engagement.
Convolutional neural network (CNN) object detection algorithms are currently used in mobile devices to estimate the position and extent of objects within a scene. Once it recognizes an object, the AR software can overlay text on it or create another object in the physical world and create an interaction between the two. Objects transferred to the real world have many uses including teaching, navigation, diet and nutrition, and many others.
For example, if a user wears AR glasses with these AI capabilities, they can walk the streets of any city and learn about each landmark in real time while viewing. The AR glasses can identify, tag and provide information about the city and its landmarks – all through the frame of the wearer. As object recognition technology improves, nutritional data such as calories, protein, fat and cholesterol will be available for every food and serving size. Meanwhile, simple QR codes on products are designed to conjure up nutritional information for users.
In addition to pure object recognition, face recognition software is also becoming commonplace for person recognition. Facial recognition is already gaining momentum in the aviation industry, as more flights use the technology to confirm a passenger’s identity – adding an extra layer of security and speeding up the boarding process. Face recognition, when used in AR glasses, could give wearers everywhere the ability to recognize them. For example, in the near future you will be able to meet up with others on social media with AR glasses and get their background information instantly before deciding whether to “befriend” or connect with them.
AI-enabled AR glasses are changing our lives and their optics and capabilities will continue to improve. In our increasingly networked world, they simplify tasks and tear down barriers that were considered insurmountable just a few years ago. Advances in artificial intelligence are happening so fast that AI will make more advances in the next 10 years than in the previous 50 years. Whether it’s government, business, or personal, artificial intelligence will soon merge with AR glasses to merge our physical and digital environments.
