Apple intends to include new continuity features in its upcoming mixed reality headset that will make switching between devices and virtual workspaces a seamless experience with a new patent on the horizon.
For anyone unfamiliar with the term, Apple uses continuity to describe all of the ways the devices in its ecosystem communicate interchangeably, allowing users to switch between them without interrupting their work.
For example, Handoff allows you to start working on one device, then switch to another nearby device and pick up where you left off in the same app. Other examples of continuity features include AirPlay to Mac, Sidecar, Universal Control, and Continuity Camera.
Last week, the European Patent Office published a patent application from Apple titled “Multi-Device Continuity for use with Extended Reality (XR) Systems,” in which Apple offers some examples of how it envisions Handoff-like interoperability between a XR will work with headset and other Apple devices.
Headset wearer looks at the iPhone display
In one example, Apple describes a scenario where a headset wearer is viewing an email on an iPhone screen, and a virtual replica of the Mail app’s user interface is displayed on the iPhone’s screen. With a wave of the hand or a glance, the user then transfers the email to a larger virtual display suspended in their environment and continues the draft by detecting their finger movements through the headset cameras.
In another example, while a song is playing in a media app on their iPhone, the headset user gestures or looks at a HomePod in the same room, thereby continuously and uninterruptedly streaming music playback to the smart speaker. without physically approaching the speaker. “This handoff logic can be done via a direct peer-to-peer connection and/or facilitated by a cloud server,” the patent authors note.
The headset draws a virtual overlay of the iPhone screen content
Various other scenarios are envisioned that show Apple’s XR headset similarly “managing the continuous transfer of control between other devices in the system that respond to three-dimensional location-based user input, and/or … one or more of the other devices.” and the device itself.”
Apple also describes another implementation where the headset augments a desktop Mac, rather than going from a physical display to a virtual one, by positioning “accessory windows” close to but outside the confines of the Mac monitor screen into an “augmented reality environment”. . ”
How far Apple will initially push Continuity with its rumored headset isn’t known, but some of the examples in the patent probably give a good general idea of what the company has been working towards.
From what we know about the headset, it doesn’t require an iPhone to function and can be used on its own. It does this by running “xrOS,” a new operating system built specifically for the AR/VR experience. xrOS will include iOS apps like Safari, Photos, Messages, Maps, Apple TV+, Apple Music, Podcasts and Calendar, as well as a FaceTime app customized for the headset.
Headset wearer pushes iPhone content into augmented reality environment
There won’t be a wearable controller for the headset, instead Apple is relying on hand gestures recognized by the device’s myriad cameras. For example, typing is done “in the air” through eye movements and hand gestures.
Apple still plans to unveil the first version of the headset, dubbed “Reality Pro,” at WWDC this June, with the device shipping no earlier than late 2023. To learn more about what to expect from the headset, we have a dedicated AR/VR roundup that rounds up all the rumors we’ve heard so far.
(Via Patently Apple.)
