Google Cloud’s Immersive Stream for XR enables interactive 3D and augmented reality experiences on mobile and will soon support augmented and virtual headsets.
Back at I/O 2022 in May, Sundar Pichai dubbed “Google Cloud Immersive Stream” so any smartphone can run the new Immersive View in Google Maps, which is just starting to launch.
Google Cloud later described Immersive Stream for XR as a way to “provide users with an immersive, interactive and photo-realistic experience without having to download an app”. The company believes that “mobile GPUs still struggle to create these photo-realistic images,” which are necessary to deliver a truly immersive experience. As such, desktop GPUs are required:
The mobile phone here sends input events to our cloud service, such as the camera position, and our cloud service then takes these camera positions and creates a photo-realistic image with powerful desktop-quality GPUs, which are then streamed to the device as video frames. The device takes these video frames and combines them with the camera image to create an immersive photo-realistic experience for this particular car model.
Today, Immersive Stream for XR is available on Android, iOS, and the desktop web. The latter platform allows you to view 3D models in Safari and Chrome, while mobile finds AR mode that allows you to place an object in your actual space and adapt it to different environmental conditions, such as lighting.
Google said earlier this week it was “actively working on an OpenXR client” to stream XR experiences to head-mounted displays (HMDs) compatible with that standard. Immersive Stream for XR on headsets supports a VR mode and Stereo Augmented Reality mode.
Use cases today mainly include product demos, especially for cars as virtual showrooms. Others involve shopping, tourism and even kitchen renovation. Looking ahead, Google offers many more examples, including training, in various categories.
Gaming – with Stadia as Immersive Stream for Games – seems to be absent unless “Augmented Reality Sports” is meant to be interactive rather than just passive watching.
This technology and ongoing maturation comes as Google’s Project Iris headset reportedly displayed “some images” in the cloud and streamed it to the device to avoid local “power limitations”. First-party hardware may still be a long way off, but it looks like Google is laying the groundwork for it, while improving the experience and monetizing it by leveraging the existing demand for non-gaming cloud streaming.
FTC: We use auto affiliate links that generate revenue. More.
Check out 9to5Google on YouTube for more news: