2017 – The Year of Mixed Reality! What You Need to Know:
2016 has been a remarkable year that’s brought continued growth and awareness to the worlds of Augmented, Virtual and Mixed Reality. Set to become a $165 Billion dollar industry by 2020, there’s still a common question that lingers among many newcomers trying to understand this fast moving digital phenomena we are just beginning to watch evolve; What’s the difference between them and how will it impact the digital world as I currently know it?
Before we jump into the mind-blowing future Mixed Reality is set to usher in over the course of 2017, let’s first discuss the distinctions between Virtual & Augmented Reality – Their technologies are very similar but have some fundamental differences.
Virtual Reality:
Virtual Reality is a digital environment that shuts out the real world.
VR is able to transpose the user. In other words, bring us someplace else. Through closed visors or goggles, VR blocks out the room and puts our presence elsewhere.
Below is a hilarious example of Conan O’Brien demoing VR on the HTC Vive in a ”Job Simulator” environment
As you can see, VR is pretty amazing because its allows users to do so many things like watch a 3D experience of Cirque du Soleil, attend an exclusive concert or sporting event, ride in a roller coaster, play games, create art projects and in some cases, hang out with friends in virtual worlds. (Conan did a poor job making friends in his virtual world in the example above by the way) Recently, Virtual Reality has even been used to help sick children escape their hospital environments and bring them to happier surroundings. So the question you might be asking is, what does this have to do with Mixed Reality? The answer; VR is a gateway for a lot of the more advanced technology that will be used in Mixed Reality.
Augmented Reality:
Augmented Reality places digital content on-top of the physical world you see around you.
AR works by adding 2D or 3D layered content on-top of real world objects or locations, allowing the user to unlock additional information that may be relevant, therefore turning the physical world around them into digital media.
Some things you can currently do within an AR experience:
- Play games
- Watch Videos
- Connect to Websites
- Listen to Music
- Enter a Sweepstakes
Right now the majority of AR is used on mobile devices, tablets and integrated into mobile apps but that is quickly changing. Mixed Reality, also a form of Augmented Reality has a way higher degree of complexity and is much more realistic as you will read below.
Below is an example of a Mercedes Benz AR app activated off a sales brochure
Mixed Reality — The next big paradigm shift
A hybrid of both AR & VR, Mixed Reality is far more advanced than Virtual Reality because it combines the use of several types of technologies including; sensors, advanced optics and next gen computing power. All of this technology bundled into a single device will provide the user with the capability to overlay augmented holographic digital content into your real-time space, creating scenarios that are unbelievably realistic and mind-blowing.
How does it work?
Mixed Reality works by scanning your physical environment and creating a 3D map of your surroundings so the device will know exactly where and how to place digital content into that space realistically, while allowing you to interact with it using gestures. Much different than Virtual Reality where the user is immersed in a totally different world, Mixed Reality experiences will invite digital content into your real-time surroundings, allowing you to interact with them.
The use of transparent lenses, spatial sound and an understanding of your physical environment will allow holograms to look, sound & behave like real objects that are able to interact with the environment around them and also with each other. Take for example the video below of Robots crashing through the walls and firing weapons at the Microsoft Hololens user. Not only mind-blowing but truly a magical experience!
“Believability is a core design principle in Mixed Reality”
Microsoft Hololens Engineer – Cris Derr
What can Mixed Reality do?
Mixed Reality will not just be another advanced gaming console to play the latest version of Halo or Madden NFL as many people might think, but instead it will add a whole new world of interactions, apps, games, and experiences we have yet to imagine. The World around you will become an entirely new canvas for you to Play, Learn, Communicate & Interact with.
Several Use Case Examples:
Communication: Holoportation will allow device users in different cities or countries to sit and interact with each other in the same space virtually, while remaining miles and miles apart.
Education: Below is an example created by Project Esper displaying how medical students will benefit and gain more comprehensive views & interactions with the human anatomy
Entertainment: Microsoft Hololens is already working with the NFL to completely change the way fans can watch and interact with players, other fans, real-time game experiences and its advertisers & sponsors.
Use-cases for this technology are endless; Sports, Music, Television, Art, Fashion, Business, Education, Medicine, Interior Design, Retail, Construction, Real Estate and virtually everything we do will be effected. During the first evolution, Mixed Reality will replace your mobile device and then slowly start to replace all remaining devices including TV’s, laptops, tablets, etc. All of the content you consume on those other devices will soon become readily available within your new world of Mixed Reality and through one single pair of lenses.
When will we see mass-consumer adoption?
The key to driving mass consumption of these platforms lies in making Mixed Reality devices that are smaller and much more affordable to everyone. Right now Microsoft Hololens is the only premium device in the market with a price tag of $3,000 and it’s really only designed for Enterprise & Engineers and not yet a mainstream consumer base. Meta 2 is supposed to start shipping soon at $949 and Magic Leap is still unannounced.
The key will be to start developing smaller affordable devices able to ensure that massive amounts of data can be transferred without being slowed down or limiting the experience. For this, 5G Networks will play a vital role and are right around the corner. Additionally, off-loading compute functions from the device to the cloud will free up significant bulk in the lens or headset. Ultimately, we need to get to the point where these capabilities become seamless and affordable, we’re getting very close and hoping to see big improvements in 2017.
What’s Affordable Right Now?
ZapBox – Cardboard Mixed Reality for $30
AR company Zappar recently announced the creation of world’s 1st cardboard Mixed Reality headset complete with cardboard controllers and a special wide-angle camera lens for only $30 on kickstarter.
Zappar’s mission is to democratize and make MR affordable to all in a similar manner that Google did with cardboard. ZapBox takes an outside-in approach to the MR experience and delivers some pretty cool engagements where the digital content is brought into your physical space to interact with you.
The Bridge MR / VR headset by Occipital for $399 (iOS only)
Start-up Occipital has also created a very cool hybrid MR / VR HMD solution called The Bridge that is designed specifically for iOS with a $399 price point. The headset comes with a 120 degree wide angle lens and a depth sensor called the “Structure Sensor” that Occipital has been working on for several years, which allows it to perform inside-out positional tracking (one that doesn’t rely on external sensors) that could provide tracking that’s fast and accurate enough to allow augmented objects to appear anchored in the real world so they actually seem real.
Microsoft has plans for Mixed Reality affordability in 2017:
Microsofts Hololens has already set the stage by being the absolute best MR device in the market place to date. That said, the current price of $3,000 is way out of reach to the average consumer.
Microsoft recently made some announcements during the opening day of the company’s WinHEC event in China on December 7th. The company has provided official PC specs that will be required to support the first VR headsets able to handle mixed reality capabilities.
Additionally, Microsoft confirmed in October that companies like Lenovo, HP, Acer and ASUS have all made commitments to developing PC tethered headsets with prices starting at $299 beginning in early 2017.
Unlike Hololens, which is a stand-alone device, to make these new headsets affordable to the mass market, the new devices will need to be tethered to a PC which is required to process much of the compute power.
What about Apple?
Apple’s entry into Mixed Reality will be an obvious and complete game changer that could literally seed the market with MR capabilities at serious scale overnight when they are ready to pull the trigger.
Tim Cook has boldly made his opinion known to the world that “AR will be much bigger than VR”.
There are also reports swirling that Apple is prepping to launch the iPhone 8 in September of 2017 and that it will be a Mixed Reality device.
Robert Scoble was first on record to report Apple is readying a completely transparent mobile device complete with a next gen OLED Display, Wireless Charging and a 3D Sensor by Primesense, which Apple has over 600 engineers currently working on in Israel. The compute will be neatly packaged and hidden within a strip at the bottom and the device will also connect to a headset. Eye sensors by Eyefluence will give the device a higher frame rate and polygon count than a PC tethered MR device with a Nvidia graphics card. . (see below patent)
To some, a transparent phone might seem crazy but Taiwanese company Polytron already developed a clear, multi-touch device back in 2013 as a prototype that still hasn’t hit the market yet. (Image below)
Recently the CEO of Foxconn, Apples manufacturing partner has confirmed the wireless charging and next-gen OLED display. Patents filed by Apple detailing the transparent mobile device have also surfaced.
What this all means is effectively, on the first weekend Apple launches the iPhone 8 in September 2017 which is the iPhones 10 year anniversary, we could potentially have over 60 million Mixed Reality capable devices in the world, overnight! Are brands, advertisers & content providers ready for this?
Note: *Robert Scoble has recently announced that brands like Samsung, Huawei & others are also readying new MR devices but details are still forthcoming.
Patent Filed for Apple’s Wearable MR Headset
Advertising in Mixed Reality
Mixed Reality will offer brands and advertisers the ability to connect, interact and sell to consumers in a way that’s never existed before and it will have a profound impact on their bottom line and brand loyalty.
Product packaging is becoming a preferred digital platform for brands wishing to tell stories & communicate with their target consumers.
For example, a Coke can will now have the ability to deliver a realistic 3D music, gaming or sporting event experience. Food products like Hellman’s mayonnaise or Extra Virgin Olive Oil will be able to deliver exclusive 3D recipe tutorials given by a Rachel Ray hologram. Or many of the items that make it into your grocery cart will have digital discounts and product benefits displayed straight from the package as you walk down the aisle and shop. Since these devices function primarily by acquiring a 3D understanding of our environment, brands are going to have incredible targeting & re-targeting opportunities with their amazing new content.
Much of this engagement will be sought after and wanted by consumers, so it will be pull-based through interactions with objects and brands making it extremely valuable to both the advertiser and consumer alike. But with all quality advertising & messaging, there will also be fierce competition by brands looking to push as much unwanted messaging directly within your line of sight as possible.
Because everything in the world will now become prime real estate for brand marketing, new filters and restrictions will need to be put in place and adhered to in an effort to protect consumers and their privacy. As Scott Belsky, the founder of Beehance & VC investor has suggested in a recent article giving 5 predictions for the future of tech, (and I agree) Adblockers may be a suitable solution for these concerns in addition to some type of governing association with a clear set of rules and guidelines for advertising within a new Mixed Reality world.
Artificial Intelligence & Machine Learning:
Because we will rely on these new MR devices for practically everything we do and wear them throughout most of our day, there will be a significant increase in the amount of time and content we consume on a daily basis. What this means to brands is more advertising opportunities and the ability to collect and learn from the data behind this behavior! What this means in the grand scheme of things, is that there will also be a vast amount of more physical objects throughout the world that will need to be indexed and have some type of relevant content associated with them. So how will messaging & content be segmented so that it’s delivered in all the right places and to the right audiences?
Using advanced computer vision, AR start-up Blippar has been working on a real-time deep learning engine that will recognize average everyday objects like fruits & veggies, flowers, plants, animals, etc. and will then scrape the web to apply a thin layer of relevant digital content on top of these objects, referred to as the “Internet on Things”.
Imagine the ability to pull nutritional content, recipes, store locators, etc. simply by looking at a piece of fruit. (see images below)
This type of engine could become an extremely valuable integration to a Mixed Reality HMD manufacturer, especially since much of what we are looking at will now be detected by Eye Sensors. Think of it like a next gen Siri assistant or Google search engine, except you will use your eyes to unlock search queries from physical objects you look at, rather than voice or typed search commands. This can also help to solve everyday problems related to unknown objects, language barriers and much more. In the end, the result could become even bigger and more expansive than the internet we know today.
Let the fun begin!
2017 will be the start of an era that will bring some of the biggest digital advancements we have ever seen on the face of the earth for all of mankind. Through Mixed Reality we will all be able to share, experience and understand the world with a whole new set of eyes. A smarter more connected and magical set of eyes!
Keith Curtin – Founder: See Digital
Follow me on Twitter @KFCurtin
Keith Curtin is an Entrepreneur, Brand Marketing Executive & Founder of See Digital, a consultancy firm specializing in Augmented Reality,Virtual Reality & Mixed Reality. He has recently been named #1 on Onayltica’s “2016 Augmented Reality Top 100 Brands & Influencers” list. Additionally, Keith has consulted with hundreds of Fortune 1000 global brands to develop and execute their first Augmented Reality & Mixed Reality campaigns including: Coca-Cola, Pepsico, Dr Pepper Snapple Group, Nestle, Kraft Foods, Kellogg’s, Mondelez, Mars Foods, Disney, Nickelodeon, Walmart, Anheuser Busch, Heineken, LVMH, Diageo and many others.
