Augmented Reality is all around you

Screengrab of news broadcast from KTRK ABC13 in Houston, Texas.

When Apple introduced the iPhone 12 in early November, the company emailed a link that allowed you to see what the phone would look like sitting on your desk.

Shoppers using the Houzz app can see what furniture will look like in their home before buying.

L’Oréal has an app that allows women to see what they look like wearing different make-up.

These examples all use computer-generated elements to add or augment what we see, and interact with, in the real world.

In case you haven’t noticed, augmented reality is everywhere. Playing and engaging with a computer-generated environment is no longer just something you see in the movies. Furthermore, as more applications and services use augmented reality, the more consumers will expect it.

BRIEF HISTORY OF AUGMENTED REALITY

The idea of using computers to enhance the real world has been around for about sixty years. In 1968, Harvard professor Ivan Sutherland created a device that combined the view through a headset with a crudely drawn computer-generated environment. By the 1990s, the Air Force and NASA used augmented reality for research and navigation. Finally, faster computers allowed the technology to expand into gaming and other commercial applications in the 2000s.

During this time, product developers at The Weather Company realized augmented reality could dramatically change how broadcast meteorologists produce their weathercasts.

“We saw the potential to truly elevate the weathercast from legacy linear presentations to virtual 3D immersive experiences, to differentiate weather presentations, and help broadcasters acquire and retain audiences,” says Mike Convey, the Offering Manager of Media Weather Products for The Weather Company, an IBM Company.

In late 2015, the company released the first version of Max Reality software, which enabled broadcast meteorologists to move weather data off the map behind them and into the real space in front of them.

“We started with augmented reality by allowing clients to place 3D objects in front of them. Then we added high-definition materials and environment maps, and finally rounded things out with full virtual sets that allow broadcasters to create animations and presentation areas for their content.”

A.R. ON THE T.V.

As a member of the field trial team for The Weather Company, I was one of the first broadcast meteorologists in the country to add Max Reality to my weather hits back in November 2015. The software was so new there wasn’t even a user manual! As a result, those early creations were simple and somewhat cartoonlike. Over time, as the software and skill of the user were upgraded and refined, the 3D computer-generated objects looked more realistic.

Like all the tools available to broadcast meteorologists, the technology is only as good as the person using it. Ryan Hoke, a meteorologist at WAVE in Louisville, KY, is one of the best. He’s been working with the software for about a year and has already wowed his viewers by making it appear to rain and snow in the studio. To demonstrate the threat of flash flooding, he “filled” the studio with several inches of water. More recently, he explained the dangers of using a turkey fryer, complete with smoke and flames. All of it produced with augmented reality.

“I think viewers benefit from seeing weather explained in augmented reality because they’re more likely to commit that information to memory,” Hoke told me. “Augmented reality graphics grab attention and excite the mind, making viewers more eager to digest what’s being explained.”

Hoke says he sometimes sketches out ideas for scenes before picking up the computer mouse to build them. This something I do as well, and it’s an integral part of the Max Reality Advanced Training. Even crude mock-ups can help broadcast meteorologists determine the data they’ll use and the appropriate size and position of the objects that will be rendered by the computer. It also forces the user to choreograph where they will physically stand in the studio while the scene is playing out.

Most Max Reality scenes take Hoke about an hour or two to build. Complicated scenes take a few hours a day over a couple of weeks to produce. This is a lot longer than it takes to create a typical weather map, but he agrees that the extra work is worth it.

“Sure, anyone on our weather team can show a cold front on a classic 2D weather map, but do viewers understand how that front will impact our weather?” he asks. “Being able to show that cold front in 3D in front of us is a game-changer. Now we can show that front lifting up the air, creating clouds, and contributing to a severe weather setup. The viewer response and the impact on our weathercasts have been overwhelmingly positive.”

AUGMENTED REALITY IS THE REAL DEAL

The Weather Channel took the concept of augmented reality to a whole new level and built an entire “Immersive Mixed Reality” studio. The software is more advanced than the Max Reality software Hoke and I use, and it harnesses the power of the Unreal Engine, the same hardware used to create special effects in movies, commercials, and games. And because it’s more complicated, the scenes can take months to design and produce.

Still, the idea is the same regardless of the software and hardware: use computer-generated elements and environments to deliver an important message about the weather in a dynamic, engaging way.

We’re likely to see more uses of augmented reality in the years ahead, be it on our computer, smartphone, or living room TV. As Convey says, “With many of us stuck at home, we are even seeing simple AR find its way into video conferencing tools. I think we will continue to see AR become more and more immersive to enhance whatever message is being presented.”

Ideally, the viewer shouldn’t notice the technology beyond recognizing what they are watching isn’t entirely real. In the end, it’s still about storytelling and delivering the essential message. At some point, weather presentations that don’t utilize augmented reality will stand out for the lack of innovation and creative storytelling.

Tim Heller is an AMS Certified Broadcast Meteorologist, Talent Coach, and Weather Content Consultant. He helps local TV stations and broadcast meteorologists communicate more effectively on-air, online, and on social media.