Why a Rockstar Games Founder Is All-In on Safety-Focused VR

How do you prepare people for an emergency situation without actually putting them in danger? What's the most practical way to train someone to operate a forklift, or how to avoid hazards on a construction site? How about training for dangerous jobs, like building bridges and tunnels or welding underwater? Increasingly, the answer is virtual reality (VR).

We're still waiting for VR headsets to take a substantive leap in the video game world as a consumer product. On the business side, however, VR training is evolving into a massive market. According to a report from ABI Research, the enterprise VR training space will generate $216 million in 2018 and grow to $6.3 billion by 2022. In more and more scenarios, VR is becoming the most efficient, cost-effective, and safest way for organizations to train workers on complicated or dangerous activities within a completely immersive experience.

One of the companies capitalizing on this opportunity is NextWave Safety Solutions, which is exiting stealth with its first few VR training modules. PCMag got to test drive them first-hand.

NextWave is far from a startup. Co-founded by alums from Lehman Brothers and Rockstar Games, NextWave is already working on training programs with a number of construction companies as well as organizations like the New York Fire Department and the New York City Department of Buildings. With more than 150 employees and counting, CEO Lorenzo Gallo explained that the operation extends far beyond just VR development. NextWave has a consulting arm, an education and certification portal called NextWave Academy, and is building out a business intelligence and collaboration platform to give enterprises a mountain of analytics and data insights.

On the VR side, today marks the release of the company's first two training modules—Hazard ID and Forklift—and NextWave has eight more modules planned for 2018 along with an upcoming mobile app for workers.

"My background is in making games," said Gary Foreman, CTO of NextWave and one of the co-founders of Rockstar Games. "As much fun as I had, I got to a point where I wanted to just do something different, now using VR and developing video game-type content for something that can actually help train people. One of my real concerns is that the novelty of VR is something that people look at and don't take seriously."

Foreman served as a technical producer, director, and ultimately CTO overseeing development of the Grand Theft Auto franchise. He had no small part in shaping the groundbreaking open worlds and story modes from GTA 1 all the way through Vice City and San Andreas, and aims to build out NextWave's VR experiences in similarly immersive fashion.

NextWave CTO Gary Foreman

"The technology's evolved so much now that we can create VR experiences with a very low bar to entry and see tremendous gains in terms of not just the immersion, but the retention in these trainings. We're trying to make these experiences as accurate as possible so it's not replacing existing training, but adding another dimension. Current OSHA [Occupational Safety and Health Administration] classes are not compelling. You're sitting there looking at old PowerPoints and videos. VR can change all of that," Foreman explained.

"We can measure and monitor every action. Every button press, every turn, where they go within an environment to track whether the user actually understands what they're being taught. Because it's VR, we can do things in a safer environment that doesn't disrupt the real world production of a warehouse or a construction site. Ultimately, the goal is to create much better training so there are fewer accidents to workers and the people around them. I think that VR can deliver that."

Mapping Out Virtual Training Worlds

NextWave gave me a hands-on look at both the Hazard ID and Forklift VR training modules. Daniel Stein, NextWave's Product Manager for Virtual Reality, strapped me into an HTC Vive headset with motion controllers as I began wandering around the virtual Hazard ID construction site.

The goal of the training program is to essentially take a worker through a certified training course, where instead of filling out a quiz, they're exploring a construction site and identifying potential hazards.

As I navigated around using the motion controllers, I was able to identify hazards like planks of wood jutting out of a dumpster, exposed electrical wires, a flimsy piece of wood over a manhole cover, etc. The environment was immersive and fairly vast, letting me walk around the entire site and around the inside of the under-construction building. Each time I uncovered a hazard, a quiz question popped up prompting me to answer with the proper way to handle the situation. I found a bunch of hazards, but it's safe to say I didn't score well enough on the quiz to be safely allowed onto a construction site just yet.

While I roamed around the construction site, Stein explained that current in-person OSHA trainings for these types of certifications are not only boring, but more or less only meet the bare minimum in terms of demonstrating competency. He took real training courses to prepare for building virtual ones, and said after only a few hours learning to drive a forklift he was handed his certification.

In VR, you can build a far more thorough experience. While I was exploring the site, NextWave's software was building a timeline of every single action I took during the simulation, plus a real-time report on how much time had elapsed, how many obstacles I'd found, and how I did on the quiz questions.

"Right now, we're gathering pretty much everything that happens," said Foreman. "We'll map the path through, say, the construction site or the warehouse in the forklift so we can generate heat maps to see where people are going and more importantly, maybe where they're not going. There could be blind spots; there could be things they're not seeing that we're seeing."

Hazard ID was mostly point-and-click exploration, but the Forklift VR module required some real driving and game mechanics. In the second demo, I wore the Vive headset and sat a seat with a few physical pedals in front of me.

The simulation put me in the middle of a large warehouse in the driver's seat of a forklift. Highlighted instructions guided me through turning on the engine, working the handbrake, and showing me how to raise and lower the forklift to specific heights. After that, it turned into a driving course.

I learned how to accelerate and reverse, and then eventually navigate the warehouse, driving around cones and obstacles, and avoiding accidents waiting to happen, like driving into another forklift or running over a passing worker. Finally, I actually started picking up boxes and dropping them precisely on different warehouse shelves. The nuance of the simulation made it feel real enough that I felt as though the forklift would tip over when I got the pallet lodged in a shelf. The module also makes you actually turn your head when backing up.

When building out the experiences, Foreman said he first has to understand where the need is and how he can improve existing training using VR.

"There's a lot more sophistication we can add into the reality," said Foreman. "VR training can track not just where your hands are, but your feet as well. We can use a real steering wheel. There's so much more detail we can measure, but it's about understanding what someone's doing and gathering enough data to recreate the whole session: where you went, where you looked, what you clicked on, what you didn't."

NextWave has primarily tested on the HTC Vive, but the VR experiences themselves are built using the cross-platform Unity 3D game development engine so most hardware is compatible. Foreman said one of the biggest challenges right now is wired systems. Headsets like the HTC Vive and Oculus Rift have a bulky cable connected to a computer that gets in the way.

"I think this year we're gonna see some launches of systems that will be wireless and with the high enough frame rates so that it can still be a compelling experience," said Foreman. "The space is maturing. I think how people deal with space and scale will be a game-changer as well."

That concept of space and scale is key for NextWave. Foreman's goal, in a very Grand Theft Auto fashion, is to build out more expansive virtual training worlds with multi-user functionality. He plans to incorporate a gamification element as well, tying in badges and scores when workers pass different training courses. The company is also working on 3D scanning and spatial-mapping technology with 3DVista to create more detailed virtual environments incorporating BIM (Building Information Modeling) to recreate physical locations.

"The goal is to build bigger environments, richer content, but also to scale up the number of users at one time in there as well. Not all in one location. We'll be able to have people in one office, can enter people in another office and actually collaborate in 3D," said Foreman. "We want a lot of interaction. Not only with controllers and clicking buttons, but other inputs like the microphone to bring in some natural language processing. So instead of pointing and clicking you could interact verbally. The typical construction worker isn't used to this kind of tech, so what we want to do is to provide a natural way for them to interact with the experience."

Inside NextWave's Safety VR Business

Pinning down exactly what NextWave does and tracing all the different arms of the business can be difficult. CEO Lorenzo Gallo explained how the company is essentially a one-stop shop for in-person or virtual safety training, on-site consulting and compliance, and a full-blown Continuing Education Unit (CEU) provider for certified training courses.

There are NextWave Academy locations in New York, Jersey City, Philadelphia, Baltimore, and San Francisco offering in-person and online safety certification classes. NextWave and its partner company The Safety Group (which Gallo also runs) are involved in projects as wide-ranging as Congressional infrastructure committees and building virtual training for modular homes in Puerto Rico.

"Our customer base right now is heavily weighted on a construction site in New York City. Outside of that it's branching out into major infrastructure," said Gallo. "We've been working with Congress where we're part of a committee and a panel for Build America. We're partnered with the Army Corp of Engineers, Skanska, AECOM, Turner, Amtrak, and Port Authority on that committee, and we're advisors and subject matter experts for Congress on all infrastructure. Our role as NextWave is all of the training, safety oversight, and risk management. We also just had an opportunity to go down to Puerto Rico and come up with some virtual training on modular homes and training people locally to help them rebuild."

As CEO of NextWave, Gallo manages site safety managers and coordinators who help sites meet building codes and safety requirements. He has drafted safety regulations for the City of Philadelphia, the fracking industry, fire and building codes for the City of New York, as well as the first active shooter trainer curriculum, which was adapted into a virtual experienced called SurviVR.

Gallo also comes from Wall Street. He spent 20 years at Lehman Brothers before the firm went under during the financial crisis, spending much of that time running equity and finance operations and with a heavy focus on analytics. He found a second career as a safety and risk management expert, but still looks at NextWave's business value through the lens of analytics and data. The company is working with Asite on its back-end infrastructure and plans to roll out an enterprise dashboard later this year, integrated with the worker app. Gallo said the idea is to digitize onsite processes like check-in and safety checks, and then aggregate all the VR training data and onsite data into reports.

"Most safety personnel on a job site don't utilize any technology whatsoever; they still hand fill out logs every day. That's years' worth of compliance and safety data, incidents, training, everything," said Gallo. "We're going to start collecting all that, digitizing it, and that's gonna be put out in a dashboard. That alone is worth its weight in gold for a property developer that has 10 projects going on at the same time."

NextWave also works with insurance providers and mines from the massive pool of insurance claims data. This is both to build more realistic VR training scenarios and to identify potential fraud and save businesses money. When you're able to track incidents by specific workers, you can spot patterns.

For example, say Bob tends to get injured right around a holiday weekend every year like clockwork. You may not catch that through a paper trail, but when it's all data points, it's easy to say "Okay, Bob twists his ankle on December 23 every year" even though he's up-to-date with all his training. That's not a real claim.

"There's no data that exists in this sector at all," said Gallo. "When you have a company [like] AIG that's sitting on 100 years of claims data in a warehouse in New Jersey in paper form, that makes no sense. I want to give [construction companies] a real-time report every single day."

The Future of Hands-On Education

VR education and training is a huge industry. NextWave is far from alone in the space, but the company's big picture goal is using VR as an immersive tool to train a new generation of workers. Particularly in dangerous or high-risk situations, Gallo sees VR as a no-brainer.

"Just because you pass a permit test doesn't mean you know how to drive a car. I can't tell you how many times we had someone take a scaffolding course and passed it, and then they went 23 stories up and froze in fear and we had to call EMS and the fire department," said Gallo. "When that happens, number one, you shut down a job. Number two, you're jeopardizing the workers you're with. VR is not only is an academic test, but it's also a psychological fit test that prepares you for that environment versus going up there for the first time. We can't change the world overnight, but we're trying to reduce those risks and accidents on those jobs. Then those are severe, if you fall, you fall, there's no recovering from that."

The next generation of job seekers looking to get technical degrees to become carpenters, electricians, mechanics, and plumbers will be accustomed to learning visually through technology. Gallo said for the kids who aren't going to college and are looking for a trade or vocation, VR training can be a way to entice them in high school as they look for career paths that may not even exist yet.

Gallo's experiences on recent Congressional infrastructure panels have also highlighted what looks to be a coming construction and infrastructure boom the likes of which we haven't seen since the New Deal in the 1930s.

"The entire country is falling apart from an infrastructure standpoint. With the infrastructure projects going on, you have 20- to 30-year jobs coming up. Not since my grandfathers, who started on a job and retired on the same job, is that ever going happen, and all of these jobs will be run with a higher reliance on the kind of technology we're developing," said Gallo.

As a pie-in-the-sky dream, Gallo talked about creating an educational VR game for high school kids where they would actually choose and simulate different career paths. Since NextWave Academy provides CEU credits and certified courses, passing a training can also be part of a digital résumé of sorts, be it a worker carrying a digital certification from job site to job site or a student starting at a technical school.

"It's almost like building your own world in a game like Sim City," said Gallo. "You go in and pick a carpenter, a plumber, an electrician, or an engineer. Then you actually have to go through the certification process in that game to get there. This kind of simulator is something we could bring into education to give kids an idea of the career they want and then help train them. It's a virtual vocational school."

This article originally appeared on PCMag.com.