June 11, 2020

— When Chris Ferguson next launches to the International Space Station, he will don a helmet and position himself in front of the control console of Boeing's new commercial spacecraft.

The same can now be said for how he will prepare for the flight, too.

Ferguson, as Boeing's first in-house astronaut, will soon begin augmenting his training with a device capable of replicating every phase of his upcoming mission in virtual reality (VR), a first for the technology. What's more, the two NASA astronauts flying with him on Boeing's Crew Flight Test (CFT), Mike Fincke and Nicole Mann, will be able to join Ferguson aboard the virtual Starliner spacecraft and train together — regardless of where they are physically at the time.

"This is a multiple person experience, so that crews can practice together while also plugged into the simulator system, including flight controllers," said Connie Miller, Boeing's lead software engineer for the company's VR training efforts.

NASA astronauts have been using VR to train for spacewalks for years, but the low resolution limits of the system precluded using the same approach for the full extent of mission scenarios, from launch and docking with the space station to undocking, re-entry and landing.

Boeing is now changing that by employing the use of human-eye resolution VR headsets made by Varjo, a Helsinki-based manufacturer of industrial-grade virtual and mixed-reality products.

"With our devices, astronauts can see and virtually interact with the switches and control panels inside their Starliner capsule and read the real-time data on their crew displays," said Niko Eiden, the CEO and co-founder of Varjo, in a statement.

The Starliner's console includes two digital displays, each about the size of an Apple iPad, which are used to provide flight data to the crew, such as the velocity and trajectory of the spacecraft as it moves in space. With earlier generation VR headsets, reading these displays within the immersive environment would require leaning in, but then the user could not also see or use their hands to push buttons, flip switches or otherwise manipulate the controls.

Starliner astronaut training in VR. Click to view and enlarge in a new, pop-up window. (Boeing)

With the human-eye resolution of Varjo's VR-2 headset — more than 60 pixels per degree in the center of the field of view — astronauts can use the virtual Starliner controls just as they would if they were sitting in the physical simulators at NASA's Johnson Space Center in Houston, or inside the spacecraft itself.

With the VR-2, the astronauts can interact with the virtual environment with real-world feedback.

"The VR system connects to the training servers and system, which means they are practicing with the same flight training software used in the simulator at Houston," wrote Miller in an email to collectSPACE. "The system uses Valve Index controllers that the system monitors [to track] hand movements. The controllers provide some haptic feedback and also have buttons on them that astronauts use to manipulate controls in the VR environment."

With the software development now largely done, Miller and her team at Boeing are making their final evaluations so that Starliner crew members can begin using the headset full time in their training.

"Boeing's Chris Ferguson and NASA astronauts have used the system during development and evaluation. Chris will soon have it set up in his office in Florida, expected sometime in the next few weeks," said Miller.

Boeing is working toward launching its second uncrewed orbital flight test later this year, after its first Starliner mission in December 2019 was plagued by software problems and failed to reach and dock with the space station. Should that flight go as planned, then Ferguson, Fincke and Mann would launch on the CFT mission in 2021, followed by Boeing's first certified mission for NASA with astronauts Sunita Williams and Josh Cassada on board.

In addition to preparing crews before they launch, including during their pre-flight quarantine periods while still in their crew quarters, future applications for the VR may include sending a version to the space station for in-flight proficiency training.

"Headsets don't work in zero gravity because of their tracking sensors, but a future headset may be able to tackle that problem and develop something that will work," said Miller.

"Advancements like this have the potential to transform the way any pilot is trained," said Eiden. "We are proud to be delivering the technology that is pushing industrial training applications to their furthest reaches — even to space."