5 Reasons Virtual Reality Will be the Future of Flight Simulators

We haven’t quite entered the times presented in ‘Ready Player One’ just yet, but it’s actually slowly getting there. Virtual Reality (VR) technology is getting cheaper and more advanced every year since companies like Oculus and Valve made it more mainstream, resulting in companies like Apple, Microsoft, and Google all pouring resources into research and development to bring VR to mainstream society. It is likely that virtual reality will be the future of flight simulators.

Why? Because with this advancement, the aviation industry gets to benefit from reduced cost and easier to implement VR technology into flight simulators and pilot training environments.

If you’re new to all of this, you’re probably wondering what the fuss is all about, and what the actual benefits are? Well let’s address that once and for all – let’s talk about the 5 biggest reasons VR will be the future of flight simulators.

Do you like these articles and want to stay up to date with the best & most recent content? Follow Pilots Who Ask Why below!

What is Virtual Reality?

VR essentially comes down to having a Head Mounted Display (HMD), that you can look at through 2 goggles featuring particular lenses (1 per eye). The headset itself is equipped with quite a few sensors that make it easier for it to determine your head’s position.

As soon as you put the headset on, it will feel as if you’ve just entered some sort of magic portal and you’re now in this completely different environment, it’s a little crazy the first time you try it. You then either have this headset track your hands so you can use your hands inside the virtual environment as well, or you have 2 controllers in your hands which are tracked by the headset automatically.

The main thing that needs to work correctly is keeping the entire 3D environment you see ‘level’ or ‘constant’ as you move your head. If by moving your head, the environment on the screen would move too, you would get very disoriented very quickly!

Everything needs to work flawlessly, otherwise it gets disorienting very quickly. If things work properly though, the things you can experience through VR are remarkable and really immersive, even with ‘just’ today’s technology.

How does Virtual Reality Work?

Each lens and screen will show a slightly different picture to each individual eye, which enables you to see depth and a lot of details. The lenses are usually of the ‘fresnel’ type to help with the bending of light from the 2 individual screens, which have (weird looking) circular patterns around the centre, they look something like this:

The headset uses gyroscopes, accelerometers, and sometimes even LED’s with cameras on the outside to scan the environment around you to make sure the position sensing is accurate. The latency (delay) of the screen has to be really low as well, usually a maximum of around 50 microseconds is used as a limit (not very long). Any more delay than this and you would get dizzy and nauseous.

Imagine moving your head, but the visuals only updating after a short delay, not a lot of fun! Any time you move your head, the hardware and software adjust the ‘virtual environment’ so that it feels like the environment does not move.

Then there are the controllers. These are used to manipulate the environment around you. While hand tracking is already successfully implemented in some cases, controllers are often more reliable. Their position are tracked accurately by the headset and allow you to see your own ‘hands’ inside the virtual environment, a little freaky sometimes!

They will usually look something like this (one in each hand):

So that’s the basic setup of most VR solutions. Now obviously when it comes to aviation, we’ll need more add-ons to make you REALLY believe you are in a cockpit such as seats, switches etc.

Have a look at this video to see how easily fully immersive experiences can be created and how you go from boring (and messy) desk, to a fully fledged cockpit experience. There are just 2 panels that are used to represent multifunction displays in the virtual cockpit, same for the joystick. Pressing any of the buttons or moving the joystick in real life corresponds to pressing them in VR!

Of course the real life panels and joystick could be completely ‘dead’ as well! All that’s needed is a headset that can detect whether or not a button is pressed by tracking your hands or controllers, the software will do the rest and displays the correct changes on the virtual flight screen in front of your eyes: tadaa – pretty nifty technology.

So imagine a scenario where a dead cockpit full of useless switches, is all you need, together with your headset, to have a fully fledged cockpit experience! Motion could be added as well, but there is no longer a need for a full aircraft cockpit replica full of integrated hardware and software for specific cockpits.

But for now let’s focus on why we’re here:

What are the benefits of Virtual Reality?

Let’s break down the benefits and the main reasons VR will take over the flight simulator industry as the technology matures and becomes more widely available:

1) Field of View

The first one is fairly obvious but extremely important. By having your goggles right in front of you eyes, the field of view is massive compared to a conventional external screen. The average VR headset field of view is already really impressive at this time of writing, but only to improve as time moves on.

On top of this, you can move your head in whichever direction you want, and it will feel like you are physically in the cockpit (as per the video above). You can even look backwards and see whatever is actually supposed to be there in real life. Want to look down the window for winching? Not a problem, it will feel lifelike.

This is a huge increase in immersion as you will be able to observe an accurate representation of lighting, cockpit details, and unlike current helicopter simulators, there won’t be such a big divide between ‘screen’ and ‘cockpit’. Instead, it will feel more like the real deal.

2) Depth Perception

Then there is depth perception. This is the current generation simulator’s biggest struggle. Anything you look at outside, will be at the same distance from your eyes on the same screen. This is the reason it is always quite immersion-breaking to actually fly visually in simulators (not to mention the potential for poor graphics).

This is not the case in VR. As mentioned before, each eye will have its own individual screen.

A bird or drone that is close, will feel close, so will a mast or other obstruction. Dynamic weather, runways, helipads, formation flying, it will all feel extremely realistic compared to just looking at a static screen without any depth.

3) Immersion and Training Efficiency

Overall immersion is much greater when wearing a VR headset. More immersion, and reaching states of ‘flow’ (more in this future articles), massively increases information retention and therefore training performance, as laid out by Edgar Dale’s cone of experience:

As per the scale, the more you move down, the more easy retaining information becomes. The blue items are rough categories, but we can elaborate on this principle in the future. Where does VR fit? All the way at the bottom!

The VR headset and software know exactly where you are in the virtual environment, and can therefore tailor the experience much better than if you were finding yourself in a normal simulator with a conventional screen as a window.

With VR, you could easily simulate smoke, bird strikes, electrical fires, broken levers, actual fires, and much much more. Specific helicopter operations that require you to look down a winch, or put fires out, are even better to simulate now and immerse the pilot properly.

NVG operations can easily be simulated as well, unlike current helicopter simulators, and if you move your head and peek around the aircraft nose, you will actually see what is behind the nose, pretty cool huh?

For airliner cockpits (their windows tend to be smaller though), but especially with fast jet and helicopters: screen size and overall visual quality is super important.

Loads of today’s simulators have way smaller screens than they should, to the point where you can clearly see it’s ‘just a screen’. This will almost always result in a loss of immersion such as in this military trainer:

For VR however, there have been plenty of instances where, even with simple VR games, the user forgets they are not actually ‘there’ and starts leaning on virtual surfaces that don’t exist in real life! They end up almost losing their balance, which looks funny, but don’t smirk too much: it could easily be you one day!

4) Cost Effectiveness

There is a massive benefit for companies who decide to use VR for their training solutions once they get fully developed into Level D simulators in the future: upfront costs as well as construction costs.

At the moment, the cockpits used (both fixed wing and rotary) are replicas of the actual cockpits, to the point where the simulator hardware cost can easily surpass the the cost of the actual aircraft, especially in case of helicopters.

After the sim has set you back let’s say $12 million, you use it for as many years as possible before donating it to a university or museum, or getting rid of it for a few cents on the dollar. As the airplane type is phased out, the sim is now pretty much useless.

Don’t even start about modifications, upgrades, new software avionic versions, and any other amendments that are going to require hardware engineers, downtime and most of all: more money. Normally, any of these things are extremely inefficient, costly and slow processes.

For an industry that is constantly looking forwards and cycling through new technological advancements, a conventional sim can be quite the headache.

With VR however, ALL functionalities for visuals, handles, buttons, lighting, and anything else takes place INSIDE the headset, and is therefore fully sorted by a dedicated software team: a LOT more efficient, quicker and cheaper.

You might have to change some physical buttons inside the cockpit that you can feel with your hands, but the functionality of the buttons, or wiring, won’t be an issue: there is no wiring for any of the buttons and switches!

5) Maintenance Friendly

This also massively simplifies maintenance, and brings most workload to the software developers, apart from the motion aspect. Due to their expensive nature, a lot of simulators worldwide tend to be operated on a 24/7 schedule in order to reach their financial break-even point as quickly as possible.

Maintenance downtime heavily delays this break-even point, no matter the type of maintenance. Not only will VR simulaters have less bells and whistles (and thefore less problem areas), their actual scheduled maintenance can be easier to schedule, requires less manpower, and will mostly be performed by software engineers who can work and fix issues from anywhere.

Virtual Reality Flight Simulators

What is even more exciting is the realisation that some of these technologies are already implemented in real life aviation training environments at various scales:

• KLM has started their VR journey, which they showed off fairly recently in this video.

• A company called Komodo Simulations has fully embraced VR as the future of flight simulation, and already has a functional implementation of their VR simulator. It can be mounted on a moving platform for full immersion as well.

• Then there are helicopter training companies such as VRM in Switzerland who managed to get full EASA certification (FNPT II) as reported here by EASA. Vertical Magazine covered it in one of their videos:

EASA Head of Standards, Jesper Rasmussen, stated:

The Agency is pursuing the modernisation of its regulation for training devices to reflect their actual capability and technology advancement. This evolution will make a wider range of cost-effective training devices available to complement Full Flight Simulators and is being driven in part by training needs for new Vertical Take Off & Landing (VTOL) aircraft.

This also aligns with the Safety Objectives of the EASA Rotorcraft Safety Roadmap to review the most critical training scenarios and promote the use of simulators for high-risk training operations.

You can read more about the first EASA VR FSTD on VRM’s website here.

• Not just the civilian industry is reaping the benefits though. It has actually been proven in multiple studies, such as the one referenced here, that VR implementations reduce the training time required for military pilots to get online, pretty convincing stuff.

• The United States Air Force has even observed students being able to hover complex types to the highest grade, right from the start after VR training, which is unheard of within the military. They are now considering making students spend over 20 hours in VR before touching any aircraft, due to all the proven benefits.

• To go a step further, NASA is using VR consistently, as shown here, to train astronauts for their Extravehicular Activity (space walks for ISS maintenance etc), as this is the only way they can provide an environment that is actually believable and close to the real thing, other than their underwater training station. They have stated numerous times that immersion is key to a beneficial training experience.

• Competentaviators.com has recently published an excellent article on how head mounted displays can aid VFR flight. This is a very relevent sector for companies looking into VR as well.

Conclusion

So with that all said, obviously there is still a lot of work to do before we can all fly our OPC’s and LPC’s in a fully fledged Level D (full motion) VR simulator. And there are also downsides and challenges to VR (to be covered in a future article). However, the training industry is in motion, and the trend and facts heavily favour VR for future training solutions.

There will definitely be companies (and people) who will try to resist this progress, for their own personal reasons. Eventually though, conventional sims will simply not have the features and capabilities that VR sims will do, and VR will outperform the others financially as well. For now, only time will tell when this actually becomes the norm.

Do you like these articles and want to stay up to date with the best fresh aviation content? Follow Pilots Who Ask Why below!