Let’s face it, we all have daydreamed of sitting in a cockpit and roaming the wild, blue yonder. It’s hard to find someone who wouldn’t have been fascinated by aviation at some point in their life. But for all the gratification that flying brings with it, no one can deny that it is also in equal measure, a dangerous thing. Now, for the number of moving parts that make up an aircraft, it is a surprisingly efficient and safe machine. The incredibly high standard to which an aircraft is made and maintained ensures that failure rates become a statistical improbability. No, the real weak link in the chain isn’t a plane’s hydraulics or engines or control surfaces as one might expect, but is in fact the pilot itself.

Current studies point that pilot error accounts for a staggering 85% of all aviation accidents. And while accident rates in commercial aviation have decreased over the past few years, in general, they have remained mostly the same. Accidents in personal flight have actually gone up by 20% in the last decade.

 

Augmented Reality in General Aviation

With all the numbers, it’s easy to just point the finger at pilots and say they didn’t do their job right. But there is more to it than just that. Richard Collins in his article – Was it Really Pilot Error – Or Was it Something Else? sums up the real problem here very succinctly – “Pilots don’t err on purpose, though, they err because they don’t know better.”

Anyone who has flown (or has even tried out a desktop flight simulator) will tell you that flying ain’t easy. Even a glancing look at the controls of a Cessna 172 can confound a student pilot, let alone those of a Boeing 737 which consists of hundreds of switches and dials.

Pilots need to consider a lot of information before making the simplest of decisions and small errors have a way of snowballing out of control. Reading instruments, terrain, and weather to make decisions can get very tedious very fast. Being a pilot myself, I know at first hand how dangerous such a scenario can be.

This is where Augmented Reality (AR) steps in. The problem of pilot error isn’t so much as information not being available, but rather, too much information presented all the time that can lead to analysis paralysis. With AR applications, timely relevant information can be presented to the pilot when it is needed in an intuitive format, so that they can focus on the task at hand.

The idea of using AR in aviation isn’t so far fetched either, in fact, it has already been successfully implemented. Today, every fourth generation onwards fighter jet comes with a standard issue Heads Up Display (HUD) that displays critical navigational, flight, targeting, and mission related information on a piece of glass in front of the pilot. The idea is to ensure the pilot need not keep looking down at the instruments while in the heat of the battle. The fifth generation F-35 Lightning 2 has taken this concept even further by installing a complete AR package within the pilot’s helmet, giving them unprecedented 360 degree situational awareness and even see-thru ability.

Now, while most technologies typically trickle down from military applications to consumer markets, startups such as Aero Glass are also disrupting the traditional aviation landscape. Today, thanks to falling hardware prices and advancements in visualization technologies, AR is finally ready to make its appearance in commercial flying as well, a development that is long overdue. Many car models from Audi, BMW and Toyota have HUDs and it’s easy to find third party add ons for regular cars as well, so it’s definitely due for flight systems.

How AR Can Help Pilots

As stated before, the primary utility of AR in aviation is its ability to overlay relevant information on demand. Today’s AR systems can visualize terrain, navigation, air-traffic, instrument, weather, and airspace information in a 360-degree, 3D overlay that is easy to understand. Here are a few ways in which AR can assist a pilot. The following are shots from a working Aero Glass prototype in action.

So, let’s say a pilot is getting ready to taxi. Their AR HMD can create a virtual checklist that can help them with their pre-flight checks. Once the check is complete, the HMD can display runway information and guide the pilot to their designated runway. The pilot can even be alerted of other aircraft that are taxiing/landing/taking off.

Likewise, when the pilot is getting ready to take off or land, the AR system can display a simple corridor overlay to show the appropriate path. This is particularly useful as taking off and landings are the riskiest part of flying. As pilots are closer to the ground, any emergency needs to be addressed quickly. By telling a pilot exactly what needs to be done, an AR system can negate oversights making take-offs and landings simpler and safer.

Finally, an AR system can prove very handy during the cruise phase of the flight as well. Important information including artificial horizons, waypoints, weather updates, flight plans, restricted areas and terrain information can be displayed to provide complete situational awareness.

The display can be customized to a pilot’s preferences and modes can be turned on and off as well. It’s worth noting that a very high degree of precision is required to make this work and even the slightest different in overlay can have drastic (and potentially fatal) consequences.

Check out the below video to see a working Aero Glass prototype in action:

AR Use Cases Beyond Piloting

While the above mentioned uses of AR are quite obvious and well tested, the technology presents opportunities elsewhere as well. Maintenance Repair and Operations (MRO) are another area that can benefit greatly from AR. Training and licensing a technician can be very expensive and time consuming. In the U.S.A., it can take up to 8 years for a maintenance professional to become fully licensed primarily because training is usually hands-on and getting access to equipment can be tough at times.

AR, VR, and Mixed Reality are already proving to be invaluable here. By creating virtual replicas of the actual components, technicians can practice their skills in a safe environment as many times as needed. They can place their hands on virtual parts and work with them just as they would on the real thing. AR/VR based instructions can reduce the amount of time and money required to train a professional, while making training completely accident-free.

Likewise while HUDs are making appearances in automobiles, they are barely scratching the surface of what’s possible. Wearable AR systems can provide 360-degree situational awareness to drivers just like pilots and help them drive safer. Landmarks, navigational information, and hazards, can all be displayed in front of a driver’s line of sight so that they don’t need to keep taking their eyes off the road.

Some people are of the opinion that automation is the future of both general and military aviation. Autopilot and sensor technology are no doubt making great strides and they will make the skies safer. That being said, technology won’t be replacing the humble pilots anytime soon, error prone as they might be.

Take for instance the case of Flight 1549 (the flight the movie Sully is based on). Heading from New York City to Charlotte, North Carolina, the plane experienced a bird strike just 3 minutes after take off which took out both the engines. Finding that he couldn’t turn back, nor could they make it to New Jersey’s Teterboro airport, the pilot decided to ditch the plane in the Hudson river, which he successfully did saving all the 155 people onboard. Now known as the “Miracle on the Hudson,” the incident is a reminder that the human element cannot be overlooked as machines cannot make decisions of such nature.

Augmented reality applications such as those being developed by Aero Glass will help pilots of the future avoid costly mistakes and make timely decisions that will save lives. While the technology is still under development, it goes without saying that the enhancements to safety they bring are well worth the time.

---

Disclosure: This is a guest post by an actual pilot named Ákos Maróy; he is also the founder of Aero Glass. The content in this article was not produced by the UploadVR staff, but was edited for grammar and flow. No compensation was exchanged for the creation of this content.