Detect And Avoid Autonomous Augmentation System

1. Roadmap Overview

This is a technology roadmap for: 3PEM - Proton Exchange Membrane This is a “Level 3” roadmap, indicating that it addresses a technology at the subsystem level. Higher level roadmaps related to this subject would address technology progression at the market level (Hydrogen Supply) and product level (Water Electrolyzer). A short video summary and introduction to this roadmap follows here.

{{#evt: service=youtube |id=https://youtu.be/mfDTZW8ekyg }}

ROADMAP OVERVIEW

The technology selected consists of a Detect And Avoid Automated Augmentation System (DA4S). The system is intended to augment Urban Air Mobility (UAM) pilots in their responsibility to see other air traffic and maneuver to remain well clear of them [14 CFR Part 91]. As studied in this roadmap, this is a “Level 4” technology (4DAA), where “Level 1” is the market level for Urban Air Mobility (1UAM), “Level 2” are the Increasingly Autonomous System (IAS) aircraft found in UAM (2IAS), and “Level 3” is the flight control system (3FCS). Even though our roadmap targets this for the UAM market, it is noted that the technology is applicable to many General Aviation (GA) aircraft.

Detect And Avoid (DAA) is a significant challenge in emerging aviation markets such as integrating Unmanned Aircraft Systems into the National Airspace System, or increasing the density of air traffic as envisioned in UAM [1]. The DA4S technology is intended to support the pilot by performing four main functions: (1) help detect and track other air traffic, (2) plan trajectories to remain well clear of the traffic and satisfy other flight constraints, (3) issue control commands into the aircraft autopilot to maneuver the aircraft along the selected collision resolution trajectory, and (4) interface with the pilot. Each of these functions are enabled through a suite of integrated subsystems. The technology is intended to be aircraft agnostic, and be able to be integrated into any small piloted GA or UAM aircraft. The system augments the pilot in performing DAA, but does not replace him/her. The pilot retains the ability to visually detect, plan, and perform their own evasive maneuvers. The pilot may also work with the DA4S autonomy in any of its first three functions to improve the overall system output. For example, the pilot may input traffic they detect themselves, tune trajectory planning parameters, and assist in selecting the best resolution. The figure below shows the OV-1 diagram for the ACAS Xr system, which is one of the technologies being pursued to assist in DAA for UAM aircraft, and visually represents some of the considerations for the roadmapped DA4S technology.

DESIGN STRUCTURE MATRIX (DSM)

ROADMAP MODEL USING OPM

FIGURES OF MERIT

One of the critical FOMs that we are looking at are the number of fatal accidents per million flights. Ultimately what DA4S aims to accomplish is increase flight safety in a future combined environment where there will be both crewed and autonomous flight vehicles. This is also one of the most well historically documented pieces of flight data that we have and is maintained by the National Transportation Safety Board (NTSB) [3]. The absolute limit would be zero fatalities for any flight. This would likely be extremely difficult to achieve. However, there is still plenty of room for improvement as there are still numerous fatalities every year. Unlike performance FOMs which typically seek to increase their metric, safety systems tend to be inverse as the utopia point is zero. This trend more closely resembles an S-curve. As we can see in Error! Reference source not found., there was a takeoff in airline safety in the mid-1970s. Then a slowing in the 1980s which has led to somewhat of a stagnation since 2015. Since 1970 this FOM has exhibited an annual rate of progress of 0.12 reduction in fatalities per million commercial flights through 2017. In terms of technology lifecycle of flight safety, we take consider visual flight rules being first generation, instrumentation flight being second generation, electronic/computer aided flight as the third and current generation. DA4S would introduce a leap forward in the next generation of flight safety with the implementation of autonomous safety systems. A future FOM will also be derived utilizing smaller craft hazard reports from commuter and on-demand flight operations conducted under 14 Code of Federal Regulation (CFR) Part 135.

ALIGNMENT WITH COMPANY STRATEGIC DRIVERS

POSITIONING OF COMPANY VS. COMPETITION

The following figure compares the DA4S system against its competitors for the following considerations: (1) traffic detection capability, (2) trajectory planning capability, (3) mechanism to control maneuvers, (4) pilot interface, (5) hours required to pilot such an aircraft, and (6) maturity of the system. The mechanism used to create this comparison is described in the technical model section. A broad variety of aircraft was included in this analysis in an effort to capture the trade with respect to the overall field of aviation. Aircraft included in the analysis include small and larger GA aircraft, a variety of business jets, an airliner, a 5th generation fighter jet, unmanned aircraft, and Urban Air Mobility concepts. A visual comparison of their flight deck technologies is shown below.

TECHNICAL MODEL

The following figure plots the tradespace of aircraft in terms of their Detect And Avoid augmentation capabilities compared to a broad variety of other aircraft. The vertical axis represents a Detect And Avoid rating systematically derived for each aircraft using the following considerations: (1) traffic detection capability, (2) trajectory planning capability, (3) mechanism to control maneuvers, (4) pilot interface, (5) hours required to pilot such an aircraft, and (6) maturity of the system. The horizontal axis represents the cost per empty weight of such aircraft. Since the DA4S is intended to augment pilots of General Aviation (GA) and Urban Air Mobility (UAM) aircraft, the design point utopia is to the top left of the chart (high DAA rating with low cost per empty weight). Unsurprisingly, this design point of DA4S is in the vicinity (but outperforms) several existing UAM prototypes.

The Empty Weight and Cost data for each aircraft were obtained using internet searches. The Detect And Avoid rating was systematically derived for each aircraft to capture attributes of importance to our technology. These attributes are reflective of the four main functions of the DA4S system: (1) traffic detection capability, (2) trajectory planning capability, (3) mechanism to control maneuvers, (4) pilot interface. In addition, two attributes were included to capture aircraft complexity of operation using (5) hours required to pilot such an aircraft, and (6) maturity of the system. Performance scores for each of these ratings were obtained using the specifications listed in the 6 tables below given expert knowledge of the aircraft assessed by our team members (2 of the team members are experienced pilots). These

6 different scores were combined into a unique rating using the following Multi-Attribute Utility Analysis Equation.

where (Ji) represents each of the six attribute scores from the tables above. ki represents normalizing weights, representing the relative important of each attribute.

The following figure below represents the Morphological Matrix for the DA4S technology. The circled items consist of one possible configuration embodiment. The forms chosen for each function are consistent with a solution that would meet the standards of the ACAS-X standards in development for GA, UAM, and UAS aircraft (see Paper 1). On-board EO/IR sensing is provided as a supplement to ADS-B widely available today, and provides a day and night detection capability that can plug and play into the broader surveillance concept of ACAS- X. A probabilistic trajectory planning algorithm is implemented using Markov Decision Processes as done with ACAS-X. It is assumed that the GA and UAM aircraft that will seek this technology will be equipped with Multi-Function Displays (MFD) as part of a glass cockpit into which the DA4S information can be integrated. DAA information will be integrated into a Synthetic Vision application as currently available for Instrument Flying on some GA aircraft. DAA information can further be projected onto a simple head mounted unit to provide additional immersion to the pilot. Resolution Advisories will be coupled to the Flight Director on the MFD to provide the pilot with low level attitude control guidance to execute the maneuvers. In order to keep the DA4S system as a “bolt on” system that can be integrated into any aircraft, we decided to leave the control inputs to be performed by the human pilot, which will follow directions provided by our system. This too is consistent with current implementations of ACAS-X.

The two FOMs highlighted in the sensitivity analysis are scanning and tracking. Scanning is the ability for the DAA system to survey the environment around the aircraft. Scanning performance is influenced by the Azimuth (A, degrees), Elevation (E, degrees), Range (R, kilometers), and Time (T, seconds). These factors determine the overall scanning ability in the equation below.

For the sensitivity analysis we will use the estimated minimum and maximum expected bounds that the system will be exposed to in the table below. The higher the sensitivity performance, the better it is for DAA.

In terms of scanning performance as a FOM, range of the DAA is the most sensitive on the system. In order to achieve the greatest performance for the system. One should try to optimize DAA to achieve the greatest sensing range. This makes sense as you increase the envelope in which you have more situational awareness. The ability to detect hazards goes up tremendously. Tracking performance is determined by the number of vehicles within the same shared airspace as the aircraft DAA is equipped on. Tracking performance is dictated by the following parameters – Number of Targets (N, dimensionless), Velocity of Targets (V, m/s), and Time (T, seconds). The relationship is depicted in the equation below.

For tracking performance, the system is most sensitive to the number of targets that are being tracked. The duration of which it is being tracked is second most sensitive while the velocity of the target is less relevant. This makes intuitive sense as the most important thing to know is if another party is sharing your airspace and where it is located. The velocity of the target is supplemental information as you could avoid the targets’ general location with its initial position information.

FINANCIAL MODEL

The financial model assumes an initial employment of four engineers with initial Product Development Project (PDP) period of four years from 2021-2024 as two more employees are added each year while the product is matured. This period will produce the first initial prototypes and be refined with beta testers. The first full retail product will be available in 2025 (year 5) which sets an expectation of 250 units being sold. The DA4S experiences further improvements, adoption, and enhancements with future features being integrated in 2027 and 2030. By 2031 (year 10) the product will have reached full-scale production.

LIST OF R&D PROJECTS

The ability to successfully develop and launch the Detect And Avoid Autonomous Augmentation System (DA4S) is dependent on its R&D Portfolio. The DA4S system supports four fundamental functions: (1) detect other aircraft, (2) plan trajectories, (3) maneuver the aircraft, and (4) interface with the pilot. As such, the R&D portfolio is diversified to span these functions at a variety of Technology Readiness Levels to support multi-generational growth of the system. The following is a list of 10 projects that are part of our portfolio. These are plotted on the R&D Portfolio framework based on their project type in the figure below.

The R&D Portfolio Plan, including the budget, schedule, and milestones associated with each individual project is shown below.

The figure below maps how the Figures of Merit (FOM) of some of the projects above transfer to the higher level FOM of the overall DA4S system based on the Technical Model previously described in the Technology Roadmap. The figure represents the current expected progress of each FOM without our R&D efforts and compares it to improvements expected from our R&D. The right side of the plot represents the overall DAA Rating FOM found in aircraft today (2021), what the expected market status is without our participation by 2030, and how our R&D projects can improve this state and achieve our targeted design point. The R&D Portfolio Plan, including the budget, schedule, and milestones associated with each individual project is shown below.

KEY PUBLICATIONS AND PATENTS

Detecting and Avoiding (DAA) other aircraft is a fundamental responsibility of an aircraft pilot as stipulated by the FAA in 14 Code of Federal Regulations (CFR) Part 91. As such, the DAA problem has been well studied in the literature, and numerous patents have been developed to support its function. It is a challenging, multifaceted problem that is illustrated by the four fundamental functions our Detect And Avoid Autonomous Augmentation System (DA4S) system must perform: (1) detect other aircraft, (2) plan trajectories, (3) maneuver the aircraft, and (4) interface with the pilot. The following review explores literature papers, patents, and their key takeaways that span this wide set of research challenges.

This is a well cited paper from MIT Lincoln Laboratory that describes the development strategies and challenges associated with the Airborne Collision Avoidance System (known as ACAS-X). The need for a collision avoidance system became nationally recognized following several infamous mid-collisions involving airliner aircraft in the 1950s, 70s, and 80s. This initially led to the creation of the FAA, processes to regulate the National Airspace System, and later to the implementation of the Traffic Collision Avoidance System (TCAS) now mandatory on all airliners. ACAS-X is a major developmental evolution of TCAS intended to handle increased traffic density, plug and play capability using multiple sensing modalities, handle varieties in pilot responses and aircraft performances, and ultimately be accessible to smaller aircraft and Unmanned Aircraft Systems (UAS). The following figure illustrates some of the functions performed by ACAS-X.

Key takeaways from this paper for our technology are the following. First, ACAS-X acknowledges the detection of aircraft should leverage a combination of multiple sensing mechanisms, to include data feeds from Automatic Dependent Surveillance Broadcast (ADS-B), which is now mandatory on all aircraft operating in certain airspaces, as well as on-board optical and radar sensing mechanisms. Second, ACAS-X uses a probabilistic framework based on Markov Decision Processes (MDPs) and Linear Programming to select the best collision resolution actions given uncertainties associated with pilot behavior. Third, ACAS-X serves as a mandatory decision aid to the pilot and provides explicit maneuver directions for the pilot to manually steer the aircraft away from conflict. Fourth, ACAS-X is currently limited to large commercial aircraft, and extensions applicable to smaller aircraft like General Aviation, Urban Air Mobility, and UAS are highly desired, but are still actively being researched. Finally, this type of system is subject to an intensive regulatory and certification framework, and time and costs associated with these items are substantial. The regulatory framework and performance standards associated with ACAS-X equipment have since been listed in RTCA DO-385 [2]. These last two takeaways strongly motivate us to design our DA4S technology to be compatible with the ACAS-X standard in development for GA, UAM, and UAS to serve as its leading technological embodiment.

This is a seminal, widely cited, paper written at MIT on an aircraft trajectory planning algorithm that avoids collisions between multiple aircraft. In it, aircraft dynamics are simplified and approximated using point masses, speed constraints, and acted on by forces of limited magnitudes. This allows a trajectory solver that can be formulated using a Mixed Integer Linear Programming framework, which is shown to be highly efficient compared to other non-linear solvers. The figure below shows how the framework is employed in simulation to deconflict three aircraft by enforcing a minimum distance rectangular region around each in their trajectory planning.

Key takeaways for our DA4S technology are the following. First, optimal trajectory planning around constraints can be very computationally demanding, if all non-linear effects are accounted for. Simplification and linearization of the problem may allow the planning function of our technology to be tractable for onboard computation. Second, regardless of how much the problem is simplified, it will take time to compute trajectories and this process must be carefully balanced against the dynamic nature of the flight environment to ensure solutions are obtained in time. Third, this paper was based on a centralized planning approach, and highlights some of the challenges in deploying the collision avoidance algorithms in a distributed case as will be needed for each aircraft to make its own trajectory decisions in the real world. This challenge is also reflected in some of the other papers in this review.

These two papers are from a related body of work. NASA has sponsored and conducted research efforts applying Formal Analysis to shift the allocation of Detect And Avoid (DAA) from the human pilot over to automated systems. Research has centered on formalizing the definition of remaining “Well Clear” of other Aircraft to enable the development of automated traffic deconfliction algorithms that are needed to safely integrate UAS into the National Airspace System (NAS). A set of DAA algorithms named DAIDALUS (Detect and Avoid Alerting Logic for Unmanned Systems) were developed by NASA in order to ensure Well Clear Volumes (WCV), which are horizontally and vertically defined, remain un-intruded. These algorithms provide predictions of the WCV, maneuver guidance, and alert levels based on the severity of the potential violation. The specifications along with the algorithms are mathematically formulated and verified in NASA's Prototype Verification System (PVS). The DAIDALUS algorithms were recently successfully demonstrated in live flight testing where test pilots followed the output resolution maneuvers. The following figure illustrates the Well Clear Volume and the display used to alert the pilot in this effort.

The key takeaways for the DA4S technology from this work are the following. First, once aircraft are detected, the need to generate a conflict resolution advisory must be mathematically formalized and also be consistent with how other aircraft are doing it. Lessons learned in flight testing highlight some of the problems that can arise in human-automation interactions. For example, the rate at which the algorithm displayed information led to intermittent decision aid displays that were confusing and misleading to highly trained test pilots. This is indicative of the general challenge of integrating automation and humans in safety critical applications.

This paper from NASA proposes a design framework to evaluate the resilience of human-automation teaming systems for aircraft control. It acknowledges the fundamental benefits to having human pilots remain in the control loop of the aircraft. It also highlights a desire from industry to shift some of the responsibilities that are currently fulfilled by human pilots over to certified automation. A process founded in resilience engineering is proposed to decompose the work currently performed by human pilots, and explore what tasks can be performed by the autonomy, what tasks should remain with the human, and how they should support each other in each of these. This process is applied different phases of flight, including “tactical operations” like Detect And Avoid scenarios where conflicts emerge and constraints on the overall mission are shed to reach a safe resolution before resuming normal flight operations. The figure below illustrates different phases for technology to become increasingly autonomous while supporting the human pilot.

Key takeaways for our DA4S technology are the following. First, there is a clear desire on the behalf of industry to reach full automation of flight control technologies that would alleviate the need to train pilots in certain functions. However, second, certification and regulatory frameworks to allow these concepts to be implemented do not exist yet, and it is clear that human pilots will need to remain in the control loop as these concepts maturate. So, third, the design of a DAA augmentation system must deliberately evaluate how the human and autonomy support each other, and consider system safety in these interactions in order for it be successful and accepted by the public. Recommendations for future R&D are provided in the paper, and it is obvious from it that future work is needed in this area.

This patent filed in 2008 is strongly related to the technology we are pursuing. It consists of an augmentation system to the existing transponder based TCAS (Traffic Collisions Avoidance System – see Paper 1). It calls for an arbitrary set of other onboard sensor modalities, including optical, infrared, lidar, radar, and acoustic to be correlated with one another and with traffic alerts called out by TCAS. The patent describes a high-level algorithm run on a system computer to perform the correlation and issue resolution advisories to a pilot or autopilot system. The intent is to augment TCAS for when TCAS does not detect potential collisions by itself, or augment pilot visual DAA, or even provide a DAA suite to UAS. The figure below represents the system data processing flow as well as its generic set of components.

Despite this patent being well aligned with our technology, there is no evidence that a detailed design has been conducted to specify the sensor suite, compute deconfliction trajectories, issue maneuver guidance, or to interface with the pilot. There is also no evidence that this technology has been brought to market. Nevertheless, a key takeaway is that other technologists have recognized the value of having multiple sensing modalities be correlated to assist a pilot in conducting DAA.

This patent describes an algorithm to detect potential air traffic conflicts without a predetermined look ahead time, and determines the time for issuing traffic resolution advisories to the pilot. This patent does not perform any traffic detection sensing by itself, and assumes track information is provided. However, given this information, it determines the relative motion trajectory between the ownship and intruder aircraft, generates multiple options for evasive maneuver recommendations called Resolution Advisories (RA), determines the right time to provide the alert associated with each RA, and then transmits them at the right time to the piloting authority of the aircraft. The patent describes a series of decision trees based on different conflict scenarios for how to execute this process, and includes the mathematic definition for some of its steps. The figure below illustrates one of the decision trees and the embodied architecture.

The key takeaways from this patent for DA4S are the following. First, a DAA augmentation system will face large variety of traffic collision scenarios, which must all be described mathematically to be handled effectively. Second, it highlights the importance of carefully selecting the time of resolution advisory to the pilot as to not overwhelm them with information. This work is well in line with the research presented in Papers 3 and 4 above.

This patent describes an obstacle avoidance system to assist a pilot in maneuvering to avoid ground obstacles. Multiple proximity sensors such as LIDAR are mounted on the exterior of the aircraft to actively detect obstacles. An onboard processor calculates the distance and trajectory to an obstacle, and calculates the treat level posed by it. A pilot interface displays this information and can provide audible alerts to warn the pilot of the potential risk and help them decide how to maneuver to avoid it. The information can also be provided to a Head Up Display or Head Mounted Display to augment the pilot’s capability to see such obstacles.

This patent does not explicitly call out this system as augmenting the detection and avoidance of other air traffic, and instead focuses on fixed ground obstacles. However, a key takeaway for the DA4S system is it illustrates how a sensor suite can be integrated on the aircraft itself, processed by a computer, and the results of this system displayed to the pilot effectively for actionable response. This type of technology can be adapted to augment traffic detection provided that the right sensor suite and processing is incorporated.

TECHNOLOGY STATEMENT

Our target is to develop a Detect and Avoid Autonomous Augmentation System with the four primary functions: (1) detect other aircraft, (2) plan trajectories, (3) maneuver the aircraft, and (4) interface with the pilot. To achieve this target, we plan to have production-intent hardware selected and installed on 10 aircraft by 2025 to begin the stakeholder customer analysis and data collection to train the multi-modal detection and trajectory planning neural networks. The 10 customer machines in 2025 will offer a heads-up display with path planning recommendations. While these units are providing value to the customer and collecting large amounts of data, the development team will be working on using the DA4S system to maneuver the aircraft, which we plan to release to the existing customer base, via an OTA software update. The aircraft maneuverability control will be released in 2027. These products will first be released to the trained early Alpha adopter and then released to the wider fleet when there are a certain number of hours with zero safety-related incidents. By 2027 our target goal is to have 10,000 hours of operation with zero safety-related disengagements. The final step of this roadmap is to operate the system and collect enough data to build and test a fully autonomous, aftermarket installable autonomous system to the existing GA fleet and new UAV aircraft that are being developed. By 2030 we plan to show our system is ready for autonomy by having flown 100,000 flight hours with 0 safety-related incidents.

OTHER REFERENCES