Editor’s note: Combing through the final accident report and related documents detailing Joby’s 2022 eVTOL crash, Vertical’s Alex Scerri provides an op-ed recapping the incident and the lessons that industry can learn from it.
In aerospace engineering, technologies and boundaries are continuously being pushed to the limit, and the possibility of an anomalous event during a flight test program is always present. While the initial collective shock, amplified by clickbait headlines, may appear as a major setback, it can serve as a precious catalyst leading to more robust designs and providing invaluable insight on how to improve the flight test process itself.
While the internal lessons learned are a given, the medium by which this knowledge is shared with the wider community is through accident reports, such as the one for the Joby JAS4-2 accident at Jolon, California, in February 2022. The National Transportation Safety Board (NTSB) provides some invaluable insights in its final report published earlier this year — two years after the event.
This review is based on the NTSB report and the documents in the associated docket, some of which are redacted.
The incident
On Feb. 16, 2022, Joby was conducting planned, remotely piloted, airspeed and altitude envelope expansion flight tests on aircraft JAS4-2. As per NTSB form 6120.1 filed by Joby, during the second test flight, the aircraft completed a three-minute, 175 knots indicated airspeed (KIAS) (201 miles per hour) test point level at 11,000 feet (3,350 meters). From the data readout, approximately 90 seconds after completing that test point, the aircraft commenced a dive test. When it reached a maximum dive speed of 181 KIAS (208 mph) at an altitude of approximately 8,900 ft. (2,712 m), a propeller blade on propulsion station 3 (located on the right wing inboard) experienced a bending failure near the blade root, which resulted in the release of the propeller blade.
The released blade impacted the propeller on propulsion station 4 (located on the right wing outboard), which subsequently resulted in a release of the impacted blade. Cascading effects resulted from the initial inflight blade failures, including the separation of multiple propulsion motor/propeller assemblies and loss of remote pilot control of the aircraft. The aircraft departed controlled flight and impacted the ground.
Examination of the high-resolution recorder (HRR) data for the accident time revealed that the variable pitch actuator for station 3 was commanding a typical cruise pitch when the blade release occurred, whereas video evidence indicated a steeper pitch on some blades immediately before the initial blade release. Accelerometer data for station 3 showed a rapid increase in vibration after reaching the accident flights test condition before the initial blade release. Tilt actuator position values for station 3 also showed an oscillation at this time.
Prior flight test data examined by Joby revealed consistent asymmetric behavior between station 2 and station 3, despite identical mirrored designs. In cruise mode, the tilt actuators on station 3 showed increased activity in all flight conditions compared to station 2. Tilt actuator linkage loads were also higher in station 3, which can be an indication of anomalous behavior in the tilt mechanism. The resonant response to this propeller mode crossing in station 3 was also consistently stronger than in station 2, indicative of a coupled interaction with the anomalous tilt mechanism. While prior flights excited the propeller mode in transition flight, the strong excitation in cruise was not predicted. Post-accident analysis revealed this strong excitation was due to aerodynamic interactions that only became significant when the airspeeds were beyond the expected operating conditions of the aircraft.
The dive speed of 181 KIAS (208 mph) reached during the flight test, in conjunction with an anomalous propeller tilt system condition at propulsion station 3, likely resulted in unanticipated aerodynamic interactions that excited a propeller mode, leading to a non-uniform blade pitch increase beyond its design limitations. This likely caused a load exceedance which resulted in the initial blade failure.
NTSB probable cause and findings
The NTSB determines the probable cause(s) of this accident to be the separation of a propeller blade during expansion flight testing that resulted in cascading effects to include the separation of multiple propulsion motor/propeller assemblies and the loss of remote pilot control of the aircraft. Contributing to the accident was the tiltrotor actuator linkage for propulsion station 3 that allowed some propeller blades to be at a steeper angle than commanded.
Incident review
The first and foremost positive takeaway from this event is the immeasurable advantage of conducting uncrewed flight tests for high-risk level tests wherever possible. The dive speed test is classified by NASA’s Flight Test Safety Database in the case of crewed aircraft. Although the aircraft was lost, there was no loss of life nor injury to the crew or third parties, and beyond the primary advantage of having no adverse human effect on families or colleagues, it also most certainly allowed a quicker resumption of the flight test program.
Recording devices and video evidence
A cockpit-mounted GoPro 360-degree video camera provided very valuable information for the report. A video study included in the docket documents showed how the imagery was used in conjunction with a computer aided design (CAD) model overlay to determine the actual pitch angles of the blades compared to the commanded value prior to the blade being released.
However, it was fortuitous that the recorded data was recovered at all, because the device was not designed to be crash-resistant. This is certainly something that could be considered for future flight test programs or the alternative of having the video data transmitted and recorded at a ground station in real-time.
An onboard flight test instrumentation data acquisition module designed to record telemetry information, such as vibration and strain gauge information, did not provide any usable data that could have been useful for the investigation. As for video data, it could be a consideration to have this information downlinked during flight.
Battery-powered flight test challenges
When I spoke to Giorgio Clementi, president of the International Test Pilots School (ITPS) Canada Ltd in early 2022 and prior to this accident, I asked him what he thought could be a specific challenge for eVTOL flight tests. One thing he remarked was the limited endurance of battery-powered eVTOLs, which could compress the time available to perform the test cards.
For the Joby event, it appears that the aircraft started the mission with some value less than 100% battery state, although this is not clear from the data readout available. The first test point, the 175 kt (201 mph), three-minute level run, was presumably quite power intensive in relative terms. By the time of the in-flight event, the battery state appears to have decreased to about 50% in less than 15 minutes total flight time.
There was approximately 90 seconds between the two tests and one can consider if this was sufficient to review the data from the first test, which could have revealed some anomaly that in turn, would have triggered postponing the high-risk dive test that followed.
Propeller failure analysis
From Joby’s internal recommendations in NTSB form 6120.1, it is stated that low-fidelity aerodynamic analysis may not predict higher-harmonic propeller loads at high-speed cruise flight. In addition to numerical aerodynamic analysis, Joby presumably did quite comprehensive propeller testing at the Bonny Doon circular track. However, it is unclear if that setup can fully simulate high-speed cruise flight conditions. Just one year after the accident, in February 2023, Joby carried out a test campaign at the National Full-Scale Aerodynamic Complex (NFAC) at NASA’s Ames Research Center, which could have added useful additional data for this flight regime.
The NTSB report also stated that there were irregularities noted in portions of the adhesive bond area on the upper skin of the initial failure blade from station 3 that were consistent with amine blush. Production, quality, and inspection processes for the propellor subcomponents would be something that Joby could be looking into. Computed tomography (CT) scans of the failed blade had shown a small anomaly on the inner surface of the spar, likely affecting a single ply. However, comparisons of this anomaly over time revealed that this feature would not have met the criteria to remove the blade from service and was not likely a factor in the accident.
Cascading failures
While distributed electric propulsion (DEP) brings an element of safety through redundancy, the possibility of cascading failures, specifically those resulting from high-energy fragments from propulsion system failures, needs to be considered in the design and certification phase.
The European Union Aviation Safety Agency’s (EASA) MOC VTOL.2240 (d) High Energy Fragments – Particular Risk Analysis includes detailed guidance. For category enhanced aircraft, the first failure should not have a catastrophic effect. If the first failure can cause a second failure of a lift/thrust unit, the probability of the second failure should be evaluated considering the probability of the occurrence of the first failure and the probability of a chain reaction.
If the overall probability is less than 10-9 per flight hour, the hazards can be considered to have been minimized and the analysis can stop there. If higher, the effect of the second failure should be considered and should not be catastrophic. The probability of the third failure should then be evaluated and continued until the overall probability of the next failure is less than 1 x 10-9 per flight hour or all lift/thrust units have been assessed.
Further detailed guidance is also available in EUROCAE ED-306, Design Considerations for VTOL Aircraft Protection from Uncontained High-Energy Fragments and Sustained Imbalance, published in October 2022.
The Joby S4 will be certified under the Federal Aviation Administration’s (FAA) special class airworthiness criteria for Joby’s JAS4-1 powered-lift, which has been updated on March 8, 2024, incorporating feedback from the industry. Multiple commenters requested that the FAA align JS4.2240(c) with EASA SC–VTOL.2240(d).
The FAA comment is that JS4.2240(c) is similar to SC–VTOL.2240(d), although SC–VTOL.2240(d) refers to “lift/thrust unit” instead of “engine.” The EASA term “lift/thrust unit” includes the engine and propeller or rotor assembly. This topic is an ongoing discussion with foreign certification authorities. For the JAS4–1, other rotating parts within the system, except for propeller blades or rotors, should be evaluated using typical rotor burst methods, including shielding where practical.
It is understandable that redesign may not be the most desirable nor practical path at this point, and other avenues to reach the required safety levels can also include increased component reliability and more frequent in-service inspections, including predictive maintenance.
The NTSB report timeline
The final report for this event was published at just about the two-year mark from the occurrence date. This is not unexpected for a report of this complexity and level of detail, especially considering that this is a novel aircraft design and the overall NTSB workload. As for any new technology in a testing phase, this type of event is not totally unexpected and a nascent and fast-moving industry such as eVTOL would benefit from a quicker turnaround. One idea in this direction is to use the French model, where incidents for prototype aircraft are investigated by a separate entity, the Bureau Enquêtes Accidents pour la sécurité de l’aéronautique d’État (BEA-É). This is a military organization that just investigates accidents for state aircraft compared to the BEA, which investigates all commercial and general aviation occurrences.
Conclusion
This investigation and report are an important confirmation of the transparency and information sharing mindset that is vital for the common growth of the ecosystem. We must make sure that the system embraces the culture of continuous improvement. Lessons learned must be shared openly and in a timely manner in order not to repeat any avoidable events later in service that could have a negative impact on the industry.
Having reached out to Joby during the compilation of this article, the company replied as follows:
“We are confident that the actions we have taken, including design changes, additional instrumentation and testing, eliminate the circumstances that caused this accident.
Since the accident, we’ve incorporated a range of improvements to our design and testing methodologies, many of which were already planned, and our second pre-production prototype aircraft has flown nearly 25,000 miles, including more than 100 flights flown by a pilot on board, as well as exhibition flights in New York City.
Experimental flight test programs are intentionally designed to determine the limits of aircraft performance and, in doing so, provide critical insight and learnings that support the safe operation of aircraft, as well as inform final design elements.
We continue to work with the FAA to ensure that the type certification process reflects our learnings, and we are committed to sharing relevant learnings with other eVTOL companies, to further support the safety of our entire industry.
We have also produced, flown, and delivered our first production prototype. This aircraft, which includes a number of upgrades compared with our pre-production prototype aircraft, is stationed at Edwards Air Force Base for flight testing with the U.S. Air Force and NASA.”
Wouldn’t smaller diameter props have less leverage applied to the blade root, thus be less likely to bend and separate?