Neo Vision: Foreign Object Debris Detector

By: Haichuan Wang

School: The Pennington School

Date: 2019-2020

1. Rationale:

Runway safety has always been concerns for major airports across the world, where Foreign Object Debris (FOD) on runway and taxiway surfaces have been the major threat to airport operation. According to the Federal Aviation Administration (FAA), FOD causes an estimated $4 billion in damage in engines and aircraft taken out of service each year. Under the current system, most airports are using manual runway inspections by humans and pilots’ reports as ways to identify FOD on critical airport surfaces. Being a licensed pilot myself, experience shows that this system cannot minimize the possible damage to the aircraft, especially when a plane is taking off over an unreported FOD. The infamous Concorde accident, for example, with hundreds of deaths in Paris was caused by FOD on its take-off runway.

Manual FOD detection can decrease operational efficiency in airports. Runway FOD inspection causes airports to stop arrival and departures traffic for an average of 3 minutes per incident, which is a big number for airports like New York JFK when planes are supposed to land and take-off every 45 seconds.

By using the yolov3 model for object detection, a subcategory in deep learning and neural network, the Foreign Object Debris Detector can detect possible FOD on the surface of the runway; automatically categorize FOD into urgent and nonurgent categories, and pick up the FOD from the surfaces. This detector connects with the airport operation system, which when a FOD is reported, can dispatch the inspection team to take appropriate actions so that they can minimize the possibility of dangerous unreported FOD. This will make the airport environment safer by minimizing the possible damage to the aircraft, and saving billions in repair fees to the airlines.

2. Engineering Goal:

The goal of this foreign object debris (FOD) detector is to find and categorize different kinds of FOD in the airport environment. The detector will also have the ability to pick up the FOD on airport runways. The detector will immediately alert the operation center if dangerous FOD such as metals and plastic bags have been detected and proceed to the active runways to pick up the FOD with permission from the air traffic control. However, if the detector identifies less urgent FOD such as grasses and thin layers of rubber resulting from normal operations, the system will not alert airport staff immediately as clearing these less urgent FOD can cause delays in airport traffic.

3. Design:

1. Procedures

1. Initial Research:

The initial research will include an investigation into different types of FOD that airports encounter. The information will be gathered from different airports in the United States by interviewing/surveying airport managers and investigating each airport’s published statistical data. The selection of the airports will be based on size, geographic location (low/high-density altitude), and any special weather conditions (snow/storm/heat, etc.) that an airport experiences. In order to ensure the statistical validity of the interview, 15 airports across the region and the FAA William J. Hughes Technical Center’s FOD analysis branch will be selected and interviewed. Of those airports, 60% should be major hubs like New York JFK, where FOD could have a potential higher effect than those smaller airports. The remaining 40% consists of regional and general aviation airports like Trenton-Mercer, where FOD could be a safety concern with regard to operations, but with less financial consequences than those major airports. Information regarding FOD’s effects in different weather conditions can be gathered through the interview with the FAA technical center. The survey questions will address the general effect of the FOD for that particular airport, common types of FOD, and their removal requirements. 

 Once this investigation is completed, a list of common FOD can be created and categorized into dangers, and non-dangerous categories.

It will also be necessary to research accuracy and detection quality when using different kinds of software and imaging equipment. This research will be conducted by looking at published performance data and interviewing engineers who work in the field. Initial research will run test data with different methods in order to find the method with the best performance.

2. About the Detection:

There will be a computer-based object detection system, using the yolov3 model, to train the detector to identify classes of objects. The data gathered from the survey of different airports will be used to classify different types of FOD into “dangerous” and “non-dangerous” categories. Once such a list is created, the images of these items can be found and labeled with labeling software to create an image database. The yolov3 model can create bounding boxes of an object class by learning from the image database, which makes the system able to recognize such a class of objects after the training. The detector will be able to detect and categorize the FOD by themselves afterward.

The results of any given detection will be input to pre-programmed software, which will show the type of FOD identified, and the category to which that FOD belongs. When the detector is connected to the airport network, it can send out a warning to the operations staff. 

There will be a LIDAR sensor installed for redundancy, which reports a detection if the computer-based system does not recognize some non-defined object on the runway. The LIDAR sensor will only activate when the computer-based system is not responding, and the airport manager will be able to observe the FOD through the camera on the detector.

3. About the Design of the Detector:

The first version (V1.0) of the detector will focus on functioning in good weather conditions (without considering snow/storms/low visibility conditions). The detector will contain three sections: base, robotic arms, and camera. 

The camera will be used for detection and to provide a live view of the runway. The camera is a 120-degree wide-angle camera that will be able to rotate 45 degrees to the left and right. The camera will have a lens protector, in order to be operable in bright and rainy conditions. Five different types of cameras will be tested to find which camera can provide the most accurate detection result. The quality tests will be based on published performance reports and image analysis.

The base of the detector will consist of three wheels for the movement of the detector between its station and the runway. The wheels will be made of rubber, and the connections between them will be made of light metals so that it will be durable but lightweight. The main section of the base will be designed through computer software and 3D printed, using plastic. The base will have a position to place the LIDAR sensor; a rotational gear on the top will rotate the camera in different directions (Figure 1).

Figure 1: Base

The arm will be connected to the central part of the base, having two joints, which make it able to pick up the FOD on the runway (Figure 2).

Figure 2: Arm

After the prototype is built, additional detection capability tests in heat, windy, and rainy conditions will be conducted. The test will also assess the freedom of the movement of the detector, and the quality of the images that it captures.

2. Risk and Safety

There are anticipated light safety risks in the process of building this detector, as the majority of this project is based on computer software, pre-built sensors, and 3D printers. However, caution should be exercised when linking wires between the detectors and the computer to avoid possible short circuits. 

A simulated runway environment (i.e. a parking lot) will be used for testing. The performance of the detector is an unknown factor, so the testing environment will be closely monitored.

The main risk for the detector itself is that it could become a FOD, so it will be important to contact regulatory officials to ensure that such a device will not be a concern for the operation safety of airports.

3. Data Analysis

1. FOD in Different Airports:

After the survey (mentioned in section c.1.a), a list of common FOD will be computed, both in “dangerous” and “non-dangerous” categories. Then, it will be necessary to create a minimum margin which determines to what kind of FOD the detector will be reacted to. This data will be used to train the detector.

2. The Practicality of the Detector:

After the survey (mentioned in section c.1.a), a chart that represents the necessity of such a device to different sized airports can be created. If more than 70% of the airport managers surveyed believe that such a system will be beneficial, it will be possible to conclude that such a detector is practical in operation.

In some airports, like Boston Logan and Seattle Tacoma, a similar system, using radar, has been installed for testing. Results can be compared with the performance records of a similar system in those two airports to identify the necessity of the automation in FOD detection. If the automatic FOD detector can increase the operation efficiency, then it may be concluded that such a device will be beneficial to the public.

3. Detection Accuracy:

When the detector is trained, test runs can follow to examine the accuracy of the computer-based system. A comparison between human and automatic detection will be done to recognize some common FOD and confirm detection results. The result of the system will be compared to a human’s result, where the accuracy of the system will be computed. A confidence interval (with a confidence level of 90%) for the two sets of data will be used to see if there is a statistical difference between the result of human detection and automated detection. If there is a significant difference, it will be important to improve the accuracy of the detector. Otherwise, the detector will be proved to be functional.

Finally, the detector will be tested in the school’s parking lot, simulating the airport runway environment, and evaluating the performance of the detector when objects are at a distance. If it is proven that such performance agrees with previous tests, and the detector has the ability to travel and pick up the FOD, it can be concluded that the goal of making a foreign object debris detector has been achieved. 

4. Bibliography

1. "Airport Foreign Object Debris (FOD) Management." Federal Aviation Administration. Last modified September 30, 2010. https://www.faa.gov/airports/resources/advisory_circulars/index.cfm/go/document.information/documentID/391902.
2. The Bureau of Enquiry and Analysis for Civil Aviation Safety. Accident on 25 July 2000 at La Patte d'Oie in Gonesse (95) to the Concorde registered F-BTSC operated by Air France. July 25, 2000. https://www.bea.aero/uploads/tx_elydbrapports/f-sc000725a.pdf.
3. "Fact Sheet – Foreign Object Debris (FOD)." Federal Aviation Administration. Last modified November 15, 2013. https://www.faa.gov/news/fact_sheets/news_story.cfm?newsId=15394.
4. "Foreign Object Debris (FOD)." Sky brary. Last modified November 26, 2019. https://www.skybrary.aero/index.php/Foreign_Object_Debris_(FOD).
5. Herricks, Edwin E., Elizabeth Woodworth, Sid Majumdar, and James Patterson, Jr. "Performance Assessment of a Radar-Based Foreign Object Debris Detection System." Federal Aviation Administration. Last modified February 2011. https://www.airporttech.tc.faa.gov/Products/Airport-Safety-Papers-Publications/Airport-Safety-Detail/ArtMID/3682/ArticleID/68/Performance-Assessment-of-a-Radar-Based-Foreign-Object-Debris-Detection-System.

Appendix 1: Project Summary

I. Description

After interviewing with relative airport operations and regulatory staff, an automated moving detector is not recommended near the runway environment. With further research, each runway contains a runway safety area where only necessary requirements are allowed to be placed in. Such a moving detector, however, may propose concerns to the regulatory agency in which may be a harm to the safety of operations.

In response to this regulation, it is more practical for the detector to move along a metal track that is installed parallel to the runway. Due to the accuracy of object detection varies from different angles, such a track-based detector can detect a possible FOD from different locations, which increases the overall accuracy of the detection.

II. Project Adjustments

1. There will be a track installed parallel to the runway, where the detector moving back and forth to make its detections;
2. The arm will be removed as the detector cannot proceed to pick up the FOD;

III. Design Summary

The design will be similar to the one that is mentioned in section III.1.c., where the wheels will be replaced by a 3D printed base that can move on a track. There will be no arms for the detector. The design of the detection system remains the same, and the design of the rotating camera remains the same.