Localization of Natural Disaster Survivors Through Drone-based Sound Source Localization

Student: Ian Lee
Table: ENG1603
Experimentation location: Home
Regulated Research (Form 1c): No
Project continuation (Form 7): No

Display board image not available

Abstract:

Most natural disaster casualties arise from entrapment underneath collapsed structures, as disaster-relief efforts can be hindered by the danger and inaccessibility. Consequently, unmanned aerial vehicles (UAVs) can be used for survivor detection. Some UAVs are mounted with microphones so that shouts or whistles can be detected in low-visibility conditions. In order to determine the location of a sound of interest, UAVs use sound source localization (SSL), but current methods can be inefficient or inaccurate. A novel method of SSL was designed using the volume ratios of the left and right channels of a stereo microphone to calculate the horizontal and vertical angles of origin of an incoming sound within a given frequency range.

A 1,760 Hz sound was played at seven locations, each a multiple of 30 degrees 30cm away from a stereo microphone. The Wolfram Mathematica programming language was used to compute seven left/right ratios that each corresponded to an angle. These served as the groundwork for an exponential model that could output an angle of sound origin when given a volume ratio. A band-pass filter was applied so that a specific range of frequencies could be analyzed. The system calculated vertical and horizontal angles of origin with an average of 4.9 degrees off from the actual position in quiet conditions, and 19.2 degrees in loud conditions. Calculations took around 0.03 seconds. The accuracy and speed of the system show the viability of this form of SSL, and its potential in life-saving applications.

Bibliography/Citations:

 

 

Feuilherade, P. (2017, July 19). ≡. Retrieved December 30, 2020, from https://robohub.org/robots-to-the-rescue/

Cara Labrador, R., & Cheatham, A. (2020, August 24). U.S. Disaster Relief at Home and Abroad. Retrieved December 30, 2020, from https://www.cfr.org/backgrounder/us-disaster-relief-home-and-abroad

Madisetti, V. K. (2010). The Digital Signal Processing Handbook. In 1225997811 911504594 R. C. Dorf (Ed.), The Digital Signal Processing Handbook (2nd ed.). Boca Raton: CRC Press.

Valin, J., Michaud, F., Rouat, J., & L´etourneau, D. (2016). Robust Sound Source Localization Using a Microphone Array on a Mobile Robot. LABORIUS - Research Laboratory on Mobile Robotics and Intelligent Systems Department of Electrical Engineering and Computer Engineering, 1. https://arxiv.org/pdf/1602.08213.pdf

Deleforge, A., Di Carlo, D., Strauss, M., Serizel, R., & Marcenaro, L. (2019). Audio-Based Search and Rescue with a Drone: Highlights from the IEEE Signal Processing Cup 2019 Student Competition. IEEE Signal Processing Magazine, 1.

        Ritchie, H. (2014, June 3). Natural Disasters. Our World in Data.     https://ourworldindata.org/natural-disasters

 


 


Additional Project Information

Project website: -- No project website --
Project web pages: -- No webpages provided --
Presentation files:
Research paper:
Additional Resources: -- No resources provided --
Project files:
Project files
 

Research Plan:

A. Rationale:

Unmanned aerial vehicles (UAVs) are currently used by disaster-relief forces of governments around the world to perform search/rescue efforts as well as supply delivery roles in the wake of natural disasters. UAVs are employed in these situations that are typically too dangerous or inaccessible for human intervention. These UAVs use a wide variety of sensors to locate areas in need of assistance, including an array of stereo microphones to detect cries for help from survivors. UAVs with mounted microphones use sound source localization (SSL) to determine the direction of incoming audio input. Current SSL techniques include machine learning algorithms, the analysis of sound arrival times, and triangulation methods. However, these approaches can be subjected to lengthy calculation times and the UAV’s own propellor noise, both of which are factors that reduce the effectiveness of the drone. This research seeks to determine the three-dimensional direction of an incoming whistle sound based on the volume inputs of two stereo cardioid microphones.

 

B. Research Questions:

  1. How effective is left/right ratio-based SSL in the calculation of direction of origin both horizontally and vertically?
  2. How does background wind noise affect the accuracy of SSL?
  3. Can the lateral and vertical angles of origin of a target frequency be calculated despite the presence of other frequencies? 
  4. How time-efficient are ratio-based SSL calculations?

C. Procedures and Data Analysis

  1. Two stereo microphones will record a 1,760Hz frequency tone played from 13 different locations, each situated 15 degrees apart and 30cm away from the microphones, with three seconds of silence in between.
  2. An automatic segmentation algorithm will be written in the Wolfram Mathematica programming language that splits the audio data into 26 discrete lists of volume data per microphone — 13 lists for the left and right channels. 
  3. The left/right volume ratio of each of the thirteen angles will be computed by dividing the average of the left channel’s angle volume values with those of the right channel’s. This will result in 13 distinct ratios (represented as numbers) for each of the two microphones.
  4. Using this information, a function will be created that receives a left/right channel volume ratio and predicts the angular direction of origin by mapping the ratio to its corresponding angle value.
  5. Another program will be written that extracts a specified frequency’s audio data in an inputted audio recording.
  6. A series of trials will be conducted to measure the speed and accuracy of the system in predicting the horizontal and vertical direction of a series of whistle tones when background noise is present in the environment.

 

D. Bibliography: List at least five (5) major references (e.g. science journal articles, books, internet sites) from your literature review. If you plan to use vertebrate animals, one of these references must be an animal care reference.

Feuilherade, P. (2017, July 19). ≡. Retrieved December 30, 2020, from https://robohub.org/robots-to-the-rescue/

Cara Labrador, R., & Cheatham, A. (2020, August 24). U.S. Disaster Relief at Home and Abroad. Retrieved December 30, 2020, from https://www.cfr.org/backgrounder/us-disaster-relief-home-and-abroad

Madisetti, V. K. (2010). The Digital Signal Processing Handbook. In 1225997811 911504594 R. C. Dorf (Ed.), The Digital Signal Processing Handbook (2nd ed.). Boca Raton: CRC Press.

Valin, J., Michaud, F., Rouat, J., & L´etourneau, D. (2016). Robust Sound Source Localization Using a Microphone Array on a Mobile Robot. LABORIUS - Research Laboratory on Mobile Robotics and Intelligent Systems Department of Electrical Engineering and Computer Engineering, 1. https://arxiv.org/pdf/1602.08213.pdf

Deleforge, A., Di Carlo, D., Strauss, M., Serizel, R., & Marcenaro, L. (2019). Audio-Based Search and Rescue with a Drone: Highlights from the IEEE Signal Processing Cup 2019 Student Competition. IEEE Signal Processing Magazine, 1.

  Ritchie, H. (2014, June 3). Natural Disasters. Our World in Data.     https://ourworldindata.org/natural-disasters

 

 

Questions and Answers

1. What was the major objective of your project and what was your plan to achieve it? 

The main purpose of the project was to create a system that would allow search and rescue drones to locate the three-dimensional direction of origin of a whistling or screaming sound. I also wanted to use a novel method that operated based on the ratio of the left and right channels of a stereo microphone. In essence, I created a program that could receive an input of a raw audio file and output two pieces of data, the horizontal and vertical degree values from 0 to 180 of where a specified frequency is arriving from. First, a series of preliminary trials were conducted with the goal of generating a set of angle-ratio pairs which in turn would serve as a basis for a mathematical function that received a ratio and outputted an angle. Then, code was written so that only the frequency range of 300 Hz to 8,000 Hz was analyzed. Finally, many test trials were conducted to assess the accuracy of the design in different conditions. All code was written in the Wolfram Mathematica language.

       a. Was that goal the result of any specific situation, experience, or problem you encountered?

I had recently read Into Thin Air by Jon Krakauer, where the author describes his perilous journey up Mount Everest as well as the catastrophic storm that resulted in the stranding and deaths of many. Inspired, I looked into the current implementations of disaster relief and the usage of drones to find trapped survivors. I had prior experience in signal processing and was aware of its practicality in barely visible environments. After reading about how these rescue drones can detect noises from survivors, I wanted to see if I could improve upon current sound detection systems.

       b. Were you trying to solve a problem, answer a question, or test a hypothesis?

A multitude of factors can inhibit a disaster relief drone's ability to locate the source of a potential survivor's shouts for help, including but not limited to long calculation times and overwhelming background noise. Consequently, I wanted to use my prior knowledge in signal processing to see if I could create a sound source localization system that could work faster and yield more accurate results.

2. What were the major tasks you had to perform in order to complete your project?

First, I needed to perform an experiment that could accomplish two goals; to verify that the microphones measured in a cardioid pattern, and to generate a series of seven angles corresponding with seven (initially 13) left/right channel volume ratios. This data was gathered in a controlled environment to ensure that the angle-ratio pairs were as accurate as possible, as the main program would be designed around this key component.   

Second, I had to write code that could process audio input and extract a specific frequency's volume data. I wanted the program to be able to hone in on one specific frequency, which in the case of search and rescue, could be a whistling noise or the human voice. This was achieved by adding a band-pass filter for frequencies from 

Finally, I merged the two together and performed several test trials at different locations to see how accurate my result was.

3. What is new or novel about your project?

       a. Is there some aspect of your project's objective, or how you achieved it that you haven't done before?

During the project, I had to calculate a mathematical function that could receive an input of left/right microphone ratios and output the predicted angle or origin. While I could use a Wolfram Mathematica command that could do this automatically, it still needed the general parent function to calculate the coefficients. This involved a lot of concepts I had learned in precalculus class, which was especially interesting. I was able to apply my knowledge of exponential functions as well as logarithmic functions to assist the computer in finding the optimal ratio-angle model. What really stood out to me was the experience of using complex math concepts in real world applications.

       b. Is your project's objective, or the way you implemented it, different from anything you have seen?

While sound source localization (SSL) systems already exist for drones, the method by which direction is determined has not been based on the ratio of volumes between the left and right channels of a stereo microphone. I wanted to see if this technique would prove to work faster and more accurately, especially as disaster relief scenarios rely heavily on rapid response times.

       c. If you believe your work to be unique in some way, what research have you done to confirm that it is?

I researched the current methods of SSL and found that most systems use either machine learning models or an analysis of the arrival times between two separate microphones. I had found no mention of the usage of pre-mapped ratio/angle pairs in SSL. 

4. What was the most challenging part of completing your project?

      a. What problems did you encounter, and how did you overcome them?

When gathering and analyzing the data that would ultimately result in the creation of the angle-ratio chart, which shouldered the weight of the entire project, I ran into difficulties that could threaten my design's accuracy. More specifically, I had been finding "bumps" in the ratio/angle graphs, meaning that one ratio had been measured at two different locations. This was problematic because if I were to use that data as a reference, the inputted ratio would return multiple outputs, which would make the system unreliable (every input within the domain of the function needs to have one corresponding output). 

In order to devise a solution, I considered the potential causes and ultimately decided to look into two. I deduced that the irregularities were arising from either defective microphones or an inadvertent introduction of additional variables, as the sound was traveling across angles by hand on a phone. To test the first hypothesis, I conducted a trial where one microphone lay on top of the other, but this time was upside-down (the left and right facing microphones were switched). If the graphs of volume ratio to angle resulted in mirror images of each other, it would show that the microphones were not defective, as they recorded audio in similar ways. This proved to be the case, and I performed another recording trial where I took extra care to lay the phone flat so that sounds would not be bouncing around and disrupting the recording.

      b. What did you learn from overcoming these problems?

The process highlighted above allowed me to take away substantial lessons on the scientific and engineering process. I had taken time to consider the root causes of the problem and then proceeded to design a solution to see if they were indeed the case. I feel that, in addition to experiencing the process of devising solutions, I have learned how to approach any problem with clear and analytical considerations to work out a solution, which can help me in any field later in life.

5. If you were going to do this project again, are there any things you would you do differently the next time?

If I were to redo the project, I would start with reducing the number of unnecessary variables. The bulk of the time spent on the project revolved around the measurement and analysis of repeated preliminary recording trials. Although I eventually diagnosed the problem, it may have been solved quicker if I had been more careful in reducing variables like my body blocking sound waves, sounds bouncing around a room, or the direction of which the speaker was facing.

6. Did working on this project give you any ideas for other projects? 

The current project gave me some ideas to further develop the design, including a 3d-printed cone to block out drone propeller noise and a machine learning algorithm that decides whether a given input of sound belongs to a human and whether action is necessary.

Staying on the topic of rescue UAVs, a machine learning program could be designed that calculates the fastest route towards a trapped survivor in harsh conditions or a system that maps out all survivors in a given array and computes the most efficient way to extract them. 

7. How did COVID-19 affect the completion of your project?

All work was completed at home, so the COVID-19 pandemic did not negatively affect the project. On the contrary, an online winter term of school allowed me to spend more time at home working on the project.