Can Scientists Train Machines to Listen for Marine Ecosystem Health?
By Leila Hatch & Rachel Plunkett
September 2021
What if we could detect a problem within a marine ecosystem just like a doctor can detect a heart murmur using a stethoscope? Listening to the heart and hearing the murmur tells the doctor there may be a more serious underlying condition that should be addressed before it gets worse. In an ocean world where things like climate change and overfishing have the ability to drastically alter the functionality of entire ecosystems, having a stethoscope to detect signs of major issues could really come in handy to marine resource managers. That’s where sound monitoring, artificial intelligence, and machine learning come in.
Since 2018, NOAA and the U.S. Navy have engaged in a multi-year effort to monitor underwater sound within the National Marine Sanctuary System. The agencies worked with numerous scientific partners to study sound within seven national marine sanctuaries and one marine national monument on the U.S. east and west coasts and in the Pacific Islands region. As the first coordinated monitoring effort of its kind for the National Marine Sanctuary System, SanctSound was designed to provide standardized acoustic data collection to document how much sound is present within these protected areas as well as potential impacts of unnatural noises to the areas’ marine taxa and habitats.
SanctSound project scientists published a paper in Frontiers in Marine Science that showcases what they’ve learned so far, and how that relates to the information needed for managing national marine sanctuaries. They show that some of the basic techniques for measuring and comparing sound characteristics recorded in different geographic locations are complicated to interpret, especially at the scale of this project.
The analysis incorporated efforts of the project’s many collaborators to identify sound signals of interest from large collections of data (datasets) at each sanctuary location. Then, researchers described the unique combination of sounds at each site that make up the soundscape, and made comparisons across sanctuaries. The methods used in this study laid the groundwork to be able to more robustly test emerging automated approaches in the future, including artificial intelligence and machine learning, to characterize soundscapes, their dominant features, and changing conditions. A review of these methods highlights how this expanding field of science will advance the application of soundscape research in national marine sanctuaries and beyond.
Using Automated Methods to Characterize Soundscapes
This project will gather over 300 terabytes (TB) of data over its three years of listening. For context, 1 TB gives you the option of storing roughly: 250,000 photos taken with a 12 megapixel camera, 250 movies, or 500 hours of HD video. Three hundred TB is a lot of data to store and process in a way that gives meaningful information about the health of the underwater environments that people care about. When working with massive datasets such as this, using automated approaches will be an important advance in producing faster and more insightful soundscape interpretations across vast networks of underwater sound recorders (hydrophones).
The application of artificial intelligence, including machine learning, accelerates the analysis of large volumes of data, while greatly reducing cost. Put simply, artificial intelligence is when machines carry out tasks in a way that humans would consider “smart.” Machine learning is one particular application of artificial intelligence that involves giving machines access to large sets of data and either training them to learn (supervised), or letting them learn on their own (unsupervised). In the context of SanctSound, the broader idea is that it may be possible to teach machines to know which sounds are part of the usual chorus of a particular marine ecosystem, and which ones are anomalies (outside of the norm). Detecting these anomalies with this smart “stethoscope” can alert marine resource managers that there may be a problem within an ecosystem if something isn’t done about it soon.
“This project is a great example of the power of artificial intelligence to transform our understanding of the marine environment,” said Gregory Dusek, senior scientist with the National Ocean Service and chair of the NOAA AI Executive Committee. “The amount of acoustic data collected is so vast, that without artificial intelligence, a thorough manual analysis would be nearly impossible.”
“SanctSound points us towards the types of advancements in quality and timeliness afforded by artificial intelligence approaches,” said Jordan Watson, acting deputy director for NOAA’s Center for Artificial Intelligence. “Furthermore, it serves as an example of the key role that cooperative partnerships play in optimizing resources, effort, and impact."
What Have We Learned So Far?
The team of researchers collaborating on this analysis used standardized tools to summarize sound levels across recording locations. They also used automated tools to identify sound features of interest that were shared across datasets, including invertebrate snapping sounds common to warmer water reefs, whale and dolphin calls made by a wide-range of species, the sounds made by wind, waves, rain, and storms, and sounds introduced by vessels and other human activities. More hands-on efforts further identified sounds unique to individual sanctuaries, including local fish populations. All of these sound-based descriptions were then pooled together to describe the dominant drivers of the underwater soundscape in each place, and, based on those core characteristics, which places across the sanctuary system were more similar vs. more different from each other.
“This analysis showed that initial methods for comparing soundscapes don’t always tell the whole story. For example, places can have higher levels of sound (loudness) at similar frequencies (tones), but for very different reasons,” said Megan McKenna, an acoustic biologist at Stanford University and the paper’s lead author. “Understanding these differences is key to advancing soundscape analysis for marine sanctuaries and beyond.”
Listen below to sounds recorded at Olympic Coast, Stellwagen Bank, and Gray’s Reef national marine sanctuaries. These sites all have higher levels of sound when summarized over the same frequencies and time periods of recording. However, the broader suite of measurements made from these data sets show that sound levels in the mouth of Puget Sound in Olympic Coast National Marine Sanctuary off the coast of Washington state are driven by tidal flow across the hydrophone at that location, while levels at Stellwagen Bank National Marine Sanctuary off the coast of Massachusetts are driven by vessels coming and going from the Port of Boston, and levels at Gray’s Reef National Marine Sanctuary off the coast of Georgia are driven by marine animals, including chorusing fish and snapping shrimp.
So far, SanctSound has completed a deep dive into the patterns of sound at each recording location. The broader goal of this comprehensive analysis, however, is to advance future methods that are more sustainable. The paper explains that using these higher resolution results as a training and ground-truthing dataset, we can now ask whether machine learning and other artificial intelligence approaches could return signals of interest more efficiently. Can computers search through years of data from widely dispersed recording locations and characterize the same dominant features identified by human researchers? Where and when the project’s work identified shifts in these dominant features or the occurrence of rare events, can these new approaches find the same changes and anomalies? Where there are gaps in using a particular artificial intelligence approach, can we develop a unique combination of approaches that can help us answer a suite of research questions?
By making the data from this project publicly available, we are hoping to leverage the community of big data specialists to continue to apply new techniques to underwater recordings in order to increase the utility of sound as a marine resource management tool for sanctuaries and beyond. Such proliferations of public, artificial intelligence-ready data sets are critical to NOAA and broader efforts to maximize innovations and efficiencies from artificial intelligence.
Leila Hatch is a marine ecologist at Stellwagen Bank National Marine Sanctuary and co-lead of the SanctSound project
Rachel Plunkett is the writer/editor for NOAA’s Office of National Marine Sanctuaries