Laurence A. Clarfeld Katherina D. Gieder Robert Abrams Christopher Bernier Joseph Cahill Susan Staats Scott Wixsom Therese M. Donovan 20250424 spreadsheet Ecological Informatics Elsevier https://doi.org/10.1016/j.ecoinf.xxxxxxxxxx https://doi.org/10.5066/P13EFLXX Autonomous recording units are increasingly being used to monitor wildlife on large geographic and temporal scales, paired with machine learning (ML) to automate detection of wildlife. However, false positive detections from ML classifiers can result in erroneous ecological models that can lead to misguided management and conservation actions. We used a two-stage general approach to understand and reduce false positive detections, a technique in which outputs of the primary classification model are passed to a secondary classification model to yield the probability that a detection from the primary model is a true positive detection. This approach is demonstrated on two open-source models, BirdNET and the Drumming Model, that detect Ruffed Grouse (Bonasa umbellus). We analyzed over 9500 hours of acoustic data collected in 2022-2023 from the Green Mountain National Forest in Vermont, USA, and found the two models detected different types of acoustic signals associated with differing life history traits. The Drumming Model yielded 4106 detections (71.5% true positives); BirdNET yielded 524 detections (17.0% true positives). Secondary logistic regression models separated true positives and false positives with high accuracy (BirdNET = 84.5%; Drumming Model = 89.8%). Our findings go beyond improving Ruffed Grouse monitoring and conservation efforts to, more broadly, illustrate how two-stage ML approaches can improve the use of model-derived detections in wildlife research. Autonomous recording units are increasingly being used to monitor wildlife on large geographic and temporal scales, paired with machine learning (ML) to automate detection of wildlife. However, false positive detections from ML classifiers can result in erroneous ecological models that can lead to misguided management and conservation actions. We used a two-stage general approach to understand and reduce false positive detections, a technique in which outputs of the primary classification model are passed to a secondary classification model to yield the probability that a detection from the primary model is a true positive detection. This approach is demonstrated on two open-source models, BirdNET and the Drumming Model, that detect Ruffed Grouse (Bonasa umbellus). We analyzed over 9500 hours of acoustic data collected in 2022-2023 from the Green Mountain National Forest in Vermont, USA, and found the two models detected different types of acoustic signals associated with differing life history traits. The Drumming Model yielded 4106 detections (71.5% true positives); BirdNET yielded 524 detections (17.0% true positives). Secondary logistic regression models separated true positives and false positives with high accuracy (BirdNET = 84.5%; Drumming Model = 89.8%). Our findings go beyond improving Ruffed Grouse monitoring and conservation efforts to, more broadly, illustrate how two-stage ML approaches can improve the use of model-derived detections in wildlife research. 20220523 20231128 ground condition Complete None planned Green Mountain National Forest -73.2300 -72.7400 44.1600 42.7300 ISO 19115 Topic Category biota None acoustic monitoring artificial intelligence bioacoustics Machine Learning wildlife monitoring USGS Metadata Identifier USGS:679392d5d34e88f5864c50b5 None Green Mountain National Forest Vermont None Ruffed Grouse Kingdom Animalia Subkingdom Bilateria Infrakingdom Deuterostomia Phylum Chordata Subphylum Vertebrata Infraphylum Gnathostomata Superclass Tetrapoda Class Aves Order Galliformes Family Phasianidae Genus Bonasa Species Bonasa umbellus TSN: 175790 None. Please see 'Distribution Info' for details. None. Users are advised to read the dataset's metadata thoroughly to understand appropriate use and data limitations. Laurence Clarfeld Vermont Cooperative Fish and Wildlife Research Unit Post-doctoral Fellow mailing and physical

University of Vermont

Burlington VT 05405 USA 802-656-3011 lclarfel@uvm.edu Data are accurate. Data have been screened for consistency. Dataset is complete. Acoustic Data Collection: Recordings were collected in Green Mountain National Forest, Vermont, USA at 50 sites between May 23 and June 16, 2022, and at 60 sites between April 4 and May 25, 2023. Fifty ARU’s were programed to record one minute out of every ten minutes between 05:00 and 09:00 and one minute of every thirty from 16:30 to 23:00 each day. The ten additional units deployed in 2023 recorded continuously between 05:30 and 08:00 daily. All sites used Wildlife Acoustics Song Meter Micro recording units. 2024 Automated Detection of Ruffed Grouse: We analyzed all recordings with BirdNET (V 2.4), parameterizing the model with a minimum confidence score of 0.1 (a default setting) and providing a species list containing 171 species likely to occur in the study area (Appendix A). Due to the later deployment in the 2022 season that missed most of the peak drumming period, only recordings collected in 2023 were analyzed with the Drumming Model, using its default settings (Appendix B). Both models were run in a cloud computing environment (Amazon Web Services Elastic Compute Cloud) powered by 3rd generation AMD EPYC processors. 2024 Each detection was independently validated by a human observer as either correct (true positive) or incorrect (false positive) (Figure 3a). Verifications were based primarily on visual inspection of spectrograms as well as listening. We further identified the type of vocalization captured, as defined in the Peterson Guide to Bird Sounds (bark, cluck, drum, purr, seer, or wail) (Pieplow, 2017). Pieplow, N., 2017. Peterson Field Guide to Bird Sounds of Eastern North America. Houghton Mifflin Harcourt, Boston. 2024 birdnet_data.csv Comma Separated Value (CSV) file containing data. Producer Defined pk_modeloutputid Unique ID for each BirdNET detection Producer Defined 63745 3924536 start_date Date of recording Producer Defined Range from 2022-05-26 to 2023-09-26 start_time Time of recording Producer Defined Range from 05:00:00 to 22:30:00 is_valid Whether a detection was a true positive (1) or false positive (0) detection Producer Defined 0 False positive detection Producer defined 1 True positive detection Producer defined off_schedule Whether a recording was made with the standardized schedule (1) or not (0), to determine whether it was included in effort-based temporal analyses. Producer Defined 0 Off-schedule recording Producer defined 1 On-schedule recording Producer defined value_num The BirdNET confidence score of the detection. Producer Defined 0.1 0.9937 drumming_model_data.csv Comma Separated Value (CSV) file containing data. Producer Defined pk_modeloutputid Unique ID for each Drumming Model detection Producer Defined 3983249 3987354 start_date Date of recording Producer Defined Range from 2023-04-04 to 2023-08-31 start_time Time of recording Producer Defined Range from 05:00:00 to 22:30:00 is_valid Whether a detection was a true positive (1) or false positive (0) detection Producer Defined 0 False positive detection Producer defined 1 True positive detection Producer defined off_schedule Whether a recording was made with the standardized schedule (1) or not (0), to determine whether it was included in effort-based temporal analyses. Producer Defined 1 On-schedule recording Producer defined 0 Off-schedule recording Producer defined dur Duration of the detected drumming sequence Producer Defined 1.05004375182299 6.89778740780866 seconds y-len Length of the drumming sequence Producer Defined 9 32 Number of detected "thumps" y-min Minimum forward time difference between "thumps" Producer Defined 0.0375015625651045 0.265011042126751 seconds y-max Maximum forward time difference between "thumps" Producer Defined 0.0500020834201393 0.742530938789116 seconds cor_p_log log of the correlation coefficient of the y sequence Producer Defined -37.6339710609729 0.0 y-01 Forward time difference of the 1st pair of "thumps" Producer Defined 0.0375015625651045 0.742530938789116 seconds y-02 Forward time difference of the 2nd pair of "thumps" Producer Defined 0.0375015625651045 0.655027292803867 seconds y-03 Forward time difference of the 3rd pair of "thumps" Producer Defined 0.0375015625651045 0.647526980290841 seconds y-04 Forward time difference of the 4th pair of "thumps" Producer Defined 0.0375015625651045 0.637526563606819 seconds y-05 Forward time difference of the 5th pair of "thumps" Producer Defined 0.0375015625651045 0.650027084461854 seconds y-06 Forward time difference of the 6th pair of "thumps" Producer Defined 0.0375015625651045 0.692528855368973 seconds y-07 Forward time difference of the 7th pair of "thumps" Producer Defined 0.0375015625651045 0.682528438684946 seconds y-08 Forward time difference of the 8th pair of "thumps" Producer Defined 0.0375015625651045 0.675028126171924 Forward time difference of the 8th pair of "thumps" y-09 Forward time difference of the 9th pair of "thumps" Producer Defined 0.0375015625651045 0.647526980290841 seconds y-10 Forward time difference of the 10th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.690028751197966 seconds y-11 Forward time difference of the 11th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.647526980290841 seconds y-12 Forward time difference of the 12th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.640026667777824 seconds y-13 Forward time difference of the 13th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.657527396974874 seconds y-14 Forward time difference of the 14th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.640026667777824 seconds y-15 Forward time difference of the 15th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.630026251093796 seconds y-16 Forward time difference of the 16th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.637526563606819 seconds y-17 Forward time difference of the 17th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.640026667777824 seconds y-18 Forward time difference of the 18th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.637526563606817 seconds y-19 Forward time difference of the 19th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.637526563606819 seconds y-20 Forward time difference of the 20th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651081 0.640026667777824 seconds y-21 Forward time difference of the 21st pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.637526563606819 seconds y-22 Forward time difference of the 22nd pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651063 0.627526146922788 seconds y-23 Forward time difference of the 23rd pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.640026667777824 seconds y-24 Forward time difference of the 24th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.637526563606817 seconds y-25 Forward time difference of the 25th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.622525938580774 seconds y-26 Forward time difference of the 26th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651081 0.637526563606817 seconds y-27 Forward time difference of the 27th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.622525938580774 seconds y-28 Forward time difference of the 28th pair of "thumps" Producer Defined NA No Data Producer defined 0.0375015625651045 0.637526563606817 seconds y-29 Forward time difference of the 29th pair of "thumps" Producer Defined NA No Data Producer defined 0.0425017709071218 0.622525938580774 seconds y-30 Forward time difference of the 30th pair of "thumps" Producer Defined NA No Data Producer defined 0.0425017709071227 0.52002166756948 seconds y-31 Forward time difference of the 31st pair of "thumps" Producer Defined NA No Data Producer defined 0.0450018750781283 0.130005416892374 seconds y-32 Forward time difference of the 32nd pair of "thumps" Producer Defined NA No Data Producer defined 0.0475019792491338 0.0500020834201429 seconds birdnet_vocalization_types_data.csv Comma Separated Value (CSV) file containing data. Producer Defined modeloutput_id Unique ID for each BirdNET detection Producer Defined 150041 3508708 vocalization_type Ruffed Grouse vocalization type Producer Defined Cluck Vocalization of type "cluck" Producer defined Bark Vocalization of type "bark" Producer defined Wail/Bark Vocalization containing both types "bark" and "wail" Producer defined Wail Vocalization of type "wail" Producer defined Purr Vocalization of type "purr" Producer defined Drum Vocalization of type "drum" Producer defined Cluck/Bark Vocalization containing both types "cluck" and "bark" Producer defined Seer Vocalization of type "seer" Producer defined Purr/Bark Vocalization containing both types "purr" and "bark" Producer defined effort_data_date.csv Comma Separated Value (CSV) file containing data. Producer Defined model Model name Producer Defined birdnet The BirdNET model Producer defined drumming model The Drumming Model Producer defined start_date Date (without year) of recording Producer Defined April 3 to August 31 total_minutes Total number of minutes of recordings (using standardized recording schedule) for each date Producer Defined 1 3192 minutes effort_data_time.csv Comma Separated Value (CSV) file containing data. Producer Defined model Model name Producer Defined birdnet The BirdNET model Producer defined drumming model The Drumming Model Producer defined start_time Time (hour) of recording Producer Defined 05:00 05:00 Producer defined 06:00 06:00 Producer defined 07:00 07:00 Producer defined 08:00 08:00 Producer defined 16:00 16:00 Producer defined 17:00 17:00 Producer defined 18:00 18:00 Producer defined 19:00 19:00 Producer defined 20:00 20:00 Producer defined 21:00 21:00 Producer defined 22:00 22:00 Producer defined total_minutes Total number of minutes of recordings (using standardized recording schedule) for each hour Producer Defined 5327 56203 minutes U.S. Geological Survey - ScienceBase mailing address

Denver Federal Center

Building 810

Mail Stop 302

Denver CO 80225 United States 1-888-275-8747 sciencebase@usgs.gov Unless otherwise stated, all data, metadata and related materials are considered to satisfy the quality standards relative to the purpose for which the data were collected. Although these data and associated metadata have been reviewed for accuracy and completeness and approved for release by the U.S. Geological Survey (USGS), no warranty expressed or implied is made regarding the display or utility of the data for other purposes, nor on all computer systems, nor shall the act of distribution constitute any such warranty. Digital Data https://doi.org/10.5066/P13EFLXX None 20250430 Laurence Clarfeld Vermont Cooperative Fish and Wildlife Research Unit Post-doctoral Fellow mailing and physical

University of Vermont

Burlington VT 05405 USA 802-656-3011 lclarfel@uvm.edu FGDC Biological Data Profile of the Content Standard for Digital Geospatial Metadata FGDC-STD-001.1-1999