README: From buzzes to bytes: A systematic review of automated bioacoustics models used to detect, classify, and monitor insects

https://doi.org/10.5061/dryad.hmgqnk9r1

The following files correspond to the results from a systematic review of automated bioacoustics models used to detect, classify, and monitor insects.

Description of the data and file structure

This submission contains five CSV files required for analyses and one R script:

1) data_extraction_metadata, which provides metadata information for each study evaluated, such as title, author, journal, and language.

Details:

duplication_flagged (whether this was likely a duplicate article already present in the dataset),

article_ID (individual article), 

inclusion (included yes/no data from this article),

exclusion_criteria (if excluded, what was the reason why),

exclusion_insect_clumped (if excluded because the aggregate of insects were clumped into a single category that was not relevant for the article),

concern_type (ACI_clump, ambiguous_results, data_reporting_descrepancy, methods_lack_detail, NA, no_error_reported, other, small_sample_size, some_data_unreported, unclear_automation, unclear_model_usage),

concerns_notes (additional notes regarding potential concerns about article),

notes (other types of notes),

title (article title),

author (article authors),

journal (journal where article published),

volume (volume in which article published),

issue (issue in which article published),

pages (pages in which article is published),

publish_year (year in which article published),

article_language (language in which article written),

tropics (whether article was conducted in the tropics),

article_country (country in which article conducted/published),

multiple_locations (whether article included multiple locations),

DOI (DOI associated with article),

full_text_link (link to full article).

2) data_extraction-methods, which provides information about the different automated bioacoustics models used in each study as well as other relevant results from the study level, including the resolution obtained from the model (e.g. species), the number of taxa classified, and the accuracy estimates obtained with each model.

Details:

study_ID (individual article)

resolution (taxonomic resolution used in this study, e.g. species, family, order, class)       

number_taxa_classified (number of taxa that the model classified)       

date_recording_data_collection (date of data collection recording) 

ecosystem (ecosystem in which the data collection took place)     

monitoring_type (the focus of the type of monitoring; beneficial_other, hive_monitoring, NA, parasite_monitoring, pest_monitoring_food, pest_monitoring_other, pest_monitoring_wood)

substrate_specific (the substrate on which the recording took place, if relevant; e.g. food, wood, etc.)

uses_raven (whether article uses the Raven software, from Cornell Lab of Ornithology)

conservation_mentioned (whether conservation was mentioned in the article)

community_science (whether article employed community/citizen science)  

using_existing_database (whether article used existing audio database)        

using_mulitple_databases (whether article used multiple audio database)       

database_name (name of audio database, if relevant)        

open_source_database (whether article used an open source audio database) 

using_existing_code (whether article used existing code for the development of their automated models)    

open_source_code (whether article provides open source code for the development of their automated models)    

create_user_interface (whether article creates a user interface)

employ_user_interface (whether article uses a user interface)

data_reporting (the general state of the data reporting, namely ambiguous_partial_or_nodata, application or unambiguous_fulldata).

train_test_ratio_clean (ratio of training to test data in the article; cleaned for analysis)

train_test_ratio_raw (raw ratio of training to test data in the article if provided)

train_test_rounded (ratio of training to test data in the article; cleaned for analysis and rounded to nearest multiple of 5)

paper_type (type of article, namely whether 1) applied, i.e. testing questions using bioacoustics; 2) methods; i.e. development of bioacoustics methods but not applied; or 3) methods_applied, developed bioacoustics methods and applied them) 

automation_type (automation type, namely DL_ML, ML, SSO, or unknown; DL = deep learning; ML = machine learning; SSO = Signals, statistics, and other).

one_taxa_model (whether the model only included one taxa)     

one_taxa_name (what the taxa was if only one taxa evaluated)

ecoacoustics_model (whether an ecoacoustics model)

detection_only_model (whether a detection only model)

model_role (model role, namely behavior, classification, detection or NA)

feature_extraction (feature extraction method)

features_used  (feature used)

ML_classification (type of ML classification used, broad category)

ML_spec (type of ML classification used, specific category)

covariate (covariate if present)

accuracy_percent (model accuracy result)

accuracy_error_type (model accuracy error type)

accuracy_error_value (value of accuracy value)

precision_percent (model precision result)  

precision_error_type (model precision error type)

precision_error_value (precision error value)

recall_percent (model recall result)  

recall_error_type (model recall error type)  

recall_error_value (recall error value)

f1 (f1 = harmonic mean of recall and precision value)

f1_error_type (f1 error type) 

f1_error_value (f1 error value)

other_metric (if any other model performance estimate was used)

other_metric_score (score of additional model)           

other_metric2 (if a second additional model performance estimate was used)

other_metric2_score (score of second additional model)

3) data_extraction-taxa, which provides information on taxa-specific model outputs (for studies that specified output for specific taxa).

Details:

study_ID (individual article)

focal_class (taxonomic class of organism evaluated in model)

focal_order (taxonomic order of organism evaluated in model)

focal_superfamily (taxonomic superfamily of organism evaluated in model)

focal_family (taxonomic family of organism evaluated in model)

focal_genus (taxonomic genus of organism evaluated in model)

focal_species (taxonomic species of organism evaluated in model)

category_common_name (common name of organism evaluated in model)

substrate_specific (substrate on which data collected)

data_reporting (the general state of the data reporting, namely ambiguous_partial_or_nodata, application or unambiguous_fulldata).

automation_type (automation type, namely DL_ML, ML, SSO, or unknown; DL = deep learning; ML = machine learning; SSO = Signals, statistics, and other).

ecoacoustics_model (whether an ecoacoustics model)

detection_only_model (whether a detection only model)

model_role (model role, namely behavior, classification, detection or NA)

feature_extraction (feature extraction method)

features_used  (feature used)

ML_classification (type of ML classification used, broad category)

ML_spec (type of ML classification used, specific category)

covariate (covariate if present)

accuracy_percent (model accuracy result)

accuracy_error_type (model accuracy error type)

accuracy_error_value (value of accuracy value)

precision_percent (model precision result)  

precision_error_type (model precision error type)

precision_error_value (precision error value)

recall_percent (model recall result)  

recall_error_type (model recall error type)  

recall_error_value (recall error value)

f1 (f1 = harmonic mean of recall and precision value)

f1_error_type (f1 error type) 

f1_error_value (f1 error value)

other_metric (if any other model performance estimate was used)

other_metric_score (score of additional model)           

other_metric2 (if a second additional model performance estimate was used)

other_metric2_score (score of second additional model)

4) data_extraction-all-ecosystems, which provides information on the ecosystems studied for each of the studies.

Details:

study ID (individual article).

ecosystem (ecosystem in which the data collection took place).     

5) data_extraction-key, which provides additional details on some of the terms used in the other .csv files.

6) Kohlberg_et_al_JAE_Final.R, R file with code for all analyses and figures presented in the main and supplementary material of this article. The R file has been annotated to provide additional details and explain logic of all decisions made. The R file calls all except the data_extraction-key .csv file.