Modelling heterogeneity in the classification process in multi-species distribution models can improve predictive performance
Data files
Mar 05, 2024 version files 707.53 KB
Abstract
Species distribution models and maps from large-scale biodiversity data are necessary for conservation management. One current issue is that biodiversity data are prone to taxonomic misclassifications. Methods to account for these misclassifications in multispecies distribution models have assumed that the classification probabilities are constant throughout the study. In reality, classification probabilities are likely to vary with several covariates. Failure to account for such heterogeneity can lead to biased prediction of species distributions.
Here, we present a general multispecies distribution model that accounts for heterogeneity in the classification process. The proposed model assumes a multinomial generalized linear model for the classification confusion matrix. We compare the performance of the heterogeneous classification model to that of the homogeneous classification model by assessing how well they estimate the parameters in the model and their predictive performance on hold-out samples. We applied the model to gull data from Norway, Denmark, and Finland, obtained from the Global Biodiversity Information Facility.
Our simulation study showed that accounting for heterogeneity in the classification process increased the precision of true species' identity predictions by 30 % and accuracy and recall by 6%. Since all the models in this study accounted for misclassification of some sort, there was no significant effect of accounting for heterogeneity in the classification process on the inference about the ecological process. Applying the model framework to the gull dataset did not improve the predictive performance between the homogeneous and heterogeneous models (with parametric distributions) due to the smaller misclassified sample sizes. However, when machine learning predictive scores were used as weights to inform the species distribution models about the classification process, the precision increased by 70%.
We recommend multiple multinomial regression to be used to model the variation in the classification process when the data contains relatively larger misclassified samples. Machine prediction scores should be used when the data contains relatively smaller misclassified samples.
README: Modelling heterogeneity in the classification process in multi-species distribution models can improve predictive performance
The uploads consist of R-scripts that are used for the simulation study and the analysis of the gulls dataset.
A. Simulation study:
i. simulationData.R
This script is used to simulate the data needed for the analysis.
ii. nimbleSimulatedData.R
This script contains the function that is used to analyse the simulated dataset from the simulationData.R script.
iii. constant.R, fixedIntercov.R, intercept.R, main.R, onlyCov.R, variable.R
These scripts are used to run a specific model specification described in the main paper. It calls the function defined in simulationData.R and the dataset followed from simulationData.R.
iv. plotSimulations.R
The scripts are used to summarise the results and generate the plots and tables presented in the main paper and the supplementary information.
B. Case study : Gull dataset
i. gull_data_formatting.R
This script prepares the data downloaded from GBIF to be used for the analysis.
ii. observations.csv
The images of the downloaded data from gbif are used to classify the species into their actual species identity. The machine leaarning algorithm returns prediction scores, which will be used to account for heterogeity in the classification process of the species distribution model. These Machine learning results are saves in the observations.csv file and has the columns:
- gbifID: the unique identifier for the occurrence record
- occurrence ID_: the link to the particular occurrence record
- scientific name: the scientific name of the occurence record
- Prediction 1: The species with the highest prediction score
- Prediction 1 score: The $F_1$ score (prediction score) associated with the species in prediction 1.
- Prediction 2: The species with the second highest prediction score
- Prediction 2 score: The $F_1$ score (prediction score) associated with the species in prediction 2.
- Prediction 3: The species with the highest prediction score
- Prediction 3 score: The $F_1$ score (prediction score) associated with the species in prediction 3.
- Prediction 4: The species with the highest prediction score
- Prediction 4 score: The $F_1$ score (prediction score) associated with the species in prediction 4.
- Prediction 5: The species with the lowest prediction score
- Prediction 5 score: The $F_1$ score (prediction score) associated with the species in prediction 5
iii. fullMLFormatting.R
This script formats the machine learning predictive scores (saved in observations.csv) for use in the data analysis.
iv. nimble.R
This script contains a function that is used to fit the species distribution models for various configurations.
v. constantData.R, fixedIntercovData.R, interceptData.R, mainData.R, onlyCovData.R, variableData.R, ML_estimatesData.R
These scripts are used to run a specific model specification described in the main paper. It calls the function defined in nimble.R and the formatted dataset from gull_data_formatting.R.
vi. plot_for_data.R
This script is used to plot and summarise the results from the application of model to gull dataset.
Methods
This study performed simulation study to test the hypothesis and applied the multi-species distribution model to gulls dataset that was accessed from the Global Biodiversity Infrastructure Facility (with DOI: https://doi.org/10.15468/dl.h24bp5). Attached here are the R-scripts required to reproduce the simulation study and the analysis of the case study (the gull dataset would have to be downloaded first). The details of the scripts are provided in README file.