3D photogrammetry and deep-learning deliver accurate estimates of epibenthic biomass
Cite this dataset
Marlow, Joseph; Halpin, John; Wilding, Thomas (2024). 3D photogrammetry and deep-learning deliver accurate estimates of epibenthic biomass [Dataset]. Dryad. https://doi.org/10.5061/dryad.1rn8pk11z
Abstract
Accurate biomass estimates are key to understanding a wide variety of ecological functions. In marine systems, epibenthic biomass estimates have traditionally relied on either destructive/extractive methods which are limited to horizontal soft-sediment environments, or simplistic geometry-based biomass conversions which are unsuitable for more complex morphologies. Consequently, there is a requirement for non-destructive, higher-accuracy methods that can be used in an array of environments, targeting more morphologically diverse taxa, and at ecological relevant scales. We used a combination of 3D photogrammetry, convolutional-neural-network (CNN) automated taxonomic identification, and taxa-specific biovolume:biomass calibrations to test the viability of estimating biomass of three species of morphologically-complex epibenthic taxa from in situ stereo 2D source imagery. Our trained CNN produced accurate and reliable annotations of our target taxa across a wide range of conditions. When incorporated into photogrammetric 3D models of underwater surveys, we were able to automatically isolate our three target taxa from their environment, producing biovolume measurements that had respective mean similarities of 99, 102, and 120% of those obtained from human annotators. When combined with taxa-specific biovolume:biomass calibrations values, we produced biomass estimates of 88, 125, and 133% mean similarity to that of the “true” biomass of the respective taxa. Our methodology provides a highly reliable and efficient method for estimating the epibenthic biomass of morphologically complex taxa using non-destructive 2D imagery. This approach can be applied to a variety of environments and photo/video survey approaches (e.g. SCUBA, ROV, AUV) and is especially valuable in spatially extensive surveys where manual approaches are prohibitively time-consuming.
README: 3D photogrammetry and deep-learning deliver accurate estimates of epibenthic biomass
https://doi.org/10.5061/dryad.1rn8pk11z
A combination of biomass and biovolume data for NE Atlantic Epibenthic Species and a machine-learning code for automated identification of these species.
Description of the data and file structure
Biomass data is collected in kg or g, dry mass or wet mass. Biovolume is collected using photogrammetric approaches measured in cm^3 or m^3. These data are in Excel format. Data can be used to generate density regressions.
Sharing/Access information
This data will also be made available as part of a submission to the NERC British Oceanographic Data Centre.
Code/Software
All training data/code for the machine learning semantic segmentation is contained in a folder called SemanticSegmentation. The training image set is in JPEG format partitioned into train/validate/test sets, and contained in folders with associated names.
The Python code is contained in a Jupyter Notebook, 'BiomassMethodsSegmentation.ipynb', that contains annotated steps for training and predicting segmentations. Trained network weights have been made available, named 'swin' and 'deeplabv3+' for two different semantic segmentation frameworks. 'swin' and 'deeplabv3+' files are the saved weights in serialised object format for different trained machine learning segmentation models, that can be opened by the open source PyTorch machine learning library, with example script contained in Jupyter Notebook, 'BiomassMethodsSegmentation.ipynb.
Methods
Biomass regressions
Biomass regressions for target taxa were conducted using in-situ and ex-situ photogrammetry to estimate biovolume and subsequent weighing using dry- or wet-weight methods, depending upon the taxa.
Field Biomass Validation
Five underwater transects were conducted using photogrammetric video surveys. On each transect, three 0.5 quadrats were placed and the biovolume of the target taxa was measured from the photogrammetric 3D model. All target taxa within each of the quadrats were subsequently collected and retained for biomass measurements. Validation of field biomass estimates was achieved by comparing the “true” biomass of the target taxa (measured from weighing subsampled quadrats) with that predicted from our biovolume conversions.
Machine-learning 3D annotation validation
To test the accuracy of the automated model annotation, we compared the biovolume from manually annotated meshes with the biovolume from meshes created using machine-learning annotated dense clouds. On each transect, three separate groups of each of the target taxa were selected to be annotated using each method (n = 15 pairs per species). For each of the paired models, the biovolume derived from machine learning was normalised to that of the manual-annotation biovolume.
Machine-learning semantic segmentation
A training/test/validation dataset was constructed from images taken during underwater work and annotated using CVAT annotation software. This was used to train DeepLabV3+ and SegFormer frameworks for semantic segmentation, using a Python script contained in a Juyter notebook. The trained SegFormer framework was used to label images, which were then used to label meshes by species. The trained weights for the framework have been made available.
Funding
Natural Environment Research Council, Award: NE/T010665/1