Data from: Towards a fully automated underwater census for fish assemblages in the Mediterranean Sea
Data files
Dec 17, 2024 version files 121.66 GB
-
autoUC_Data.zip
121.66 GB
-
README.md
7.29 KB
Abstract
Assessing underwater biodiversity is labour-intensive and costly, but is crucial for measuring the extent of the decline in local fish stock. In most cases, Underwater Visual Census (UVC) is the preferred method, however this can be costly in terms of human effort and is limited by meteorological and logistical factors. Advances in technology allows the utilisation of more autonomous video recording methods (i.e. Remote Operated Vehicles (ROV)) which addresses these limitations. This study used a transect-wise UVC coupled with diver operated videos (DOV). For the video analysis, a comprehensive fully automated pipeline was developed to extract frames from DOV and perform colour correction. This pipeline integrates a YOLO-based model to detect 20 Mediterranean fish species and validate the presence or absence of each species within individual transects. This study was conducted to evaluate the feasibility of using video-based methods for UVC with minimal human-input. The result of automated video analysis were in agreement with manual video counting, validating the autonomous and bias-free procedure for video assessment. In conclusion, utilising a minimal-human-input video method liberates the data acquisition from limiting factors (i.e. meteorological and logistical) and automation of this video analysis significantly reduces the labour and time required. For future fieldwork campaigns, the video data collection protocol needs to be modified to better resemble traditional UVC and enhance this acquisition method.
README: Data from: Towards a fully automated underwater census for fish assemblages in the Mediterranean Sea
https://doi.org/10.5061/dryad.f7m0cfz6f
Description of the data and file structure
1. Study area and data collection
The training dataset (DATA_T) was gathered in eight different locations in the Mediterranean Sea along the French Riviera, following the same UVC protocol on each site. The depth ranged from 1-37m and was carried out during the whole year in 2022 (cold and warm season) to cover the full range of conditions and possibilities of fish occurrences.
The experimental dataset (DATA_E) was recorded in October 2023 in and around two protected areas, one no-take zone (Cap Roux) and one Natura2000 site (Corniche Varoise), which both have elevated biodiversity. A total of 64 videos, each corresponding to a transect, from 14 sites (8 on seagrass meadows and 6 on rocky substrates) were evaluated and compared. Each site consists of 3 to 6 transects, depending on the availability of video recordings and UVC data from the divers.
The videos were obtained with GoPro HERO 9 cameras, mounted on the clipboards used by the divers to note the number of fish per species with their respective size category (variable number of categories per species). The videos were recorded with a framerate of 24 frames per second (FPS) and full high definition resolution (1920x1080px). Frames were extracted from these recordings with a framerate of 1 FPS for DATA_T and 5 FPS for DATA_E.
Table 1: Showing the location of where the videos for Data_E were collected as well as site and transect indication.
location | site | transect | videoname |
---|---|---|---|
Cap Roux | 4 | 1 | GH010022 |
Cap Roux | 4 | 2 | GH010023 |
Cap Roux | 4 | 3 | GH010024 |
Cap Roux | 4 | 4 | GH010025 |
Cap Roux | 6 | 1 | GH010026 |
Cap Roux | 6 | 2 | GH010027 |
Cap Roux | 6 | 3 | GH010028 |
Cap Roux | 6 | 4 | GH010029 |
Cap Roux | 6 | 4 | GH010030 |
Cap Roux | 6 | 5 | GH010031 |
Cap Roux | 6 | 6 | GH010032 |
Cap Roux | 4 | 1 | GH010396 |
Cap Roux | 4 | 2 | GH010397 |
Cap Roux | 4 | 3 | GH010398 |
Cap Roux | 4 | 1 | GH010399 |
Cap Roux | 4 | 2 | GH010400 |
Cap Roux | 4 | 3 | GH010401 |
Cap Roux | 6 | 1 | GH010403 |
Cap Roux | 6 | 2 | GH010404 |
Cap Roux | 6 | 3 | GH010405 |
Cap Roux | 1 | 4 | GH010417 |
Cap Roux | 1 | 5 | GH010418 |
Cap Roux | 1 | 6 | GH010419 |
Corniche varoise | 1 | 1 | GH010446 |
Corniche varoise | 1 | 2 | GH010448 |
Corniche varoise | 1 | 3 | GH010449 |
Corniche varoise | 1 | 1 | GH010451 |
Corniche varoise | 1 | 2 | GH010452 |
Corniche varoise | 1 | 3 | GH010453 |
Cap Roux | 1 | 1 | GH010736 |
Cap Roux | 1 | 2 | GH010737 |
Cap Roux | 1 | 3 | GH010739 |
Cap Roux | 5 | 1 | GH010740 |
Cap Roux | 5 | 2 | GH010742 |
Cap Roux | 5 | 3 | GH010743 |
Corniche varoise | 2 | 4 | GH010757 |
Corniche varoise | 2 | 5 | GH010758 |
Corniche varoise | 2 | 1 | GH011368 |
Corniche varoise | 2 | 2 | GH011369 |
Corniche varoise | 2 | 3 | GH011370 |
Corniche varoise | 1 | 4 | GH011371 |
Corniche varoise | 1 | 5 | GH011372 |
Corniche varoise | 1 | 6 | GH011373 |
Cap Roux | 2 | 1 | GX010752 |
Cap Roux | 2 | 2 | GX010753 |
Cap Roux | 2 | 3 | GX010754 |
Cap Roux | 2 | 4 | GX010755 |
Cap Roux | 2 | 5 | GX010756 |
Cap Roux | 3 | 1 | GX010777 |
Cap Roux | 3 | 2 | GX010778 |
Cap Roux | 3 | 3 | GX010779 |
Corniche varoise | 3 | 1 | GX010783 |
Corniche varoise | 3 | 2 | GX010784 |
Corniche varoise | 3 | 3 | GX010785 |
Corniche varoise | 3 | 4 | GX010786 |
Corniche varoise | 3 | 5 | GX010787 |
Corniche varoise | 3 | 6 | GX010788 |
Corniche varoise | 4 | 1 | GX012705 |
Corniche varoise | 4 | 2 | GX012707 |
Corniche varoise | 4 | 3 | GX012709 |
Corniche varoise | 4 | 4 | GX012710 |
Corniche varoise | 4 | 5 | GX012712 |
Corniche varoise | 4 | 6 | GX012714 |
2. Image preprocessing
The frames were processed in a next step by a marine biology expert to guarantee correctly identified species in the videos. To ensure a good species coverage, 19 different species and an ’Other’ class were labeled manually in the frames resulting in 13,033 images (131 videos in the training set and 47 independent videos in the test set) in DATA_T with a total of 68,573 (train = 40,379, test = 28,194) individual fish labels and 8,739 miscellaneous labels such as background and diver. The ’Other’ class includes 21 species hat have insufficient occurrences in the test videos (n < 100). Since there was a wide range of conditions in the videos, a preprocessing was applied to both datasets to enhance each image colour range. For this purpose, a pretrained UIEC2-Net model was utilised to enhance the images.
3. Folder Structure
The zip file contains two folders Data_E and Data_T. As described in our publication, the folder Data_T contains images from the training dataset and the folder Data_E contains images from the experimental dataset.
Data_T contains two folders – ‘testSet’ and ‘trainSet’ both containing three folders. The test set folder was used in the study to test the DL model and the train folder was used to train the DL model. The folder ‘images’ contains the images. The two labels folder contain labels in the YOLO format (class_id center_x center_y width height). The folder ‘labels_allSpecies’ contains the labels of 40 species labeled with some species being underrepresented. The other folder ‘labels_Reduced’ combines the 21 less important classes into one class called ‘Other’.
Data_E contains two folders. The folder ‘images’ contains all the images of the experimental dataset. The folder ‘detections’ contains the DL model predictions from the study with the YOLO format plus the prediction confidence (class_id center_x center_y width height confidence).
There are lose files in the root directory. The file called count_classes.py can be used to extract information on the labels of how many of each species are in the datasets. The two files ‘labels_Reduced.txt’ and ‘labels_allSpecies.txt’ give the names of the different classes Data_T was trained on.
Methods
1. Study area and data collection
The training dataset (DATAT ) was gathered in eight different locations in the Mediterranean Sea along the French Riviera, following the same UVC protocol on each site (Harmelin-Vivien et al., 1985). The depth ranged from 1-37m and was carried out during the whole year in 2022 (cold and warm season) to cover the full range of conditions and possibilities of fish occurrences.
The experimental dataset (DATAE) was recorded in October 2023 in and around two protected areas, one no-take zone (Cap Roux) and one Natura2000 site (Corniche Varoise), which both have elevated biodiversity. The specific coordinates and meta data can be found in the supplementary material (Table S1). A total of 64 videos, each corresponding to a transect, from 14 sites (8 on seagrass meadows and 6 on rocky substrates) were evaluated and compared. Each site consists of 3 to 6 transects, depending on the availability of video recordings and UVC data from the divers.
The videos were obtained with GoPro HERO 9 cameras, mounted on the clipboards (Fig. 1) used by the divers to note the number of fish per species with their respective size category (variable number of categories per species). The videos were recorded with a framerate of 24 frames per second (FPS) and full high definition resolution (1920x1080px). Frames were extracted from these recordings with a framerate of 1 FPS for DATAT and 5 FPS for DATAE. Fish visible for less than 1 second (less than 5 frames) in the videos of DATAE will not be considered in the methodology evaluation as they were unlikely to be actual detections.
2. Image preprocessing
The frames were processed in a next step by a marine biology expert to guarantee correctly identified species in the videos. To ensure a good species coverage, 19 different species and an ’Other’ class were labelled manually in the frames resulting in 13,033 images (131 videos in the training set and 47 independent videos in the test set) in DATAT with a total of 68,573 (train = 40,379, test = 28,194) individual fish labels (species breakdown in Table S3) and 8,739 miscellaneous labels such as background and diver. The ’Other’ class includes species (Table S2) that have insufficient occurrences in the test videos (n < 100). Since there was a wide range of conditions in the videos, a preprocessing was applied to both datasets to enhance each image colour range. For this purpose, a pretrained UIEC2-Net model (Y. Wang et al., 2021) was utilised to enhance the images. As the last preprocessing step, images were rescaled to 960x960px.