1. Convolutional neural networks (CNNs) and deep learning are powerful and robust tools for ecological applications, and are particularly suited for image data. Image segmentation (the classification of all pixels in images) is one such application and can for example be used to assess forest structural metrics. While CNN-based image segmentation methods for such applications have been suggested, widespread adoption in ecological research has been slow, likely due to technical difficulties in implementation of CNNs and lack of toolboxes for ecologists.

2. Here, we present R package imageseg which implements a CNN-based workflow for general-purpose image segmentation using the U-Net and U-Net++ architectures in R. The workflow covers data (pre)processing, model training, and predictions. We illustrate the utility of the package with image recognition models for two forest structural metrics: tree canopy density and understory vegetation density. We trained the models using large and diverse training data sets from a variety of forest types and biomes, consisting of 2877 canopy images (both canopy cover and hemispherical canopy closure photographs) and 1285 understory vegetation images.

3. Overall segmentation accuracy of the models was high with a Dice score of 0.91 for the canopy model and 0.89 for the understory vegetation model (assessed with 821 and 367 images, respectively). The image segmentation models performed significantly better than commonly used thresholding methods, and generalized well to data from study areas not included in training. This indicates robustness to variation in input images and good generalization strength across forest types and biomes.

4. The package and its workflow allow simple yet powerful assessments of forest structural metrics using pre-trained models. Furthermore, the package facilitates custom image segmentation with single or multiple classes and based on color or grayscale images, e.g. for applications in cell biology or for medical images. Our package is free, open source, and available from CRAN. It will enable easier and faster implementation of deep learning-based image segmentation within R for ecological applications and beyond.

This data set contains training data, R scripts and pre-trained models for two image segmentation models for forest structural metrics: canopy density and understory vegetation density. Models were implemented in R via Keras using a TensorFlow backend.

imageseg_canopy_model.zip

Contains model file (.hdf5) and "examples" folder for canopy density image segmentation model. In the examples, left column is model input, central column is raw model output, right column is binarized output.

imageseg_canopy_training_data.zip

Training data used for canopy density model (masks and images, 256x256 pixels), split into three folders. See the info.txt file for details.

imageseg_canopy_training_run.R

R script used for canopy model training.

imageseg_understory_model.zip

Contains model file (.hdf5) and "examples" folder for understory vegetation density image segmentation model. In the examples, left column is model input, central column is raw model output, right column is binarized output.

imageseg_understory_training_data.zip

Training data used for understory vegetation density model (masks and images, 160x256 pixels).

imageseg_understory_training_run.R

R script used for understory vegetation model training.