Dataset DOI: 10.5061/dryad.wh70rxx0z

Description of the data and file structure

This archive contains data presented in our paper, including the masks and imagery used to train our Mask R-CNN sweetpotato instance segmentation model, and data used to validate it (plot-level dataset, one-to-one dataset, and the commercial optical sorter baseline dataset).

Files and variables

File: Dryad_SweetAPPS_Data.zip

Description: Contains the contents of the one-to-one dataset, plot-level dataset, and the sorter baseline data. Decompressing this file will create the following directories:

1. 3D_set: Monte-Carlo simulation data. Contains a Google Colab file, “process_data.ipynb”, that was used to generate the figures and analyses used for the free-space, plane, and roller data. Monte-Carlo datasets that were used are contained in the folders “free_space”, “plane”, and “roller” and are .txt files that are formatted as comma-separated value (CSV) data.
2. Cellphone_Images: One-to-one dataset. Contains folders, numbered 1-30, that contains the images that were used for the one-to-one dataset. Each folder contains two images: one of the roots with the labels face-down, and another with the labels face-up. This is so that each sweetpotato can be associated with its corresponding measurement on the Exeter (commercial) sorter. Additionally, each folder contains a file "pair_dict.xlsx", which contains columns "region", defining the MaskRCNN-based region index for each sweetpotato in the two folders' images (e.g., the file contains columns "region", "IMG_7172", and "IMG_7175" for the folder "10"). Each region has an index that refers back to the Exeter one-to-one dataset entry (see #3 below), and allows each region in each image to be matched (paired) to the Exeter's dataset on a instance-by-instance basis.
3. Exeter: One-to-one dataset. Contains a CSV and excel (xlsx) spreadsheet that has the individual sorter’s measurements for length, width, and weight. Note that length and width are measured in 1/100ths of an inch (e.g., 200 would be 2 inches). A folder “Images” contains the two-view images from the Exeter sorter. The file manual.xlsx contains:
   1. plotname: ID number associated with each individual instance. Note: manual.xlsx contains this number and is used by the script, while manual.csv contains the original image names for traceability back to the original image files.
   2. estimateddiameter: Diameter in 1/100ths of an inch.
   3. Estimatedweight: Weight in 10ths of ounces (e.g., a number 358 would be 35.8 oz);
   4. estimatedlength: Length in 1/100ths of an inch.
4. Mask_RCNN_Calibrated: One-to-one dataset. Contains the Mask R-CNN processed data for the images contained in the folder “Cellphone_Images,” which was used to establish the length, width, and weight estimates for the one-to-one dataset. This folder contains many sub-folders that are image names from the “Cellphone_Images” folders (e.g., IMG_7150). Within this folder are a qualitative image from Mask R-CNN showing the outlines of each detected mask, a zip file containing all the individual binary masks (numbered by region), and a csv file for each mask (e.g., csv_IMG_7150) with columns:
   1. region. A integer denoting which region (mask ID number) that each row of metrics is associated with. This number matches the number in the “masks_IMG_XXXX.zip” file and the numbers in the qualitative mask detection image (e.g., “masked_IMG_XXXX.jpg”).
   2. area (px^2). Area in square pixels.
   3. width (px). Width in pixels.
   4. length (px). Length in pixels.
   5. volume (px^3). Volume in pixels from ellipsoid model.
   6. area (in^2). Area in square inches.
   7. width (in). Width in inches.
   8. length (in). Length in inches.
   9. volume (in^3). Cubic inches estimated from ellipsoid model.
   10. solidity strict_solidity. A computer vision metric.
   11. Enrique_weight (g). This is an estimate provided for the emperical model presented in a different manuscript (10.1016/j.atech.2024.100469)
5. processed_data: Plot level dataset. This folder contains a series of folders containing image names. The image names and data are formatted exactly as (4) above and represent Mask R-CNN outputs for each image taken in for the plot-level data assessment.
6. test: Plot level dataset. This folder contains the full resolution images for the plot level dataset in (5) above.
7. Kylie_Complete.xlsx: This file contains the observed Exeter length, width, and weights as compared to manually measured length, width, and weight for the one-to-one Exeter dataset.
8. 2020Sweetapps roots.csv: Contains plot-level Exeter data containing columns as:
   1. plotname: A unique ID for the plot in the field.
   2. estimateddiameter, estimatedweight, and estimatedlength are defined the same as #3 above.
9. exeter_stats.py: Code to parse the Exeter data's file names.
10. SweetAPPS_MaskRCNN_DataValidation.ipynb and SweetAPPS_MaskRCNN_DataValidation.py: See below, runs the comparison and plotting / analysis code.
11. run_del_datedfolder.py: See below, runs inference.
12. Readme.docx: This information.

File: Image_Datasets.zip

Description: Contains labels and cellphone imagery that was used to train the Mask R-CNN model.

1. SP_dataset: contains two folders:
   1. annotations: labels for individual sweetpotato regions in json format, as produced by the VGG Image Annotator (VIA) tool. Documentation on this format is provided at their github: https://github.com/ox-vgg/via
   2. Images: contains four folders:
      1. all: All data in one place. These are cellphone images that were used to train the Mask R-CNN model and can be associated with the annotations in the annotations folder.
      2. test: 20% testing data that was used for model developing and assessments.
      3. train: 60% training data that was used for model developing and assessments.
      4. validate: 20% validation data that was used for model developing and assessments.

File: model_final.zip

Description: Contains the model coefficients of the trained Mask R-CNN data, for use with detectron2. This is a single file with the format “.pth”. Once detectron2 package is installed, this file is pointed to instead of the default. This would be initialized as follows:

\#Initialize configuration for Mask-RCNN

cfg = get_cfg()

cfg.merge_from_file("/home/user/Downloads/detectron2/configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")

cfg.DATALOADER.NUM_WORKERS = 2

cfg.MODEL.DEVICE = "cpu" #change to “cuda” if nvidia gpu is present.

cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1

cfg.MODEL.WEIGHTS = r'/home/user/Downloads/detectron2/model_final.pth'

Code/software

Code is present in "SweetAPPS_MaskRCNN_DataValidation". This file was tested with Google Colab using the “SweetAPPS_MaskRCNN_DataValidation.ipynb” file. As an alternative, users can setup a Python environment and run “SweetAPPS_MaskRCNN_DataValidation.py” with the following Anaconda installation command:

conda create -n sweetapps_analysis_env python=3.10 pandas matplotlib numpy scipy scikit-learn opencv -c conda-forge

conda activate sweetapps_analysis_env

However, the following commands assume the use of Google Colab:

1. Import data into a Google Drive folder. For this example, we assume creation of a folder “SweetAPPS_Colab” under your “My Drive”. Note that folder locations will have to be adjusted in the Colab code if it is different.
2. Go to “Add Workspace Apps” and install “Colaboratory”. Once complete, select “Make Google Colaboratory the default app for files it can open.”
3. Right click “Hoang_code_rewrite.ipynb” from Google Drive, select “Open With” and choose “Google Colaboratory”. This will launch a new tab with the notebook opened.
4. Once running, execute the cells in order to generate the figures or access/export the paper’s data. Upon executing the first cell, you do need to grant Google Colab permissions to your Google Drive by selecting “Connect to Google Drive” when prompted to “Permit this notebook to access your Google Drive files?”

Note: an example “run_del_datedfolder.py” is also included. This example will load the model coefficients for inference. Users are directed to detectron2’s github (https://github.com/facebookresearch/detectron2) to configure and run this code. In this example, images that appears in the folder specified in “input_path” (in the example, “/home/user/Downloads/detectron2/MaskRCNN/Input”) will be processed by Mask R-CNN and placed into a new folder in “/home/user/Downloads/detectron2/MaskRCNN/Output”. The folder name will match the file name that was inserted in the “Input”. This setup can enable a real-time flow of images to be processed by the model.