Sweetpotatoes (SPs) (Ipomoea batatas) are crops valued for their color, flavor, and nutrition.  Harvesting is labor-intensive, requiring hand-picking to maintain skin quality.  Mechanical harvesting often causes skin damage, known as “skinning,” where skin is cut, scraped, or torn, leading to lower quality during packing. To manage this, packers may conduct a costly “field-switch” to reduce skinning in the production line. Currently, skinning levels are visually assessed by the packers and stakeholders.  Field-switches involve transitioning between multiple fields during harvest to meet specific customer orders (e.g., supermarkets, processing plants, or end users) that require higher-quality SPs. This process aims to minimize skinning and ensure the SPs meet the desired quality standards for those orders.  This study introduces a computer vision (CV) pipeline to automate skinning assessment using a ResNet50-based DeepLabV3+ semantic segmentation model. The CV system was trained to identify three classes: skinning, intact skin\textbackslash SP, and background. A machine vision camera, mounted above a conveyor belt, captured images throughout several full production days. The pipeline calculated the percentage of skinning PS as the ratio between the predicted skinning area and the total SP surface area (As + Ap). Using this method, daily production trends and field-switch decisions were studied with image data from six production days, chosen by our grower collaborator. Percent skinning ratings calculated from random sub-samples of imaged SPs—where only a portion of the full image set was analyzed—showed no significant differences compared to those derived from complete image sets. This demonstrates that sub-sampling can reduce computational processing times by 90% while maintaining accuracy. When data was binned in 30-minute intervals, field-switches occurred when there was approximately 1.5 PS. Across 2,417,907 SP instances, the model achieved a root mean square error (RMSE) of 0.55%, R^2 of 0.84, 80.03% recall, 99.99% specificity, 77.44% F1 score, and 99.98% grading accuracy. This offers a promising improvement for automatic skinning detection on a commercial scale.

Data Collection

The data collection process involved reviewing millions of SPs over a two-year period, from late winter 2022 to late winter 2024. Based on discussions with our industry collaborator, we identified specific days when skinning issues were most apparent. SPs for the training dataset were chosen for their fresh skinning marks, which were likely caused by harvesting, rough handling, improper curing, washing, or damage from machinery. These skinning issues typically occurred within one to two weeks of the harvest, guiding the selection of images.

The images collected were stored in .jpg format with a ground sample distance (GSD) of 0.521 mm/pixel. Most images had a size of 866x1599 pixels, though a few were smaller and resized to 866x1599 pixels using MATLAB. The data collection process was ongoing throughout the study, with adjustments made to the dataset after each performance evaluation to ensure that only the most relevant and high-quality images were retained.

Data Curation and Labeling

A dataset of individual SP instances was curated by selecting high-quality images with distinct skinning marks. These images were chosen based on input from our industry collaborator, who helped identify periods when skinning issues were most prevalent. The curated dataset consisted of high-quality images from five distinct days spanning late winter 2022 to late winter 2024. The SPs were selected based on their fresh skinning marks, which were presumed to have occurred within the past one to two weeks due to factors such as harvesting, rough handling, improper curing, washing, and mechanical damage (bumps, scrapes, or falls).

Each image was manually annotated with three categories: skinning, SP, and background. The skinning marks ranged in size from 0.36 mm² to 316.17 mm². The images were taken with a machine vision camera at the eliminator table. Once annotated, the images were processed further for model training. To augment the dataset, each image was randomly partitioned into 20 regions of interest (ROIs). This process increased the dataset size from 547 to 10,940 images. Each ROI was resized to $224\times224$ pixels to match the input size of the ResNet50 model, and labeled images were converted into a single-channel 8-bit format. The resulting dataset was split into an 80/20 ratio for training and validation, with 8,752 images used for training and 2,188 for validation.

Model Pipeline

For our study, a joint model consisting of a DeepLabV3+ model with a ResNet50 backbone was used to enhance feature extraction for semantic segmentation tasks. DeepLabV3+ was chosen due to its strong performance in pixel-level segmentation tasks and its availability in MATLAB's Deep Learning Toolbox, which provided a straightforward implementation path. While newer architectures such as YOLO or SAM/SAM2 have demonstrated superior speed and performance, DeepLabV3+ remains highly effective in various segmentation applications, and it offers a reliable baseline for comparison in this study.