Code and example images from: recolorize: An R package for flexible color segmentation of biological images

Weller, Hannah1 ; Hiller, Anna2; Lord, Nathan2; Van Belleghem, Steven3

Published Jan 31, 2024 on Dryad. https://doi.org/10.5061/dryad.9kd51c5r3

Data files

Jan 31, 2024 version files 41.86 MB

README.md

8.90 KB
recolorize_code_dryad.zip

41.86 MB

Abstract

Color pattern variation provides biological information in fields ranging from disease ecology to speciation dynamics. Comparing color pattern geometries across images requires color segmentation, where pixels in an image are assigned to one of a set of color classes shared by all images. Manual methods for color segmentation are slow and subjective, while automated methods can struggle with high technical variation in aggregate image sets. We present recolorize, an R package toolbox for human-subjective color segmentation with functions for batch-processing low-variation image sets and additional tools for handling images from diverse (high variation) sources. The package also includes export options for a variety of formats and color analysis packages. This paper illustrates recolorize for three example datasets, including high variation, batch processing, and combining with reflectance spectra, and demonstrates the downstream use of methods that rely on this output.

README: Code and example images from 'recolorize: An R package for flexible color segmentation of biological images'

https://doi.org/10.5061/dryad.9kd51c5r3

Summary

This repository contains example files and code from the methods paper (Weller, Hiller, Lord, and Van Belleghem, 2024) describing how to use the recolorize R package (https://github.com/hiweller/recolorize).

Software requirements

All examples were developed and tested using R version 4.3.2 (2023-10-31, "Eye Holes"), and R version >3.5.0 is recommended.

Overview

The main directory contains folders for each of the examples demonstrated in the paper (01_beetles, 03_cichlids, 04_wasps, and 05_birds), as well as two R scripts (patPCA_total.R and 00_installation_RUN_ME_FIRST.R) and an R project file (recolorize_examples.Rproj).

Before running any of the examples, it is recommended to run the 00_installation_RUN_ME_FIRST.R script, as this will install the development versions of the relevant packages for the examples.
The .Rproj file is not necessary to run the examples but is convenient for users of RStudio (double-clicking the file will open RStudio with the recolorize_examples folder set as the working directory).
The patPCA_total.R script is called by other scripts in the repository and does not need to be run by the user. This script contains the function needed for running a whole-colour-pattern PCA with recolorize and patternize (03_cichlids and 04_wasps).
Each folder (01_beetles, 03_ cichlids, 04_wasps, and 05_birds) contains data (images) and code for recreating the analysis shown in the corresponding paper example. The number of the folder matches the figure number from the paper (e.g. figure 3 from the paper can be recreated using the code and images in the 03_cichlids folder). Some folders also contain landmark data and image masks (plain text files generated in ImageJ), as well as RDS files (.rds). The RDS files save intermediate steps of the analysis so that, for example, you can evaluate only the last step of an analysis without having to perform the more time-consuming prior steps such as image alignment; this means you can run the last script in a folder without needing to run the prior scripts. All analyses take you from original images to final analyses.

File structure

Explanation for each file or group of files is included where needed in parentheses with the format 'filename - explanation'

00_installation_RUN_ME_FIRST.R
recolorize_examples.Rproj
patPCA_total.R
01_beetles
- 01_beetles.R
- chrysochroa_corbetti.png
- mniszechii_diffuse.png
- mniszechii_direct.png
03_cichlids
- 03_a_alignment.R
- 03_b1_segmentation_recolorize.R
- 03_b2_segmentation_kmeans.R
- 03_c_color_pattern_PCA.R
- 03_d_repeatability_tests.R
- images - Original, unmasked images of Neolamprologus from Ad Konings.
  - Neo_brichardi_BililaIsland.jpg
  - ...
  - Neo_pulcher_toby02.jpg
- landmarks - XY coordinates corresponding to the 'images' subfolder above.
  - Neo_brichardi_BililaIsland.txt
  - ...
  - Neo_pulcher_toby02.txt
- masks - Polygons (also image XY coordinates) used for masking in patternize.
  - brichardi_nambeyeye_mask1.txt
  - brichardi_nambeyeye_mask2.txt
- rds_files
  - imageList_aligned.rds - Aligned images generated by 03_a_alignment.R
  - image_arrays.rds - Aligned images in array format.
  - image_extents.rds - Image extents (used for recolorize-patternize conversion).
  - kmeans_aggregate.rds - Result of aggregate k-means clustering (1st iteration).
  - kmeans_aggregate_2.rds - Result of aggregate k-means clustering (2nd iteration, see supplement).
  - kmeans_standard.rds - Result of standard k-means clustering (1st iteration).
  - kmeans_standard_2.rds - Result of standard k-means clustering (2nd iteration).
  - recolorize_list.rds - List of recolorize objects (segmented images) from Fig. 3 of the paper.
  - recolorize_list_user1.rds - Segmentation performed by the same user for repeatability (see supplement).
  - recolorize_list_user2.rds - Segmentation performed by different user (see supplement).
04_wasps
- 04_a_align.R
- 04_b_recolorize.R
- 04_c_patPCA.R
- landmarks - XY coordinates corresponding to the 'images' subfolder above.
  - polistes_01_landmarks.txt
  - ...
  - polistes_20_landmarks.txt
- masks - Polygons (also image XY coordinates) used for masking in patternize.
  - polistes_05_Lantenna.txt
  - ...
  - polistes_05_outline.txt
- original_images - Original images from Tumulty et al. 2023.
  - polistes_01.jpg
  - ...
  - polistes_20.jpg
- rds_files
  - imageList_aligned.rds - Images aligned in patternize.
  - palette.rds - Shared color palette to which all images are mapped. Generated by 04_b_recolorize.R.
  - patternize_list.rds - List of aligned patternize images.
  - recolorize_fits.rds - List of color-segmented images as recolorize objects.
- wasp_metadata.csv - Original photo ID, image name, year taken, species, state, nest, latitude, and longitude for each image.
05_birds
- 05_a_alignment.R
- 05_b_recolorize.R
- 05_c_spectra.R
- StockmanSharpe2000_human_receptors_400-700.csv - Human cone sensitivity data from Stockman and Sharpe (2000). Columns = wavelength (wl), and, long-, medium-, and short-wavelength sensitive cone sensitivities.
- hsl_comparison_df.csv - Dataframe generated by 05_c_spectra.R comparing image- and spectra-derived statistics. Coluns = image name, mean luminance, whether value was calculated from image (rgb) or spectra, museum ID, luminance relative to Fig. 5A/Bi, and image ID.
- images - Images used in analyses for Fig. 5.
  - Canon80D-no-flash-129286.JPG - Fig. 5Bii
  - Canon80D-no-flash-196409.JPG - Fig. 5Aii
  - Canon80D-w-flash-129286.JPG - Fig. 5Biii
  - Canon80D-w-flash-196409.JPG - Fig. 5Aiii
  - Diglossa_Aug2021_LSUMZ129286_B_VIS.JPG - Fig. 5Bi
  - Diglossa_Aug2021_LSUMZ196409_B_VIS.JPG - Fig. 5Ai
  - iPhoneXR-w-flash-129286.JPG - Fig. 5Biv
  - iPhoneXR-w-flash-196409.JPG - Fig. 5Aiv
- patternize_coordinates - XY coordinates corresponding to the 'images' subfolder above.
  - Canon80D-no-flash-129286.txt
  - ...
  - iPhoneXR-w-flash-196409.txt
  - mask.txt - Mask for patternize.
- patternize_images - Patternize-aligned images for segmentation with recolorize.
  - Canon80D-no-flash-129286.png
  - ...
  - iPhoneXR-w-flash-196409.png
- rds_files
  - imageList_aligned.rds - Images aligned in patternize.
  - impose_list.rds - Segmented, aligned images from recolorize.
  - spectra.rds - Patch spectra shown in Fig. 5 C and E.

Usage notes

For the folders, it is recommended to execute the scripts in their alphabetic order, e.g. 03_a_alignment.R, then 03_b1_segmentation_recolorize.R, 03_b2_segmentation_kmeans.R, then 03_c_color_pattern_PCA.R to understand how the examples go from input images to final output. However, for each example, the intermediate files generated by each step are saved as RDS files in an rds_files folder, so you can also run them out of order and they should still work.

Data sources

01_beetles: Images of Chrysochroa beetles by Nathan P. Lord (coauthor).

03_cichlids: Images of Neolamprologus fishes taken by Ad Konings, used with his kind permission.

04_wasps: Images of Polistes fuscatus wasps taken by James Tumulty, and are a subset of the images used for analysis in Tumulty et al. (2023): https://doi.org/10.1016/j.cub.2023.11.032

05_birds: Images of Diglossa birds taken by Anna E. Hiller (coauthor). Human cone-catch sensitivity data from Stockman and Sharpe (2000): https://doi.org/10.1016/S0042-6989(00)00021-3

Comments

This dataset is also available on the recolorize examples GitHub repository: https://github.com/hiweller/recolorize_examples. This repository also includes other examples for the R package. Note that figure 2 from the paper uses an image from John E. Randall distributed under a CC BY-NC 4.0 license and cannot be redistributed on Dryad. This image is available on GitHub or from the Bishop Museum (http://pbs.bishopmuseum.org/images/JER/detail.asp?size=i&cols=10&ID=1432937402), and the code for producing the segmentation from figure 2 of the paper is available on GitHub.