Data from: Crowds replicate performance of scientific experts scoring phylogenetic matrices of phenotypes

O'Leary MA, Alphonse K, Arce H. M, Cavaliere D, Cirranello A, Dietterich TG, Julius M, Kaufman S, Law E, Passarotti M, Reft A, Robalino J, Simmons NB, Smith SY, Stevenson DW, Theriot E, Velazco PM, Walls RL, Yu M, Daly M

Date Published: June 7, 2017

DOI: http://dx.doi.org/10.5061/dryad.766cp.2

 

Files in this package

Content in the Dryad Digital Repository is offered "as is." By downloading files, you agree to the Dryad Terms of Service. To the extent possible under law, the authors have waived all copyright and related or neighboring rights to this data. CC0 (opens a new window) Open Data (opens a new window)

Title Appendix 1 - List of six matrices developed for this study.
Downloaded 4 times
Description Online Appendix 1. List of six matrices developed for this study, including character names, types of characters, and assessment of character difficulty by experts. See also Morphobank (www.morphobank.org) projects P950, P2463, P2502, P2490, P2491, and P2577.
Download Appendix 1 Run3onlyVer5.xlsx (46.74 Kb)
Details View File Details
Title Appendix 2 characterDifficultyMP_3-15-16_MAH edited
Downloaded 1 time
Description Online Appendix 2. List of characters evaluated and their simplified anatomical descriptions used in the Evolution Project. The latter were presented to non-experts in the crowd.
Download Appendix 2 characterDifficultyMP_3-15-16_M....xlsx (41.17 Kb)
Details View File Details
Title Appendix 3 Anemones-character-taxon-results
Downloaded 1 time
Description Online Appendix 3. Sea anemones character scores. For each character and taxon in the anemones matrix, we show the probability (“Estimate”) that a crowd member’s score would agree with the majority vote of the crowd. We also show the lower confidence interval on this probability (ci.lower), which is the crowd confidence score. Finally, we indicate whether the majority vote was correct, and compute an ROC curve for the crowd’s scores. The Threshold Plot worksheet provides a visualization of this information.
Download Appendix 3 Anemones-character-taxon-results.xlsx (36.23 Kb)
Details View File Details
Title Appendix 4 Anemones-user-results
Description Online Appendix 4. Sea anemones user scores. For each crowd member, we report the number of scores they provided and the number that were correct. The “Estimate” column is the probability that this crowd member voted correctly, and the “ci.lower” column gives the 95% lower confidence bound on this probability. These scores are for all characters (evaluation and test).
Download Appendix 4 Anemones-user-results.csv (9.323 Kb)
Details View File Details
Title Appendix 5 Bats-character-taxon-results
Downloaded 1 time
Description Online Appendix 5. Bats character scores.
Download Appendix 5 Bats-character-taxon-results.xlsx (38.83 Kb)
Details View File Details
Title Appendix 6 Bats-users-results
Description Online Appendix 6. Bats user scores.
Download Appendix 6 Bats-users-results.csv (7.402 Kb)
Details View File Details
Title Appendix 7 Catfish-character-taxon-results
Downloaded 1 time
Description Online Appendix 7. Catfish character scores.
Download Appendix 7 Catfish-character-taxon-results.xlsx (37.30 Kb)
Details View File Details
Title Appendix 8 Catfish-user-results
Downloaded 1 time
Description Online Appendix 8. Catfish user scores.
Download Appendix 8 Catfish-user-results.csv (10.34 Kb)
Details View File Details
Title Appendix 9 Diatoms-character-taxon-results
Downloaded 2 times
Description Online Appendix 9. Diatom character scores
Download Appendix 9 Diatoms-character-taxon-results.xlsx (39.79 Kb)
Details View File Details
Title Appendix 10 Diatoms-user-results
Downloaded 3 times
Description Online Appendix 10. Diatom user scores
Download Appendix 10 Diatoms-user-results.csv (10.67 Kb)
Details View File Details
Title Appendix 11 Lilies-character-taxon-results
Downloaded 2 times
Description Online Appendix 11. Lilies character scores.
Download Appendix 11 Lilies-character-taxon-results.xlsx (35.98 Kb)
Details View File Details
Title Appendix 12 Lilies-user-results
Downloaded 1 time
Description Online Appendix 12. Lilies user scores.
Download Appendix 12 Lilies-user-results.csv (8.885 Kb)
Details View File Details
Title Appendix 13 Marine-Shrimp-character-taxon-results
Downloaded 1 time
Description Online Appendix 13. Marine shrimp character scores.
Download Appendix 13 Marine-Shrimp-character-taxon-....xlsx (34.90 Kb)
Details View File Details
Title Appendix 14 Marine-Shrimp-user-results
Description Online Appendix 14. Marine shrimp user scores.
Download Appendix 14 Marine-Shrimp-user-results.csv (9.006 Kb)
Details View File Details
Title Appendix 15 all-users-results
Downloaded 3 times
Description line Appendix 15. Combined results for all users. Details for Figure 3.
Download Appendix 15 all-users-results.xlsx (75.38 Kb)
Details View File Details
Title Appendix 16 joint-predicted-difficulty
Downloaded 3 times
Description Online Appendix 16. Joint predicted difficulty. Predicted and Observed difficulty of each character based on a linear model fit to all of the character score data. Details for Figure 5.
Download Appendix 16 joint-predicted-difficulty.xlsx (15.81 Kb)
Details View File Details
Title Appendix 17 Diatoms-user-thresh-curve
Downloaded 1 time
Description Online Appendix 17. Results of the parameter tuning experiment on the Diatoms. Details for Figure 6.
Download Appendix 17 Diatoms-user-thresh-curve.xlsx (19.47 Kb)
Details View File Details
Title Appendix 18 final-results-summary_MAH edited
Downloaded 1 time
Description Online Appendix 18. Final results. Details for Figure 3.
Download Appendix 18 final-results-summary_MAH edited.xlsx (17.21 Kb)
Details View File Details
Title Appendix 19 Data and R Scripts_MAH modified
Description Online Appendix 19. R scripts for analysis as a zipped folder.
Download Appendix 19 Data and R Scripts_MAH modified.zip (518.2 Kb)
Details View File Details
Title Appendix 20 Instructions to Crowd
Downloaded 21 times
Description Online Appendix 20. Instruction sheets for the study participants (undergraduate students at Ohio State University).
Download Appendix 20 Instructions to Crowd.pdf (757.9 Kb)
Details View File Details

When using this data, please cite the original publication:

O’Leary MA, Alphonse K, Arce H. M, Cavaliere D, Cirranello A, Dietterich T, Julius M, Kaufman S, Law E, Passarotti M, Reft A, Robalino J, Simmons NB, Smith S, Stevenson D, Theriot E, Velazco PM, Walls R, Yu M, Daly M (2017) Crowds replicate performance of scientific experts scoring phylogenetic matrices of phenotypes. Systematic Biology, online in advance of print. http://dx.doi.org/10.1093/sysbio/syx052

Additionally, please cite the Dryad data package:

O'Leary MA, Alphonse K, Arce H. M, Cavaliere D, Cirranello A, Dietterich TG, Julius M, Kaufman S, Law E, Passarotti M, Reft A, Robalino J, Simmons NB, Smith SY, Stevenson DW, Theriot E, Velazco PM, Walls RL, Yu M, Daly M (2017) Data from: Crowds replicate performance of scientific experts scoring phylogenetic matrices of phenotypes. Dryad Digital Repository. http://dx.doi.org/10.5061/dryad.766cp.2
Cite | Share
Download the data package citation in the following formats:
   RIS (compatible with EndNote, Reference Manager, ProCite, RefWorks)
   BibTex (compatible with BibDesk, LaTeX)

Version History

Item Version Date Summary

* Selected Version

Search for data

Be part of Dryad

We encourage organizations to: