Data from: Fun surveys? Developing an innovative approach to assessing learning through citizen science
Data files
Sep 03, 2025 version files 72.40 KB
-
README.md
10.59 KB
-
survey_data_final.csv
61.81 KB
Sep 09, 2025 version files 73.26 KB
-
README.md
11.45 KB
-
survey_data_final.csv
61.81 KB
Abstract
Assessing the impact of citizen science participation on volunteers is increasingly important. However, traditional methods for evaluating volunteer experiences—such as quantitative surveying—have drawbacks. Response rates are often low, pre- and post-tests provide a coarse metric for assessing learning, and static evaluation techniques have failed to align with the participatory spirit of citizen science, where volunteers are involved in understanding their role in scientific inquiry. In response to these problems, we developed a more engaging type of longitudinal survey aimed at achieving higher response rates and better involving volunteers in understanding the scientific value of their survey efforts. We first reviewed the literature on “fun” and found that fun activities embody six characteristics: autonomy, social connection, playfulness, challenge, transport, and curiosity. We used this “fun heuristic” to design “SciQuest,” an exploratory survey tool that evaluated volunteers’ pro-environmental attitudes and behaviors across citizen science projects hosted on the platform SciStarter.org. SciQuest proved to be a reliable instrument that captured common learning outcome–related constructs measured by conventional, validated instruments. Although most beta-testers had no preference between SciQuest and a conventional survey, among those expressing a preference, nearly twice as many favored SciQuest. SciQuest also achieved much higher retention rates (49–67%) than past conventional surveys on SciStarter (19%). Embedded, creative approaches to citizen science evaluation are thus a viable, and perhaps preferable, complement to traditional surveys, particularly on citizen science platforms. Citizen science facilitators should extend their volunteer-centric perspectives to include participant evaluation and seek evidence-based strategies for enriching the citizen science experience.
https://doi.org/10.5061/dryad.7d7wm3843
Description of the data and file structure
This CSV contains survey data from 243 Qualtrics Panelists (paid survey takers). These Panelists completed a survey evaluating their pro-environmental attitudes and beliefs. The Panelists completed two different survey types assessing the same pro-environmental constructs. The first survey type used existing, validated instruments (e.g. Manfredo's "Wildlife Value Orientation"). The second survey type used variations of these existing instruments designed to be more engaging, fun, use interesting graphic elements, etc., for a new type of citizen science survey tool called "SciQuest." The goal of this survey effort was to quasi-validate the survey questions used in SciQuest against the existing, conventional instruments.
There is also an attached R script for replicating our data analysis.
There is also an attached supplemental video file that is a screen recording of someone completing the "SciQuest" citizen science survey modules. We have included this here because the journal we are submitting to doesn't allow video files to be uploaded in their manuscript submission software. Including the video here (as Supplemental Information) allows us to create a stable link to which we can reference this file.
Lastly, there are three Supplemental Files of file type .qsf corresponding to the three survey modules referenced in our study (Journey Prep module is in Supplemental File 4, Module 1 is in Supplemental File 5, and Module 2 is in Supplemental File 6). The file type .qsf is the Qualtrics Survey file type, and this file can be directly loaded into the Qualtrics software to view or adapt our survey modules. The files can also be viewed using a text editor.
Files and variables
File: survey_data_final.csv
Description: Results of Qualtrics Panelist survey effort evaluating respondents' pro-environmental attitudes and beliefs and quasi-validating new survey instruments for a citizen science survey tool called "SciQuest." Many of the columns start with either "SI" or "VI". This indicates whether the survey question(s) is a conventional validated survey instrument ("VI") or a new, experimental SciQuest instrument ("SI"). Citations related to validated instruments will be provided in the supplemental table included in the publication.
Variables
- gender: Numeric value representing the gender of survey respondent ("Man" or "Woman"). Genders were randomly assigned a numeric value (1 or 2) to anonymize respondents.
- race: Race/ethnicity of survey respondent
- age: Age of survey respondent, binned to anonymize respondents.
- education: Highest educational attainment of survey respondent
- politics: Political views of survey respondent on a five point scale from "Conservative Republican" to "Liberal Democrat". Note that we randomly assigned the direction of this scale (either from Conservative Republican (1) to Liberal Democrat (5) or from Liberal Democrat (1) to Conservative Republican (5) to anonymize responses.
- career: Binary variable indicating whether the respondent worked in a STEM (science, technology, engineering, mathematics) career
- cs_experience: Variable indicating whether the respondent has prior experience participating in citizen science
- live_in_usa: Binary variable indicating whether the respondent lives in the United States
- prefer_ab: Variable indicating whether the respondent preferred the SciQuest version ("A") or the conventional version ("B") of the first set of survey questions.
- prefer_cd: Variable indicating whether the respondent preferred the SciQuest version ("C") or the conventional version ("D") of the second set of survey questions.
- enjoy_ab: Variable indicating whether the respondent enjoyed the SciQuest version ("A") or the conventional version ("B") of the first set of survey questions.
- enjoy_cd: Variable indicating whether the respondent enjoyed the SciQuest version ("C") or the conventional version ("D") of the second set of survey questions.
- SI_aff_enviro_concern: Average of three items assessing respondents' affective environmental concern using a set of content to distraught emoji faces. The scale for this set of items is 1 (low environmental concern) to 7 (high environmental concern).
- VI_enviro_concern: Average of nine items assessing respondents' environmental concern using a conventional instrument from Cruz and Manata. The scale for this set of items is 1 (low environmental concern) to 7 (high environmental concern).
- SI_dist_from_nature: Average of three items assessing respondents' connection to nature by asking respondents to place themselves close to or far from a nature-related word. The scale for this set of items is 1 (low nature connection) to 9 (high nature connection).
- VI_ill_inclusion_nature_in_self: Single item measuring respondents' connection to nature by asking them to move two circles representing them and nature either close to or far from one another. Based on the Illustrated Inclusion of Nature in Self Scale by Kleespies. The scale for this item is 1 (low nature connection) to 7 (high nature connection).
- SI_ecocentrism_slider: Single item measuring respondents' wildlife valuation by having them indicate which of a set of images represents their valuation of nature. The images represent a spectrum of ecocentrism to anthropocentrism. The scale for this item is 1 (low wildlife valuation) to 5 (high wildlife valuation).
- SI_basketfish_narrative: Single item measuring respondents' wildlife valuation by having them respond to a fictional narrative about the perils posed to a rare fish by a dam-building project. The scale for this item is 1 (low wildlife valuation) to 4 (high wildlife valuation).
- SI_basketfish_slider: Single item measuring respondents' wildlife valuation by having them move a slider indicating whose rights they thought were most important (people or a rare fish). The scale for this item is 0 (low wildlife valuation) to 100 (high wildlife valuation).
- VI_wildlife_value_orient: Average of eight items measuring respondents wildlife valuation based on the Wildlife Value Orientation instrument by Manfredo. The scale for these items is 1 (low wildlife valuation) to 7 (high wildlife valuation).
- SI_this_or_that_pref: Average of eight items measuring respondents' interest in nature by having them choose which activities they would prefer-- activities involving nature/the environment and those not involving nature/the environment. The scale for these items is 0 (low nature interest) to 1 (high nature interest).
- SI_cartoon_nat_int: Average of four items measuring respondents' interest in nature by having them respond to a cartoon person inviting them to do an activity related to nature/the environment. Possible responses range from high to low excitement to participate. The scale for these items is 1 (low nature interest) to 5 (high nature interest).
- SI_nature_photo_feel: Average of six items measuring respondents' nature affinity by having them respond to a nature-related photo. Respondents indicate whether the photo provokes a negative or positive emotional state. The scale for these items is 0 (low nature affinity) to 100 (high nature affinity).
- VI_interest_affinity_nat: Average of ten items measuring respondents' interest in nature/nature affinity using adaptations of established scales from Kals et al., Cheng and Monroe, and Phillips et al. The scale for these items is 1 (low nature interest) to 5 (high nature interest).
- SI_dnd_conservation: Average of two items measuring respondents' conservation behavior by having them indicate how likely they are to do a conservation behavior in a set of different circumstances. The scale for these items is 0 (low conservation behavior) to 1 (high conservation behavior).
- SI_food_behavior: Single item measuring respondents' conservation behavior by asking them how frequently they eat meat. The scale for this item is 1 (low conservation behavior) to 5 (high conservation behavior).
- SI_consumer_choice: Average of two items measuring respondents' conservation behavior by asking them about their likelihood of making various consumer choices that help the environment. The scale for these items is 0 (low conservation behavior) to 1 (high conservation behavior).
- VI_conservation_behav: Average of four items measuring respondents' conservation behavior using a validated instrument from Larson et al. The scale for these items is 1 (low conservation behavior) to 5 (high conservation behavior).
- SI_land_steward_graphics: Average of three items measuring respondents' land stewardship by responding to graphic questions asking questions about the frequency with which they engage in different land stewardship activities. The scale for these items is 1 (low land stewardship) to 5 (high land stewardship).
- VI_land_stewardship: Average of three items measuring respondents' land stewardship using a validated instrument from Larson et al. The scale for these items is 1 (low land stewardship) to 5 (high land stewardship).
- SI_avatar_soc_enviro_slider: Single item measuring respondents' social environmentalism by reading about different cartoon people with different environmental habits and indicating which person best represents their views. The scale for this item is 1 (low social environmentalism) to 5 (high social environmentalism).
- VI_social_enviro: Average of three items measuring respondents' social environmentalism using a validated instrument from Larson et al. The scale for these items is 1 (low social environmentalism) to 5 (high social environmentalism).
- SI_avatar_enviro_cit_slider: Average of two items measuring respondents' environmental citizenship by reading about different cartoon people with different environmental citizenship behaviors and indicating which person best represents their habits. The scale for this item is 1 (low environmental citizenship) to 5 (high environmental citizenship).
- VI_enviro_cit: Average of four items measuring respondents' environmental citizenship using a validated instrument from Larson et al. The scale for these items is 1 (low environmental citizenship) to 5 (high environmental citizenship).
Note that we removed the column "end_date" (the date the survey was completed) from this file to help with anonymizing these survey responses (suggestion from datadryad).
File: Supplemental_File_4.qsf (Zenodo)
Description: This is the "Journey Prep" Qualtrics survey.
File: Supplemental_File_5.qsf (Zenodo)
Description: This is the "Module 1" Qualtrics survey.
File: Supplemental_File_6.qsf (Zenodo)
Description: This is the "Module 2" Qualtrics survey.
Code/software
Data can be viewed in Excel, R, etc. (it's a simple CSV file). Anyone is also welcome to use the included R script to run the analyses used in this submission to calculate correlation coefficients, and other summary statistics.
Access information
Other publicly accessible locations of the data:
- NA
Data was derived from the following sources:
- NA
Change Log
- 9 Sept 2025: Uploaded Qualtrics files to Supplemental Information
The attached CSV contains 243 anonymized survey responses from Qualtrics "Panelists" (paid surveytakers). These surveytakers completed a survey in May, 2021 where they answered a variety of questions about their connection to nature, their pro-enviornmental behaviors, and other conservation-related constructs. Each respondent completed two different types of survey analyzing the same construct: one using existing, validated constructs, and one using a new, untested instrument called "SciQuest."
The goal of collecting this survey data was to quasi-validate the SciQuest version of these constructs. SciQuest is designed to provide a more fun, engaging way of answering survey questions for citizen science evaluators. However, to make sure the SciQuest questions are assessing the same constructs as conventional instruments, we compared responses to the same constructs among these Panelists who took both a conventional survey and a SciQuest version of the same survey constructs.
This dataset has already been processed to remove 197 incomplete/low-quality survey responses.
We have also attached an R script that analzes this dataset to calculate the results presented in our manuscript.
We have also included a supplemental video file. This file is a screen recording of the SciQuest landing page and annotated walk-throughs of each survey module on SciQuest. The video shows a participant completing the three available survey modules on SciQuest: Journey Prep, Module 1: Nature, and Module 2: Environment. The journal to which we are submitting this manuscript does not allow for video-type files. I am uploading this supplemental video file on Dryad so that we can have a stable reference to this supplemental video.
Lastly we have included three .qsf files of each of the three Qualtrics survey modules (Supplemental Files 4 corresponds to the Journey Prep module, Supplemental File 5 corresponds to Module 1 and Supplemental File 6 corresponds to Module 2). These files can be directly loaded into Qualtrics by others interested in using the surveys. The journal to which we are submitting this manuscript does not allow for submission of the .qsf file type, so I am uploading these supplemental files on Dryad.
