Data and code from: Coordinated distributed experiments in ecology do not consistently reduce heterogeneity in effect size
Data files
Mar 04, 2024 version files 12.39 KB
-
README.md
-
rousk_et_al_2013_table_3_data_-_INCREASE.csv
Abstract
Ecological meta-analyses usually exhibit high relative heterogeneity of effect size: most among-study variation in effect size represents true variation in mean effect size, rather than sampling error. This heterogeneity arises from both methodological and ecological sources. Methodological heterogeneity is a nuisance that complicates the interpretation of data syntheses. One way to reduce methodological heterogeneity is via coordinated distributed experiments, in which investigators conduct the same experiment at different sites, using the same methods. We tested whether coordinated distributed experiments in ecology exhibit a) low heterogeneity in effect size, and b) lower heterogeneity than meta-analyses, using data on 17 effects from eight coordinated distributed experiments, and 406 meta-analyses. Consistent with our expectations, among-site heterogeneity typically comprised <50% of the variance in effect size in distributed experiments. In contrast, heterogeneity within and among studies typically comprised >90% of the variance in effect size in meta-analyses. However, this difference largely reflected the small size of most coordinated distributed experiments, and was no longer significant after controlling for size (number of studies or sites). These results are consistent with the hypothesis that methodological heterogeneity rarely comprises a substantial fraction of variance in effect size in ecology. We also conducted pairwise comparisons of absolute heterogeneity between coordinated distributed experiments and meta-analyses on the same topics. Coordinated distributed experiments did not consistently exhibit lower absolute heterogeneity in effect size than meta-analyses on the same topics. Our findings suggest that coordinated distributed experiments rarely increase uniformity of results by reducing methodological heterogeneity. Our results help refine the numerous distinct reasons for conducting coordinated distributed experiments.
README: Coordinated distributed experiments in ecology do not consistently reduce heterogeneity in effect size
Included here is a data file for a distributed experiment, and code which analyses the heterogeneity of many coordinated distributed experiments and meta-analyses. The R code file reproduces the results of this study, called meta-analyses vs distd expts - R code for sharing v 2.R.
## Description of the data and file structure
Data File:
rousk et al 2013 table 3 data - INCREASE.csv: data from the INCREASE distributed experiment by Rousk et al. (2013)
All other data used in code is automatically sourced from URLs, but relevant variables are still described below.
Other variables in datasets were not used in our analysis, and so are not explained in this README file. Cells with missing data have "NA" values.
Variables used in code:
Costello & Fox variables:
meta.analysis.id: Unique ID number for each meta-analysis
eff.size: Effect size
var. eff.size: Variance in effect size
study: Unique names for all studies within all meta-analyses
id.effect.within.study: Unique ID number for each effect within a study. Starts over at 1 for each study.
PANAMEX variables:
Sites: Sites where experiments were conducted
Lat: Latitude coordinate
Atl.Pac: Factor for whether site was in Atlantic (A) or Pacific (P)
Treat: Treatments coded, 1: open control, 2: full predator exclusion cage, 3: part cage, 4: exposure. 1 and 3 are considered the controls.
Plate: Akin to an experimental plot, unique for each replicate
Wt: Biomass, wet weight in g of each plate
Smith et al. 2024, DroughtNet variables:
se: Sampling error
site_code: Character code for each site
mean_DS3: Mean drought response at each site, effect size
n: Number of replicates at each site
Rousk et al 2013 Table 3 data, INCREASE variables:
Ttmt: Control, warming, or drought treatment of plot
bacterial.growth.mean: Mean estimated bacterial growth based on leucine incorporation
bacterial.growth.sd: Standard deviation of estimated bacterial growth based on leucine incorporation
fungal.growth.mean: Mean estimated fungal growth based on acetate incorporation into ergosterol
fungal.growth.sd: Standard deviation of estimated fungal growth based on acetate incorporation into ergosterol
fungal.PFLA.mean: Mean concentration fungal-biomass-specific PLFA markers
fungal.PFLA.sd: Standard deviation of concentration fungal-biomass-specific PLFA markers
site: Sites where experiments were conducted
Fronhofer et al. 2018, Dispnet variables:
lab: Lab where experiments took place
effect: Treatment, effect of either predation or resources on dispersal
id.within.lab: Unique identifier for each experiment conducted at a single lab
no_dispersers: Number of individuals who dispersed
no_residents: Number of individuals who remained
sp.lab.rsrc: Species name, lab name, and resource availability level together, separated by “.”
PRED: Yes or no to presence of predators
RA: Resource availability level
Collins et al. 2021 ITEX data variables:
site_name: Name of site where experiments took place
subsite: Subsites where experiments took place
year: Year experiments took place
phen_stage: Phenological stages
simple.phenophase: Simpler categories of phenological stages, ex. “Senesce”
definition: Definitions of phen_stage values
treatment: CTL control or OTC open top chamber (warming treatment)
early.or.late: Early or late phenological events
Spp: Species
doy: Day of year
Milcu et al. 2018 variables:
Lab: “L#” Unique for each lab that conducted experiments
Treat..legume: B means only Brachypodium grass and no legume, BM means grass with legume Medicago present
Biom.dm..g...shoot.: Aboveground biomass of plot in g
Method.comm..type.of.setup: Where the experiment took place, either in a greenhouse or growth chamber
Spehn et al. 2005, BIODEPTH variables:
plot: Unique to each experimental plot
location: Site of experimental plot
year: 1 through 3, only year 3 was used
legumes: Yes/no for legume presence/absence
species.richness: Number of species in plot
biomass: Aboveground plant biomass in g/m2
Borer et al. 2020, Nutnet variables:
site: Site name where experiments conducted
block: Block number of site where experiments conducted
plot: Plot number where given treatment imposed
year: Year data collected from plot
year_trt: Number of years treatment has been imposed at plot
lm_lg: Log10 aboveground plant biomass in g in plot for a given year
trt: Control (no treatment), NPK (nutrient addition), Fence (herbivore exclusion), or NPK+Fence (both) imposed on plot
Guasconi et al. 2023 variables:
shoot_biomass_control: Mean aboveground plant biomass in control plots
stand_dev_shoot_biomass_control: Standard deviation of aboveground plant biomass in control plots
n_replicates_shoot_samples: Sample size of control plots
shoot_biomass_treatment: Mean aboveground plant biomass in drought treated plots
stand_dev_shoot_biomass_treatment: Standard deviation of aboveground plant biomass in drought treated plots
n_replicates_shoot_samples: Sample size of drought treated plots
study: Unique number for each study included in the meta-analysis
datapoint: Unique number for each effect size
Jia et al. 2018 variables:
X: Unique number for each effect size
Article: Unique number for each study included in the meta-analysis
Vegatation_response: Response variable used for effect size, we only use “standing biomass”
Unfenced_mean: Mean response variable (biomass) for unfenced plots
Unfenced_se: Standard error for unfenced plots
Unfenced_n: Number of unfenced plots
Fenced_n: Number of fenced comparator plots
Fenced_se: Standard error of biomass for fenced plots
Fenced_mean: Mean response variable (biomass) for fenced plots
Cardinale et al. 2006 variables:
Study: Unique for each published paper
Entry: Unique for each experiment
N1: Number of observations in a study with 1 species
N2: Number of observations in a study with 2 species
Y1: Mean aboveground plant biomass in g/m2 for observations with 1 species
Y2: Mean aboveground plant biomass in g/m2 for observations with 2 species
SD1: Standard deviation of aboveground plant biomass in g/m2 for observations with 1 species
SD2: Standard deviation of aboveground plant biomass in g/m2 for observations with 2 species
LnRR2: Log response ratio effect size of the presence of 2 species vs 1 species on aboveground plant biomass
SV2: Sampling variance of each effect of the presence of 2 species vs 1 species on aboveground plant biomass
Stuble et al 2021 variables:
LastYear: Yes/no for last year of experiment
EarlyLate: Early or late phenophase
CitationID: Unique for each published paper
effect.id: Unique for each experiment
yi: Hedge’s g effect size of warming on timing of phenological event
vi: Sampling variance of each effect of warming on timing of phenological event
## Sharing/Access information
Data used in code was derived from the following sources:
Milcu et al. 2018: https://doi.pangaea.de/10.1594/PANGAEA.880980
Borer et al. 2020: https://doi.org/10.6073/pasta/a318fe0fb11eb43c1a2c8233b2e3494f
Stuble et al. 2021: https://onlinelibrary.wiley.com/doi/abs/10.1111/gcb.15685
Costello & Fox 2022: https://doi.org/10.5061/dryad.zkh1893b7
Collins et al. 2021: https://github.com/cour10eygrace/OTC_synthesis_analyses.git
Cardinale et al. 2006: https://www.pnas.org/doi/10.1073/pnas.0709069104#supplementary-materials
Rousk et al. 2013: Table 3 of Rousk et al. 2013
Ohlert and Collins 2021: doi:10.6073/pasta/5c2eb1c145a3babf399a0587d0189c2e
Jia et al. 2018: https://doi.org/10.5061/dryad.s7g70j0
Ashton et al. 2022: https://doi.org/10.25573/serc.19469900.
Fronhofer et al. 2018: https://doi.org/10.5281/zenodo.1344579
Guasconi et al 2023: https://doi.org/10.1016/j.scitotenv.2023.166209
Smith et al. 2024: https://datadryad.org/stash/dataset/doi:10.5061/dryad.3j9kd51rb
## Code/Software
We conducted all statistical analyses using R 3.6.3 running within R Studio 1.3.1093 (R Core Team 2020). We fit all models using functions from the metafor package, version 2.4-0 (Viechtbauer, 2010). The likelihood ratio tests for differences in heterogeneity between meta-analyses vs. coordinated distributed experiments were based on the following example from metafor package author Wolfgang Viechtbauer: https://www.metafor-project.org/doku.php/tips:different_tau2_across_subgroups. If data does not read into R from the provided URLs, users can download the the data at these URLs personally.