The partitioned respiration dataset, together with the perturbed training data and the training code, for KGML-RaRh, which partitions daily ecosystem respiration into aboveground autotrophic, belowground autotrophic and heterotrophic components at 105 CONUS FLUXNET sites.

Contents

• us_rarh_KGML_site_finetune_pred.sav — the dataset itself: the ensemble mean and standard deviation of the seven daily fluxes and two annual stock changes at each of the 105 sites, alongside the FLUXNET observations and the per-site skill metrics (57 MB). Produced by stage 4 below; the fields are documented in Table 1 of the paper.
• LHS_FLUXNET/ — Fluxnet_0.sav .. Fluxnet_99.sav, 100 perturbed draws of the FLUXNET daily NEE, GPP and R_eco targets (3.7 GB).
• LHS_SRDB/ — SRDB_0.sav .. SRDB_99.sav, 100 perturbed draws of the SRDB annual R_s, R_a and R_h targets (540 MB).
• LHS_COSORE/ — COSORE_0.sav .. COSORE_99.sav, 100 perturbed draws of the COSORE daily R_h, R_s and R_eco targets (243 MB).
• training_processing_codes/ — the five scripts described below.
• additional_training_input_files/ — three small inputs the scripts need:
  • scalers_124_v1.sav — z-normalization statistics for the predictors and targets.
  • us_rarh_v2e5_KGML.sav — the pretrained initialization M0.
  • all_sites_info_fluxnet.csv — site table for the 105 FLUXNET sites.

Draw k of all three datasets is one joint realization, so Fluxnet_k, SRDB_k and COSORE_k are always used together. Each .sav is a torch.save dict whose values are lists indexed by record.

Variables in all_sites_info_fluxnet.csv

The only tabular file in this record. One row per site, 105 rows, comma-separated, with a header row and empty fields for missing values. Sites are located by row position (0-based), and that position is the value stored in the first column of X_info in every .sav file, so the row order must not be changed.

• SITEID — AmeriFlux site identifier, e.g. US-Ha1. Text, 105 unique values.
• LAT — latitude, decimal degrees north.
• LON — longitude, decimal degrees east.
• IGBP — plant functional type, given as an International Geosphere-Biosphere Programme land-cover class. Four values occur: DBF deciduous broadleaf forest, GRA grassland, OSH open shrubland, CSH closed shrubland. OSH and CSH are pooled into one shrubland class, abbreviated SRB.
• ELEV — elevation above sea level, metres.
• MAT — mean annual air temperature, degrees Celsius.
• MAP — mean annual precipitation, millimetres per year.
• STARTYEAR — first calendar year of the site's flux record.
• ENDYEAR — last calendar year of the site's flux record.
• AGE — forest stand age at the start of the record, years. Zero at every grassland and shrubland site.

The model is driven by the daily meteorology and soil properties held in X_dynamic and X_static inside the .sav files, not by the columns of this table. The scripts read SITEID, IGBP and STARTYEAR from it to label sites and to date the annual values; the remaining columns describe the sites for the reader.

Code

Each entry gives what the script does, then what it needs.

• finetune.py — library holding the Structured_GRU model, the knowledge-guided losses and the normalization helpers. Imported by the others and never run on its own.
  • Needs: nothing.
• finetuning_crossval.py — stage 1. Trains the global model for one cross-validation fold on one realization.
  • Needs: scalers_124_v1.sav, us_rarh_v2e5_KGML.sav, Fluxnet_k.sav, SRDB_k.sav, COSORE_k.sav.
• finetuning_site.py — stage 2. Finetunes all five fold models at each FLUXNET site individually, for one realization.
  • Needs: scalers_124_v1.sav, Fluxnet_k.sav, all_sites_info_fluxnet.csv, and the stage 1 checkpoints.
• finetuning_site_post_process.py — stage 3. Reduces the five folds per site and computes per-site skill, for one realization.
  • Needs: all_sites_info_fluxnet.csv and the stage 2 outputs.
• aggregate_lhs_ensemble.py — stage 4. Reduces the 100 realizations into the ensemble mean and standard deviation per site-day, attaches the unperturbed observations and the per-site skill, and writes us_rarh_KGML_site_finetune_pred.sav.
  • Needs: the stage 3 outputs, all_sites_info_fluxnet.csv, and Ameriflux_fluxnet.sav, the unperturbed observation torch, which is not included here.

Running order

Run stages 1 to 3 once per realization k = 0..99, with stage 1 additionally once per cross-validation fold, so 500 jobs for stage 1 and 100 jobs each for stages 2 and 3. Then run aggregate_lhs_ensemble.py once to produce us_rarh_KGML_site_finetune_pred.sav.

Stages 1 and 2 need a CUDA GPU; stages 3 and 4 are CPU only. Requires torch, numpy, pandas, scipy and matplotlib.

Data usage

The observation fields are daily aggregations of FLUXNET data, which are distributed under CC-BY-4.0; users of those fields should cite FLUXNET and the contributing site teams in accordance with the FLUXNET data policy.