Data from: STREAM-Sat: a novel near-realtime quasi-global satellite-only ensemble precipitation
Data files
Dec 17, 2023 version files 85.41 GB
-
README.md
-
STREAMSat_20170101.nc
-
STREAMSat_20170102.nc
-
STREAMSat_20170103.nc
-
STREAMSat_20170104.nc
-
STREAMSat_20170105.nc
-
STREAMSat_20170106.nc
-
STREAMSat_20170107.nc
-
STREAMSat_20170108.nc
-
STREAMSat_20170109.nc
-
STREAMSat_20170110.nc
-
STREAMSat_20170111.nc
-
STREAMSat_20170112.nc
-
STREAMSat_20170113.nc
-
STREAMSat_20170114.nc
-
STREAMSat_20170115.nc
-
STREAMSat_20170116.nc
-
STREAMSat_20170117.nc
-
STREAMSat_20170118.nc
-
STREAMSat_20170119.nc
-
STREAMSat_20170120.nc
-
STREAMSat_20170121.nc
-
STREAMSat_20170122.nc
-
STREAMSat_20170123.nc
-
STREAMSat_20170124.nc
-
STREAMSat_20170125.nc
-
STREAMSat_20170126.nc
-
STREAMSat_20170127.nc
-
STREAMSat_20170128.nc
-
STREAMSat_20170129.nc
-
STREAMSat_20170130.nc
-
STREAMSat_20170131.nc
-
STREAMSat_20170201.nc
-
STREAMSat_20170202.nc
-
STREAMSat_20170203.nc
-
STREAMSat_20170204.nc
-
STREAMSat_20170205.nc
-
STREAMSat_20170206.nc
-
STREAMSat_20170207.nc
-
STREAMSat_20170208.nc
-
STREAMSat_20170209.nc
-
STREAMSat_20170210.nc
-
STREAMSat_20170211.nc
-
STREAMSat_20170212.nc
-
STREAMSat_20170213.nc
-
STREAMSat_20170214.nc
-
STREAMSat_20170215.nc
-
STREAMSat_20170216.nc
-
STREAMSat_20170217.nc
-
STREAMSat_20170218.nc
-
STREAMSat_20170219.nc
-
STREAMSat_20170220.nc
-
STREAMSat_20170221.nc
-
STREAMSat_20170222.nc
-
STREAMSat_20170223.nc
-
STREAMSat_20170224.nc
-
STREAMSat_20170225.nc
-
STREAMSat_20170226.nc
-
STREAMSat_20170227.nc
-
STREAMSat_20170228.nc
-
STREAMSat_20170301.nc
-
STREAMSat_20170302.nc
-
STREAMSat_20170303.nc
-
STREAMSat_20170304.nc
-
STREAMSat_20170305.nc
-
STREAMSat_20170306.nc
-
STREAMSat_20170307.nc
-
STREAMSat_20170308.nc
-
STREAMSat_20170309.nc
-
STREAMSat_20170310.nc
-
STREAMSat_20170311.nc
-
STREAMSat_20170312.nc
-
STREAMSat_20170313.nc
-
STREAMSat_20170314.nc
-
STREAMSat_20170315.nc
-
STREAMSat_20170316.nc
-
STREAMSat_20170317.nc
-
STREAMSat_20170318.nc
-
STREAMSat_20170319.nc
-
STREAMSat_20170320.nc
-
STREAMSat_20170321.nc
-
STREAMSat_20170322.nc
-
STREAMSat_20170323.nc
-
STREAMSat_20170324.nc
-
STREAMSat_20170325.nc
-
STREAMSat_20170326.nc
-
STREAMSat_20170327.nc
-
STREAMSat_20170328.nc
-
STREAMSat_20170329.nc
-
STREAMSat_20170330.nc
-
STREAMSat_20170331.nc
-
STREAMSat_20170401.nc
-
STREAMSat_20170402.nc
-
STREAMSat_20170403.nc
-
STREAMSat_20170404.nc
-
STREAMSat_20170405.nc
-
STREAMSat_20170406.nc
-
STREAMSat_20170407.nc
-
STREAMSat_20170408.nc
-
STREAMSat_20170409.nc
-
STREAMSat_20170410.nc
-
STREAMSat_20170411.nc
-
STREAMSat_20170412.nc
-
STREAMSat_20170413.nc
-
STREAMSat_20170414.nc
-
STREAMSat_20170415.nc
-
STREAMSat_20170416.nc
-
STREAMSat_20170417.nc
-
STREAMSat_20170418.nc
-
STREAMSat_20170419.nc
-
STREAMSat_20170420.nc
-
STREAMSat_20170421.nc
-
STREAMSat_20170422.nc
-
STREAMSat_20170423.nc
-
STREAMSat_20170424.nc
-
STREAMSat_20170425.nc
-
STREAMSat_20170426.nc
-
STREAMSat_20170427.nc
-
STREAMSat_20170428.nc
-
STREAMSat_20170429.nc
-
STREAMSat_20170430.nc
-
STREAMSat_20170501.nc
-
STREAMSat_20170502.nc
-
STREAMSat_20170503.nc
-
STREAMSat_20170504.nc
-
STREAMSat_20170505.nc
-
STREAMSat_20170506.nc
-
STREAMSat_20170507.nc
-
STREAMSat_20170508.nc
-
STREAMSat_20170509.nc
-
STREAMSat_20170510.nc
-
STREAMSat_20170511.nc
-
STREAMSat_20170512.nc
-
STREAMSat_20170513.nc
-
STREAMSat_20170514.nc
-
STREAMSat_20170515.nc
-
STREAMSat_20170516.nc
-
STREAMSat_20170517.nc
-
STREAMSat_20170518.nc
-
STREAMSat_20170519.nc
-
STREAMSat_20170520.nc
-
STREAMSat_20170521.nc
-
STREAMSat_20170522.nc
-
STREAMSat_20170523.nc
-
STREAMSat_20170524.nc
-
STREAMSat_20170525.nc
-
STREAMSat_20170526.nc
-
STREAMSat_20170527.nc
-
STREAMSat_20170528.nc
-
STREAMSat_20170529.nc
-
STREAMSat_20170530.nc
-
STREAMSat_20170531.nc
-
STREAMSat_20170601.nc
-
STREAMSat_20170602.nc
-
STREAMSat_20170603.nc
-
STREAMSat_20170604.nc
-
STREAMSat_20170605.nc
-
STREAMSat_20170606.nc
-
STREAMSat_20170607.nc
-
STREAMSat_20170608.nc
-
STREAMSat_20170609.nc
-
STREAMSat_20170610.nc
-
STREAMSat_20170611.nc
-
STREAMSat_20170612.nc
-
STREAMSat_20170613.nc
-
STREAMSat_20170614.nc
-
STREAMSat_20170615.nc
-
STREAMSat_20170616.nc
-
STREAMSat_20170617.nc
-
STREAMSat_20170618.nc
-
STREAMSat_20170619.nc
-
STREAMSat_20170620.nc
-
STREAMSat_20170621.nc
-
STREAMSat_20170622.nc
-
STREAMSat_20170623.nc
-
STREAMSat_20170624.nc
-
STREAMSat_20170625.nc
-
STREAMSat_20170626.nc
-
STREAMSat_20170627.nc
-
STREAMSat_20170628.nc
-
STREAMSat_20170629.nc
-
STREAMSat_20170630.nc
-
STREAMSat_20170701.nc
-
STREAMSat_20170702.nc
-
STREAMSat_20170703.nc
-
STREAMSat_20170704.nc
-
STREAMSat_20170705.nc
-
STREAMSat_20170706.nc
-
STREAMSat_20170707.nc
-
STREAMSat_20170708.nc
-
STREAMSat_20170709.nc
-
STREAMSat_20170710.nc
-
STREAMSat_20170711.nc
-
STREAMSat_20170712.nc
-
STREAMSat_20170713.nc
-
STREAMSat_20170714.nc
-
STREAMSat_20170715.nc
-
STREAMSat_20170716.nc
-
STREAMSat_20170717.nc
-
STREAMSat_20170718.nc
-
STREAMSat_20170719.nc
-
STREAMSat_20170720.nc
-
STREAMSat_20170721.nc
-
STREAMSat_20170722.nc
-
STREAMSat_20170723.nc
-
STREAMSat_20170724.nc
-
STREAMSat_20170725.nc
-
STREAMSat_20170726.nc
-
STREAMSat_20170727.nc
-
STREAMSat_20170728.nc
-
STREAMSat_20170729.nc
-
STREAMSat_20170730.nc
-
STREAMSat_20170731.nc
-
STREAMSat_20170801.nc
-
STREAMSat_20170802.nc
-
STREAMSat_20170803.nc
-
STREAMSat_20170804.nc
-
STREAMSat_20170805.nc
-
STREAMSat_20170806.nc
-
STREAMSat_20170807.nc
-
STREAMSat_20170808.nc
-
STREAMSat_20170809.nc
-
STREAMSat_20170810.nc
-
STREAMSat_20170811.nc
-
STREAMSat_20170812.nc
-
STREAMSat_20170813.nc
-
STREAMSat_20170814.nc
-
STREAMSat_20170815.nc
-
STREAMSat_20170816.nc
-
STREAMSat_20170817.nc
-
STREAMSat_20170818.nc
-
STREAMSat_20170819.nc
-
STREAMSat_20170820.nc
-
STREAMSat_20170821.nc
-
STREAMSat_20170822.nc
-
STREAMSat_20170823.nc
-
STREAMSat_20170824.nc
-
STREAMSat_20170825.nc
-
STREAMSat_20170826.nc
-
STREAMSat_20170827.nc
-
STREAMSat_20170828.nc
-
STREAMSat_20170829.nc
-
STREAMSat_20170830.nc
-
STREAMSat_20170831.nc
-
STREAMSat_20170901.nc
-
STREAMSat_20170902.nc
-
STREAMSat_20170903.nc
-
STREAMSat_20170904.nc
-
STREAMSat_20170905.nc
-
STREAMSat_20170906.nc
-
STREAMSat_20170907.nc
-
STREAMSat_20170908.nc
-
STREAMSat_20170909.nc
-
STREAMSat_20170910.nc
-
STREAMSat_20170911.nc
-
STREAMSat_20170912.nc
-
STREAMSat_20170913.nc
-
STREAMSat_20170914.nc
-
STREAMSat_20170915.nc
-
STREAMSat_20170916.nc
-
STREAMSat_20170917.nc
-
STREAMSat_20170918.nc
-
STREAMSat_20170919.nc
-
STREAMSat_20170920.nc
-
STREAMSat_20170921.nc
-
STREAMSat_20170922.nc
-
STREAMSat_20170923.nc
-
STREAMSat_20170924.nc
-
STREAMSat_20170925.nc
-
STREAMSat_20170926.nc
-
STREAMSat_20170927.nc
-
STREAMSat_20170928.nc
-
STREAMSat_20170929.nc
-
STREAMSat_20170930.nc
-
STREAMSat_20171001.nc
-
STREAMSat_20171002.nc
-
STREAMSat_20171003.nc
-
STREAMSat_20171004.nc
-
STREAMSat_20171005.nc
-
STREAMSat_20171006.nc
-
STREAMSat_20171007.nc
-
STREAMSat_20171008.nc
-
STREAMSat_20171009.nc
-
STREAMSat_20171010.nc
-
STREAMSat_20171011.nc
-
STREAMSat_20171012.nc
-
STREAMSat_20171013.nc
-
STREAMSat_20171014.nc
-
STREAMSat_20171015.nc
-
STREAMSat_20171016.nc
-
STREAMSat_20171017.nc
-
STREAMSat_20171018.nc
-
STREAMSat_20171019.nc
-
STREAMSat_20171020.nc
-
STREAMSat_20171021.nc
-
STREAMSat_20171022.nc
-
STREAMSat_20171023.nc
-
STREAMSat_20171024.nc
-
STREAMSat_20171025.nc
-
STREAMSat_20171026.nc
-
STREAMSat_20171027.nc
-
STREAMSat_20171028.nc
-
STREAMSat_20171029.nc
-
STREAMSat_20171030.nc
-
STREAMSat_20171031.nc
-
STREAMSat_20171101.nc
-
STREAMSat_20171102.nc
-
STREAMSat_20171103.nc
-
STREAMSat_20171104.nc
-
STREAMSat_20171105.nc
-
STREAMSat_20171106.nc
-
STREAMSat_20171107.nc
-
STREAMSat_20171108.nc
-
STREAMSat_20171109.nc
-
STREAMSat_20171110.nc
-
STREAMSat_20171111.nc
-
STREAMSat_20171112.nc
-
STREAMSat_20171113.nc
-
STREAMSat_20171114.nc
-
STREAMSat_20171115.nc
-
STREAMSat_20171116.nc
-
STREAMSat_20171117.nc
-
STREAMSat_20171118.nc
-
STREAMSat_20171119.nc
-
STREAMSat_20171120.nc
-
STREAMSat_20171121.nc
-
STREAMSat_20171122.nc
-
STREAMSat_20171123.nc
-
STREAMSat_20171124.nc
-
STREAMSat_20171125.nc
-
STREAMSat_20171126.nc
-
STREAMSat_20171127.nc
-
STREAMSat_20171128.nc
-
STREAMSat_20171129.nc
-
STREAMSat_20171130.nc
-
STREAMSat_20171201.nc
-
STREAMSat_20171202.nc
-
STREAMSat_20171203.nc
-
STREAMSat_20171204.nc
-
STREAMSat_20171205.nc
-
STREAMSat_20171206.nc
-
STREAMSat_20171207.nc
-
STREAMSat_20171208.nc
-
STREAMSat_20171209.nc
-
STREAMSat_20171210.nc
-
STREAMSat_20171211.nc
-
STREAMSat_20171212.nc
-
STREAMSat_20171213.nc
-
STREAMSat_20171214.nc
-
STREAMSat_20171215.nc
-
STREAMSat_20171216.nc
-
STREAMSat_20171217.nc
-
STREAMSat_20171218.nc
-
STREAMSat_20171219.nc
-
STREAMSat_20171220.nc
-
STREAMSat_20171221.nc
-
STREAMSat_20171222.nc
-
STREAMSat_20171223.nc
-
STREAMSat_20171224.nc
-
STREAMSat_20171225.nc
-
STREAMSat_20171226.nc
-
STREAMSat_20171227.nc
-
STREAMSat_20171228.nc
-
STREAMSat_20171229.nc
-
STREAMSat_20171230.nc
-
STREAMSat_20171231.nc
Abstract
Satellite-based precipitation observations provide near-global coverage with high spatiotemporal resolution in near-realtime. Their utility, however, is hindered by oftentimes large errors that vary substantially in space and time. Since precipitation uncertainty is, by definition, a random process, probabilistic expression of satellite-based precipitation product uncertainty is needed to advance their operational applications. Ensemble methods, in which uncertainty is depicted via multiple realizations of precipitation fields, have been widely used in other contexts such as numerical weather prediction, but rarely in satellite contexts. Creating such an ensemble dataset is challenging due to the complexity of errors and the scarcity of “ground truth” to characterize it. This challenge is particularly pronounced in ungauged regions, where the benefits of satellite-based precipitation data could otherwise provide substantial benefits. In this study, we propose the first quasi-global (covering all continental land masses within 50°N-50°S) satellite-only ensemble precipitation dataset, derived entirely from NASA’s Integrated Multi-SatellitE Retrievals for Global Precipitation Measurement (IMERG) and GPM’s radar-radiometer combined precipitation product (2B-CMB). No ground-based measurements are used in this generation and it is suitable for near-realtime use, limited only by the latency of IMERG. We compare the results against several precipitation datasets of distinct classes, including global satellite-based, rain gauge-based, atmospheric reanalysis, and merged products. While our proposed approach faces some limitations and is not universally superior to the datasets it is compared to in all respects, it does hold relative advantages due to its combination of accuracy, resolution, latency, and utility in hydrologic and hazard applications.
README: STREAM-Sat: A novel near-realtime quasi-global satellite-only ensemble precipitation
Kaidi Peng et al.
This repository is to create near-realtime global precipitation ensembles that condition on satellite observations (e.g., IMERG: Integrated Multi-satellitE Retrievals for GPM; https://gpm.nasa.gov/data/imerg). We unified the methods proposed in Li et al. (2023) doi:10.1109/tgrs.2023.3235270 and Hartke et al. (2022) https://doi.org/10.1029/2021WR031650. We tried to solve the challenge of Near-Realtime (NRT) global precipitation generation due to the lack of ground-based gauge network and the complex error of satellite precipitation.
The highlights of this method are
1. no ground-based measurement is needed. The performance (e.g., ensemble spread and accuracy) is independent of gauge density;
2. It could be generated in Near-Realtime, which means its time latency is only affected by satellite products (e.g., IMERG Early has 4-hour latency);
3. It can be done globally, while the performance will be affected by satellite retrieval accuracy over different regions.
The inputs of STREAM-Sat are
1. a gridded input precipitation dataset, such as IMERG Early;
2. an error model for that input dataset at individual grid cell scale (CSGD in this example);
3. motion vector: MERRA2 U850/V850 or IMERG's motion vector;
4. covariates (optional): It depends on the covariates you used in the error model (WAR in this example);
The output is a user-defined number of precipitation and noise ensemble (20 in this example).
The performance of the ensemble can be evaluated by CR_calc.py and CRPS_calc.py.
Details about CSGD are on https://github.com/KaidiWisc/CSGD_error_model.git
Description of the data and file structure
The period of the dataset is 2017 (more can be produced upon request). The coverage is land area within 50°N-50°S. The time resolution is half an hour. The spatial resolution is 0.1 deg. The name of each file is "STREAMSat_YYYYMMDD.nc". "YYYYMMDD" is the corresponding year/month/day of the data. The data format is Netcdf4 with dimensions time, latitude, and longitude. Each file has 48 indices over time dimension, the value of which is the start time of each half hour. Latitude and longitude are the center of each grid cell.
For any questions, feel free to contact Kaidi Peng (kaidi.peng@wisc.edu).
Methods
This repository is to create near-realtime global precipitation ensembles that condition on satellite observations (e.g., IMERG: Integrated Multi-satellitE Retrievals for GPM; https://gpm.nasa.gov/data/imerg). We unified the methods proposed in Li et al. (2023) doi:10.1109/tgrs.2023.3235270 and Hartke et al. (2022) https://doi.org/10.1029/2021WR031650. We tried to solve the challenge of Near-Realtime (NRT) global precipitation generation due to the lack of ground-based gauge network and the complex error of satellite precipitation.
The highlights of this method are
-
no ground-based measurement is needed. The performance (e.g., ensemble spread and accuracy) is independent of gauge density;
-
It could be generated in Near-Realtime, which means its time latency is only affected by satellite products (e.g., IMERG Early has 4-hour latency);
-
It can be done globally, while the performance will be affected by satellite retrieval accuracy over different regions.
The inputs of STREAM-Sat are
-
a gridded input precipitation dataset, such as IMERG Early;
-
an error model for that input dataset at individual grid cell scale (CSGD in this example);
-
motion vector: MERRA2 U850/V850 or IMERG's motion vector;
-
covariates (optional): It depends on the covariates you used in the error model (WAR in this example);
The output is a user-defined number of precipitation and noise ensemble (20 in this example).
Details about CSGD are on https://github.com/KaidiWisc/CSGD_error_model.git