Turkmenistan methane point source detections and retrievals from Landsat 5 (1986-2011)

He, Tai-Long1 ; Boyd, Ryan2 ; Varon, Daniel3 ; Turner, Alexander1

Published Feb 02, 2024 on Dryad. https://doi.org/10.5061/dryad.4mw6m90hp

Data files

Feb 02, 2024 version files 9.90 GB

Abstract

Global atmospheric methane concentrations rose by 10-15 ppb/yr in the 1980s before abruptly slowing to 2-8 ppb/yr in the early 1990s. This period in the 1990s is known as the "methane slowdown" and has been attributed to the collapse of the former Soviet Union (USSR) in December 1991, which may have decreased the methane emissions from oil and gas operations. Here we develop a methane plume detection system based on probabilistic deep learning and human-labelled training data. We use this method to detect methane plumes from Landsat 5 satellite observations over Turkmenistan from 1986 to 2011. We focus on Turkmenistan because economic data suggest it could account for half of the decline in oil and gas emissions from the former USSR. We find an increase in both the frequency of methane plume detections and the magnitude of methane emissions following the collapse of the USSR. We estimate a national loss rate from oil and gas infrastructure in Turkmenistan of more than 10% at times, which suggests the socioeconomic turmoil led to a lack of oversight and widespread infrastructure failure in the oil and gas sector. Our finding of increased oil and gas methane emissions from Turkmenistan following the USSR's collapse casts doubt on the long-standing hypothesis regarding the methane slowdown, begging the question: "what drove the 1992 methane slowdown?"

README: Turkmenistan methane point source detections and retrievals from Landsat 5 (1986-2011)

https://doi.org/10.5061/dryad.4mw6m90hp

This dataset contains replication data for our paper entitled "Space-borne assessment of the Soviet Union collapse on the methane slowdown".

Usage notes

To open and visualize the data files in augmented_labelled_data.tar.gz, a Python environment with the NumPy and PyTorch libraries is required. Files archived in retrieval_and_masks.tar.gz could be opened and visualized using NASA's Panoply data viewer.

Description of the data and file structure

1) augmented_labelled_data.tar.gz contains human-labelled plume masks and the corresponding Landsat 5 measurements.

The package includes both positive (with plume) and negative (without plume) satellite scenes. The methane plumes were labeled by the two leading authors of this dataset. Four folders are archived:
- 'neg_labeller_1' contains 2326 negative satellite scenes labelled by labeler 1 in '.npz' format.
- 'neg_labeller_2' contains 2505 negative satellite scenes labelled by labeler 2 in '.npz' format.
- 'pos_labeller_1' contains 1864 positive satellite scenes labelled by labeler 1 in '.npz' format.
- 'pos_labeller_2' contains 1449 positive satellite scenes labelled by labeler 2 in '.npz' format.
The name convention for '.npz' files is "Satellite_Longitude_Latitude_Date_Localtime_MonthOverpassNum_AugmentID.npz". Satellites include Landsat 7 (L7), Landsat 8 (L8), and Sentinel-2 (S2). MonthOverpassNum stands for the number of satellite overpass for this location in the month. Filenames with an AugmentID are datasets augmented following the method described in the paper. Filenames without an AugmentID are raw data sets without any augmentation.
The '.npz' files could be loaded using the NumPy Python library use these commands:
```
import numpy as np
npzfile = np.load(‘file.npz’)
npzfile.files
```
The commands will return a list of the NumPy arrays, which can be individually opened by:
```
data = npzfile['variable'][:]
```
Four variables are archived in each '.npz' file: 'pred', 'mask', 'lon', and 'lat'.
The 'pred' variable contains 13 128x128 images sampled by the satellites. The order of the 13 images is the dR channel, the red channel, the green channel, the blue channel, the near infrared (NIR) channel, the first shortwave infrared (SWIR) channel, the second SWIR channel, the estimated cloud percent, the background (bg) scene for dR, the grayscale RGB, the normalized difference vegetation index (NDVI), and two difference fields between dR and dRbg. More details could be found in the paper mentioned above.
The 'mask' variable is 128x128 binary images, where "1" stands for plume and "0" stands for no plume.
'lon' and 'lat' stand for the longitude and latitude information, respectively.
All variables are NumPy arrays.

2) retrieval_and_masks.tar.gz contains the methane retrievals and the predicted methane plume masks from the Landsat 5 satellite.

One folder is archived:
- 'retrieval_and_masks' contains 688 '.nc' files for the retrieved methane columns and the predicted plume masks.
Filename convention is "Date_Longitude_Latitude.nc". The '.nc' files are netCDF4 files that could be directly opened and visualized using NASA's Panoply data viewer.
Four variables are listed:
- the retrieved CH4 columns (variable name 'ch4') in units of 'mol/m2',
- the estimated background CH4 columns (variable name 'ch4bg') in units of 'mol/m2',
- the binary plume masks (variable name 'plume_mask') predicted by the deep learning models,
- and the date and time information (variable name 'time') as days since 1950-01-01.

3) L5_detections.csv is a comma separated list file containing the methane point source detected over Turkmenistan from Landsat 5. The file could be opened and viewed directly using Excel, LibreOffice Calc, etc. Reported are location, date and time, estimated flux rates, and the uncertainties of flux rates associated with the detections.

Five variables are listed: Longitude, Latitude, Datetime, Q and dQ.
Datetime is in format of MM/DD/YYYY.
Q is the estimated flux rates of the detected methane point sources, reported in units of 'kg/hr'.
dQ is the uncertainties associated with the estimated flux rates, reported in units of 'kg/hr'.

Sharing/Access information

The top-of-atmosphere (TOA) reflectance measurements from Landsat 5, Landsat 8, and Sentinel-2 are processed from Google Earth Engine (GEE, https://developers.google.com/earth-engine/datasets).

The codebase to reproduce results in the paper is available at https://github.com/tailonghe/L5_methane_detection.
The methane plume mask annotation tool is available at https://github.com/tailonghe/methane_labeller.