Dataset DOI: 10.5061/dryad.j6q573nwc

Description of the data and file structure

This dataset contains the data, atomic models, configuration files, and scripts required to reproduce the results presented in the paper Cryo-Electron Microscopy Structural Ensemble Optimization Using Individual Particles. Reproducing the results requires installing the cryoJAX Ensemble Optimization (cryojax_eo) Python library (see below), which is released under the MIT license. Some results may only be reproducible with version 0.2.3, as the API might change in the future. Several files in the dataset include .yaml files used as input for the CLI scripts available with cryojax_eo.

Files and variables

This deposit consists of 5 compressed archives (.zip). Each archive corresponds to one of the datasets used in the paper. Decompress each archive before use.

File: rectangle_results.zip

This file contains a script to simulate data for a 4-atom rectangular system, as well as a Jupyter notebook to run the ensemble optimization and produce the plots and statistics shown in the publication.

Folder Structure Overview:

rectangle/
├── README.md
├── config_simulation.yaml
├── rectangle_toy_model.ipynb
├── dataset/
├── outputs/
├── plots/
└── scripts/

README.md

This README provides instructions for running the data generation script and installing the dependencies required to reproduce the results.

config_simulation.yaml

A YAML file containing the parameters used to simulate the cryo-EM data for the rectangle system.

rectangle_toy_model.ipynb

Jupyter notebook that loads the simulated data and runs the Ensemble Optimization method for the rectangle toy model.

dataset/

Simulated data used to obtain the results presented in the paper.

outputs/

Output trajectories obtained by the Ensemble Optimization method for each setting in .npy format.

plots/

Plots of the results for each setting, matching the results shown in Figure 2 of the paper.

scripts/

Code utilities for simulating data and rectangle-specific functions necessary to run the Ensemble Optimization method.

File: alanine_results.zip

This file provides atomic structures and scripts to simulate cryo-EM images of tri-alanine in two different conformations and to run the ensemble optimization pipeline. The simulated images and trajectories used for the results in the publication are also included.

Folder Structure Overview:

alanine/
├── README.md
├── config_simulation.yaml
├── config_optimization.yaml
├── config_optimization_noprior.yaml
├── analysis.ipynb
├── ensemble_optimization_no_prior.py
├── nll_vs_alanine.png
├── rmsd_walkers.png
├── atomic_models/
├── dataset/
└── optimization_results/

README.md

This README provides instructions for running the data generation script and installing the dependencies required to reproduce the results.

config_simulation.yaml

A YAML file containing the parameters used to simulate the cryo-EM data for the tri-alanine system.

config_optimization.yaml

A YAML file containing the parameters used to run the cryo-EM Ensemble Optimization method for the tri-alanine system.

config_optimization_noprior.yaml

A YAML file containing the parameters used to run the cryo-EM Ensemble Optimization method for the tri-alanine system when no physical prior is used (i.e., no molecular dynamics).

analysis.ipynb

Jupyter notebook for analyzing the cryo-EM Ensemble Optimization results and generating the plots used to construct Figure 3 in the paper.

ensemble_optimization_no_prior.py

Script and utilities for running the Ensemble Optimization method for a system without a physical prior (custom classes and functions).

nll_vs_alanine.png and rmsd_walkers.png

Subfigures used to make Figure 3 in the paper.

atomic_models/

Atomic models used to simulate cryo-EM data and initialize the cryo-EM Ensemble Optimization method.

dataset/

Simulated data used to obtain the results presented in the paper.

optimization_results/

Output trajectories, ensemble weights, and atomic structures obtained by the Ensemble Optimization method for each setting. In addition, this contains a copy of the parameters used to obtain the results. Trajectories are provided in .xtc format, and atomic structures in .pdb format. Ensemble weights and atomic coordinates are provided in .npz format.

File: groel_results.zip

This file provides atomic structures and scripts to simulate cryo-EM images of chain A of GroEL in two different conformations and to run the ensemble optimization pipeline. The simulated images and trajectories used for the results in the publication are also included. This file contains both homogeneous and heterogeneous datasets under different conditions.

Folder Structure Overview:

groel/
├── README.md
├── atomic_models/
├── batch_size_benchmark/
├── heterogeneity_benchmark/
├── opt_steps_benchmark/
├── snr_benchmark/
└── steered_benchmark/

README.md

This README provides instructions for running the data generation script and installing the dependencies required to reproduce the results.

atomic_models/

Atomic models used to simulate cryo-EM data and to initialize the cryo-EM Ensemble Optimization method. The models are in .pdb format and consist of chain A of GroEL in the apo and holo states.

Every other subdirectory in this file follows a structure similar to that of alanine_results.zip. For brevity, we describe the common structure here and explain the purpose of each subdirectory below. Each subdirectory contains one or more config_optimization_*.yaml files specifying the settings used to run the Ensemble Optimization method. The file names reflect the setting being varied; for example, in the SNR benchmark, the suffixes _snr10, _snr1, _snr01, and _snr001 indicate the signal-to-noise ratio of the dataset used. Each subdirectory also contains an analysis.ipynb notebook used to produce the figures for the paper (typically in .png or .svg format) and an optimization_results/ directory containing output trajectories (.xtc), atomic structures (.pdb), ensemble weights and coordinates (.npz), and copies of the parameters used (.yaml).

batch_size_benchmark/

This subdirectory contains results for running the cryo-EM Ensemble Optimization method with different random batch sizes in the stochastic gradient descent and reweighting steps.

heterogeneity_benchmark/

This subdirectory contains results for running the cryo-EM Ensemble Optimization method using different numbers of walkers.

opt_steps_benchmark/

This subdirectory contains results for running the cryo-EM Ensemble Optimization method for different numbers of unconstrained optimization steps.

snr_benchmark/

This subdirectory contains results for running the cryo-EM Ensemble Optimization method on datasets simulated at different signal-to-noise ratios. The config_optimization_*.yaml files use the suffixes _snr10, _snr1, _snr01, and _snr001 to indicate the signal-to-noise ratio of the dataset.

steered_benchmark/

This subdirectory contains results for running the cryo-EM Ensemble Optimization method while varying the strength of the steering potential in the Steered Molecular Dynamics simulations.

File: hsp90_results.zip

This file provides atomic structures and picked particles from EMPIAR 11028 for the Hsp90–p23 subset. Scripts to run the ensemble optimization pipeline from a distant initial structure are also available. The trajectories used for the results in the publication are also included.

Folder Structure Overview:

hsp90/
├── README.md
├── analysis.ipynb
├── analysis_multiM.ipynb
├── config_optimization_1walker.yaml
├── config_optimization_4walkers.yaml
├── initial_state.xml
├── noh_noloops_indices.txt
├── run_job.job
├── atomic_models_and_volumes/
├── dataset/
├── optimization_results_1walker/
└── optimization_results_4walkers/

README.md

This README provides instructions for running the data generation script and installing the dependencies required to reproduce the results.

analysis.ipynb and analysis_multiM.ipynb

For this benchmark, we ran two experiments: one with 1 walker and one with 4 walkers. These notebooks load the results for each case, respectively, and generate the plots and other analysis metrics reported as results in the paper.

config_optimization_1walker.yaml and config_optimization_4walkers.yaml

Configuration files containing the parameters used to run the cryo-EM Ensemble Optimization method.

initial_state.xml

Initial state file in OpenMM format. Contains velocities and positions from an equilibrated simulation. Useful for reducing computation time and producing reproducible results.

noh_noloops_indices.txt

We prepared the .pdb files for MD simulation using PDBFixer, which adds long disordered loops to the Hsp90 atomic structure. These loops are too dynamic to appear in the cryo-EM images, so we do not include them when comparing with images from the cryo-EM Ensemble Optimization pipeline. The indices denote which atom indices should be filtered.

run_job.job

Slurm script file for running the method on an HPC cluster.

atomic_models_and_volumes/

Atomic models used to initialize the cryo-EM Ensemble Optimization method, in .pdb format. This directory also includes the volumes produced by the ab initio reconstruction and refinement procedure used to estimate the pose parameters (the consensus volume and the halfmaps), as well as a volumetric mask that filters out the p23 region (this complex is ignored for benchmarking purposes).

dataset/

Picked particles from the EMPIAR 11028 dataset, downsampled to a 128-pixel box size. Particles were picked using the coordinates provided in the dataset and restricted to those associated with Hsp90–p23.

optimization_results_1walker/ and optimization_results_4walkers/

Trajectories (.xtc), atomic structures (.pdb), ensemble weights and coordinates (.npz) obtained for each setting.

File: spike_results.zip

This file provides atomic structures and picked particles for the N501Y variant of the SARS-CoV-2 spike ectodomain in the open (1-up) and closed (3-down states. Scripts to run the ensemble optimization pipeline from different initial conformations are also available. The trajectories used for the results in the publication are also included.

Folder Structure Overview:

spike/
├── README.md
├── analysis.ipynb
├── config_optimization_from_closed.yaml
├── config_optimization_from_open.yaml
├── rbd_com_displacement.png
├── rmsd_to_initial.png
├── run_job_closed.job
├── run_job_open.job
├── atomic_models/
├── downsampled_dataset/
├── original_dataset/
├── optimization_results_from_closed/
├── optimization_results_from_open/
└── rbd_info/

README.md

This README provides instructions for running the data generation script and installing the dependencies required to reproduce the results.

analysis.ipynb

For this benchmark, we ran two experiments starting from different states of the SARS-CoV-2 Spike: one from the 3-down state and one from the 1-up state. This notebook loads the results for each case and generates the plots and other analysis metrics reported as results in the paper.

config_optimization_from_closed.yaml and config_optimization_from_open.yaml

Configuration files containing the parameters used to run the cryo-EM Ensemble Optimization method.

rbd_com_displacement.png and rmsd_to_initial.png

Subfigures used to construct Figure 6 in the paper.

run_job_*.job

Slurm script files for running the method on an HPC cluster.

atomic_models/

Atomic models used to initialize the cryo-EM Ensemble Optimization method. The models are in .pdb format. In addition, .xtc files are provided for each atomic structure, containing OpenMM State information (positions and velocities) from an equilibrated MD simulation. The files correspond to the PDB:7QDG (1-up) and PDB:8P9Y (3-down) states, obtained from the Protein Data Bank.

original_dataset/ and downsampled_dataset/

The original picked particles used to obtain the results in the paper. The downsampled dataset contains particles downsampled to a 128-pixel box size.

optimization_results_from_open/ and optimization_results_from_closed/

Trajectories (.xtc), atomic structures (.pdb), ensemble weights and coordinates (.npz) obtained for each setting.

rbd_info/

Contains the extracted receptor-binding domain (RBD) for each PDB file (.pdb), indices defining which atoms correspond to the RBD for each PDB (.txt), and a volumetric mask that focuses the RBD region (used for focusing the Ensemble Optimization pipeline). In addition, an analysis.ipynb notebook is provided that loads the results and checks various statistics related to the RBD for each simulation.

Code/software

These results were produced with version 0.2.3 of cryojax_eo, a module of the cryoJAX library (JAX- and Equinox-based differentiable cryo-EM forward models). This version of the code requires version 0.5.5 of cryoJAX and version 0.1.2 of cryoSPAX.

Installation instructions, including CPU, GPU (conda/mamba), and Apptainer/Singularity options for HPC clusters, are documented in the code repository. A minimal GPU install:

mamba create -n cryojax_eo_env python==3.11
mamba activate cryojax_eo_env
mamba install -c conda-forge openmm cuda-version==12.4   # match your CUDA version
pip install --upgrade "jax[cuda12]"
pip install cryojax_eo==0.2.3

Note: OpenMM is only required for the Steered Molecular Dynamics projection step. Data simulation and ensemble reweighting require only JAX.

Access information

Other publicly accessible locations of the data:

• This data is also available in Zenodo: https://zenodo.org/records/19224943

Data was derived from the following sources:

• The original Hsp90 dataset (Micrographs) is available EMPIAR. This dataset was initially provided with the publication: Noddings, C. M., Wang, R. Y. R., Johnson, J. L., & Agard, D. A. (2022). Structure of Hsp90–p23–GR reveals the Hsp90 client-remodelling mechanism. Nature, 601(7893), 465-469.