Diffraction-limited optical imaging through scattering media has the potential to transform many applications such as airborne and space-based imaging (through the atmosphere), bio-imaging (through skin and human tissue), and fiber-based-imaging (through fiber bundles). Existing wavefront shaping methods can image through scattering media and other obscurants by optically correcting wavefront aberrations using high-resolution spatial light modulators—but these methods generally require (a) guidestars, (b) controlled illumination, (c) point scanning and/or (d) statics scenes and aberrations. We propose Neural Wavefront Shaping (NeuWS), a scanning-free wavefront shaping technique which integrates maximum likelihood estimation, measurement modulation, and neural signal representations to reconstruct diffraction-limited images through strong static and dynamic scattering media without guidestars, sparse targets, controlled illumination, nor specialized image sensors. We experimentally demonstrate guidestar-free, wide-field-of-view, high-resolution, diffraction-limited imaging of extended, non-sparse, static/dynamic scenes captured through static/dynamic aberrations.

Data:

The NeuWS_data directory contains all the measurements and corresponding SLM patterns of all the experiments shown in the article. The data are divided into four groups based on imaging dynamic/static objects through dynamic/static aberrations. The data from the automatic refocusing experiments is also provided.

In each folder, the Zernike_SLM_data subfolder includes the patterns projected on the SLM (SLM_sim) and the raw measurements captured by the camera (SLM_raw). 

For static scenes and aberrations, the "uncorrected" image is a measurement with aberrations and without SLM modulation. For dynamic scenes and aberrations, "Uncorrected GIF" is generated by combining the images from the corresponding Zero subfolder (uncorrected measurements). The Zero subfolders contain the measurements captured without any SLM modulation; i.e., the uncorrected measurements one would observe through the aberration. 

The "optical_correct" images and GIFS show the measurements captured through an aberration when the conjugate of the estimated aberration was projected on the SLM. For the dynamic experiments, the corresponding raw data is provided in the "optical_correct_frames" folder.

"GT" is the ground truth image captured without any aberrations or SLM modulation.

We generated each modulation pattern by creating a 256 × 256 image made from the weighted sum of the first 15 Zernike basis functions, where each weight was sampled from an independent Gaussian distribution with a standard deviation of 5 radians. These patterns were then evenly cropped on the top and bottom to 144×256 to match our SLM’s aspect ratio. 

The optically corrected measurements (image data in optical_correct_frames directories) are unprocessed 1384 × 1036 images captured by our camera. 

The modulated and uncorrected measurements (image data in Zernike_SLM_data and Zero directories) are 256x256 images. These were formed by first resizing 1280x1026 camera measurements to 387 × 290 and then cropping a 256 × 256 area in the center. The images were resized to reduce the computational cost of running NeuWS.

The data collection process is described in detail in our paper.

Code:

The NeuWS_code directory contains all the code necessary to run NeuWS on the provided data. Please see the included README.md file for information on setting up the code. It assumes you have access to a recent Nvidia GPU. 

For a more complete, up-to-date, and optimized implementation of NeuWS please refer to the NeuWS repository hosted on GitHub.

The SLM patterns are stored as Matlab .mat files. They can be opened with Python's open-source SciPy module. This is handled by "dataset.py" in our code.  

Information on how to set up your Python environment to run NeuWS can be found in the README.md file.