Data for: Motion adaptive deblurring with single photon cameras

Seets, Trevor1 ; Ingle, Atul1; Laurenzis, Martin2; Velten, Andreas1

Published Apr 08, 2024 on Dryad. https://doi.org/10.5061/dryad.xpnvx0kpc

Data files

Apr 08, 2024 version files 143.65 MB

Abstract

Single-photon avalanche diodes (SPADs) are a rapidly developing image sensing technology with extreme lowlight sensitivity and picosecond timing resolution. These unique capabilities have enabled SPADs to be used in applications like LiDAR, non-line-of-sight imaging and fluorescence microscopy that require imaging in photon-starved scenarios. In this work we harness these capabilities for dealing with motion blur in a passive imaging setting in low illumination conditions. Our key insight is that the data captured by a SPAD array camera can be represented as a 3D spatio-temporal tensor of photon detection events which can be integrated along arbitrary spatio-temporal trajectories with dynamically varying integration windows, depending on scene motion. We propose an algorithm that estimates pixel motion from photon timestamp data and dynamically adapts the integration windows to minimize motion blur. Our simulation results show the applicability of this algorithm to a variety of motion profiles including translation, rotation and local object motion. We also demonstrate the real-world feasibility of our method on data captured using a 32 × 32 SPAD camera.

README: Data for: Motion adaptive deblurring with single photon cameras

https://doi.org/10.5061/dryad.xpnvx0kpc

Description of the data and file structure

This dataset contains the data for Motion Adaptive Deblurring with Single Photon Cameras. It contains raw data from a InGaAs 32x32 Single Photon Avalanche Diode (SPAD) array captured under ambient light. SPAD cameras are able to detect and time tag individual photons. This data comes from a frame readout SPAD which represents temporal data on a coarse and fine time scale. The coarse time scale is similar to a frame of video and is called a "photon frame" whereas the fine time scale represents the arrival time of photons within each photon frame. This dataset is the raw SPAD video capture of a spinning fan captured at 50k FPS.

The data represents a 100000 x 32 x 32 16-bit array. The last two dimensions represent the 32 x 32 pixels of the camera. The first dimension represents the index of a photon frame or the coarse time dimension, each photon frame represents a 2us window. Within each photon frame, for each pixel, up to 1 photon is detected and the photon's time of arrival is time tagged relative to the start of the photon frame. Each value in the array represents the time of a arrival of a photon within the photon frame window, with timing resolution of 256ps, so a value of 4 represents a photon detected 4*256ps = 1024ps after the start of the photon frame. Any value above 8000 indicates no photon was detected in that pixel during that photon frame.

Code/Software

The linked code is available to process the data and demonstrate reading this datafile in python.

This data must be read as a 16 bit binary file with a shape of [100000,32,32]. This can be done with python with the following code snipit. Data will appear very noisy without any processing.

import numpy as np

NFRAMES = 100000
XPIX = 32
YPIX = 32
PIXELS_PER_FRAME = XPIX*YPIX

def load_data(filename, nframes, nx, ny):
    """read a binary file generated by the 32x32 GM-APD array
    """

    mat = np.zeros((nframes, nx, ny))
    with open(filename, 'rb') as f:
        for i in range(nframes):
            cur_frame = np.fromfile(f, dtype=np.uint16, count=1056)
            cur_frame[cur_frame>8000] = 8001
            npix = nx*ny
            mat[i,:,:] = cur_frame[0:npix].reshape(nx,ny)
    return mat

fanData =load_data("20170503094851_pTOP_fan_move_high.bin", NFRAMES, XPIX, YPIX)