Single-Shot Efficient Depth Imaging Based on Time-Compressive CMOS Image Sensor

Abstract

This article introduces a novel framework that effectively resolves multipath interference (MPI) in time-of-flight (ToF) depth measurements by utilizing a single-shot time-compressive complementary metal–oxide– semiconductor (CMOS) image sensor, a U-Net-based neural network, and joint end-to-end optimization. Conventional indirect ToF (iToF) methods inevitably suffer from MPI artifacts because each detection tap integrates incoming photons over a wide exposure window, making it difficult to isolate the direct reflection component from multiple reflections or scattering. Our approach, termed coded ToF (cToF), builds upon the iToF principle but employs coded exposure patterns to capture the scene response in a single shot. This time-compressive acquisition allows for the effective separation of MPI components, achieving direct ToF (dToF)- like high accuracy using a low-cost iToF-based sensor. Furthermore, the reconstruction network and the coded shutter are jointly optimized in an end-to-end framework to maximize depth estimation accuracy while explicitly considering shutter control constraints.

Publication
IEEE Sensors Journal
Hajime Nagahara
Hajime Nagahara
Professor

He is working on computer vision and pattern recognition. His main research interests lie in image/video recognition and understanding, as well as applications of natural language processing techniques.