Journal article
N-DEPTH: Neural Depth Encoding for Compression-Resilient 3D Video Streaming
Electronics (Basel), Vol.13(13), 2557
06/29/2024
DOI: 10.3390/electronics13132557
Abstract
Recent advancements in 3D data capture have enabled the real-time acquisition of high-resolution 3D range data, even in mobile devices. However, this type of high bit-depth data remains difficult to efficiently transmit over a standard broadband connection. The most successful techniques for tackling this data problem thus far have been image-based depth encoding schemes that leverage modern image and video codecs. To our knowledge, no published work has directly optimized the end-to-end losses of a depth encoding scheme sandwiched around a lossy image compression codec. We present N-DEPTH, a compression-resilient neural depth encoding method that leverages deep learning to efficiently encode depth maps into 24-bit RGB representations that minimize end-to-end depth reconstruction errors when compressed with JPEG. N-DEPTH’s learned robustness to lossy compression expands to video codecs as well. Compared to an existing state-of-the-art encoding method, N-DEPTH achieves smaller file sizes and lower errors across a large range of compression qualities, in both image (JPEG) and video (H.264) formats. For example, reconstructions from N-DEPTH encodings stored with JPEG had dramatically lower error while still offering 29.8%-smaller file sizes. When H.264 video was used to target a 10 Mbps bit rate, N-DEPTH reconstructions had 85.1%-lower root mean square error (RMSE) and 15.3%-lower mean absolute error (MAE). Overall, our method offers an efficient and robust solution for emerging 3D streaming and 3D telepresence applications, enabling high-quality 3D depth data storage and transmission.
Details
- Title: Subtitle
- N-DEPTH: Neural Depth Encoding for Compression-Resilient 3D Video Streaming
- Creators
- Stephen Siemonsma - University of IowaTyler Bell - University of Iowa
- Resource Type
- Journal article
- Publication Details
- Electronics (Basel), Vol.13(13), 2557
- DOI
- 10.3390/electronics13132557
- ISSN
- 2079-9292
- eISSN
- 2079-9292
- Language
- English
- Date published
- 06/29/2024
- Academic Unit
- Electrical and Computer Engineering
- Record Identifier
- 9984652254902771
Metrics
37 Record Views