Depth map compression, streaming, and reconstruction for immersive and accessible 3D telepresence

In our increasingly digital world, video calls have become essential for work, education, and socializing. Yet traditional video conferencing lacks the natural depth and presence of face-to-face interaction. This dissertation develops innovative solutions to make three-dimensional (3D) video conferencing more accessible and practical for everyday use. This research explores 3D telepresence, which involves transmitting realistic 3D imagery of individuals to remote locations in order to make digital communication feel more immersive. However, sending detailed 3D video data requires a lot of internet bandwidth, and specialized equipment has often been expensive. This dissertation introduces new methods to make 3D telepresence more efficient and accessible. First, we developed HoloKinect, a system using affordable, common devices (like the Microsoft Kinect sensor and a special 3D display) to enable live 3D video calls over standard internet connections. Second, we created new computer algorithms, N-DEPTH and GraDE, to cleverly compress the 3D depth information into regular video formats. These methods significantly reduce the internet speeds needed to maintain good visual quality. N-DEPTH uses artificial intelligence, while GraDE is a faster, non-AI version suitable for less powerful devices. Finally, we built a platform allowing multiple smartphones to capture a scene in 3D together and stream it into virtual reality, demonstrating how this technology can work flexibly with everyday mobile devices. This work demonstrates that realistic 3D communication is achievable with today’s consumer technology, paving the way for more natural and engaging digital interactions.