Research
Real-time 3D reconstruction and foundation-model adaptation for surgery — depth estimation and SLAM where ground truth is scarce and the scene refuses to hold still.
Bingwen Dong†, Gan Liu†, Xiaoxi Lu†, Guangcheng Chen, Jialu Zhang, Yan Hu, Xiaoqing Zhang, Jiang Liu
Real2Sim2Real: RetinalDepth-64K for Depth Estimation in Posterior Segment Ophthalmic Surgery
IEEE/CVF CVPR 2026pp. 26899–26908† Equal contributionA Real2Sim2Real pipeline that closes the domain gap for monocular depth estimation inside the posterior segment of the eye during vitreoretinal surgery — where real ground-truth depth is nearly impossible to capture. We build RetinalDepth-64K, a large-scale dataset, and learn depth predictors that transfer back to real surgical footage.
Depth in intraocular surgery is hard to supervise: the working distance is millimetres, the scene is wet and specular, and there is no external depth sensor. Real2Sim2Real bridges this by moving between real captures and a controllable simulation, then back to real data.
RetinalDepth-64K provides paired imagery and depth at a scale that makes learned monocular depth practical for the posterior segment, supporting downstream reconstruction and navigation.
Monocular depth3D visionSim-to-realSurgical AIXiaoxi Lu†, Gan Liu†, Bingwen Dong†, Guangcheng Chen, Mingdao Gong, Yan Hu, Xiaoqing Zhang, Jiang Liu
SurgFM-SLAM: Robust Surgical Scene Reconstruction via 3D Foundation Model Adaptation
Research project 2025† Equal contributionA SLAM system for reconstructing dynamic surgical scenes by adapting a 3D foundation model, rather than training a bespoke geometry model from scratch. The goal is robust, real-time scene reconstruction that survives the deformation, texture-poor tissue, and abrupt motion typical of endoscopic surgery.
Foundation models for 3D give strong geometric priors; SurgFM-SLAM adapts those priors to the surgical domain so tracking and mapping stay stable where classical SLAM drifts.
The result is a reconstruction pipeline aimed at the real conditions of surgery — occlusion, non-rigid tissue, and specular highlights — targeting downstream navigation and measurement.
SLAM3D reconstructionFoundation-model adaptationSurgical AI