EndoSurf: Neural surface reconstruction of deformable tissues with stereo endoscope videos

Ruyi Zha, Xuelian Cheng, Hongdong Li, Mehrtash Harandi, Zongyuan Ge

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

2 Citations (Scopus)

Abstract

Reconstructing soft tissues from stereo endoscope videos is an essential prerequisite for many medical applications. Previous methods struggle to produce high-quality geometry and appearance due to their inadequate representations of 3D scenes. To address this issue, we propose a novel neural-field-based method, called EndoSurf, which effectively learns to represent a deforming surface from an RGBD sequence. In EndoSurf, we model surface dynamics, shape, and texture with three neural fields. First, 3D points are transformed from the observed space to the canonical space using the deformation field. The signed distance function (SDF) field and radiance field then predict their SDFs and colors, respectively, with which RGBD images can be synthesized via differentiable volume rendering. We constrain the learned shape by tailoring multiple regularization strategies and disentangling geometry and appearance. Experiments on public endoscope datasets demonstrate that EndoSurf significantly outperforms existing solutions, particularly in reconstructing high-fidelity shapes. Code is available at https://github.com/Ruyi-Zha/endosurf.git.

Original languageEnglish
Title of host publicationMedical Image Computing and Computer Assisted Intervention – MICCAI 2023 - 26th International Conference Vancouver, BC, Canada, October 8–12, 2023 Proceedings, Part IX
EditorsHayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor
Place of PublicationCham Switzerland
PublisherSpringer
Pages13-23
Number of pages11
ISBN (Electronic)9783031439964
ISBN (Print)9783031439957
DOIs
Publication statusPublished - 2023
EventMedical Image Computing and Computer-Assisted Intervention 2023 - Vancouver, Canada
Duration: 8 Oct 202312 Oct 2023
Conference number: 26th
https://link.springer.com/book/10.1007/978-3-031-43901-8 (Proceedings)
https://conferences.miccai.org/2023/en/ (Website)

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume14228
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceMedical Image Computing and Computer-Assisted Intervention 2023
Abbreviated titleMICCAI 2023
Country/TerritoryCanada
CityVancouver
Period8/10/2312/10/23
Internet address

Keywords

  • 3D Reconstructon
  • Neural Fields
  • Robotic Surgery

Cite this