Towards augmented reality: Intraoperative prostate modelling for biopsy and brachytherapy

Mohammad Ali Jan Ghasab, Andrew Paplinski, John Betts, Hayley Reynolds, Annette Haworth

    Research output: Contribution to conferenceAbstract


    Background and Purpose:
    Visualization of the prostate is being used increasingly to improve the accuracy of biopsies and seed placement during brachytherapy. One approach is to create a 3D model of the patient’s prostate from a stack of MR images pre-operatively and to fuse this inter-operatively with real time TRUS imaging.

    The current practice of manually segmenting the 2D MR images pre-operatively and manual co-registration with TRUS images inter-operatively requires a very high level of expertise and is time-consuming.

    In order to improve on current practice, we propose to create an augmented reality 3D prostate model automatically from the pre-operative MR images and real time TRUS, suitable for a clinician to use inter-operatively.

    To achieve this, we improve upon the current state of the art in prostate boundary detection and 3D model creation for MR and TRUS imaging, and introduce fast optimisation models for the deformable registration of the 3D prostate models created in each modality.

    Beginning with the patient’s pre-operative 2D MR images we create a 3D voxel-based image stack from which a 3D model describing the patient’s prostate surface and interior is created directly by fitting a 3D Active Appearance Model (AAM) [1] using a Modified Inverse Compositional Model Alignment (ICMA) approach. This forms the MR reference model. TRUS images, made intra-operatively, are then used to update the 3D reference model in real time. This enables the detailed MR image data to be fused with real-time TRUS image in any plane. In this way we create a realistic 3D deformable prostate model, which can provide full volumetric and surface information to surgeons inter-operatively.

    Results and Conclusion:
    We have addressed two necessary steps in achieving the augmented reality model outlined above in the following ways: We have compared our 3D segmentation method with the best results presented in the MICCAI competition [2] and achieved better 3D segmentation accuracy with reduced computational time. Our results also show that the ICMA approach also produces segmentation results for TRUS slices to a high degree of accuracy and computational efficiency.

    [1] Mitchell, Steven C et al. “3-D active appearance models: segmentation of cardiac MR and ultrasound images,” in IEEE Transactions on Medical Imaging, vol. 21, no. 9, pp. 1167-1178, Sept. 2002.

    [2] Litjens, Geert et al. “Evaluation of Prostate Segmentation Algorithms for MRI: The PROMISE12 Challenge.” Medical image analysis 18.2 (2014): 359–373. PMC. Web. 14 Nov. 2016.
    Original languageEnglish
    Number of pages1
    Publication statusPublished - 24 Feb 2017
    Event26th Annual Scientific Meeting of the Australasian Brachytherapy Group - Crown Conference Centre, Melbourne Vic, Australia
    Duration: 23 Feb 201725 Feb 2017
    Conference number: 26


    Conference26th Annual Scientific Meeting of the Australasian Brachytherapy Group
    Abbreviated titleABG ASM
    CityMelbourne Vic

    Cite this