Scene interpretation of multimodal data capture metrological assessment for minimally invasive surgical procedures of the hand

Alwin Kumar Rathinam, Yunli Lee, David Ngo Chek Ling, Rajesh Singh, Lakshmi Selvaratnam, Narendra Pamidi

Research output: Contribution to conferenceAbstract


Introduction: Medical imaging such as ultrasound coupled with visual analytics provides important pre-surgical, intra-operative and post-surgical evaluation information for surgeons performing Minimally Invasive Surgical Procedures(MISP) on the hand. This research focuses on the motion of the metacarpophalangeal or phalangeal joints affected by either trauma,arthritis, synovitis or tendonitis. Common evaluation includes a visual inspection of the motion of digits such as theability to touch opposing digits with the thumb.Scene interpretation is a branch of artificial intelligence that deals with the analysis of multiple imagingsources. Variation of viewpoints or data sources provide ways to develop logical flow statements.

Methods: A device was developed with customised software application and fixtures for two cameras at 90° to each otherand an ultrasound probe. The cameras captured the palmar and lateral view of the hand. The ultrasound visualised theinternal hand anatomy. The camera data streams were cropped, dithered, thresholded, the largest object segmented andanalysed to derive the rate of change of motion. Clinical trial participants provided consent for data recording and wererequested to make a fist or move certain digits. Surgeons provided logic to consolidate the streams.

Results: The logic flow was generated using scene interpretation from the three data streams to develop a tool that provides analytics to assess motion of the digits of the hand. Analytics of the camera view provided the rate of changediagrams.

Conclusion: The use of such measurement devices should be expounded to provide a systematic evaluation of MISP. Such evaluation would result in improvement of techniques, extended utilisation and endorsement of MISP as a plausible option. Future work includes metrological studies on hand motion subsequent to trauma so as to categorise traumatypes, to provide surgeons with distinct indicators enabling intra-operative and postoperative evaluation and reporting of surgical outcomes
Original languageEnglish
Number of pages1
Publication statusPublished - 2021
EventAnnual Conference of the German Society for Biomedical Engineering 2021 - Hannover, Germany
Duration: 5 Oct 20217 Oct 2021
Conference number: 55th


ConferenceAnnual Conference of the German Society for Biomedical Engineering 2021
Abbreviated titleBMT 2021

Cite this