File Download

There are no files associated with this item.

  Links for fulltext
     (May Require Subscription)
Supplementary

Article: SV-RCNet: Workflow recognition from surgical videos using recurrent convolutional network

TitleSV-RCNet: Workflow recognition from surgical videos using recurrent convolutional network
Authors
Keywordslong short-term memory
joint learning of spatio-temporal features
surgical workflow recognition
very deep residual network
Recurrent convolutional network
Issue Date2018
Citation
IEEE Transactions on Medical Imaging, 2018, v. 37, n. 5, p. 1114-1126 How to Cite?
AbstractWe propose an analysis of surgical videos that is based on a novel recurrent convolutional network (SV-RCNet), specifically for automatic workflow recognition from surgical videos online, which is a key component for developing the context-aware computer-assisted intervention systems. Different from previous methods which harness visual and temporal information separately, the proposed SV-RCNet seamlessly integrates a convolutional neural network (CNN) and a recurrent neural network (RNN) to form a novel recurrent convolutional architecture in order to take full advantages of the complementary information of visual and temporal features learned from surgical videos. We effectively train the SV-RCNet in an end-to-end manner so that the visual representations and sequential dynamics can be jointly optimized in the learning process. In order to produce more discriminative spatio-temporal features, we exploit a deep residual network (ResNet) and a long short term memory (LSTM) network, to extract visual features and temporal dependencies, respectively, and integrate them into the SV-RCNet. Moreover, based on the phase transition-sensitive predictions from the SV-RCNet, we propose a simple yet effective inference scheme, namely the prior knowledge inference (PKI), by leveraging the natural characteristic of surgical video. Such a strategy further improves the consistency of results and largely boosts the recognition performance. Extensive experiments have been conducted with the MICCAI 2016 Modeling and Monitoring of Computer Assisted Interventions Workflow Challenge dataset and Cholec80 dataset to validate SV-RCNet. Our approach not only achieves superior performance on these two datasets but also outperforms the state-of-the-art methods by a significant margin.
Persistent Identifierhttp://hdl.handle.net/10722/299565
ISSN
2023 Impact Factor: 8.9
2023 SCImago Journal Rankings: 3.703
ISI Accession Number ID

 

DC FieldValueLanguage
dc.contributor.authorJin, Yueming-
dc.contributor.authorDou, Qi-
dc.contributor.authorChen, Hao-
dc.contributor.authorYu, Lequan-
dc.contributor.authorQin, Jing-
dc.contributor.authorFu, Chi Wing-
dc.contributor.authorHeng, Pheng Ann-
dc.date.accessioned2021-05-21T03:34:41Z-
dc.date.available2021-05-21T03:34:41Z-
dc.date.issued2018-
dc.identifier.citationIEEE Transactions on Medical Imaging, 2018, v. 37, n. 5, p. 1114-1126-
dc.identifier.issn0278-0062-
dc.identifier.urihttp://hdl.handle.net/10722/299565-
dc.description.abstractWe propose an analysis of surgical videos that is based on a novel recurrent convolutional network (SV-RCNet), specifically for automatic workflow recognition from surgical videos online, which is a key component for developing the context-aware computer-assisted intervention systems. Different from previous methods which harness visual and temporal information separately, the proposed SV-RCNet seamlessly integrates a convolutional neural network (CNN) and a recurrent neural network (RNN) to form a novel recurrent convolutional architecture in order to take full advantages of the complementary information of visual and temporal features learned from surgical videos. We effectively train the SV-RCNet in an end-to-end manner so that the visual representations and sequential dynamics can be jointly optimized in the learning process. In order to produce more discriminative spatio-temporal features, we exploit a deep residual network (ResNet) and a long short term memory (LSTM) network, to extract visual features and temporal dependencies, respectively, and integrate them into the SV-RCNet. Moreover, based on the phase transition-sensitive predictions from the SV-RCNet, we propose a simple yet effective inference scheme, namely the prior knowledge inference (PKI), by leveraging the natural characteristic of surgical video. Such a strategy further improves the consistency of results and largely boosts the recognition performance. Extensive experiments have been conducted with the MICCAI 2016 Modeling and Monitoring of Computer Assisted Interventions Workflow Challenge dataset and Cholec80 dataset to validate SV-RCNet. Our approach not only achieves superior performance on these two datasets but also outperforms the state-of-the-art methods by a significant margin.-
dc.languageeng-
dc.relation.ispartofIEEE Transactions on Medical Imaging-
dc.subjectlong short-term memory-
dc.subjectjoint learning of spatio-temporal features-
dc.subjectsurgical workflow recognition-
dc.subjectvery deep residual network-
dc.subjectRecurrent convolutional network-
dc.titleSV-RCNet: Workflow recognition from surgical videos using recurrent convolutional network-
dc.typeArticle-
dc.description.naturelink_to_subscribed_fulltext-
dc.identifier.doi10.1109/TMI.2017.2787657-
dc.identifier.pmid29727275-
dc.identifier.scopuseid_2-s2.0-85040080064-
dc.identifier.volume37-
dc.identifier.issue5-
dc.identifier.spage1114-
dc.identifier.epage1126-
dc.identifier.eissn1558-254X-
dc.identifier.isiWOS:000431544500004-

Export via OAI-PMH Interface in XML Formats


OR


Export to Other Non-XML Formats