VesNet-RL: Simulation-based Reinforcement Learning for Real-World US Probe Navigation

by   Yuan Bi, et al.

Ultrasound (US) is one of the most common medical imaging modalities since it is radiation-free, low-cost, and real-time. In freehand US examinations, sonographers often navigate a US probe to visualize standard examination planes with rich diagnostic information. However, reproducibility and stability of the resulting images often suffer from intra- and inter-operator variation. Reinforcement learning (RL), as an interaction-based learning method, has demonstrated its effectiveness in visual navigating tasks; however, RL is limited in terms of generalization. To address this challenge, we propose a simulation-based RL framework for real-world navigation of US probes towards the standard longitudinal views of vessels. A UNet is used to provide binary masks from US images; thereby, the RL agent trained on simulated binary vessel images can be applied in real scenarios without further training. To accurately characterize actual states, a multi-modality state representation structure is introduced to facilitate the understanding of environments. Moreover, considering the characteristics of vessels, a novel standard view recognition approach based on the minimum bounding rectangle is proposed to terminate the searching process. To evaluate the effectiveness of the proposed method, the trained policy is validated virtually on 3D volumes of a volunteer's in-vivo carotid artery, and physically on custom-designed gel phantoms using robotic US. The results demonstrate that proposed approach can effectively and accurately navigate the probe towards the longitudinal view of vessels.


page 1

page 3

page 5

page 6


Image-Guided Navigation of a Robotic Ultrasound Probe for Autonomous Spinal Sonography Using a Shadow-aware Dual-Agent Framework

Ultrasound (US) imaging is commonly used to assist in the diagnosis and ...

Autonomous Navigation of an Ultrasound Probe Towards Standard Scan Planes with Deep Reinforcement Learning

Autonomous ultrasound (US) acquisition is an important yet challenging t...

Ultrasound-Guided Robotic Navigation with Deep Reinforcement Learning

In this paper we introduce the first reinforcement learning (RL) based r...

Parallel Reinforcement Learning Simulation for Visual Quadrotor Navigation

Reinforcement learning (RL) is an agent-based approach for teaching robo...

Cross-View Policy Learning for Street Navigation

The ability to navigate from visual observations in unfamiliar environme...

Visual-Policy Learning through Multi-Camera View to Single-Camera View Knowledge Distillation for Robot Manipulation Tasks

The use of multi-camera views simultaneously has been shown to improve t...

Persuading to Prepare for Quitting Smoking with a Virtual Coach: Using States and User Characteristics to Predict Behavior

Despite their prevalence in eHealth applications for behavior change, pe...

Please sign up or login with your details

Forgot password? Click here to reset