NVIDIA releases Cosmos3-H-Dreams for real-time surgical simulation
NVIDIA’s downloadable 15.17B action-conditioned video model simulates robot-assisted cholecystectomy, but remains a research system requiring more than 32 GB of GPU memory and clinical validation.
2 min read
NVIDIA has released Cosmos3-H-Dreams, an action-conditioned video model designed to simulate robot-assisted surgery in real time. The company describes it as the first real-time generative surgical simulator trained on clinical cholecystectomy recordings, a claim that has not yet been independently validated.
The 15.17-billion-parameter model was fine-tuned from Cosmos3-H-Embodied-Nano using video and 28-dimensional kinematic data from CMR Surgical’s Versius system. It generates 832-by-480 video conditioned on robot actions sampled at 10 Hz. NVIDIA provides four-step and two-step student checkpoints; the faster two-step version sacrifices some visual quality for lower latency.
The release is downloadable from Hugging Face under the NVIDIA Open Model License. Its accompanying code is maintained by the Isaac for Healthcare organization on GitHub under Apache 2.0, so users should distinguish the code license from the separate model-weight terms. The repository specifies Linux, Python 3.12, an NVIDIA CUDA GPU and more than 32 GB of VRAM; NVIDIA says interactive use is possible on a single RTX PRO 6000.
This is a research simulator, not a clinically validated patient-care system. NVIDIA says the generated worlds still need surgeon evaluation and numerical constraints before they can support safety-critical use. The project is therefore best read as an embodied-AI research release for training and evaluating surgical robotics, rather than evidence of deployment readiness.
Image credit: Photo by Navy Medicine on Unsplash, used under the Unsplash License.
Sources: NVIDIA release · Model card · Canonical repository · Image · Unsplash License
Conversation
0 approved commentsNo comments yet. Start a thoughtful conversation.
Join the conversation.
Sign in with ChatGPT to leave a comment or rate this article.