Surgical NAvigation Platform with PYthon (SNAPPY)
SNAPPY is collection of software developed by the Wellcome EPSRC Centre for Surgical Interventions and Sciences. The aim of SNAPPY software echos the aims of WEISS; to accelerate the translation of surgical research to clinical use.
Delivered Packages
Here's a quick lookup for things that are available:
| Project | Notes | ReadTheDocs | Public PyPi | Internal PyPi |
|---|---|---|---|---|
| scikit-surgery | Placeholder to install other packages | Yes | Yes | N/A |
| scikit-surgerycore | Core package, minimal dependencies, just numpy and six? | Yes | Yes | N/A |
| scikit-surgeryimage | Image processing, based on opencv-python | Yes | Yes | N/A |
| scikit-surgeryvtk | Overlay models/text, based on vtk | Yes | Yes | N/A |
| scikit-surgerynditracker | Library for NDI trackers | Yes | Yes | N/A |
| scikit-surgeryarucotracker | Library for ArUco trackers | Yes | Yes | N/A |
| scikit-surgerybk | Library for communicating with BK Ultrasound scanners | No | yes | No |
| scikit-surgeryutils | Various video apps, render apps | Yes | Yes | N/A |
| scikit-surgerybard | Basic Augmented Reality Demo (BARD) | No | No | No |
| scikit-surgery-torso-simulator | Ultrasound acquisition simulator | yes | yes | No |
| scikit-surgerydavinci | DaVinci overlay app | Yes | Yes | No |
| scikit-surgery-sphere-fitting | Fits sphere to VTK data, example module from tutorial 2 | Yes | Yes | No |
| scikit-surgeryopencvcpp | OpenCV functions, in C++ | No | Yes | No |
| scikit-surgerypclcpp | PCL functions, in C++ | No | Yes | No |
| scikit-surgerygpucpp | GPU functions, in C++ | No | No | No |
| scikit-surgeryspeech | Speech services for image-guided surgery | No (due to porcupine) | Yes | No |
| scikit-surgerytf | Tensor flow examples and utilities for image-guided surgery | No | No | No |
| Python Template | Used to generate new projects | N/A | N/A | N/A |
| C++ Template | Used to generate new projects | N/A | N/A | N/A |
And here are links to tutorials and teaching materials.
| Project | Notes | ReadTheDocs | Public PyPi | Internal PyPi |
|---|---|---|---|---|
| SNAPPY Tutorial 01 | Build a simple augmented reality application | Yes | N/A | N/A |
| SNAPPY Tutorial 02 | Implement, document, test, and publish your work to PyPi | Yes | N/A | N/A |
| SNAPPY Workshop 01 | Slides from the first SNAPPY workshop and tutorial. | N/A | N/A | N/A |
Aim: SNAPPY is Not a Platform with PYthon
SNAPPY is not a platform in the style of many of the existing alternative medical imaging or IGI platforms. Existing surgical and medical imaging platforms usually provide an easy way to prototype new algorithms for surgery, for example slicer or NifTK. However when it comes time to translate the algorithm to clinic the researcher is left with a very large dependency on third party software, most of which is not used by the specific application and sometimes is of dubious quality. A typical approach at this stage is to re-engineer the software before further clinical translation, which is costly and time consuming, leading to the "valley of death" between research and application. SNAPPY aims to avoid the valley of death by providing a well engineered collection of modular, light weight libraries and application that can be used for both research and clinical application development.
By providing a well engineered collection of software libraries and suitable infrastructure, the SNAPPY platforms aims to;
- Allow rapid prototyping of novel algorithms for the initial stage of research.
- Support the translation of research software from bench to bedside in less than two years.
- Support researchers in the dissemination of their research by allowing algorithms to be made widely available on standard platforms.
SNAPPY is inspired by the unix software philosophy of providing small libraries and applications that do one task well. The resulting applications can be rapidly connected together by taking advantage of a well defined and standardised interfaces. SNAPPY uses the python programming language to connect its various libraries and applications. Use of the python language enables the easy handling of the various data types encountered in surgical research, and the language widely used and understood by the research community. So whilst individual SNAPPY components can be written in any language, they must provide a python interface to allow use in an application.
Minimum Requirements
"Software process: the key to developing robust, reusable and maintainable open-source software", explains how good software process supports the development of robust, reusable and useful software for medical applications. In line with good software process all SNAPPY software should meed the following minimum requirements.
- Must provide documentation to explain what the software is expected to do and the software's intended inputs and outputs.
- Should provide unit tests to check that the software is performing as intended.
- Must provide an interface to python to enable use in applications.
- Must be kept in a version controlled repository.
- Must support issue tracking and bug reporting.
- Where possible, should support all common platforms used in research (Windows, Linux, and Mac.)
Minutes
Applications
A list of applications that SNAPPY could be used for.
Alternative Platforms
A list of other IGI or medical imaging platforms.
Integrations
- Use PLUS if PLUS supports any device out of the box, stream to SNAPPY via OpenIGTLink
- Use ROS if ROS supports devices out of the box, ROS has an OpenIGTLink bridge, or we parse ROS messages
So, if we use ROS or PLUS as data-sources, then we can easily receive images/tracking via the network. This is good for code modularity, but is more fiddly to launch. There will be cases where we want a single application to just run, or need everything in the same memory space, or for performance reasons we code our own python module. So, the packages below will still be needed in some way.
Data Capture
| Feature | Notes | Where? | Status |
|---|---|---|---|
| USB3 video | Base on opencv-python | scikit-surgeryimage | Ready for contributions |
| NDI Spectra, Veiga, Auora | Ask Steve | scikit-surgerynditracker | Ready for contributions |
| Atracsys | Atracsys released Python API | Need developer | |
| NDI Certus | Obsolete? | ||
| BK 5000 | Network based? Pure python? | scikit-surgerybk | Ready for contributions |
| Ultrasonix | Python wrapping C++ library, 32 bit windows only. | Can be done, but do we need it. | |
| Buffering |
Algorithms
| Feature | Notes | Where? | Status |
|---|---|---|---|
| Point based registration | Implemented | scikit-surgerycore/algorithms | Ready for contributions |
| Rigid Body Transformation Manager | Implemented | scikit-surgerycore/transforms | Ready for contributions |
| Pivot calibration | Implemented | scikit-surgerycore/algorithms | Ready for contributions |
| ICP | Implemented | scikit-surgerypcl | Ready for contributions |
| Global ICP | |||
| Video intrinsic calibration | Just use OpenCV | ||
| Video stereo calibration | Just use OpenCV | ||
| Video hand-eye calibration | One in SmartLiver, but protocol is fairly application dependent? | Discuss? | |
| Maria IPCAI2018 | |||
| Surface recon (Stoyanov, Totz, OpenCV/MATLAB) | Stoyanov 2010 MICCAI in scikit-surgeryopencvcpp | Ready for contributions | |
| Ultrasound calibration (Point, Z-wire, Xiao) | |||
| Ultrasound reconstruction | |||
| FCN segmentation | |||
| Timing calibration |
Visualisation Types
| Feature | Notes | Where? | Status |
|---|---|---|---|
| Surface/Wireframe - colour, opacity | scikit-surgeryvtk | Ready for contributions | |
| Point cloud - RGB colour per point | scikit-surgeryvtk | Ready for contributions | |
| Image plane in 3D space, e.g. ultrasound | scikit-surgeryvtk | Ready for contributions | |
| Calibrated video view | scikit-surgeryvtk | Ready for contributions | |
| Stereo visualisation | interlaced for 3D monitor | scikit-surgeryvtk | kinda-done, but better to use hardware adaptors |
User Interfaces
| Feature | Notes | Where? | Status |
|---|---|---|---|
| Voice recognition | scikit-surgeryspeech | Ready for contributions | |
| Gesture recognition | |||
| Foot pedals | on order |
Configuration
Its difficult to be fiddling with config params in the operating room. You might not have a keyboard to hand, or you might be using a touchscreen. What about just starting an app via a pre-configured config file.
| Feature | Notes | Where? | Status |
|---|---|---|---|
| After this discussion, decided .json | Implemented | scikit-surgerycore | Ready for contributions |
Guidance Utilities
| Feature | Notes | Where? | Status |
|---|---|---|---|
| Coordinate System Conversion - imaging, tracking, physical, Cartesian, polar | |||
| Triangulated Surface Manipulation - smoothing, simplify, intersection with planes | |||
| Image Re-sampling - non-orthogonal slicing |
Internal PyPI server
A local test PyPI server is up and running to host internally developed pip packages. For privacy and security, the server is accessible only from inside the UCL and not available from external network. To view the package list visit http://128.16.6.189:8080. The username is compass and password is Compass@007. Once logged-in you can list the packages and install it using command e.g. for flask pip install -i http://localhost:8080/simple/ flask. Give the mentioned username and password when requested.