Hand tracking for surgical skills... with just a webcam

Motion capture systems are notoriously complex. I'm on a quest to try to achieve hand tracking with as minimalistic of a setup as possible; ideally, a single camera from the optimal vantage point – the surgeon's head. Here's my progress so far.
For the uninitiated, please see my prior work here on the importance of the head-mounted camera setups and the first-person perspective for capturing operative video.
This setup currently is not fully head-mounted, but does work with the overhead camera on the boom arm.
Codepen: https://codepen.io/mediapipe/pen/RwGWYJw
The big picture of what's next for this project:
- Learn how to save / export data after capture
- Figure out how to analyze
- Capture various skill levels: no experience, novice, intern, chief, attending, 'master'
- Figure out how to predict experience level given a new recording
B Q M D Newsletter
Join the newsletter to receive the latest updates in your inbox.