Software replacement for Kinect3d pose esimator using Grasshopper3D
Analysis, SImulation, Robots
(p5.js, ml5.js), C#
In 2020, for any type of real-time pose estimation in an environment like Grasshopper3D, an Xbox Kinect was the most efficient method but unfortunately not as accessible to creators to be a viable or affordable option for everyone.
This plugin employs Machine Learning and lets you use just your Webcam to estimate human figures similar to the XBox Kinect.
The project was started as a software replacement for the Kinect sensor in the Grasshopper3D environment. To achieve this, certain challenges had to be overcome:
1. We first had to determine whether p5.js/ml5.js could run inside the Grasshopper3D environment
It essentially runs an instance of the Google Chrome browser (Chromium, to be precise) within Grasshopper by simply referencing an open source library — CefSharp.The webpage runs a JS script, but instead of displaying the pose in a p5.js canvas , it simply exposes the raw pose data through a JS function (CefSharp allows calling functions through it’s C# API).
So what can you use applications like catwalk for? The possibility of use cases is unlimited but here we provide a few use cases to stimulate your imagination!
Project Catwalk enables users to conduct pose estimation using only a webcam in conjugation with Grasshopper3D. It acts as the starting point for the development of intricate solutions for a wide variety of use cases.