Help with realtime motion tracking for ASP?
Posted: Mon Apr 15, 2013 8:49 am
I have this crazy dream of creating some sort of scripted realtime link from video markers on a face or body to bones in an Anime Studio character. My thought is that this would have to be a bit complicated and would involve using the "external" lua integration. Lua can "talk" to outside applications. I did this a long time ago for recent documents before that feature was added.
The idea would be based on my own 2.5D head rigs using bones for all face motions; eyes, eyebrows, mouth, etc etc.
The concept seems so simple; capture video of say, a face with bright markers for eyes mouth etc. Calculate a xy location of the points, assign those points to bone translations.
The one thing that has me stumped from even starting is linking "realtime video" directly to Anime Studio. I created a script a long time ago that loads image sequences directly into a single image layer based on frame number. I created this for frame by frame image animation in ASP.
So maybe a video app could be set to update a single image file in a folder (like a really "fast webcam"). This script would actually do the trick, as it would simply load the same image over and over. Lots of webcam software simply updates a single jpg image. It's usually based on like, 1 image per minute. But if you set this to one image per second... you might be able to capture enough to have sort of smooth motion.
The other trick would be getting the XY locations of "markers" and applying to the bones. My head rigs use bone location for mouth and eye changes. The realtime feedback is instantaneous, even with scripting that controls XY locations of MANY MANY bones at the same time. If I can get realtime feedback pumped into a script in AS the bones could be moved easily and quickly. I still need those XY positions though.
My dream would be to be able to have a realtime view of the ASP preview of a "head" or face (not saved or keyed) that I could use for video conferencing or tutorials. Of course this could also be used to key frame "acting" with heads or even bodies.
Any suggestions or input from those who know more than I do about video conversions or webcam software that might be able to communicate with ASP would be appreciated.
The idea would be based on my own 2.5D head rigs using bones for all face motions; eyes, eyebrows, mouth, etc etc.
The concept seems so simple; capture video of say, a face with bright markers for eyes mouth etc. Calculate a xy location of the points, assign those points to bone translations.
The one thing that has me stumped from even starting is linking "realtime video" directly to Anime Studio. I created a script a long time ago that loads image sequences directly into a single image layer based on frame number. I created this for frame by frame image animation in ASP.
So maybe a video app could be set to update a single image file in a folder (like a really "fast webcam"). This script would actually do the trick, as it would simply load the same image over and over. Lots of webcam software simply updates a single jpg image. It's usually based on like, 1 image per minute. But if you set this to one image per second... you might be able to capture enough to have sort of smooth motion.
The other trick would be getting the XY locations of "markers" and applying to the bones. My head rigs use bone location for mouth and eye changes. The realtime feedback is instantaneous, even with scripting that controls XY locations of MANY MANY bones at the same time. If I can get realtime feedback pumped into a script in AS the bones could be moved easily and quickly. I still need those XY positions though.
My dream would be to be able to have a realtime view of the ASP preview of a "head" or face (not saved or keyed) that I could use for video conferencing or tutorials. Of course this could also be used to key frame "acting" with heads or even bodies.
Any suggestions or input from those who know more than I do about video conversions or webcam software that might be able to communicate with ASP would be appreciated.
