3D Motion Tracked Interactivity with Edge Animate and Greensock

 

As a continuation of my research into motion tracking interactive elements onto HTML5 video, and in order to achieve the 3D effects I want, I’ve added perspective.

The process is a little convoluted. It involves Greensock, Edge Animate, Flash CC, After Effects CC and Mocha Pro.

Once I’ve shot the video I bring that into After Effects and do a corner pin track using Mocha. Then I copy the tracking data from Mocha back into After Effects as transform properties. They’re then converted to Layer Markers using a script and the video is then output to FLV. I’ve written an Air program that opens, plays and harvests the cue point (tracking) data from the FLV and spits out a JSON file with sets of 8 data points (upper left X and Y, upper right X and Y, lower left X and Y etc).

I then consume the JSON file using jQuery/JavaScript and convert the points to Greensock tweens which are then added to a master TimelineMax timeline. This is then synchronised to the video’s timecode – the whole she-bang is controlled by a Greensock master timer that gets the current video time code (FYI the video’s time update event is unreliable at best/rubbish) and tells the TimelineMax’s playhead where to be. You can also scrub through the entire thing using Greensock’s Draggable.

I think that’s it!

(…and breathe…)

The 3D tracked elements are interactive and the performance is great on the desktop (Mac and PC) and iPad Air although I’ve tried it on my Nexus 7 (2012) and the coloured circles that animate when you tap look blurred and terrible and the tracking isn’t very tight.

Still it’s a start and the performance is (hopefully) only going to get better on all platforms/devices.

3 thoughts on “3D Motion Tracked Interactivity with Edge Animate and Greensock

    1. Thanks Dan – because of the complex of nature of all the software and processes involved I’m looking to train people instead.

      If you (or your employer) are interested in learning about these techniques (there is a fair bit to learn) then you can book a training session with me directly.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s