Magic Mocap

From RGB vide recording to professional Motion Capture — harnessing Apple's ARKit platform to put a full motion capture studio in your pocket.

Challenge

This initiative harnesses Apple's ARKit and Video Editing ecosystem to revolutionize motion capture workflows, traditionally expensive and time-consuming, into an accessible and intuitive experience that enables immediate creation of motion capture content from any iOS device.

Results

The successful prototype combines an iPad app with a media processing server capable of transforming RGB footage into videos up to 300 frames long, completely removing the human actor and precisely replacing them with selected USDZ models.

Next Steps

Looking forward, the vision includes evolving the system to function independently without servers—keeping all data on-device for enhanced privacy—while expanding tracking capabilities to include eyes and fingers for close-up scenarios, and refining shadow rendering to achieve more photorealistic results that seamlessly blend digital models into real-world environments.