Design Interaction

23/04/2024 - 30-04-2024

In my recent project, I set out to capture my body movements using MediaPipe and apply these to a 3D model in Blender. While I haven’t yet succeeded in fully rigging a 3D model with this captured motion, I did manage to import the motion data and play it back. Here’s a brief overview of the challenges I faced

The Process

I used MediaPipe’s Pose and Hands modules to capture my body and hand movements via my webcam. Using Python, I extracted the captured landmarks and saved them as JSON files. After that, I converted the JSON files into BVH format to make the motion data compatible with Blender. Finally, I imported the BVH file into Blender, which allowed me to visualize the captured motion within a 3D environment.

Challenges and Current Status

Applying the captured motion to a 3D model has been complex due to issues with bone mappings and animation constraints. Despite these challenges, I managed to import and play back the BVH animation in Blender, marking it a significant progress.

Reflection and Future Directions

This project is part of my larger initiative to use AI to preserve traditional craftsmanship. By analyzing artisans' movements, I aim to create an educational platform that teaches these skills to new generations. While the integration of motion capture and 3D animation is still in progress, the ability to capture and visualize motion is a huge step at the moment.