Project Page

ArtiMo: Agent-Driven Articulated Mesh Animation

Chunyu Zou1 Peng Dai1,2,3† Yihua Huang1 Ze Yuan1 Jingwei Huang4 Yeming Yao1 Xiaojuan Qi1†
1University of Hong Kong, China 2University of British Columbia, Canada 3Vector Institute, Canada 4Tencent, China

† Corresponding authors.

A zero-shot agent framework for text-guided articulated mesh animation with kinematic constraints, causal part reasoning, and visual self-improvement.

arXiv HuggingFace Code

Abstract

Animating articulated 3D meshes via text requires satisfying strict kinematic constraints, modeling causal interactions between parts, and achieving instruction fidelity. Due to a scarcity of training data, existing data-driven mesh animation methods are largely inapplicable. To address this, we propose ArtiMo, a novel agent-driven framework for text-guided articulated mesh animation. Operating in a zero-shot manner, ArtiMo employs an autonomous agent powered by Large Language and Vision-Language Models to orchestrate motion generation. By synergizing the explicit kinematic constraints of URDF with the agent's reasoning and planning capabilities, it effectively produces causally coherent part motions and interactions without requiring model fine-tuning. To ensure high-fidelity results, the agent additionally utilizes a visual self-improvement mechanism: generated animations are rendered into compact keyframes and motion cues, enabling the VLM to iteratively diagnose and correct errors. Furthermore, we contribute a new benchmark dataset spanning 21 object categories with high-quality motion annotations. Extensive experiments demonstrate that ArtiMo significantly outperforms existing baselines, particularly on complex, causally-driven motions.

Method

ArtiMo combines structured articulated assets, visual-language diagnosis, and executable motion planning in a closed loop.

ArtiMo method overview
Overview of our agent-driven articulated mesh animation framework. Given a URDF asset and an action prompt, our system performs link analysis, causal motion planning, and critic--actor refinement to generate a causally correct 4D articulated animation.

Full Demo

A concise end-to-end video covering qualitative results, evaluation, and ablation study.

ArtiMo Application Rollouts

These physical rollouts are generated by the ArtiMo application from language instructions, articulated meshes, and contact-aware execution. Swipe left or right to view one rollout at a time.

Swipe horizontally, or use the left and right arrow keys, to switch between rollouts. Videos load as their cards enter the viewport.