Microduck RL on a Home PC: Install, Drive, and Train a Walking Policy on a 6 GB RTX 2060

  • Post author:
  • Post published:
  • Post category:Technology

Pollen Robotics — the French team behind the open-source Reachy humanoid — released the complete reinforcement-learning stack for their new ~25 cm / ~800 g bipedal robot Microduck: 44+ RL task environments, the whole sim2real recipe, and every policy they run on the real robot. You don’t need the hardware to use it. This post is my own record of installing it on a home PC, driving three of its environments in simulation, and training a new walking policy locally on a 6 GB RTX 2060 — plus the honest numbers and the traps I hit along the way.

Pollen Robotics(開源人形機械人 Reachy 嘅法國團隊)為佢哋新嘅 ~25cm / ~800g 兩腳機械人 Microduck 開源咗成套強化學習訓練系統:44 個以上 RL 任務環境、完整 sim2real 配方、仲有真機上用到嘅全部 policy。無實體機械人都玩到。呢篇文係我自己喺屋企部電腦安裝佢、喺模擬器玩三個環境、再喺 6GB 嘅 RTX 2060 上由零訓練一條行走 policy 嘅全紀錄 —— 包括真實數字同踩過嘅坑。

① Velocity walk — the alpha_walking reference policy sprinting on flat ground. Commanded 0.35 m/s, straight line: 5.9 m in 11.5 s.
② VelStand — the walk-and-recover policy takes a 1.15 m/s shove (the hardest push in its training curriculum), topples, and climbs back up in about one second.
③ BallKick — the 70 mm ball is placed in front of the kicking foot; two trained episodic policies (left/right) swing, strike and re-balance.
④ My locally trained Velocity-Flat policy (PPO, 2048 parallel envs, RTX 2060, ~2 h) exported to ONNX and driven by the same inference loop.

1 · What you need

Part Minimum Notes
GPU (training) Any CUDA GPU, ~6 GB VRAM RTX 2060 6 GB ran 2048 parallel envs at 3.7 GB. 8 GB+ fits 4096 envs (the upstream default).
GPU (play only) None — CPU works Policy inference and the 3D viewer run fine without CUDA.
RAM / disk 16 GB / ~10 GB free The first uv sync pulls ~2 GB of CUDA wheels (torch 2.9.1 cu128, MuJoCo Warp).
OS Ubuntu (native or WSL2) Everything here ran inside WSL2 on Windows 11 with the NVIDIA WSL driver.

1 · 要啲乜

部件 最低要求 備註
GPU(訓練) 任何 CUDA 顯示卡,~6GB VRAM RTX 2060 6GB 行 2048 個並行環境只食 3.7GB;8GB+ 可以支援 4096 個(上游默認值)。
GPU(只玩唔訓練) 唔要 —— CPU 都得 Policy 推理同 3D viewer 無 CUDA 都照行。
RAM / 磁碟 16GB / ~10GB 空間 第一次 uv sync 會拉 ~2GB CUDA wheels(torch 2.9.1 cu128、MuJoCo Warp)。
作業系統 Ubuntu(原生或 WSL2) 本文全部喺 Windows 11 + WSL2 + NVIDIA WSL 驅動環境行。

2 · Install (15 minutes)

Three commands. uv is the Python package manager the project uses; the long HTTP timeout matters because the first sync downloads ~2 GB of pinned CUDA wheels.

curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/pollen-robotics/microduck_rl
cd microduck_rl
export UV_HTTP_TIMEOUT=600
uv sync          # ~2 GB first run, < 10 min on fibre
uv run list-envs # the gym registry — 44 tasks

list-envs is the full task table: walking, stand-up, sit/stand, ground-pick, ball kick, forward roll (roulade), roller skating (six roller tasks), all with Backlash variants that add realistic gear-play to every servo. Every one is a self-contained PPO gym.

On WSL, the NVIDIA driver comes from Windows — check nvidia-smi inside WSL before training. My card showed up as NVIDIA GeForce RTX 2060, 6144 MiB and uv run python -c "import torch; print(torch.cuda.is_available())" → True.

2 · 安裝(15 分鐘)

三句搞掂。uv 係呢個 project 用嘅 Python 套件管理器;/個 HTTP timeout 要拉長,因為第一次 sync 會下載 ~2GB 指定版本 CUDA wheels。

curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/pollen-robotics/microduck_rl
cd microduck_rl
export UV_HTTP_TIMEOUT=600
uv sync          # 第一次 ~2GB,光纖下 < 10 分鐘
uv run list-envs # 任務登記表 —— 44 個任務

list-envs 會列出全部任務:行走、起身、坐企、蹲低拾物、踢波、前滾翻(roulade)、滾軸溜冰(六個 roller 任務),仲有每個都有的 Backlash 版本(每邊齒輪加入真實齒隙)。每個任務都係一個獨立嘅 PPO gym。

WSL 嘅 NVIDIA 驅動由 Windows 側提供 —— 訓練前先喺 WSL 內行 nvidia-smi 確認。我部機顯示 NVIDIA GeForce RTX 2060, 6144 MiB,import torch 後 torch.cuda.is_available() == True。

3 · Load the policies the real robot runs

Pollen publishes everything the physical Microduck runs to a Hugging Face model card — eleven ONNX files, ~800 KB each. You download the exact networks that live inside the robot and drive them in the simulator.

uv run python - <<'EOF'
from huggingface_hub import snapshot_download
snapshot_download(repo_id="pollen-robotics/microduck-policies",
                  local_dir="/home/<you>/microduck_policies")
EOF

3 · 下載真機用緊嘅 policy

Pollen 將實體 Microduck 行緊嘅網絡全部放上 Hugging Face —— 11 個 ONNX 檔,每個 ~800KB。即係你可以喺模擬器入面,開啲一個「真機腦海中嘅神經網絡」嚟行。

uv run python - <<'EOF'
from huggingface_hub import snapshot_download
snapshot_download(repo_id="pollen-robotics/microduck-policies",
                  local_dir="/home/<你>/microduck_policies")
EOF

4 · Drive a policy with the keyboard

The repo's scripts/infer_policy.py loads the scene, swaps in the ONNX policy and steps MuJoCo at 50 Hz in a normal desktop viewer — the same actuator model, control rate and observation contract as the physical robot:

uv run scripts/infer_policy.py \
  --walking ~/microduck_policies/velstand.onnx \
  --new-cmd-obs
All the current policies use a 61-dim observation, so --new-cmd-obs is required (the older 51-dim mode predates the unified command contract).
Key Action
↑ / ↓ forward speed +/− (max 0.3 m/s)
← / → strafe (velstand tracks lateral commands)
A / E turn rate ±1.5 rad/s
SPACE zero command — the same policy stands
P shove the trunk 1.0 m/s in a random direction (the training curriculum's hardest push)
K / L / R trigger kick-left / kick-right / roulade (with those ONNX loaded)
Q quit

The official ball-kick scene is loaded automatically when you pass --kick-left / --kick-right (it needs --new-cmd-obs too) — that is exactly demo ③.

4 · 用鍵盤駕駛 policy

Repo 入面嘅 scripts/infer_policy.py 會載入場景、掛上 ONNX policy、以 50Hz 喺普通桌面 viewer 行 MuJoCo —— 同真機辦同一套 actuator 模型同觀察協議:

uv run scripts/infer_policy.py \
  --walking ~/microduck_policies/velstand.onnx \
  --new-cmd-obs
而家全部 policy 都行 61 維觀察,所以一定要加 --new-cmd-obs (舊 51 維模式係統一指今埋指之前嘅版本)。
按鍵 作用
↑ / ↓ 前進速度 +/−(上限 0.3m/s)
← / → 側移(velstand 有做橫向追蹤)
A / E 轉彎 ±1.5rad/s
空白鍵 歸零指令 —— 同一條 policy 會企定
P 將軀幹橫掃 1.0m/s 隨機方向(訓練中最重嘅推撞)
K / L / R 踢波(左/右)/前滾翻(要載埋對應 ONNX)
Q 離開

只要加 --kick-left / --kick-right,官方嘅踢波場景會自動載入(記得 --new-cmd-obs)—— 示範③ 就是這樣做到。

5 · How I recorded the three videos (headless, no GUI)

The videos above are straight MuJoCo renders of the same loop — I wrote a ~180-line script that reuses the repo's own PolicyInference class, runs the identical physics steps (4 sub-steps per control tick, BAM voltage actuator), screens offscreen with MuJoCo's EGL backend, and pipes frames straight into ffmpeg:

MUJOCO_GL=egl uv run python cloof_demo.py --demo walk    # / recovery / kick
# 1152x864, 25 fps, every 2nd control frame, libx264 CRF21
The essential trick on a GPU-less-display server: MUJOCO_GL=egl plus model.vis.global_.offwidth / offheight raised from the 640×480 default, and scene.flags[mjRND_SHADOW] = 0 to kill the strip-artefact shadows. I've kept the script; a cleaned-up version you can drop into scripts/ is at the bottom of this post.

5 · 三條片點錄(無畫面 headless 出)

上面三條片全部係同一個 loop 嘅直接 MuJoCo 渲染 —— 我寫咗個 ~180 行 script:重用 repo 本身嘅 PolicyInference 類別、行同一套物理(每個控制 tick 4 個子步、BAM 電壓 actuator),用 MuJoCo EGL 後端離屏渲染、frame 直接 pipe 入 ffmpeg:

MUJOCO_GL=egl uv run python cloof_demo.py --demo walk    # / recovery / kick
# 1152x864、25fps、每 2 個控制格出 1 frame、libx264 CRF21
無顯示器嘅關鍵三件事:MUJOCO_GL=egl、將 model.vis.global_.offwidth / offheight 調大過默認 640×480,再加 scene.flags[mjRND_SHADOW] = 0 殺掉條紋影子瑕疵。完整 script 我放在文末嘅 code block,直接貼入 scripts/ 就用得。

6 · Train your own walking policy on a 6 GB card

One command starts PPO on 2048 parallel ducks:

uv run train Mjlab-Velocity-Flat-MicroDuck \
  --env.scene.num-envs 2048 \
  --agent.logger tensorboard \
  --agent.max-iterations 2000 \
  --agent.run-name cloof_local_2048

Real numbers from my RTX 2060:

Setting Value
1024-envs probe 3.0 s/iter · 1.25 GB VRAM
2048-envs run (final) ~4.1 s/iter · 3.7 GB / 6.1 GB VRAM · GPU util ~49%
2000 iterations ~2 h15m total (2048 envs × 24 steps × 2000 ≈ 100M simulator steps)
Checkpoints logs/rsl_rl/<run>/model_NNNN.pt (writing continues while training; safe to Ctrl-C and resume with --agent.load-checkpoint)
Set --agent.logger tensorboard: the default logger is wandb and would want an account/API key. TensorBoard leaves everything on disk — run uv run tensorboard --logdir logs/rsl_rl to watch the reward climb live.

Export the winner (the exporter bakes the observation normalizer into the ONNX graph — this is the one path to a deployable file):

uv run scripts/export.py Mjlab-Velocity-Flat-MicroDuck \
  --checkpoint-file logs/rsl_rl/<yours>/model_1999.pt
# then drive it exactly like the official ones:
uv run scripts/infer_policy.py \
  --walking my_new_walk.onnx --new-cmd-obs --lin-vel-x 0.22

The fourth video above is my model from this run — same inference loop as the official policies.

6 · 喺 6GB 顯卡自己訓練行走 policy

一句 command 就會喺 2048 個並行環境開 PPO:

uv run train Mjlab-Velocity-Flat-MicroDuck \
  --env.scene.num-envs 2048 \
  --agent.logger tensorboard \
  --agent.max-iterations 2000 \
  --agent.run-name cloof_local_2048

RTX 2060 真實數字:

設定 數值
1024 環境試探 3.0秒/iter · 1.25GB VRAM
2048 環境正式跑 ~4.1秒/iter · 3.7/6.1GB VRAM · GPU 利用率 ~49%
2048 環境正式跑 3.5–3.7秒/iter · 3.7/6.1GB VRAM · GPU 利用率 ~49%
2000 個 iteration(完成) 2小時01分 · 98,304,000 步模擬 · 終值 mean reward 108 / episode 長度 940 / 跌倒次數 0.375
Checkpoints logs/rsl_rl/<run>/model_NNNN.pt(邊訓練邊存;Ctrl-C 停咗可以用 --agent.load-checkpoint 續訓)
一定要 --agent.logger tensorboard:默認 logger 是 wandb、會要求帳號/API key。TensorBoard 全部留喺本機 —— 行 uv run tensorboard --logdir logs/rsl_rl 可以看 reward 升(即「機械人學到行」)。

匯出(exporter 會將觀察 normalizer 燒落 ONNX graph —— 呢個先係可以部署嘅一條正途):

uv run scripts/export.py Mjlab-Velocity-Flat-MicroDuck \
  --checkpoint-file logs/rsl_rl/<yours>/model_1999.pt
#匯出之後,可以用返官方 policy 同一套推理 loop 開住玩:
uv run scripts/infer_policy.py \
  --walking my_new_walk.onnx --new-cmd-obs --lin-vel-x 0.22

上面第四條片就係我用佢自己訓練出嚟嘅 model —— 同官方 policy 用同一段推理 loop。

7 · What the numbers taught me

  • My 2-hour local PPO run: stable but conservative. The final checkpoint never fell in any probe run, tracked ~0.13 m/s at a 0.30 command and ~0.28 m/s at 0.50 (~56% tracking), and even has a dead zone — below ~0.25 command it just stands there. The official sprint reference still walks faster (0.53 m/s), but with 98 M simulation steps of my own on a consumer 6 GB card, "trained it overnight" here took one relaxed evening.
  • The shipped "walk" policy is deliberately lazy. velstand.onnx (the default robot brain, trained on rough terrain + backlash) under-tracks linear commands in the quiet flat scene: commanded 0.22 m/s → achieved ~0.07 m/s. I reproduced the same ratio with the official inference loop, so it isn't my wiring: robustness was traded for tracking. It still turns and recovers beautifully.
  • Use the reference sprint policy when you need speed. Pollen publishes reference policies separately (e.g. microduck-sprint-2m) — the sprint reference hit 0.53 m/s at a 0.35 m/s command (it overshoots; it was trained for a 2 m sprint) and walks a clean straight line, but falls if you stack a hard turn on it.
  • Fall recovery is the hero trick. VelStand's v8 update trains getting up from a fall inside the walking policy — in my clip the duck goes from lying-on-its-side to standing in about 1 s after a 1.15 m/s shove.
  • Everything is ONNX-first. Policies train on GPU (MuJoCo Warp) then export to a 61→14-dim ONNX — the same file runs in the simulator, on the real robot, and anywhere ONNX Runtime runs. That's why "download official→run locally" works with zero glue code.

7 · 數字教曉我嘅事

  • 我兩個鐘嘅本地 PPO 訓練:好穩但保守。最後 checkpoint 喺所有試探 run 都冇跌過,0.30 指令追到 ~0.13m/s、0.50 指令追到 ~0.28m/s(約 56%),仲有個死位 —— 指令低過 ~0.25 就企喺度唔郁。官方 sprint 參考版行得快啲(0.53m/s),不過屋企一張 6GB 顯卡模擬咗 9,830 萬步 —— 「過夜訓練」其實只係一個悠閒嘅晚上。
  • 出廠「行走」policy 是刻意保守的。velstand.onnx(真機默認腦,崎嶇地形+齒隙版)喺平整場景下令 0.22m/s 只得返 ~0.07m/s。我用官方推理 loop 重現咗同一個比率,證明唔係我接錯線:佢用追蹤表現換嚟強壯性。但佢轉彎同起身一樣頂呱呱。
  • 要速度就要用參考 sprint policy。Pollen 會另外出参考 policy(例如 microduck-sprint-2m)—— sprint 參考版喺 0.35m/s 指令下完全 0.53m/s(佢係為 2 米短跑訓練所以會超速),直線好靚,不過疊個硬轉彎就會跌。
  • 起身先係主角技。VelStand v8 將「跌倒起身」訓練入行走 policy 本體 —— 片中隻鴨俾 1.15m/s 撞埋去側晒落地,約 1 秒企返起身。
  • 系統以 ONNX 為中心。Policy 喺 GPU(MuJoCo Warp)訓練,匯出成 61→14 維 ONNX —— 同一個檔案在模擬器、真機同任何 ONNX Runtime 環境照行。「下載官方→本地玩」因而零膠水代碼。

8 · Ideas from here

  • Warm-start my local model from a Polle­n checkpoint and fine-tune on flat + a tiny reward nudge — a "policy remix" that then deploys on the real robot if you own one.
  • Run the rough-terrain and slope tasks; the environment code randomly re-rolls terrain every reset.
  • Classroom angle: the same 61→14 contract that swaps walk→recover→kick behind one button on the real robot is a lovely concrete example of model switching for AI students.
  • The roller-skating family (six tasks!) trains a duck on passive wheels under its feet — the funnest demo material, next on my list.

8 · 由呢度去到邊

  • 用 Pollen 嘅 checkpoint 做 warm-start、再喺平面微調 —— 「policy remix」,有真機的話可以部署返去真身上。
  • 試 rough terrain 同斜坡任務;環境碼每次 reset 會隨機重擲地形。
  • 課堂角度:真機上 61→14 嘅協議容許一粒按鈕切換 walk→recover→kick —— 對 AI 學生嚟講係「model switching」好具體嘅活例。
  • 滾軸溜冰任務家族(六個!)訓練機械人腳下裝被動輪 —— 最有娛樂性嘅示範素材,喺我嘅待辦內。

Credits & licenses

來源同授權

Bonus · the full headless render script

cloof_demo.py — click to unfold (paste into the repo's scripts/ directory)
#!/usr/bin/env python3
"""Headless demo-video recorder for microduck_rl (Cloof tooling).

Runs the official ONNX policies through the SAME loop as
scripts/infer_policy.py (BAM actuators, 50 Hz control, decimation 4),
renders offscreen with MuJoCo EGL, and pipes frames into ffmpeg.

Demos:
  walk      velstand protocol: stand -> forward -> strafe -> turn -> forward
  recovery  velstand: get pushed 1 m/s (curriculum cap) and recover
  kick      scene_ball.xml: kick left, then kick right
"""
import argparse
import json
import math
import os
import subprocess
import sys

os.environ.setdefault("MUJOCO_GL", "egl")

import numpy as np  # noqa: E402

REPO = "/home/cloofsiu/microduck_rl"
POLICIES = "/home/cloofsiu/microduck_policies"
sys.path.insert(0, os.path.join(REPO, "scripts"))
import infer_policy as ip  # noqa: E402
import mujoco  # noqa: E402

DEMO_DATA = {
    "walk": {
        "scene": ip.MICRODUCK_XML,
        "sessions": {"walking_onnx_path": f"{POLICIES}/../microduck_sprint/policy.onnx"
                     if os.path.exists(f"{POLICIES}/../microduck_sprint/policy.onnx")
                     else f"{POLICIES}/velstand.onnx"},
        "plan": [
            (0.5, "vel", (0.35, 0.0, 0.0)),
            (9.8, "vel", (0.0, 0.0, 0.0)),
        ],
        "seconds": 11.5,
    },
    "local": {
        "scene": f"{REPO}/src/mjlab_microduck/robot/microduck/scene_walk.xml",
        "sessions": {"walking_onnx_path": f"{REPO}/output.onnx"},
        "plan": [
            (0.5, "vel", (0.30, 0.0, 0.0)),
            (9.8, "vel", (0.0, 0.0, 0.0)),
        ],
        "seconds": 11.5,
    },
    "recovery": {
        "scene": ip.MICRODUCK_XML,
        "sessions": {"walking_onnx_path": f"{POLICIES}/velstand.onnx"},
        "plan": [
            (1.5, "push", (1.15, math.radians(105))),
            (7.5, "vel", (0.35, 0.0, 0.0)),
            (9.8, "vel", (0.0, 0.0, 0.0)),
        ],
        "seconds": 10.5,
        "push": 1.15,
    },
    "kick": {
        "scene": ip.MICRODUCK_BALL_XML,
        "sessions": {
            "walking_onnx_path": f"{POLICIES}/velstand.onnx",
            "kick_left_onnx_path": f"{POLICIES}/ball_kick_left.onnx",
            "kick_right_onnx_path": f"{POLICIES}/ball_kick_right.onnx",
            "kick_duration": 3.0,
        },
        "plan": [
            (1.2, "kick", "kick_left"),
            (4.5, "kick", "kick_right"),
        ],
        "seconds": 7.5,
    },
}


def trunk_addresses(model):
    jid = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_JOINT, "trunk_base_freejoint")
    return int(model.jnt_qposadr[jid]), int(model.jnt_dofadr[jid])


def build(spec, action_scale=1.0, width=1152, height=864):
    bam_model = ip.load_bam_model(ip.BAM_KP_FW, 7.4, ip.BAM_MAX_CURRENT)
    model, data, bam_ctrl, _names = ip.load_mujoco_with_bam(
        spec["scene"], bam_model, 0.005, None, ip.BAM_VIN_MIN)
    model.vis.global_.offwidth = max(width, 640)
    model.vis.global_.offheight = max(height, 480)
    policy = ip.PolicyInference(
        model, data, bam_ctrl=bam_ctrl, action_scale=action_scale,
        use_projected_gravity=True, new_cmd_obs=True,
        **spec["sessions"])
    qpos_adr, qvel_adr = trunk_addresses(model)
    data.qpos[qpos_adr:qpos_adr + 2] = 0.0
    data.qpos[qpos_adr + 2] = 0.125
    data.qpos[qpos_adr + 3:qpos_adr + 7] = [1, 0, 0, 0]
    for i, idx in enumerate(policy.joint_qpos_indices):
        data.qpos[idx] = policy.default_pose[i]
    if spec["scene"] == ip.MICRODUCK_BALL_XML:
        bj = mujoco.mj_name2id(model, mujoco.mjtObj.mjOBJ_JOINT, "ball_free")
        bqa = int(model.jnt_qposadr[bj])
        data.qpos[bqa:bqa + 3] = [-0.5, 0.35, 0.035]
        data.qpos[bqa + 3:bqa + 7] = [1, 0, 0, 0]
    bam_ctrl.reset(data.qpos)
    policy.set_position_targets(policy.default_pose)
    mujoco.mj_forward(model, data)
    policy.vel_max_x, policy.vel_min_x = 0.3, -0.3
    policy.vel_max_y, policy.vel_min_y = 0.2, -0.2
    policy.vel_max_ang = 1.5
    return model, data, bam_ctrl, policy, qpos_adr, qvel_adr


def plan_op(plan, t):
    for (pt, kind, args) in plan:
        if abs(t - pt) < 0.01:
            return kind, args
    return None, None


def run(demo, out_dir, width, height, fps_div, push_mag=None):
    spec = dict(DEMO_DATA[demo])
    if push_mag is not None:
        spec["plan"] = [(t, k, (push_mag, a) if k == "push" else v)
                        for (t, k, v) in spec["plan"]]
    model, data, bam_ctrl, policy, qpos_adr, qvel_adr = build(spec, width=width, height=height)
    os.makedirs(out_dir, exist_ok=True)
    mp4 = os.path.join(out_dir, f"duck_{demo}.mp4")

    from mujoco import Renderer
    metrics = {"demo": demo, "steps": 0, "control_dt": 0.02,
               "vel_avg": [], "frames": 0}
    renderer = None
    ffmpeg = None
    try:
        renderer = Renderer(model, height=height, width=width)
        cam = mujoco.MjvCamera()
        cam.azimuth = 130
        cam.elevation = -14
        cam.distance = 0.78

        ffmpeg = subprocess.Popen([
            "ffmpeg", "-y", "-loglevel", "error",
            "-f", "rawvideo", "-pix_fmt", "rgb24",
            "-s", f"{width}x{height}", "-r", "25", "-i", "pipe:",
            "-c:v", "libx264", "-preset", "medium", "-crf", "21",
            "-pix_fmt", "yuv420p", "-movflags", "+faststart", mp4,
        ], stdin=subprocess.PIPE)

        decim = 4
        control_dt = decim * model.opt.timestep
        n_steps = int(spec["seconds"] / control_dt)
        metrics["steps"] = n_steps
        metrics["control_dt"] = control_dt
        frame_every = max(1, fps_div)
        vel_hist = []
        start_pos = None
        for step in range(n_steps):
            t = step * control_dt
            kind, args = plan_op(spec["plan"], t)
            if kind == "vel":
                policy.set_vel_cmd(*args)
                print(f"[{t:5.1f}s] cmd -> {args}", flush=True)
            elif kind == "push":
                mag, ang = args
                data.qvel[qvel_adr + 0] = mag * math.cos(ang)
                data.qvel[qvel_adr + 1] = mag * math.sin(ang)
                print(f"[{t:5.1f}s] PUSH {mag:.2f} m/s @ {math.degrees(ang):.0f} deg", flush=True)
            elif kind == "kick":
                policy.trigger_behavior(args)
                print(f"[{t:5.1f}s] trigger {args}", flush=True)

            policy.update_ground_pick_phase(control_dt)
            policy.update_behavior(control_dt)
            action = policy.infer()
            policy.apply_action(action)
            for _ in range(decim):
                bam_ctrl.update()
                mujoco.mj_step(model, data)

            quat = data.qpos[qpos_adr + 3:qpos_adr + 7].astype("float32")
            v_w = data.qvel[qvel_adr:qvel_adr + 3].astype("float32")
            v_b = policy.quat_rotate_inverse(quat, v_w)
            vel_hist.append((float(v_b[0]), float(v_b[1]), float(t)))
            if len(vel_hist) > 50:
                vel_hist.pop(0)
            if step % 50 == 0:
                avg = tuple(float(sum(v[i] for v in vel_hist) / len(vel_hist)) for i in (0, 1))
                metrics["vel_avg"].append({"t": round(float(t), 2), "fwd": round(avg[0], 3),
                                           "lat": round(avg[1], 3),
                                           "cmd": [float(policy.vel_cmd[0]), float(policy.vel_cmd[1])],
                                           "trunk_z": round(float(data.qpos[qpos_adr + 2]), 4),
                                           "policy": policy.current_policy})
            if start_pos is None:
                start_pos = np.array(data.qpos[qpos_adr:qpos_adr + 2])
            if step % frame_every == 0:
                p = data.qpos[qpos_adr:qpos_adr + 3]
                cam.lookat[:] = [p[0], p[1], max(0.02, float(p[2]) * 0.55)]
                renderer.update_scene(data, camera=cam)
                renderer.scene.flags[mujoco.mjtRndFlag.mjRND_SHADOW] = 0
                img = renderer.render()
                ffmpeg.stdin.write(img.tobytes())
                metrics["frames"] += 1
        ffmpeg.stdin.close()
        ffmpeg.wait()
        metrics["travelled_m"] = round(float(np.abs(data.qpos[qpos_adr:qpos_adr + 2] - start_pos).sum()), 3)
        with open(os.path.join(out_dir, f"duck_{demo}_metrics.json"), "w") as f:
            json.dump(metrics, f, indent=2)
    finally:
        if renderer is not None:
            try:
                renderer.close()
            except Exception as exc:
                print(f"renderer close warning: {exc}", flush=True)
        if ffmpeg is not None and ffmpeg.stdin is not None and not ffmpeg.stdin.closed:
            ffmpeg.stdin.close()
            ffmpeg.wait(timeout=60)
    print(f"DONE {mp4}  travelled={metrics.get('travelled_m')}", flush=True)


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--demo", required=True, choices=list(DEMO_DATA))
    ap.add_argument("--out", required=True)
    ap.add_argument("--width", type=int, default=1152)
    ap.add_argument("--height", type=int, default=864)
    ap.add_argument("--fps-div", type=int, default=2)
    ap.add_argument("--push", type=float, default=None)
    a = ap.parse_args()
    run(a.demo, a.out, a.width, a.height, a.fps_div, a.push)


if __name__ == "__main__":
    main()