.. _data: Motion data =========== Motion libraries for WBC tracking live under **`data///`**. .. tip:: First time? Convert the bundled samples and train with ``Wbc-G1`` — see :doc:`workflows/quickstart`. Errors like empty ``npz/``: :doc:`troubleshooting`. A small **`samples/`** folder is version-controlled under each robot (e.g. ``g1/samples/``) with a few clips from public datasets for smoke tests. All other dataset folders stay local — download full libraries from Hugging Face (see robot guides) or convert your own clips. Only READMEs, ``samples/``, and ``.gitkeep`` placeholders are version-controlled; other clip and bundle files stay gitignored. Robot guides ------------ .. list-table:: :header-rows: 1 * - Robot - Guide * - **g1** (in-tree example) - ``data/g1/README.md`` — downloads, joint order, example datasets Directory layout ---------------- .. code-block:: text data/// *.csv / *.pkl # source clips (or under raw/) npz/.npz # per-clip exports (source of truth for training) .npz # optional cached stack (--cache-motion-bundle) - Put source clips in the dataset folder or in **`raw/`** — converters prefer ``raw/`` when it contains ``.csv`` or ``.pkl`` files. - **`npz/`** and **`*.npz`** are **never committed** — run ``wbc-mjlab-data-to-npz`` after adding source clips. - **`.npz`** is optional: written only when you pass **`--cache-motion-bundle`** on train/play. - **`params/motion_library.yaml`** is written automatically on **play** from the loaded motion bundle. Per-clip NPZ schema ------------------- ``wbc-mjlab-data-to-npz`` writes one ``.npz`` per source clip under ``npz/``. ``MotionCommand`` / ``MotionLoader`` load either a directory of those clips or a pre-stacked bundle. Required arrays (time axis ``T``, joints ``J``, bodies ``B``): .. list-table:: :header-rows: 1 :widths: 28 72 * - Key - Shape / meaning * - ``fps`` - Scalar or length-1 array — export rate (default **50** Hz) * - ``joint_pos`` - ``(T, J)`` — joint positions (rad) * - ``joint_vel`` - ``(T, J)`` — joint velocities (rad/s) * - ``body_pos_w`` - ``(T, B, 3)`` — body positions in world frame (m) * - ``body_quat_w`` - ``(T, B, 4)`` — body orientations, **xyzw** * - ``body_lin_vel_w`` - ``(T, B, 3)`` — body linear velocities (m/s) * - ``body_ang_vel_w`` - ``(T, B, 3)`` — body angular velocities (rad/s) Bodies ``B`` follow the robot model body order from FK. Training selects a **keybody subset** via ``MotionCommandCfg.body_names`` (robot entity / preset). Optional stacked bundle (``--cache-motion-bundle``) concatenates clips and stores segment start/length metadata so RSI can sample across the library. Sagittal mirroring (``--mirror``) --------------------------------- Bilateral humanoids often benefit from **left-right mirrored reference motion**: it doubles the motion library without new mocap and improves left/right generalization during RSI sampling. Pass ``--mirror`` to ``wbc-mjlab-data-to-npz`` to export an extra NPZ per source clip: .. code-block:: text data/g1/samples/npz/ walk1_subject1.npz walk1_subject1_mirror.npz # sagittal mirror of walk1_subject1 Training discovers both files automatically (``npz/*.npz``). No train/play flag is required. What gets mirrored ~~~~~~~~~~~~~~~~~~ Mirroring runs **after FK** on the exported arrays (``motion/motion_mirror.py``): .. list-table:: :header-rows: 1 :widths: 28 72 * - Channel - Transform * - ``joint_pos``, ``joint_vel`` - Swap left/right joint pairs; negate roll/yaw axes (see robot symmetry config) * - ``body_pos_w``, ``body_lin_vel_w`` - Swap left/right bodies; flip world **y** (reflection across the XZ plane) * - ``body_quat_w`` (xyzw) - Swap bodies; negate quaternion **x** and **z** * - ``body_ang_vel_w`` - Swap bodies; negate **x** and **z** components (axial vector under reflection) * - ``fps``, ``joint_names``, ``robot`` - Unchanged The robot's **`RobotSymmetryConfig``** defines joint swap pairs and signs. G1 ships in-tree at ``robots/g1/symmetry.py``. Extension robots register their own config (see :doc:`extensions/robot_entity` and :doc:`extensions/extensions`). CLI notes ~~~~~~~~~ - ``--mirror`` requires a symmetry config for ``--robot``; otherwise conversion errors. - Mirrored output stem: ```` (default suffix ``_mirror``). - Sources whose stem already ends with the mirror suffix are skipped (avoids ``walk_mirror_mirror.npz`` on re-convert). .. code-block:: bash uv run wbc-mjlab-data-to-npz --robot g1 --dataset samples --batch-size 8 --mirror Online mirroring during training (phase 2) will reuse the same symmetry config inside ``MotionCommand``; offline ``--mirror`` is the recommended starting point. Supported formats ----------------- Source layouts are defined in ``wbc_mjlab.motion.motion_formats``. The converter infers format from the file extension; pass ``--format`` to pick a registered name. .. list-table:: :header-rows: 1 * - Format key - Extension - Role * - ``default`` - ``.csv`` - LAFAN / retarget CSV (no header, m + quat xyzw + rad) * - ``gmr_pkl`` - ``.pkl`` - GMR / ``bvh_to_robot`` pickle dict * - **NPZ** - train / play - Per-clip exports or a pre-stacked training bundle CSV ``default`` (LAFAN / retarget) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Comma-separated rows, no header:: root_pos(3) + root_rot_xyzw(4) + joint_pos(n_dof) # meters, radians Joint count must match the target robot. Default input rate is 30 Hz unless ``--input-fps`` is set. PKL (``gmr_pkl``) ^^^^^^^^^^^^^^^^^ Python dict with at least:: fps, root_pos (T×3), root_rot (T×4, xyzw), dof_pos (T×n_dof) Optional keys: ``joint_names``, ``dof_joint_names``, or ``joint_order``. Workflow -------- .. code-block:: bash # 1. Add clips under data/// uv run wbc-mjlab-data-to-npz --robot --dataset # large libraries: parallel FK on GPU uv run wbc-mjlab-data-to-npz --robot --dataset --batch-size 8 # optional: sagittal mirror (doubles npz/ — see "Sagittal mirroring" above) uv run wbc-mjlab-data-to-npz --robot g1 --dataset samples --mirror # 2. Train uv run wbc-mjlab-train --task Wbc- --dataset # 3. Optional cached bundle uv run wbc-mjlab-train --task Wbc- --dataset --cache-motion-bundle ``--batch-size N`` runs **N parallel FK workers** on GPU. Use ``--batch-size 1`` for ``--render`` preview. Conversion requires **`--robot``** (MuJoCo asset). Train/play use **`--task``** (robot inferred). Train / play motion source -------------------------- .. list-table:: :header-rows: 1 * - Flag - Resolves to * - ``--dataset `` - ``data///`` (loads ``npz/*.npz`` in memory) * - ``--dataset-path `` - load ``npz/*.npz`` in that folder * - ``--dataset-path .npz`` - that file directly * - ``--motion-file .npz`` - explicit NPZ path * - ``--cache-motion-bundle`` - write/read ``/.npz`` on disk Version control --------------- Everything under ``data/`` is gitignored except READMEs, ``samples/``, and ``.gitkeep`` files. See the repository ``.gitignore``.