The Mocap Pipeline and Solving

A motion-capture stage delivers something surprisingly useless: for every frame, a flickering cloud of anonymous 3-D dots. The cameras saw bright reflective markers and triangulated where each dot floated in space — but they don't know that this dot is a left knee and that one is the right shoulder, some dots vanished behind an arm for half a second, and a stray reflection off a watch spawned a ghost point that never should have existed. Raw mocap is not an animation. It is evidence.

This page walks the solving pipeline — the chain of steps that turns that noisy dot cloud into a clean, retargetable clip of joint-angle curves that a rigged character can actually play back. Before the dots ever arrive you need motion capture systems to record them; here we assume the capture happened and ask the harder question: now what?

The pipeline at a glance

Solving is a fixed sequence of stages. Each one hands cleaner data to the next, and skipping any of them shows up as garbage later — a foot that slides, an elbow that pops, a spine that shivers.

Calibrate & reference — capture a T-pose (or A-pose) and a range-of-motion take so the software learns each marker's fixed offset from the bone it rides on, and learns the actor's limb lengths.
Label — assign every anonymous 3-D dot to its named body location (LSHO, RKNE, …), automatically where possible, by hand where the auto-labeller is unsure.
Clean — gap-fill occluded markers, denoise jitter, delete ghost markers, and remove spikes.
Solve — fit the actor's skeleton to the labelled marker cloud each frame, an IK-like optimisation recovering the joint transforms.
Retime & polish — resample to the target frame rate, then fix foot-skate and contacts by locking planted feet.
Export — write out joint-angle (rotation) curves per bone, ready to retarget.

Calibration and the reference pose

Everything downstream leans on calibration. The actor stands in a known T-pose — arms out, legs straight — for a beat. In that pose the software knows roughly where each named bone should be, so it can measure how far each marker sits from the bone it's taped to. Those marker offsets — a marker is glued to skin a couple of centimetres off the bone — become fixed constants used by the solve on every later frame. A range-of-motion take (the actor cycling every joint through its extremes) then nails down limb lengths and joint limits. Get calibration wrong and the whole clip inherits the error, silently.

Labelling: naming the dots

The cameras output unlabelled trajectories — dot #47 in one frame may be dot #12 in the next. The labeller assigns each dot to a named marker slot (LSHO, RANK, C7, …) using the reference marker layout and continuity between frames: a marker can't teleport, so the dot nearest to where LSHO was last frame is probably LSHO now. Auto-labelling handles the easy majority; the artist steps in when markers cross, occlude, or swap — a common failure is two nearby markers trading identities as the actor's wrists cross, which the solve would read as a limb suddenly snapping to the wrong side.

Cleaning: from evidence to signal

Cleaning is four distinct chores, each fixing a different sin of the raw capture:

Gap-filling — when a marker is occluded (hidden behind a limb or prop) its trajectory has a hole; fill it by interpolation, or predict it from a rigid-body neighbour that stayed visible.
Denoising — smooth high-frequency jitter (sub-millimetre camera noise) that would otherwise read as a buzzing tremor, without smearing genuine fast motion.
Ghost removal — delete spurious points from stray reflections (jewellery, wet floor) that belong to no real marker.
Spike removal — kill single-frame teleports where a marker's position jumps impossibly far and snaps back.

The chart below shows the most common of these, a gap-fill: one marker's height y(t) over time, with a stretch of frames dropped out by occlusion (the raw trace), and the smooth interpolated reconstruction laid over the hole.

The solve: fitting a skeleton to the cloud

With labelled, clean markers in hand, the solve answers one question per frame: what pose of the actor's skeleton best explains where all the markers are right now? The skeleton is a tree of bones with joint transforms (mostly rotations, one root translation). From calibration we know each marker's offset from its parent bone, so given a candidate pose we can predict where every marker ought to be. The solve searches over the joint transforms to make those predictions match the measured markers as closely as possible — an optimisation very much like inverse kinematics, but driven by dozens of marker targets at once.

For each frame, find the joint transforms \theta minimising the total squared distance between predicted and measured marker positions: \theta^\star = \arg\min_{\theta}\; \sum_{i=1}^{M} \big\lVert\, p_i(\theta) - m_i \,\big\rVert^2 where m_i is the measured position of marker i and p_i(\theta) is where the skeleton in pose \theta places that marker (bone transform composed with the calibrated offset).

Because there are more markers than joint degrees of freedom, the fit is over-determined — no pose matches every marker perfectly, so we settle for the least-squares best. That redundancy is a feature: it averages out residual noise and survives a marker or two dropping out.

Worked example: solving the pelvis from three back markers

The pelvis is usually tracked by a small rigid cluster — say three markers on the lower back that don't move relative to each other. Call their calibrated reference positions (in the pelvis's own frame, learned from the T-pose) r_1, r_2, r_3, and their measured world positions this frame m_1, m_2, m_3. Because the cluster is rigid, some rotation R and translation t should carry the reference points onto the measured ones. We want the (R, t) — the pelvis transform — minimising

E(R, t) = \sum_{i=1}^{3} \big\lVert\, R\,r_i + t - m_i \,\big\rVert^2, \qquad R^\top R = I,\ \det R = 1.

This is the classic rigid-alignment (Procrustes / Kabsch) problem, and it has a clean recipe:

Compute the two centroids \bar r = \tfrac{1}{3}\sum_i r_i and \bar m = \tfrac{1}{3}\sum_i m_i, and centre both sets: r_i' = r_i - \bar r, m_i' = m_i - \bar m.
Form the cross-covariance H = \sum_i r_i'\, m_i'^{\top} and take its SVD H = U \Sigma V^\top.
The best rotation is R = V\,\mathrm{diag}(1, 1, \det(VU^\top))\,U^\top (the \det term forbids an accidental reflection).
Recover the translation from the centroids: t = \bar m - R\,\bar r.

That (R, t) is the pelvis pose for this frame — the root of the whole skeleton — obtained by minimising squared marker error over just three points. Repeat per frame and you have the pelvis's motion curve; the full-body solve does the same idea for every joint at once, chained down the skeleton tree.

Retiming and the final export

Capture often runs at an odd or high rate — 120, 240 fps — while the shot needs 24 or 30. The solved motion is resampled to the target frame rate by interpolating the joint curves (not the raw markers). Finally the clip is exported as joint-angle curves: one rotation channel per bone over time, a compact representation any rigged character can consume and retarget onto a differently-proportioned skeleton. The dot cloud is gone; what remains is clean, editable animation.

Because markers ride on skin, not bone. Skin slides, jiggles and bulges as muscles flex, so a marker's world path is the bone's motion plus a wobble of soft-tissue artefact. Bolting a joint directly to a marker would import all that wobble. The whole point of the skeleton solve is to recover the rigid bone motion underneath — by fitting many markers per limb, the soft-tissue slop averages out and you get the clean joint rotation the skin was only hinting at. It also lets you retarget: a skeleton of joint angles maps onto any character, whereas raw marker positions are locked to one actor's exact proportions.

The most notorious mocap artefact is foot skate (or "foot sliding"). When a foot is planted on the ground, it should be perfectly still — but tiny errors in the solve, in marker noise, or in retargeting onto a character whose legs are a different length nudge the solved foot a few millimetres every frame. Played back, those nudges accumulate into a visible glide across the floor: the character looks like it's skating on ice, and once you notice it you can't unsee it.

The fix is contact detection plus constraint. Detect the frames where the foot is genuinely planted (low foot height, near-zero foot velocity), then IK-lock that foot to a fixed ground position for the whole plant, letting inverse kinematics bend the knee and hip to keep the rest of the leg attached. The planted foot stops moving, the skate vanishes, and the weight of the step reads correctly. Never ship a walk cycle without checking the feet.