What is the most important quality metric for robot training data?

There is no single one, but camera sync and 3D hand keypoint error come first for human video: if frames do not line up or hands are misplaced in 3D, the trajectories a policy learns from are wrong.

What sync error is acceptable between cameras?

For wearable head and wrist cameras sharing a clock, a few milliseconds is a reasonable bar. For a fixed camera at a lower frame rate, within half a frame is typical.

Why does consent documentation matter to a buyer?

Because the buyer inherits the risk. A per-clip record of the briefing, consent, payment and blurring lets you answer diligence questions from investors, customers and regulators.

Should I evaluate on a sample before buying?

Yes. Load one episode in your format, check the measurements, and run it through your own evals before committing to volume.

How to evaluate robot training data: a 12-point checklist

Before buying human or robot training data, ask for twelve measurements: sync error, calibration, keypoint error, label agreement, provenance, consent and formats among them. A good vendor reports each one per delivery.

26 August 2026 · 4 min read · neocognix

Key takeaways

Ask for numbers, not adjectives: every claim about a dataset should come with a measured value and the method behind it.
The four technical checks that matter most are camera sync, calibration residuals, 3D hand keypoint error and label agreement.
Provenance is now a diligence item: who recorded each clip, what they were told, what they were paid, and what was blurred.
Run a sample through your own evals before you sign for volume.

Why a checklist

Datasets for robot learning are sold by the hour, but hours are the least informative number on the invoice. Two datasets of equal size can differ tenfold in what a policy learns from them, depending on how the cameras were synchronized, how well hands were labeled and what work was recorded. Open datasets now reach a million hours,¹ so the question is no longer how much, but how good.

Capture

Stream sync. Ask for the measured offset between cameras and how it was checked. Frames from the head, wrist and fixed cameras must line up, or 3D hand tracking is wrong. A few milliseconds between wearable cameras is a reasonable bar.
Calibration. Each camera should be calibrated per session, ideally at the start and end, with the mean reprojection error reported. Sub-pixel error is achievable with a calibration board.
Camera placement. Head height, pitch and field of view should be logged per take, so you can filter or reproject toward your robot’s own cameras.
Viewpoints. A head camera alone cannot see under the palm. Look for wrist and fixed cameras on the same clock.

Labels

Hand keypoint error. Ask how 3D keypoints were checked, against what reference, and the mean error in millimeters.
Label agreement. Action boundaries and classes should be double-labeled on a blind sample, with boundary offset in milliseconds and Cohen’s kappa reported per batch.
Label schema. Check that actions, contacts, grasp types and task steps match what your policy consumes, and that you can drop the layers you do not need.
Failure handling. Ask what happens to an episode that fails a threshold. Dropped is the right answer; patched with a caveat is not.

Provenance and consent

Buyers are now asked where their data came from. In August 2026 Bloomberg reported that workers in India were paid about $2.62 an hour to wear head cameras, and that many were not told the footage would train robots.²

Consent record per clip. Who recorded it, the briefing they received and in which language, and when they consented.
Payment. How operators were paid, and whether it was on top of their wages.
De-identification. Whether faces were blurred in every view, how that was audited, and the measured miss rate.

Delivery

Formats. The data should load in the format you train on, such as LeRobot, RLDS, MCAP, HDF5 or WebDataset, with calibration and provenance files alongside.

How to run the evaluation

Request one sample episode with every stream and label layer, in your format.
Load it, check the twelve items above, and read the QA and provenance files.
Run it through your own evals next to the data you already have.
Only then scope a pilot: one site, one task family, fixed hours and fixed thresholds.

For background on what these streams are, read what egocentric data is. To see a full example, our datasheet publishes every threshold we deliver against, and you can request a sample episode.

Questions

What is the most important quality metric for robot training data?: There is no single one, but camera sync and 3D hand keypoint error come first for human video: if frames do not line up or hands are misplaced in 3D, the trajectories a policy learns from are wrong.
What sync error is acceptable between cameras?: For wearable head and wrist cameras sharing a clock, a few milliseconds is a reasonable bar. For a fixed camera at a lower frame rate, within half a frame is typical.
Why does consent documentation matter to a buyer?: Because the buyer inherits the risk. A per-clip record of the briefing, consent, payment and blurring lets you answer diligence questions from investors, customers and regulators.
Should I evaluate on a sample before buying?: Yes. Load one episode in your format, check the measurements, and run it through your own evals before committing to volume.

Sources

1Egocentric-1M announcement · Build AI, Apr 2026
2Thousands of India workers are helping AI firms train robots to replace them · Bloomberg, Aug 2026