Why first-person data, and why now.
Robot learning is moving from what can be teleoperated to what people already do every day at work. The argument, its sources and its limits.
Human video now transfers to robots.
Physical Intelligence found that once a vision-language-action model has been pretrained on enough scenes, tasks and robot bodies, adding human video recorded with head and wrist cameras nearly doubles its performance on generalization tasks that appear only in that human data.1
Two details matter. The transfer showed up only after diverse robot pretraining, and the human data was labeled with 3D hand tracks and dense language. Human video extends what robot data teaches. It does not replace it, and it has to be labeled well to count.
It scales on a curve.
NVIDIA trained on 20,854 hours of action-labeled first-person video and measured a log-linear relationship between hours and results: average task completion rose from 0.30 at 1,000 hours to 0.71 at 20,000.2 GR00T N1.7 is pretrained on that data.
A predictable curve turns data into a budget line: more hours of the right kind buy a known amount of capability.
Wearables beat teleoperation on throughput.
UMI collected demonstrations more than three times faster than teleoperation on its cup task.3 DexWild recorded 201 demonstrations an hour with a wearable rig, against 43 with a teleoperation setup.4
Teleoperation also needs a robot for every operator. For a sense of the labor cost alone, Tesla has advertised $25.25 to $48 an hour for operators who wear motion-capture suits.5 Recording people who already do the job turns an existing shift into data, without building a fleet of robots first.
Hours are becoming a commodity. Skill is not.
Build AI released 10,000 hours of first-person factory video from more than 2,100 workers in 85 factories under an open license,6 open-sourced 100,000 hours from 14,228 workers a month later,7 and announced a million hours in April 2026.8 Generalist trains on more than 270,000 hours of manipulation data from wearable capture devices.9
When hours are plentiful, a dataset is worth what it contains that others do not:
- Skilled work, recorded from people who do it daily, with their trade and tenure attached.
- Errors and recoveries, labeled as actions of their own.
- Calibrated head, wrist and cell cameras on one clock, so hands can be tracked in 3D.
- Labels a policy consumes: hand pose, contacts, steps of the work instruction.
Consent is now a diligence question.
In August 2026 Bloomberg reported that workers in India were paid about $2.62 an hour to wear head cameras, and that many were not told the footage would train robots.10
Buyers will be asked where their data came from. Every neocognix clip carries a provenance record: who recorded it, what they were told and in which language, what they were paid, and what was blurred.
The spend is already there.
Bessemer estimates that robot data costs across the industry will exceed $3 billion over the next two years.11 The labs buying that data need it to be comparable, auditable and in the format they train on.
Where first-person data falls short.
- Hands are not grippers. Retargeting keeps wrist pose and grip aperture. It loses force and fingertip contact detail.
- Some tasks need the robot’s own body. Tight insertions and forceful tool use still call for teleoperated or on-robot data when fine-tuning.
- Heads miss things. A head camera cannot see under the palm. That is why every station adds wrist and cell cameras.
- Transfer needs a base. The published gains appear on models already pretrained on diverse robot data.
The name.
neocognix takes its name from the Neocognitron, Kunihiko Fukushima’s 1980 model of how the visual cortex recognizes patterns, and a direct ancestor of today’s convolutional networks.12 It learned to see from examples. So do the robots our data trains.
Sources
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
See the data behind the argument.
We send a sample episode with every stream and label layer, in the format you train on. No NDA needed.