From human work to robot data

We collect data on how people do real work, then turn it into structured datasets for robot training and evaluation. Together we define the tasks, the annotations, and how to review the data before delivery.

Discuss a data collaboration
Hands assembling electronic parts and wiring

How we turn worksite data into robot data

  1. Collect

    Real tasks and their variations

  2. Review

    Task coverage, quality, and privacy

  3. Annotate

    Actions, objects, tools, and outcomes

  4. Deliver

    Datasets and quality records

What the data can describe

Task demonstrations

Show how people perform a task from start to finish, including the tools and objects they use. We agree on which variations and exceptions to collect.

Actions and outcomes

Label when each action happens, what it involves, and whether the person completes it, retries it, or runs into a problem.

Hand and object motion

When the project calls for it, estimate hand landmarks and object regions over time. Review visibility and limitations in the estimates.

Quality and acceptance

Check coverage, visibility, and label accuracy against the agreed criteria. We track what we collect, what passes review, and what we can deliver.

What your dataset can include

A task and its annotations

Each label points to a specific moment in the task demonstration. The diagram shows how a dataset can link actions, hand motion, and review notes.

Task demonstration

A real task, start to finish

Action labels

Pick / align / assemble / check

Hand motion

Left and right hand landmarks and estimate status

Review records

Quality, gaps, and approved uses

Task recordings
Examples of complete tasks with agreed recording specifications
Action annotations
Timing, objects, tools, and task outcomes
Motion data
Estimated hand landmarks or object regions, when requested
Review and rights records
Quality notes, approved uses, and delivery scope
Gloved hands inspecting a machined metal part at a workbench

Collect data during real work

We start with tasks such as assembly, packaging, or inspection. Together we choose which steps, variations, and exceptions to document and how to collect the data within the site’s normal workflow.

  • Agree on safe tasks and collection boundaries
  • Check that the relevant actions and objects are visible
  • Set rules for confidential areas and external use

Collect data with worksite partners

Site safety

Check protective equipment, hygiene requirements, and mounting comfort. Stop recording if safety issues arise.

People and confidentiality

Confirm participation and review personal or confidential information. Footage is not used for individual performance assessment.

Approved use

Share material the site reviewed and approved within the agreed purpose and scope.

From task definition to data delivery

  1. Define what the robot needs to learn

    Choose the tasks and variations that matter to your training or evaluation goals.

  2. Review a sample together

    Check whether it covers the relevant actions and exceptions. Agree on annotations and acceptance criteria.

  3. Collect, annotate, and deliver

    Scale collection to the agreed scope and check the dataset against the delivery requirements.

What does your robot need to learn?

Tell us which tasks matter, what the data needs to describe, and how you plan to use it.

Discuss a data collaboration