---

name: cdf-curation-and-retrieval description: >- NVIDIA CDF curation and retrieval. Cosmos Curator video and image pipelines (split, dedup, shard, annotate) with filtering, captioning, and embedding. Cosmos Dataset Search ingest, search, and capability qualification. TAO data mining embeddings and neighbor matching. Ships domain event catalogs, including workplace employee conduct monitoring, which requires legal review before use. Use when a request names Cosmos Curator, Cosmos Dataset Search, CDF, or their Make and CLI surfaces. Not for model training, orchestration, or unrelated dataset questions. license: CC-BY-4.0 AND Apache-2.0 owner: NVIDIA service: cosmos-data-factory reviewed: 2026-10-20 metadata: author: "NVIDIA opensource@nvidia.com" version: 1.2.0 tags: - data-curation - dataset-retrieval - cosmos-curator - tao - physical-ai

CDF Curator Operator Skill

GPU-accelerated video and image curation via NVIDIA Cosmos Curator inside Cosmos Data Factory — Curation and Retrieval (cdf-curation-and-retrieval). This skill is a short Curator index. Embedding handoff boundaries live in data-mining.md and curation-retrieval-workflow.md; mining execution is make help and the repository cookbooks.

• Video: split, dedup, shard.
• Image: annotate (load → filter → embed → caption → write).
• Handoff: Curator IV2 or CE1 parquet that downstream mining can consume. See data-mining.md and curation-retrieval-workflow.md.
• CDS stitch: Curator → CDS ingest/search. Operator Path D (public GA 1.2.0; not supported on Blackwell GPUs): see cosmos-dataset-search.md.

Purpose

Turn raw video and image collections into curated, training-ready datasets. This skill configures and runs cosmos-curator pipelines (clip splitting, filtering, captioning, embeddings, SAM3 event verification, dedup, WebDataset sharding, image annotate) and supports KPI-driven, distribution-aware, and restrictive curation.

Instructions

1. Classify the request as advisory, config, run, TAO handoff, or CDS ingest/search. Do not mix those routes. A config request with no KPI output or samples goes through the calibration interview, which can stop with unresolved requirements before any YAML is written.
2. For config work, complete the mandatory pre-flight below before writing YAML.
3. For an explicit run, load running-pipelines.md, validate the config, obtain credentials only through approved injection, then execute after authorization.
4. For a TAO handoff, validate Curator output and the declared embedding family using data-mining.md before mining.
5. For CDS ingest or search, confirm CDS is healthy first, then follow cosmos-dataset-search.md.
6. Return the Output Format below. Never print secret values.

Examples

• Advisory: "How much SHM should I set?" → read running-pipelines.md and report guidance. Do not run Docker.
• Config with no KPI: complete the calibration interview in calibration-config.md, then emit YAML.
• Run: after sample clips are staged, make run-pipeline with the traffic split-minimal cookbook recipe only after preflight and user authorization.
• FFmpeg missing in the container: install the host sidecar (make ffmpeg-install) per ffmpeg-sidecar.md.

Inputs

Required inputs depend on the route:

• Advisory request: the question plus relevant repository and config context.
• Config request: input and output locations, domain and goal, available KPI output or representative samples, and hardware constraints. If neither KPI output nor samples exist, complete the calibration interview before writing YAML.
• Run request: reviewed config path, data and model paths, runtime, GPU, and SHM constraints, and explicit authorization to execute.
• TAO handoff: validated artifact paths and declared embedding family.

Optional inputs include target distributions, event taxonomy, prompt choices, existing output metadata, and user-approved operational constraints.

Resolve inputs in this order: repository configuration and validated run artifacts; explicit prompt arguments and corrections; available agent context; then the broad user prompt. Explicit user instructions remain authoritative unless unsafe or incompatible, in which case stop and explain the conflict. Never infer secret values: credentials come only from approved runtime injection.

Prerequisites

• Repository access: the agent must be able to read the Makefile, the configs/ and cookbook/ trees, and every file under references/.
• GPU host with NVIDIA drivers + nvidia-container-toolkit; Docker. SHM sized from host RAM (SHM_SIZE, default 24gb).
• cosmos-curator image: make pull uses the pin configured by the example env file and Make. No separate product engine image. Source builds are developer-only; see cosmos-curator.md.
• FFmpeg host sidecar for distributable images (make ffmpeg-install) — they do not bundle FFmpeg. See ffmpeg-sidecar.md.
• Credentials as needed: inject S3 and captioning API keys at runtime through an approved secret manager or operator deployment mechanism. Never put secret values in repository files, commands, logs, or examples. The env file is for non-secret image and CDS profile overrides copied from the example env file.

Mandatory pre-flight: do NOT emit a pipeline config without context

Before writing any *.yaml pipeline config, the agent MUST verify that one of the following is true:

1. KPI run output exists -- read it and use distribution-analysis.md, distribution-aware-curation.md, and configuration-decision-tree.md.
2. KPI sample videos are available for inspection / discovery -- follow context-understanding.md Phase 1.
3. No KPI of any kind -- no baseline exists. Read calibration-config.md and complete its Phase 1 interview (Inputs / Domain / Goal / Hardware / Calibration) BEFORE emitting a config. The interview is binding, not advisory.

If none of these holds, or required paths, intent, or hardware constraints remain unresolved, stop and report the unresolved requirements as blocked.

Workplace legal-review gate. Workplace safety and employee conduct monitoring are privacy-sensitive. The gate applies when the user names either domain or a catalog match selects employee_conduct_monitoring.yaml:

• If the gate does not apply → continue with the route below.
• If it applies and the user confirms legal review, a lawful basis, and any required worker notice → emit the reviewed configuration and its calibration disclosure.
• If it applies and any of those confirmations is absent → do not write a config; return blocked with the evidence and the next safe step (obtain the missing confirmation).

If the user requests a config with only a one-line description ("configure cosmos-curator for my videos"), assume the calibration workflow and ask the Phase 1 interview questions in one batched message. Emit the config only after the answers come back, and always include the calibration disclosure table that flags every defaulted field.

Canonical Flow

Choose one route; do not collapse advisory and execution branches. Every stop or failed gate below returns the Output Format with status blocked, the validation evidence, and the next safe step.

1. Advisory only (sizing, monitoring, troubleshooting, expected commands): inspect repository, config, and run evidence, load running-pipelines.md, and report guidance. Do not prepare credentials or execute.
2. Create or change config:
   • KPI output exists → analyze it, choose standard, distribution-aware, or restrictive curation, then emit a reviewed config.
   • Representative samples exist → inspect them before selecting defaults. A discovery run starts a pipeline, so it follows the explicit run route, including its preflights and authorization.
   • Neither exists → complete the binding calibration interview (calibration-config.md); emit config and disclosure only after answers. Catalog matching follows context-understanding.md.
   • Catalog match is employee_conduct_monitoring.yaml, on any of the paths above → apply the workplace legal-review gate. Without the confirmation, do not select the catalog; return blocked. Stop if required paths, intent, or hardware constraints remain unresolved, or if the workplace legal-review gate above is not confirmed.
3. Explicit run request, including discovery and calibration runs, in this order:
   1. Prepare the runtime (Docker, GPU, SHM, FFmpeg sidecar) and run the execution preflights in Validation.
   2. Obtain credentials through approved injection.
   3. Validate the config as described in Execution & Troubleshooting.
   4. If the config came from the calibration workflow, confirm the user has reviewed its calibration disclosure, including every defaulted field. If not, return blocked with the next step "review the disclosure".
   5. Request approval if not already granted. If approval is granted, execute. If it is pending or denied, return blocked without executing.
   6. Validate outputs.
4. Downstream TAO handoff: validate Curator output and embedding family. If validation succeeds, prepare compatible inputs for Data Mining and mine. If it fails, return blocked without mining.
5. CDS ingest or search: CDS is deployed outside this repository. Run make ds-health first. If CDS is not deployed or not healthy, return blocked with the next step "deploy CDS per the user guide's Cosmos Dataset Search page". Otherwise follow cosmos-dataset-search.md: ingest CE1 artifacts only, address collections by id rather than name, and treat HTTP 404 from /insert-data as a failure. Ingest and bulk insert change a collection, so request authorization first, as for a run.

Configs are flat YAML with pipeline: split|dedup|shard|annotate and upstream snake_case argument names. Operator first-run recipes live under the cookbook tree (split-minimal then full split, dedup, and shard YAML). The configs directory is the full flag reference and the Makefile default when CONFIG_FILE is omitted. split writes clips, metadata, and embeddings; dedup consumes embeddings; shard writes WebDataset archives; annotate processes still images (image annotate flag-reference YAML; no image cookbook).

Execution & Troubleshooting

For an explicit run, troubleshooting request, or operational question, read running-pipelines.md. The local run commands are below.

MANDATORY: run the config preflight before every make run-pipeline or make run_image_pipeline invocation. It is not optional, because it is the only guard against silently skipping SAM3 on the Make path.

make run-pipeline CONFIG_FILE=<split-config>
make run_image_pipeline IMAGE_CONFIG_FILE=<image-config>

In the config preflight, reject deprecated enable_sam3 and enable_event_captioning and return blocked; use