---

name: cdf-anomalygen description: >- Use when running the CDF AnomalyGen pipeline over a defect dataset — mask placement, fine-tuning, synthetic defect-image generation (SDG), evaluation, quality refinement, and pseudo-labeling — even when the user names only one stage. license: Apache-2.0 compatibility: >- Requires a CUDA GPU and the built env (native .venv or the Docker image); Hugging Face auth + network access to download the model checkpoints. metadata: owner: NVIDIA service: anomaly generation version: 1.2.0 author: NVIDIA nvidia@nvidia.com tags: - physical-ai - anomaly-generation - cosmos - fine-tuning languages: [python] frameworks: [pytorch, cosmos, diffusion]

Skill: cdf-anomalygen

Purpose

AnomalyGen fine-tunes the frozen Cosmos base into an anomaly-generation model: given a clean image + defect mask, it paints a realistic defect into that region (synthetic data for detectors). Pick a mode, collect its params, run the steps — all commands from the repo root.

When to Use This Skill

Use for the CDF AnomalyGen pipeline over a defect dataset — any single stage (mask placement, fine-tuning, generation, evaluation, refinement, or pseudo-labeling) or the full run.

• "Run the AnomalyGen pipeline on phone_screen"
• "Fine-tune an anomaly-generation model for PCB defects"
• "Generate 25 synthetic PCB anomaly images"
• "Place defect masks", "evaluate quality", or "pseudo-label the generated defects"

Do not use this skill for generic image generation, or for synthetic data that isn't defect images built from the {texture}/{defect} clean-image + defect-mask contract.

Prerequisites

• A CUDA GPU and the project's built env (native .venv or the Docker image).
• Hugging Face auth — run hf auth login — plus network access; the downloader fetches the model checkpoints.
• A dataset_dir in the Step 1 contract layout (clean images + {defect} masks; plus defect_spec.jsonl only when mask_backend=amp — the default amg never reads it, and nothing after Step 2 does either).

Preflight the env + checkpoints (GPU, Python, CUDA/torch, deps, HF auth, model checkpoints):

scripts/preflight_env_ckpt.sh

Read which group failed before reacting — the summary separates them, and only one is fixed by downloading:

• missing checkpoints → scripts/download_checkpoints.sh (run hf auth login first).
• env / Python / deps → rebuild the environment (bash scripts/env_setup.sh, then activate it).

Third group — model access for mask placement (checked HERE, never mid-flow). The default mask_backend=amg needs a vision model, and a run that discovers this at Step 2 has already spent the user's time. A VLM API is a base, a key and a model name — and all three live in ONE place: ~/.anomalygen/endpoints.yaml, the model registry (nothing is built into the code; the template is scripts/agentic_mask_generation_serving/endpoints.example.yaml). It sits under $HOME, not in the repo, so it works from any directory, serves every checkout, and is writable inside the read-only product containers. The moment inputs are locked with mask_backend=amg (the default), probe:

[ -f ~/.anomalygen/endpoints.yaml ] && echo "amg model: ~/.anomalygen/endpoints.yaml present" \
    || echo "amg model: no ~/.anomalygen/endpoints.yaml — NO model can resolve; copy scripts/agentic_mask_generation_serving/endpoints.example.yaml there"

The model resolves in the registry → use it. That is the whole check: --gen-model and --judge-model are REQUIRED flags (there is no default model), and every name they carry must have a registry entry (model + base_url + key) — the entry point refuses to start when one does not; there is no environment fallback. A user's own OpenAI-compatible endpoint is the same case: one endpoints.yaml entry with their documented base URL, their key, and their model name in --gen-model/--judge-model.

Anything less → stop and ask the user now, before Step 1 — unless the user's request already names the model (e.g. "use local Gemma"): that IS the pick, act on it without re-asking. Three options — whichever it is, set AMG_MODEL to the chosen name; the Step 2 command passes it to both required flags:

• (a) supply an API — one endpoints.yaml entry: a model name, a base and a key (e.g. AMG_MODEL=azure/openai/gpt-6-astra).
• (b) let the skill start the model service — in a container, as a background process inside it; on a dev host, from scripts/agentic_mask_generation_serving/. Needs a GPU; the weights are a default checkpoint — scripts/download_checkpoints.sh gemma fetches them and preflight verifies them (the group argument fetches only this model; bare, the script fetches all eleven). mkdir -p ~/.anomalygen && cp scripts/agentic_mask_generation_serving/endpoints.example.yaml ~/.anomalygen/endpoints.yaml wires it: the template's gemma entry already routes AMG_MODEL=nvidia/google/gemma-4-31b-it to http://localhost:8902/v1, and the run carries --beam-multi 4 --beam-pairwise --exemplar-captions --exemplar-visual-prompt --coverage-note — the local model always pairs with the in-loop beam and the three information channels. Bootstrap, lifecycle and the why: agentic-mask-generation.md.
• (c) set mask_backend=amp for this run.

Never fall back silently. Whichever they pick, the run's own capability probe confirms the model can take images and answer in JSON before Step 2 spends anything.

Inputs

Collect the run's inputs from the user before starting. If the mode or any input it requires is missing, ask the user first — never guess a required input or silently fall back to its default.

Lock the mode first — it fixes which steps run and which inputs are required (ask for any not provided):

• full — train + generate, Steps 1 → 7. Required: name, dataset_dir, num_sdg.
• finetune_only — train only, Steps 1 → 3. Required: name, dataset_dir, num_sdg. Here num_sdg sizes only the validation set.
• generation_only (a.k.a. inference_only) — generate from an existing checkpoint, Steps 2, 4 → 7. Required: name, checkpoint, recipe, dataset_dir, num_sdg. The checkpoint's geometry stamp must equal recipe's. A legacy checkpoint (trained before the stamp existed) has none: pass its run-dir recipe copy and it is stamped from that recipe at load (generation only; it cannot be resumed).

name is required in every mode — an experiment label that names the recipe, run dir, logs and the generated mask-set dirs. It is never a dataset location: where the data lives is dataset_dir, always passed separately and never derived from name.

Optional in every mode — all have working defaults, so ask only if the user raises them: num_gpus (1), num_search_run (3), score (nn), task (texture_ft), base_seed (1), mask_backend (amg), legacy_fixed_size (off), per_clean_pairing (off). Anything not listed as Required above is optional.

legacy_fixed_size switches Step 3 to the legacy fixed-input-size training mechanism. It is off by default: every fine-tune uses the current variable-input-size mechanism unless the user explicitly asks for the legacy one ("FT using the legacy fixed-size mechanism", "old fixed input size setting", "previous release FT mode", etc.). Never infer it from the dataset, the image sizes, or KPI numbers, and never ask about it when the user has not raised it. When set, the run takes job_name: anomalygen_texture_ft_legacy — in the recipe and as --job_name to set_pipeline_vars.sh — so it never shares a run dir with a default run of the same name, and Step 3 appends the legacy block to the recipe.

mask_backend picks how Step 2 places masks: amg (default) — the agentic engine proposes, refines and judge-accepts every mask, all defect types included; amp — the allocate → pair → place pipeline; no VLM beyond text ROIs. AMG needs model access (preflight checks before spending); if the user cannot supply it and does not choose, stop and ask — never fall back silently.

Parameter tables (each mapped to its CLI arg) and the step map: inputs.md. Confirm the inputs, set the shared variables below, then run straight through — in full, checkpoint is derived after Step 3, so don't pass it.

Instructions

Run the steps your mode selects (see Inputs), in order — each step's header notes when to skip it.

A step is done only when its output exists on disk — not when its command is launched. Every step ends with a named gate (dataset, amp, train, …); run it before moving on. Never treat starting a long job as a stopping point, nor proceed on a partial artifact. Gate commands and the fix for each failure: references/verification.md.

Set the shared variables every step below refers to. Re-source this in each new shell — variables do not survive one — and read the values it echoes back (${DATASET_DIR}, ${VAL_DIR}, ${GEN_DIR}, ${OUT}, ${RUN_DIR}, ${LOG_DIR}, …):

source scripts/skill_utility/set_pipeline_vars.sh --name <name> --dataset_dir <dataset_dir> \
    --num_sdg <num_sdg> [--num_gpus 1] [--num_search_run 3] [--task texture_ft] \
    [--job_name anomalygen_texture_ft_legacy]   # legacy_fixed_size only
export SCORE="<score>"  # repeat the confirmed choice in every shell; use nn when none was requested

--dataset_dir is required and is never derived from --name. The name is an experiment label; the dataset is a separate input that comes from the user. They differ in the normal case — a label of uc1pega-opus5-e2e over a dataset at datasets/uc1pega — so a name-derived path would be wrong far more often than right, and wrong silently: the recipe's dataset_path is set separately, so the recipe can point at the right dataset while ${DATASET_DIR} hands Steps 5 and 6 a --real_root that scores against nothing. The script does not check that the path exists; gate dataset does, one step later.

Every step's log goes under ${LOG_DIR} (results/), never the repo root. Use ${LOG_DIR}/<step>_${NAME}.log. The same goes for PID files — the launcher writes its own under results/; do not add one.

The steps below pipe through tee to keep the log and the console. Run set -o pipefail first — without it the pipeline's exit status is tee's, so a step that died reports success:

set -o pipefail

set_pipeline_vars.sh rejects a name/task outside [A-Za-z0-9][A-Za-z0-9._-]* and any non-integer count rather than sanitizing them — both reach command lines below. Apply the same care to user-supplied paths (dataset_dir, recipe, checkpoint): quote every expansion. texture_ft is the only task_type today; a new task family adds one ${TASK}→${SCRIPTS} line to that script plus a matching task_type: in the recipe.

Step 3 sets CKPT and repoints RECIPE at the run-dir copy for Steps 4–7. generation_only skips Step 3, so set both yourself before Step 4 (use the user's checkpoint verbatim if given; CKPT and RECIPE must come from one run or class ids drift):

RUN_DIR=<existing run dir>                      # only if it is not the ${RUN_DIR} already set above
CKPT="${RUN_DIR}/checkpoints/model/$(cat "${RUN_DIR}/checkpoints/best_checkpoint.txt")"
RECIPE="${RUN_DIR}/exp_${TASK}_${NAME}.yaml"    # run-dir copy — MUST match the checkpoint

Step 1 — dataset (contract layout)

All modes need dataset_dir on disk in this layout — training and eval read the anomaly_image/+mask/ pairs; both Step 2 backends read clean_image/ (AMG also samples those same pairs as exemplars); defect_spec.jsonl has exactly one con