SkillAgentSearch skills...

portaljs-migrate

Migrate (harvest) datasets between open-data platforms. Reads CKAN, a DCAT-US /data.json catalog (DKAN, ArcGIS Hub, data.gov), a DCAT / DCAT-AP RDF feed (JSON-LD, Turtle, or RDF/XML — data.europa.eu, national DCAT-AP portals, GeoDCAT-AP), Socrata, OpenDataSoft, or an ArcGIS FeatureServer, and writes…

Install / Use

npx skills add datopian/portaljs

Installs into whichever agent you are using.

About this skill
⚡

Claude Commands

Claude Code slash commands

Quality Score

94/100

Category

Marketing

Supported Platforms

Claude Code

Our assessment of portaljs-migrate

portaljs-migrate scores 94/100 on our quality scale, 110th of 642 Marketing skills we index (top 18%).

Its Claude Commands is 36 KB long, well organised into 16 sections with 7 code examples: a thorough specification that gives an agent plenty to work with.

With 2,356 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
30/30
Structure
20/20
Description
15/15
Adoption
14/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 2 months ago, so portaljs-migrate is actively maintained.
  • Our last check on 2026-11-16 found the source still online.
  • It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
  • Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-09-29. Automated pattern scan on 2026-09-29. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

portaljs-migrate compared with similar skills

All 4 of these similar skills score higher than portaljs-migrate; compare them before choosing.

SkillScoreStarsUpdatedFormat
portaljs-migrate (this skill)by datopian942.4k2mo agoClaude Commands
claude-memby thedotmack100117.6ktodayCLAUDE.md
taste-skillby Leonxlnx100116.8k8d agoCLAUDE.md
Agent-Reachby Panniantong100112.3k9d agoCLAUDE.md
Understand-Anythingby Egonex-AI10092.8k1d agoCLAUDE.md

Frequently asked questions

How do I install portaljs-migrate?
Run npx skills add datopian/portaljs. The install tabs above show the steps for each supported agent.
Which AI agents does portaljs-migrate work with?
It is written for Claude Code, as a Claude Commands file. Other agents that read the same format can often use it too.
Is portaljs-migrate safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is portaljs-migrate still maintained?
The repository was last updated about 2 months ago, so portaljs-migrate is actively maintained.

description: Migrate (harvest) datasets between open-data platforms. Reads CKAN, a DCAT-US /data.json catalog (DKAN, ArcGIS Hub, data.gov), a DCAT / DCAT-AP RDF feed (JSON-LD, Turtle, or RDF/XML — data.europa.eu, national DCAT-AP portals, GeoDCAT-AP), Socrata, OpenDataSoft, or an ArcGIS FeatureServer, and writes them to a static PortalJS catalog (datasets.json, link-by-URL or download data files into Cloudflare R2 via Git LFS / Giftless) or pushes them into a CKAN instance over its API. allowed-tools: Read, Write, Edit, Bash, WebFetch

/portaljs-migrate

Harvest datasets from an external open-data platform into an existing portaljs-catalog portal. The source's datasets are read over its API, mapped to the portal's canonical dataset shape, and written into datasets.json (the static catalog's single source of truth) — so the /search catalog and the /@<namespace>/<slug> showcases render them like any hand-added dataset.

This is the copy-into-the-portal path. It is the inverse of /portaljs-connect-ckan: connect-ckan keeps the source authoritative and reads it live at build time; /portaljs-migrate takes a one-time (re-runnable) snapshot into the static catalog, so the portal stands alone and needs no backend.

Hub-and-spoke model

Every source is read into one canonical shape (the template's Dataset/Resource type — a Frictionless-aligned { slug, namespace, name, description, resources[] }), then written to the target from that canonical form. Add a source once and it migrates to every target.

Sources:

| Source | --source | How it's read | Covers | | ------ | ---------- | ------------- | ------ | | CKAN | ckan | REST API (package_search / package_show) | any CKAN instance | | DCAT-US /data.json | dcat | one catalog document (plain JSON) | DKAN, ArcGIS Hub, data.gov, other DCAT-US publishers | | DCAT / DCAT-AP RDF feed | dcat-rdf | RDF catalog in JSON-LD, Turtle, or RDF/XML | data.europa.eu, national DCAT-AP portals (SE/CH/DE), GeoDCAT-AP, any DCAT 2/3 RDF feed | | Socrata | socrata | Discovery API + per-dataset resource exports | Socrata-powered open-data sites | | OpenDataSoft | ods | Explore API v2 catalog + exports | ODS-powered portals | | ArcGIS FeatureServer / MapServer | arcgis | layer metadata + GeoJSON query | individual ArcGIS map/feature services |

DKAN, ArcGIS Hub, and data.gov publish a DCAT-US /data.json — use the dcat source for those whole catalogs. Use arcgis for an individual FeatureServer/MapServer (each layer becomes a GeoJSON dataset, which /data.json doesn't expose).

dcat vs dcat-rdf. dcat reads the flat DCAT-US /data.json (Project Open Data JSON — dataset[] with distribution[]). dcat-rdf reads a DCAT / DCAT-AP RDF graph serialized as JSON-LD (.jsonld), Turtle (.ttl), or RDF/XML (.rdf) — the form data.europa.eu and national DCAT-AP portals publish. This is the inbound counterpart of /portaljs-add-dcat (which EXPOSES the portal as DCAT-AP RDF): the two make a portal a full two-way DCAT interop node — expose to, and harvest from, national DCAT-AP portals. Both read through the SAME profile registry (lib/metadata/dcat-profiles.ts), so expose and consume stay in sync.

Targets:

| Target | --target | Writes | | ------ | ---------- | ------ | | Static PortalJS catalog (default) | static | datasets.json in a portaljs-catalog portal (+ in download mode, data files pushed to Cloudflare R2 via Git LFS / Giftless) | | CKAN instance | ckan | datasets/resources into a CKAN backend via package_create / resource_create (needs a write API key) |

The CKAN target enables platform-to-platform moves — CKAN→CKAN and DKAN→CKAN — since any reader can feed any writer through the canonical shape.

Required input — ask, don't error

Source:

  • Source type — ckan, dcat, dcat-rdf, socrata, ods, or arcgis (auto-detected from the URL if omitted; see step 3).
  • Source URL (required) — e.g. a CKAN base URL, a DCAT-US /data.json URL, a DCAT-AP RDF feed URL (…/catalog.jsonld / .ttl / .rdf, or a portal page that autodiscovers one via <link rel="alternate">), a Socrata or OpenDataSoft site root, or an ArcGIS …/FeatureServer (or …/MapServer) URL.
  • Filters (optional) — CKAN: org / group names. Socrata/ODS: pass a search term or category to scope large catalogs.

Target — --target static (default) or --target ckan:

  • static: Portal directory (optional, default current dir); copy mode link (default) or download (step 5b).
  • ckan: target CKAN URL (required) and a write API key read from the CKAN_API_KEY env var (required — never pass it on the command line or hardcode it); an optional owner org to file every dataset under (step 7b).

Common:

  • --dry-run (optional) — preview what would be written, change nothing.
  • --replace (optional, static target) — clear existing datasets.json entries first (default: upsert alongside what's already there, e.g. the sample datasets).

If the source URL is missing, ask for it (and the source type if unclear) — never dead-end with a missing-input error. For --target ckan, if the target URL or CKAN_API_KEY is missing, ask rather than failing.

Steps

1. Gather input from $ARGUMENTS (interview if thin)

Extract:

  • SOURCE_TYPE — ckan | dcat | dcat-rdf | socrata | ods | arcgis (default: auto-detect in step 3).
  • SOURCE_URL — required; strip any trailing slash.
  • ORG_FILTER / GROUP_FILTER — lists (CKAN source only; default empty).
  • TARGET — static | ckan (default static).
  • PORTAL_DIR — default . (static target).
  • COPY_MODE — link | download (default link; static target).
  • TARGET_CKAN_URL — required for ckan target; strip any trailing slash.
  • OWNER_ORG — optional CKAN org to file datasets under (ckan target).
  • DRY_RUN — boolean (default false).
  • REPLACE — boolean (default false; static target).

The write API key for a ckan target is read from process.env.CKAN_API_KEY at run time — never from $ARGUMENTS.

If SOURCE_URL is missing, ask and wait:

To migrate datasets I need:
1. Source URL — a CKAN base URL, or a DCAT /data.json URL (required)
2. Source type — ckan or dcat (Enter to auto-detect)
3. Portal directory (Enter for current directory)
4. Copy mode — link (reference source URLs, default) or download (copy files in)

2. Validate the target

Static target (--target static). Confirm PORTAL_DIR/datasets.json, PORTAL_DIR/package.json, and PORTAL_DIR/pages/[owner]/[slug].tsx exist. If datasets.json is missing, tell the user this isn't the catalog template (it may be the minimal single-page template) and ask how to proceed rather than failing silently. Read NAMESPACE_TYPE from PORTAL_DIR/lib/datasets.ts — it doesn't change the harvest (every dataset still carries a namespace), but it tells you whether namespaces read as subjects (theme) or publishers (owner) so you can explain the result.

CKAN target (--target ckan). Confirm TARGET_CKAN_URL is a working CKAN API and the key authenticates: call package_search?rows=1 (must be success: true), then verify the key with an authenticated read such as organization_list_for_user (pass the key in the Authorization header). If the key is missing or rejected, tell the user and stop — never write without a confirmed key. If OWNER_ORG is set, confirm it exists (organization_show?id=OWNER_ORG); offer to create it (step 7b) or pick an existing one.

3. Detect the source type and verify it's reachable

If SOURCE_TYPE is unset, auto-detect:

  • URL contains /FeatureServer or /MapServer → arcgis.
  • URL ends in .jsonld, .ttl, or .rdf, or contains /catalog. (an RDF feed) → dcat-rdf.
  • URL ends in .json or contains /data.json → dcat (DCAT-US Project Open Data JSON).
  • URL contains /api/explore/ → ods; /api/catalog/ → socrata.
  • Otherwise probe CKAN: curl -s -m 20 "SOURCE_URL/api/3/action/package_search?rows=1" → if JSON with "success": true, it's ckan.
  • Else probe in turn: SOURCE_URL/api/explore/v2.1/catalog/datasets?limit=1 (ods), SOURCE_URL/data.json (dcat), then content-negotiate RDF: curl -fsSL -H "Accept: application/ld+json,text/turtle,application/rdf+xml" SOURCE_URL — if the body is JSON-LD/Turtle/RDF-XML (starts with {/[, @prefix/PREFIX, or <?xml/<rdf:RDF), it's dcat-rdf. For an HTML page, look for a feed <link rel="alternate" type="application/ld+json" …> (or text/turtle / application/rdf+xml) and follow its href — this is the autodiscovery target /portaljs-add-dcat emits, so a portal's homepage URL is enough.
  • If still nothing resolves, tell the user the URL didn't look like a supported source and ask them to confirm the URL / pick the --source type — don't dead-end. (Socrata is read through the central Discovery API, so for a Socrata site pass --source socrata with the site root, e.g. https://data.cityofnewyork.us.)

For ckan with an ORG_FILTER, validate each org exists via organization_show?id=ORG; if one is missing, list valid orgs (organization_list) and ask which they meant or to drop the filter.

4. Read the source into canonical datasets

Use WebFetch or curl for requests. Build an in-memory list of canonical entries shaped like the template's Dataset:

{
  "slug": "...",            // URL-safe, unique within a namespace
  "namespace": "...",       // groups the dataset (CKAN org / DCAT publisher or theme)
  "name": "...",            // human title
  "description": "...",     // one paragraph (optional) — sanitized, never a placeholder
  "keywords": ["..."],      // optional
  "licenses": [             // optional; map from source, never fabricate (see hygiene below)
    { "name": "...", "title": "...", "path": "<url>" }
  ],
  "created": "...",         // optional ISO 8601 — source "issued"/first-published date
  "modified": "...",        // optional ISO 8601 — source "last-modified" date (the DATA's)
  "resources": [
    { "name": "...", "path": "<url-or-filename>", "format": "csv", "title": "..." }
  ]
}

created/modified are source-provenance dates copied from the origin — they describe the data, and drive the showcase "Last updated". Never set them to the harvest time; when you copy files in (download mode / a migrator), record the harvest time as migratedAt instead, which the showcase renders as a separate "Migrated" field.

A single-file dataset may instead use the file/format sugar, but prefer resources[] so multi-file datasets are uniform.

CKAN mapping (package_search paginated by rows/start, then per dataset the search result already carries resources, so a second package_show is only needed if a field is missing):

| Canonical | CKAN field | | --------- | ---------- | | slug | name (already URL-safe) | | namespace | organization.name (fallback dataset) | | name | title || name | | description | notes | | keywords | tags[].name | | resources[].name | resource name || id | | resources[].path | resource url (link mode) | | resources[].format | resource format lowercased |

Page with rows=200 until you've read count (or a sane cap — tell the user if you cap). Apply ORG_FILTER/GROUP_FILTER via the fq query (organization:("a" OR "b") groups:("c")).

DCAT-US mapping (one GET of the /data.json; datasets live under dataset[]):

| Canonical | DCAT field | | --------- | ---------- | | slug | slugified identifier || slugified title (from the RAW title — slugs/URLs stay stable even after title cleanup) | | namespace | slugified publisher.name (fallback first `

Truncated for display — read the full file on GitHub.

Related Skills

View on GitHub
GitHub Stars2.4k
CategoryMarketing
Updated2mo ago
Forks335

Languages

TypeScript

Trust signals

100/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

No cautions