portaljs-migrate
Migrate (harvest) datasets between open-data platforms. Reads CKAN, a DCAT-US /data.json catalog (DKAN, ArcGIS Hub, data.gov), a DCAT / DCAT-AP RDF feed (JSON-LD, Turtle, or RDF/XML), Socrata, OpenDataSoft, or an ArcGIS FeatureServer, and writes them to a static PortalJS catalog or pushes them into…
Install / Use
npx skills add datopian/portaljs --skill portaljs-migrateInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
MarketingSupported Platforms
Our assessment of portaljs-migrate
portaljs-migrate scores 93/100 on our quality scale, 142nd of 642 Marketing skills we index (top 23%).
Its SKILL.md is 6.9 KB long, well organised into 12 sections with 4 code examples: a thorough specification that gives an agent plenty to work with.
With 2,356 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 2 months ago, so portaljs-migrate is actively maintained.
- Our last check on 2026-10-26 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-09-30. Automated pattern scan on 2026-09-29. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
portaljs-migrate compared with similar skills
All 4 of these similar skills score higher than portaljs-migrate; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| portaljs-migrate (this skill)by datopian | 93 | 2.4k | 2mo ago | SKILL.md |
| Agent-Reachby Panniantong | 100 | 112.3k | 8d ago | CLAUDE.md |
| headroomby headroomlabs-ai | 100 | 79.8k | today | CLAUDE.md |
| Scraplingby D4Vinci | 100 | 103.6k | 2d ago | MCP Server |
| crawl4aiby unclecode | 100 | 87.4k | 3d ago | MCP Server |
Frequently asked questions
- How do I install portaljs-migrate?
- Run
npx skills add datopian/portaljs --skill portaljs-migrate. The install tabs above show the steps for each supported agent. - Which AI agents does portaljs-migrate work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is portaljs-migrate safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is portaljs-migrate still maintained?
- The repository was last updated about 2 months ago, so portaljs-migrate is actively maintained.
Skill content
View source on GitHubname: portaljs-migrate description: Migrate (harvest) datasets between open-data platforms. Reads CKAN, a DCAT-US /data.json catalog (DKAN, ArcGIS Hub, data.gov), a DCAT / DCAT-AP RDF feed (JSON-LD, Turtle, or RDF/XML), Socrata, OpenDataSoft, or an ArcGIS FeatureServer, and writes them to a static PortalJS catalog or pushes them into a CKAN instance over its API. Use when moving datasets from an external open-data platform into a PortalJS portal, or bridging one CKAN instance to another. allowed-tools: Read, Write, Edit, Bash(npm:), Bash(npx:), Bash(node:), Bash(curl:), Bash(git:*), WebFetch version: 1.0.0 author: Datopian hello@datopian.com license: MIT compatibility: Claude Code with PortalJS portals (Next.js 14, React 18, Node 18+). Runs from any project via the plugin, a personal ~/.claude/commands install, or a portaljs clone. tags:
- portaljs
- data-portal
- migration
- harvest
- dcat
- ckan
PortalJS — Migrate
Overview
Harvest datasets from an external open-data platform into a PortalJS portal, or
push them into a CKAN instance. Every source reads into one canonical shape —
{ slug, namespace, name, description, resources[] } — then writes to the target
from that shape, so adding a source covers every target automatically.
Supported sources: CKAN, a DCAT-US /data.json catalog (DKAN, ArcGIS Hub,
data.gov), a DCAT / DCAT-AP RDF feed, Socrata, OpenDataSoft, or an ArcGIS
FeatureServer/MapServer. Supported targets: a static PortalJS catalog
(datasets.json, resources linked by URL or downloaded into Cloudflare R2) or a
CKAN instance over its write API.
This is the copy-into-the-portal path, the inverse of /portaljs-connect-ckan
(which reads the source live at build time): a one-time, re-runnable snapshot so
the portal stands alone and needs no backend.
Prerequisites
- An existing PortalJS portal for the static target:
datasets.json,package.json, andpages/[owner]/[slug].tsxmust exist (theportaljs-catalogtemplate). - Node 18+ and npm, with portal dependencies installed (
npm install). - Network access to the source platform's API or feed URL.
- For a CKAN target: a write API key in
CKAN_API_KEY— never hardcoded. - For
downloadcopy mode:git-lfsand a PortalJS Arc account (or a self-hosted Giftless endpoint) to push resource files to R2. - For
dcat-rdfsources:lib/metadata/dcat-harvest.tsanddcat-profiles.ts(copy fromexamples/portaljs-catalog/lib/metadata/if missing), run vianpx tsx.
Instructions
Full step-by-step workflow:
.claude/commands/portaljs-migrate.md.
- Gather input — source type, source URL, target (
static/ckan), portal directory, copy mode (link/download), filters. Interview instead of erroring. - Validate the target — confirm the static catalog files exist, or that the CKAN URL and API key authenticate.
- Detect the source type from the URL (or
--source) and verify reachability. - Read the source into canonical dataset entries, per the source's field table in
the reference doc, applying the metadata hygiene rules (license sentinel — never
fabricate CC-BY; description sanitizer + fallback chain; source dates preserved
with
migratedAtseparate;cleanTitledisplay-name cleanup with slugs from the raw title;categoryfrom a meaningful theme). - For a static target, resolve resource paths by copy mode:
linkkeeps source URLs;downloadcopies files into the repo under Git LFS, pushed to R2. - Print a dry-run preview; stop if
--dry-runwas passed. - Write
datasets.json(upsert or--replace), or push packages/resources into CKAN over its action API. - Verify —
npm run build(static) or re-querypackage_search(CKAN). - Report datasets migrated, namespaces touched, and next steps.
Output
Static target: an updated datasets.json (upserted on (namespace, slug)),
optionally with resource files in Git LFS served from Cloudflare R2, plus a
passing npm run build. CKAN target: created/updated packages and resources;
per-dataset failures are logged and skipped, not fatal.
Error Handling
| Symptom | Cause | Fix |
| --- | --- | --- |
| "not a portaljs-catalog template" | datasets.json missing | Confirm the portal directory, or run /portaljs-new-portal |
| CKAN target rejects every write (403) | Key missing, expired, or lacks org permission | Set a valid CKAN_API_KEY; stop rather than partially migrating |
| Source URL matches no known format | Auto-detection failed | Pass --source explicitly (ckan, dcat, dcat-rdf, socrata, ods, arcgis) |
| DCAT-RDF fetch returns HTML, not a feed | URL is a portal page, not the feed | Follow its <link rel="alternate"> href, or use the direct feed URL |
| Build fails after write | Malformed entry (missing slug/namespace/name) | Inspect /tmp/portaljs-migrate-build.log, fix, rebuild |
| download mode push fails | git-lfs missing, or no Arc/Giftless token | Install git-lfs; mint a token via the Arc API or self-hosted Giftless |
| Duplicate datasets after re-running | --replace used unintentionally, or slug collisions | Omit --replace to upsert; keep slugs unique per namespace |
Examples
Example 1 — CKAN to static catalog, link mode
/portaljs-migrate --source ckan --source-url https://demo.dev.datopian.com \
--target static --portal-dir . --copy-mode link
Upserts all packages from the CKAN demo instance into datasets.json, referencing
source resource URLs directly.
Example 2 — data.gov DCAT-US catalog, downloaded into R2
/portaljs-migrate --source dcat --source-url https://catalog.data.gov/data.json \
--target static --copy-mode download --org-filter epa-gov
Harvests one publisher and copies every resource file into the repo under Git LFS, served from Cloudflare R2.
Example 3 — DCAT-AP RDF feed from a national portal
/portaljs-migrate --source dcat-rdf \
--source-url https://data.europa.eu/api/hub/search/catalog.jsonld \
--target static --dry-run
Fetches the JSON-LD feed through the RDF harvester and previews the canonical datasets without writing anything.
Example 4 — CKAN to CKAN (platform-to-platform)
/portaljs-migrate --source ckan --source-url https://old-portal.example.org \
--target ckan --target-url https://new-portal.example.org --owner-org research
Pushes every package from the old CKAN instance into the new one, filed under
research.
Resources
- Command source:
.claude/commands/portaljs-migrate.md - Field mappings and troubleshooting:
references/reference.md - Related skills:
/portaljs-connect-ckan,/portaljs-add-dcat,/portaljs-check-data-quality,/portaljs-define-schema - CKAN Action API reference: https://docs.ckan.org/en/latest/api/
Related Skills
Agent-Reach
112.3kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
headroom
79.8kCompress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Scrapling
103.6k🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
crawl4ai
87.4kOfficial Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
