portaljs-add-dataset
Add a dataset (CSV, TSV, JSON, or GeoJSON) to an existing PortalJS portal. Appends an entry to datasets.json so the catalog and showcase render it automatically; routes the data by source (local file vs remote URL) — R2 via Git LFS by default, remote URLs by passthrough.
Install / Use
npx skills add datopian/portaljsInstalls into whichever agent you are using.
Claude Commands
Claude Code slash commands
Quality Score
Category
AutomationSupported Platforms
Our assessment of portaljs-add-dataset
portaljs-add-dataset scores 94/100 on our quality scale, 416th of 2,790 Automation skills we index (top 15%).
Its Claude Commands is 17 KB long, well organised into 30 sections with 11 code examples: a thorough specification that gives an agent plenty to work with.
With 2,356 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 2 months ago, so portaljs-add-dataset is actively maintained.
- Our last check on 2026-11-16 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-09-30. Automated pattern scan on 2026-09-29. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
portaljs-add-dataset compared with similar skills
All 4 of these similar skills score higher than portaljs-add-dataset; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| portaljs-add-dataset (this skill)by datopian | 94 | 2.4k | 2mo ago | Claude Commands |
| claude-memby thedotmack | 100 | 117.6k | today | CLAUDE.md |
| taste-skillby Leonxlnx | 100 | 116.8k | 8d ago | CLAUDE.md |
| Agent-Reachby Panniantong | 100 | 112.3k | 8d ago | CLAUDE.md |
| Understand-Anythingby Egonex-AI | 100 | 92.8k | 1d ago | CLAUDE.md |
Frequently asked questions
- How do I install portaljs-add-dataset?
- Run
npx skills add datopian/portaljs. The install tabs above show the steps for each supported agent. - Which AI agents does portaljs-add-dataset work with?
- It is written for Claude Code, as a Claude Commands file. Other agents that read the same format can often use it too.
- Is portaljs-add-dataset safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is portaljs-add-dataset still maintained?
- The repository was last updated about 2 months ago, so portaljs-add-dataset is actively maintained.
Skill content
View source on GitHubdescription: Add a dataset (CSV, TSV, JSON, or GeoJSON) to an existing PortalJS portal. Appends an entry to datasets.json so the catalog and showcase render it automatically; routes the data by source (local file vs remote URL) — R2 via Git LFS by default, remote URLs by passthrough. allowed-tools: Read, Write, Edit, Bash, WebFetch
/portaljs-add-dataset
Add a dataset to an existing PortalJS (portaljs-catalog) portal. Appends one entry to
datasets.json — the single source of truth for the catalog — and routes the data to
the right place by source first, then size. No per-dataset page is created: the catalog
at /search lists it and the dynamic showcase route pages/[owner]/[slug].tsx renders it
automatically at /@<namespace>/<slug>.
The routing matrix — branch on SOURCE, then size
Where the bytes end up depends first on where they come from, then on size/intent. The
manifest's resource.path (or single-file file) is the unifying abstraction:
resourceUrl() in lib/datasets.ts serves a repo-relative path from /public/data and
passes an absolute URL straight through. So every route below just decides what string
to write into path.
| Source | Default | What happens | Manifest path |
|--------|---------|--------------|-----------------|
| Local file | R2 via Git LFS | mv into the repo, git lfs track <path>, commit a ~134 B pointer, git lfs push → Giftless → R2 (no GitHub remote needed) | absolute R2 URL (browser fetches R2 directly) |
| Local file (fenced) | inline | Only bundled SAMPLE data or an OSS self-host with no R2: cp into public/data/ (stays inline per .gitattributes fence) | bare filename → /data/<file> |
| Remote URL | passthrough | record the URL as-is — no download, no upload, zero duplication | the absolute URL, unchanged |
| Remote URL | adopt (opt-in) | user wants it hosted/versioned under the portal: fetch → route as a local file (R2/LFS) | absolute R2 URL |
Storage decision (epic po-g9y): the default for added data is R2, regardless of size or format. Inline is not a size threshold — it is a fenced exception for bundled sample data and the OSS-no-R2 fallback. Remote URLs default to passthrough (copy nothing); adopting one into R2 is an explicit opt-in.
Remote-URL passthrough caveat: serving and linking always work. But in-browser range / DuckDB queries against a 3rd-party URL need CORS + range support on that host, which we don't control. If the user needs querying (not just preview/download) and the remote lacks CORS/range, recommend the adopt-into-R2 option.
Supported formats for the showcase preview/registration: CSV, TSV, JSON (array),
GeoJSON. (LFS itself is format-agnostic — any binary can be tracked — but the showcase
Table/map only previews these.)
Required input — ask, don't error
- Source — a local file path (
./data/file.csv) or a public URL (https://…/data.csv) - Portal directory — path to the portal project (defaults to current directory)
- Namespace — the dataset's namespace value (the portal's
NAMESPACE_TYPEgroup: a subject for'theme'portals, a publisher for'owner'portals)
If the source is missing, ask for it — never dead-end. The user can say "use defaults" to accept the defaults below.
Steps
1. Gather input from $ARGUMENTS (interview if thin)
Extract what's present:
SOURCE— file path or URLPORTAL_DIR— portal directory (default:.)DATASET_NAME— human-readable name (default: derived from filename)DATASET_SLUG— URL slug (default: lowercase hyphenated filename without extension)DESCRIPTION— optional one-line descriptionNAMESPACE— namespace value (default: read the existing first entry'snamespacefromdatasets.json, elsereference)ADOPT— for a remote URL only: whether to adopt the file into R2 (default: no — passthrough). Only ask if the URL route is taken and querying is plausibly needed.
If SOURCE is missing, ask (one focused prompt) and wait:
To add a dataset I need:
1. Source: local file path or public URL (required)
2. Portal directory (Enter for current directory)
3. Dataset name (Enter to use the filename)
4. Namespace value — the group this dataset belongs to
(subject if the portal is "theme" mode, publisher if "owner" mode; Enter to reuse the catalog's existing namespace)
Check the portal's namespace mode if helpful: read NAMESPACE_TYPE from
PORTAL_DIR/lib/datasets.ts so you can phrase the namespace question correctly ("subject"
vs "publisher").
2. Validate the portal directory
The target must be a portaljs-catalog portal. Confirm PORTAL_DIR/datasets.json,
PORTAL_DIR/package.json, and PORTAL_DIR/pages/[owner]/[slug].tsx exist. If
datasets.json is missing, tell the user this portal isn't the catalog template (it may
be an older single-page template) and ask how to proceed rather than failing silently.
3. Detect the format
If SOURCE is a URL: fetch headers (or a HEAD) and check the status. If not 200, tell
the user the fetch failed (with the HTTP status) and ask them to confirm the URL is publicly
accessible, then retry. Detect format from Content-Type or the URL extension. (For the
default passthrough route you do not download the body — only enough to detect format.)
If SOURCE is a local file path: check the file exists. If not, tell the user the path wasn't found and ask for a correct path. Detect format from the file extension.
Format detection rules:
.csvortext/csv→ CSV.tsvortext/tab-separated-values→ TSV.geojsonorapplication/geo+jsonor (JSON-parseable andparsed.type === "FeatureCollection") → GeoJSON.jsonorapplication/json→ JSON array- Vector geo formats —
.zip(zipped Shapefile),.gpkg,.kml/.kmz,.fgb, or a.csvwith a geometry/lat-lon column → hand off to/portaljs-add-geo. That skill auto-ingests geo sources into a dual-tier dataset (PMTiles render + GeoParquet query) on the user's machine. This tabular skill only previews flat files; it does not tile or build GeoParquet. (A small GeoJSON you just want listed with a download link can stay here as thegeojsonformat; for a map/query view, prefer/portaljs-add-geoor/portaljs-add-map.) - Anything else: tell the user the showcase can't preview this format and ask them to convert to CSV, TSV, JSON array, or GeoJSON first.
4. Route the data (the core of this skill)
Pick the branch from the matrix above. Decide on SOURCE first.
4a. Remote URL — passthrough (DEFAULT for URLs)
Do nothing to the bytes. The manifest path is the absolute URL itself; resourceUrl()
passes it through and the browser fetches it directly. Skip to Step 5 with
MANIFEST_PATH=<the URL>.
Mention the CORS/range caveat if the user may need querying, and offer the adopt route.
4b. Remote URL — adopt into R2 (OPT-IN only)
Only when the user explicitly wants the file hosted/versioned under the portal. Download it to a temp path, then fall through to 4c treating that temp file as the local source:
curl -fL "$SOURCE" -o "/tmp/$DATASET_SLUG.$EXT"
4c. Local file — R2 via Git LFS (DEFAULT)
This versions a tiny pointer in git and streams the bytes to Cloudflare R2 through Giftless.
The browser then fetches the bytes straight from R2 (manifest path = absolute R2 URL).
cd PORTAL_DIR
mkdir -p data
# mv (NOT cp) so an external local file isn't tripled on disk:
mv "$SOURCE" "data/$DATASET_SLUG.$EXT" # for 4b, $SOURCE is the temp download
# Ensure the LFS clean/smudge filters are installed for THIS repo before tracking.
# On a fresh machine `git lfs install` may never have run (brew install git-lfs does
# NOT auto-run it), so without this `git add` commits the RAW bytes instead of a
# ~134 B pointer and the later `git lfs push` has nothing to stream. Idempotent.
if ! git config --get filter.lfs.clean >/dev/null 2>&1; then
git lfs install --local
fi
# Per-file, format-agnostic LFS tracking (appends a path entry to .gitattributes,
# NOT an extension glob):
git lfs track "data/$DATASET_SLUG.$EXT"
git add .gitattributes "data/$DATASET_SLUG.$EXT"
git commit -m "data: add $DATASET_SLUG via LFS"
mv, not cp. When bringing an external local file into the repo, move it in place — a
cpleaves a 3rd on-disk copy (original + working tree +.git/lfscache).
Authenticate git-lfs (phase-2 gotcha, po-g9y.1). The committed .lfsconfig carries the
Giftless URL but no credentials. Mint a repo-scoped JWT and set the credentialed URL in
local git config only (never commit it). Use the _jwt HTTP Basic piggyback — do not
set a broad http.extraHeader, which git-lfs would replay onto the presigned R2 PUT (R2 →
400) and the verify callback.
Mint the token from the Arc API (default — no private key on your machine). The Arc API
is the token issuer (po-g9y.13): it holds the RS256 signer as a Worker secret and mints a
scoped, short-TTL LFS token for any authenticated Arc user. Reuse the same Arc token
/portaljs-deploy resolves (PORTALJS_TOKEN, else ~/.portaljs/credentials); if you have
neither, run /portaljs-deploy's sign-in step once. The endpoint returns a ready-to-use
credentialed lfs_url:
API="${PORTALJS_ARC_API:-https://api.arc.portaljs.com}"
ARC_TOKEN="${PORTALJS_TOKEN:-$(node -e "try{process.stdout.write(JSON.parse(require('fs').readFileSync(process.env.HOME+'/.portaljs/credentials','utf8')).token||'')}catch{}")}"
# Claim the slug under your Arc account (idempotent; safe to re-run). Minting never
# creates a slug (po-g9y.13), so claim once before the FIRST push on a portal that
# hasn't been deployed yet — this is what makes create → add-data → deploy seamless.
# 403 = the slug belongs to another account.
curl -fsS -X POST "$API/v1/repos/<project-slug>/claim" -H "Authorization: Bearer $ARC_TOKEN" >/dev/null
# Pushing data needs write: request ?actions=read,write,verify (default is read-only,
# pull). ?ttl=<secs> to tune.
LFS_URL=$(curl -fsS -X POST "$API/v1/repos/<project-slug>/lfs-token?actions=read,write,verify" \
-H "Authorization: Bearer $ARC_TOKEN" \
| node -e "process.stdin.on('data',d=>process.stdout.write(JSON.parse(d).lfs_url))")
git config lfs.url "$LFS_URL" # local only — carries the token, never committed
OSS self-host fallback. Running your own Giftless without an Arc account? Sign the token locally with the issuer's private key instead:
TOKEN=$(python3 ../../giftless/mint-token.py --org datopian --repo <project-slug> \ --ttl 3600 --algorithm RS256 --key-file ../../giftless/jwt_private_key) git config lfs.url "https://_jwt:$TOKEN@lfs.portaljs.com/datopian/<project-slug>"
(See giftless/README.md → "Authenticating a Git LFS client".)
Push the bytes to R2 (no GitHub remote required), then reclaim local disk. The
bytes route to wherever lfs.url points (set above) — a git remote is not a
prerequisite. git lfs push needs only a remote name to enumerate objects from the
branch; with lfs.url set, that remote's own URL is never contacted for the transfer.
Scaffolded portals are local-only (create-portaljs does git init + an initial
commit, no remote; a bare git push would fail with "No configured push
destination"), so branch on whether a real remote exists:
BRANCH=$(git rev-parse --abbrev-ref HEAD)
if git remote get-url --push origin >/dev/null 2>&1; then
# A GitHub remote exists (collaboration setup): a normal push carries commits +
# LFS pointers to GitHub AND streams the bytes to R2 via the pre-push hook.
git push -u origin "$BRANCH"
else
# Local-only scaffold (the DEFAULT): no GitHub remote. Add a throwaway local
# remote just so git-lfs has an object source, then push the LFS objects — the
# bytes go
Truncated for display — read the full file on GitHub.
Related Skills
claude-mem
117.6kPersistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + More
taste-skill
116.8kAgent skills that give coding agents taste: art direction, reference images and a browser check instead of generic AI pages
Agent-Reach
112.3kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Understand-Anything
92.8kGraphs that teach > graphs that impress. Turn any code into an interactive knowledge graph you can explore, search, and ask questions about. Works with Claude Code, Codex, Cursor, Copilot, Gemini CLI, and more.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
