SkillAgentSearch skills...

portaljs-add-dataset

Add a dataset (CSV, TSV, JSON, or GeoJSON) to an existing PortalJS portal. Appends an entry to datasets.json so the catalog and showcase render it automatically; routes the data by source (local file vs remote URL) — R2 via Git LFS by default, remote URLs by passthrough.

Install / Use

npx skills add datopian/portaljs

Installs into whichever agent you are using.

About this skill
⚡

Claude Commands

Claude Code slash commands

Quality Score

94/100

Category

Automation

Supported Platforms

Claude Code

Our assessment of portaljs-add-dataset

portaljs-add-dataset scores 94/100 on our quality scale, 416th of 2,790 Automation skills we index (top 15%).

Its Claude Commands is 17 KB long, well organised into 30 sections with 11 code examples: a thorough specification that gives an agent plenty to work with.

With 2,356 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
30/30
Structure
20/20
Description
15/15
Adoption
14/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 2 months ago, so portaljs-add-dataset is actively maintained.
  • Our last check on 2026-11-16 found the source still online.
  • It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
  • Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-09-30. Automated pattern scan on 2026-09-29. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

portaljs-add-dataset compared with similar skills

All 4 of these similar skills score higher than portaljs-add-dataset; compare them before choosing.

SkillScoreStarsUpdatedFormat
portaljs-add-dataset (this skill)by datopian942.4k2mo agoClaude Commands
claude-memby thedotmack100117.6ktodayCLAUDE.md
taste-skillby Leonxlnx100116.8k8d agoCLAUDE.md
Agent-Reachby Panniantong100112.3k8d agoCLAUDE.md
Understand-Anythingby Egonex-AI10092.8k1d agoCLAUDE.md

Frequently asked questions

How do I install portaljs-add-dataset?
Run npx skills add datopian/portaljs. The install tabs above show the steps for each supported agent.
Which AI agents does portaljs-add-dataset work with?
It is written for Claude Code, as a Claude Commands file. Other agents that read the same format can often use it too.
Is portaljs-add-dataset safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is portaljs-add-dataset still maintained?
The repository was last updated about 2 months ago, so portaljs-add-dataset is actively maintained.

description: Add a dataset (CSV, TSV, JSON, or GeoJSON) to an existing PortalJS portal. Appends an entry to datasets.json so the catalog and showcase render it automatically; routes the data by source (local file vs remote URL) — R2 via Git LFS by default, remote URLs by passthrough. allowed-tools: Read, Write, Edit, Bash, WebFetch

/portaljs-add-dataset

Add a dataset to an existing PortalJS (portaljs-catalog) portal. Appends one entry to datasets.json — the single source of truth for the catalog — and routes the data to the right place by source first, then size. No per-dataset page is created: the catalog at /search lists it and the dynamic showcase route pages/[owner]/[slug].tsx renders it automatically at /@<namespace>/<slug>.

The routing matrix — branch on SOURCE, then size

Where the bytes end up depends first on where they come from, then on size/intent. The manifest's resource.path (or single-file file) is the unifying abstraction: resourceUrl() in lib/datasets.ts serves a repo-relative path from /public/data and passes an absolute URL straight through. So every route below just decides what string to write into path.

| Source | Default | What happens | Manifest path | |--------|---------|--------------|-----------------| | Local file | R2 via Git LFS | mv into the repo, git lfs track <path>, commit a ~134 B pointer, git lfs push → Giftless → R2 (no GitHub remote needed) | absolute R2 URL (browser fetches R2 directly) | | Local file (fenced) | inline | Only bundled SAMPLE data or an OSS self-host with no R2: cp into public/data/ (stays inline per .gitattributes fence) | bare filename → /data/<file> | | Remote URL | passthrough | record the URL as-is — no download, no upload, zero duplication | the absolute URL, unchanged | | Remote URL | adopt (opt-in) | user wants it hosted/versioned under the portal: fetch → route as a local file (R2/LFS) | absolute R2 URL |

Storage decision (epic po-g9y): the default for added data is R2, regardless of size or format. Inline is not a size threshold — it is a fenced exception for bundled sample data and the OSS-no-R2 fallback. Remote URLs default to passthrough (copy nothing); adopting one into R2 is an explicit opt-in.

Remote-URL passthrough caveat: serving and linking always work. But in-browser range / DuckDB queries against a 3rd-party URL need CORS + range support on that host, which we don't control. If the user needs querying (not just preview/download) and the remote lacks CORS/range, recommend the adopt-into-R2 option.

Supported formats for the showcase preview/registration: CSV, TSV, JSON (array), GeoJSON. (LFS itself is format-agnostic — any binary can be tracked — but the showcase Table/map only previews these.)

Required input — ask, don't error

  • Source — a local file path (./data/file.csv) or a public URL (https://…/data.csv)
  • Portal directory — path to the portal project (defaults to current directory)
  • Namespace — the dataset's namespace value (the portal's NAMESPACE_TYPE group: a subject for 'theme' portals, a publisher for 'owner' portals)

If the source is missing, ask for it — never dead-end. The user can say "use defaults" to accept the defaults below.

Steps

1. Gather input from $ARGUMENTS (interview if thin)

Extract what's present:

  • SOURCE — file path or URL
  • PORTAL_DIR — portal directory (default: .)
  • DATASET_NAME — human-readable name (default: derived from filename)
  • DATASET_SLUG — URL slug (default: lowercase hyphenated filename without extension)
  • DESCRIPTION — optional one-line description
  • NAMESPACE — namespace value (default: read the existing first entry's namespace from datasets.json, else reference)
  • ADOPT — for a remote URL only: whether to adopt the file into R2 (default: no — passthrough). Only ask if the URL route is taken and querying is plausibly needed.

If SOURCE is missing, ask (one focused prompt) and wait:

To add a dataset I need:
1. Source: local file path or public URL (required)
2. Portal directory (Enter for current directory)
3. Dataset name (Enter to use the filename)
4. Namespace value — the group this dataset belongs to
   (subject if the portal is "theme" mode, publisher if "owner" mode; Enter to reuse the catalog's existing namespace)

Check the portal's namespace mode if helpful: read NAMESPACE_TYPE from PORTAL_DIR/lib/datasets.ts so you can phrase the namespace question correctly ("subject" vs "publisher").

2. Validate the portal directory

The target must be a portaljs-catalog portal. Confirm PORTAL_DIR/datasets.json, PORTAL_DIR/package.json, and PORTAL_DIR/pages/[owner]/[slug].tsx exist. If datasets.json is missing, tell the user this portal isn't the catalog template (it may be an older single-page template) and ask how to proceed rather than failing silently.

3. Detect the format

If SOURCE is a URL: fetch headers (or a HEAD) and check the status. If not 200, tell the user the fetch failed (with the HTTP status) and ask them to confirm the URL is publicly accessible, then retry. Detect format from Content-Type or the URL extension. (For the default passthrough route you do not download the body — only enough to detect format.)

If SOURCE is a local file path: check the file exists. If not, tell the user the path wasn't found and ask for a correct path. Detect format from the file extension.

Format detection rules:

  • .csv or text/csv → CSV
  • .tsv or text/tab-separated-values → TSV
  • .geojson or application/geo+json or (JSON-parseable and parsed.type === "FeatureCollection") → GeoJSON
  • .json or application/json → JSON array
  • Vector geo formats — .zip (zipped Shapefile), .gpkg, .kml/.kmz, .fgb, or a .csv with a geometry/lat-lon column → hand off to /portaljs-add-geo. That skill auto-ingests geo sources into a dual-tier dataset (PMTiles render + GeoParquet query) on the user's machine. This tabular skill only previews flat files; it does not tile or build GeoParquet. (A small GeoJSON you just want listed with a download link can stay here as the geojson format; for a map/query view, prefer /portaljs-add-geo or /portaljs-add-map.)
  • Anything else: tell the user the showcase can't preview this format and ask them to convert to CSV, TSV, JSON array, or GeoJSON first.

4. Route the data (the core of this skill)

Pick the branch from the matrix above. Decide on SOURCE first.

4a. Remote URL — passthrough (DEFAULT for URLs)

Do nothing to the bytes. The manifest path is the absolute URL itself; resourceUrl() passes it through and the browser fetches it directly. Skip to Step 5 with MANIFEST_PATH=<the URL>.

Mention the CORS/range caveat if the user may need querying, and offer the adopt route.

4b. Remote URL — adopt into R2 (OPT-IN only)

Only when the user explicitly wants the file hosted/versioned under the portal. Download it to a temp path, then fall through to 4c treating that temp file as the local source:

curl -fL "$SOURCE" -o "/tmp/$DATASET_SLUG.$EXT"

4c. Local file — R2 via Git LFS (DEFAULT)

This versions a tiny pointer in git and streams the bytes to Cloudflare R2 through Giftless. The browser then fetches the bytes straight from R2 (manifest path = absolute R2 URL).

cd PORTAL_DIR
mkdir -p data
# mv (NOT cp) so an external local file isn't tripled on disk:
mv "$SOURCE" "data/$DATASET_SLUG.$EXT"     # for 4b, $SOURCE is the temp download

# Ensure the LFS clean/smudge filters are installed for THIS repo before tracking.
# On a fresh machine `git lfs install` may never have run (brew install git-lfs does
# NOT auto-run it), so without this `git add` commits the RAW bytes instead of a
# ~134 B pointer and the later `git lfs push` has nothing to stream. Idempotent.
if ! git config --get filter.lfs.clean >/dev/null 2>&1; then
  git lfs install --local
fi

# Per-file, format-agnostic LFS tracking (appends a path entry to .gitattributes,
# NOT an extension glob):
git lfs track "data/$DATASET_SLUG.$EXT"
git add .gitattributes "data/$DATASET_SLUG.$EXT"
git commit -m "data: add $DATASET_SLUG via LFS"

mv, not cp. When bringing an external local file into the repo, move it in place — a cp leaves a 3rd on-disk copy (original + working tree + .git/lfs cache).

Authenticate git-lfs (phase-2 gotcha, po-g9y.1). The committed .lfsconfig carries the Giftless URL but no credentials. Mint a repo-scoped JWT and set the credentialed URL in local git config only (never commit it). Use the _jwt HTTP Basic piggyback — do not set a broad http.extraHeader, which git-lfs would replay onto the presigned R2 PUT (R2 → 400) and the verify callback.

Mint the token from the Arc API (default — no private key on your machine). The Arc API is the token issuer (po-g9y.13): it holds the RS256 signer as a Worker secret and mints a scoped, short-TTL LFS token for any authenticated Arc user. Reuse the same Arc token /portaljs-deploy resolves (PORTALJS_TOKEN, else ~/.portaljs/credentials); if you have neither, run /portaljs-deploy's sign-in step once. The endpoint returns a ready-to-use credentialed lfs_url:

API="${PORTALJS_ARC_API:-https://api.arc.portaljs.com}"
ARC_TOKEN="${PORTALJS_TOKEN:-$(node -e "try{process.stdout.write(JSON.parse(require('fs').readFileSync(process.env.HOME+'/.portaljs/credentials','utf8')).token||'')}catch{}")}"
# Claim the slug under your Arc account (idempotent; safe to re-run). Minting never
# creates a slug (po-g9y.13), so claim once before the FIRST push on a portal that
# hasn't been deployed yet — this is what makes create → add-data → deploy seamless.
# 403 = the slug belongs to another account.
curl -fsS -X POST "$API/v1/repos/<project-slug>/claim" -H "Authorization: Bearer $ARC_TOKEN" >/dev/null
# Pushing data needs write: request ?actions=read,write,verify (default is read-only,
# pull). ?ttl=<secs> to tune.
LFS_URL=$(curl -fsS -X POST "$API/v1/repos/<project-slug>/lfs-token?actions=read,write,verify" \
  -H "Authorization: Bearer $ARC_TOKEN" \
  | node -e "process.stdin.on('data',d=>process.stdout.write(JSON.parse(d).lfs_url))")
git config lfs.url "$LFS_URL"     # local only — carries the token, never committed

OSS self-host fallback. Running your own Giftless without an Arc account? Sign the token locally with the issuer's private key instead:

TOKEN=$(python3 ../../giftless/mint-token.py --org datopian --repo <project-slug> \
  --ttl 3600 --algorithm RS256 --key-file ../../giftless/jwt_private_key)
git config lfs.url "https://_jwt:$TOKEN@lfs.portaljs.com/datopian/<project-slug>"

(See giftless/README.md → "Authenticating a Git LFS client".)

Push the bytes to R2 (no GitHub remote required), then reclaim local disk. The bytes route to wherever lfs.url points (set above) — a git remote is not a prerequisite. git lfs push needs only a remote name to enumerate objects from the branch; with lfs.url set, that remote's own URL is never contacted for the transfer. Scaffolded portals are local-only (create-portaljs does git init + an initial commit, no remote; a bare git push would fail with "No configured push destination"), so branch on whether a real remote exists:

BRANCH=$(git rev-parse --abbrev-ref HEAD)
if git remote get-url --push origin >/dev/null 2>&1; then
  # A GitHub remote exists (collaboration setup): a normal push carries commits +
  # LFS pointers to GitHub AND streams the bytes to R2 via the pre-push hook.
  git push -u origin "$BRANCH"
else
  # Local-only scaffold (the DEFAULT): no GitHub remote. Add a throwaway local
  # remote just so git-lfs has an object source, then push the LFS objects — the
  # bytes go 

Truncated for display — read the full file on GitHub.

Related Skills

View on GitHub
GitHub Stars2.4k
CategoryAutomation
Updated2mo ago
Forks335

Languages

TypeScript

Trust signals

100/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

No cautions