data-scientist
Processes and analyzes data with resident-kernel engines (DuckDB, Polars) and one-shot tools. Use for CSV/parquet/JSON analysis, group-by/join/aggregation, time series, distributions, cleaning, or plotting a dataset.
Install / Use
npx skills add code-yeongyu/oh-my-openagent --skill data-scientistInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
Data & AnalyticsSupported Platforms
Our assessment of data-scientist
data-scientist scores 89/100 on our quality scale, 327th of 604 Data & Analytics skills we index.
Its SKILL.md is 5.6 KB long, well organised into 8 sections and no code examples: a solid amount of guidance for an agent.
With 69,362 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 2 months ago, so data-scientist is actively maintained.
- Our last check on 2026-10-31 found the source still online.
- No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
- Its trust signals score 88/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-09-25. Automated pattern scan on 2026-09-25. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
data-scientist compared with similar skills
All 4 of these similar skills score higher than data-scientist; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| data-scientist (this skill)by code-yeongyu | 89 | 69.4k | 53d ago | SKILL.md |
| Agent-Reachby Panniantong | 100 | 112.3k | 8d ago | CLAUDE.md |
| headroomby headroomlabs-ai | 100 | 79.8k | today | CLAUDE.md |
| Scraplingby D4Vinci | 100 | 103.6k | 2d ago | MCP Server |
| crawl4aiby unclecode | 100 | 87.4k | 3d ago | MCP Server |
Frequently asked questions
- How do I install data-scientist?
- Run
npx skills add code-yeongyu/oh-my-openagent --skill data-scientist. The install tabs above show the steps for each supported agent. - Which AI agents does data-scientist work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is data-scientist safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 88/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is data-scientist still maintained?
- The repository was last updated about 2 months ago, so data-scientist is actively maintained.
Skill content
View source on GitHubname: data-scientist description: "Processes and analyzes data with resident-kernel engines (DuckDB, Polars) and one-shot tools. Use for CSV/parquet/JSON analysis, group-by/join/aggregation, time series, distributions, cleaning, or plotting a dataset."
Data Scientist: Hybrid-Engine Data Processing
Answer data questions through the cheapest engine and surface that can prove the answer, and decide where the computation should live before touching the data.
Execution surfaces: resident kernel first
A persistent REPL/eval kernel (many harnesses expose one for JavaScript and Python) is the default surface. Reason: each one-shot process pays roughly a second of spawn-plus-import overhead and re-scans the input file, while a resident connection amortizes both — after a one-time load, repeat queries return in milliseconds. Exploration is repeat queries, so this difference dominates the session.
- JavaScript kernel (Bun): run
scripts/ensure-js-deps.shonce; it prints the absolute import path for@duckdb/node-api. Dynamic-import it, connect once, query across cells. - Python kernel: the default surface for Python work. duckdb/numpy/matplotlib are
typically resident; Polars and pyarrow come from
scripts/ensure-py-deps.sh, which installs them once into a user cache keyed to the kernel's interpreter —sys.path.insertthe printed directory and import. The interpreter itself is never mutated. - uv lane (
uv run --with ...): isolation for a heavy or crash-prone one-shot that should not take the kernel down. - No kernel (plain-shell harness): the same engines as one-shots —
bun -efor DuckDB-js,uv run python -cfor the Python stack — batching several questions per process.
Per-surface patterns and pitfalls: read references/execution-surfaces.md before first use.
Engine selection
- DuckDB for SQL-shaped work: direct file queries, joins, aggregation, subqueries, window functions. It queries CSV/Parquet/JSON in place without loading, spills to disk past its memory limit, and reads remote files with the same syntax.
- Polars when the pipeline is DataFrame-shaped: expression-chain transforms, reshapes,
streaming datasets past RAM — resident in the Python kernel via
ensure-py-deps.sh. Readreferences/polars-lane.md— the current 1.x API differs from widely-memorized older spellings. - numpy when numeric work goes beyond SQL/DataFrame aggregation: statistical tests, linear algebra, FFT, random sampling.
- matplotlib for every chart — read
references/visualization.mdfirst; it carries the quality bar and a mandatory visual check.
Performance folklore ("X is Nx faster at filtering") varies with data shape, cardinality, and hardware. When the engine choice materially matters, measure on the actual data instead of trusting remembered multipliers.
Placement: decide where the computation lives
Probe before you compute — one cell: file size, free RAM, and (when unclear) a row count via a direct scan. Then place the work:
- Load into memory when the working set stays within roughly a quarter of free RAM AND
the session will run repeated queries:
CREATE TABLE t AS SELECT ...(or a collected DataFrame) once, then iterate. One scan up front converts every later query from a file re-scan into milliseconds. - Query in place / stream when the question is single-pass, or the data exceeds RAM:
DuckDB reads files directly (
FROM 'data.csv'); past RAM, cap DuckDB's memory and let it spill, or use Polars' streaming engine in the Python kernel. NEVER load a larger-than-RAM dataset fully into memory — swapping stalls the whole machine, while streaming merely takes longer. - Query remotely, in place when the data lives elsewhere: DuckDB reads http(s)/S3 Parquet and CSV with projection and predicate pushdown, so fetch the columns and rows the question needs, never the whole file. When data sits on another machine you can execute on, ship the query to the data and return the small result. Rule: result much smaller than data — move the query; repeated local iteration planned — move a pruned copy of the data once.
Sizing heuristics and recipes: references/placement.md.
Hard rules
- NEVER use pandas. DuckDB and Polars beat it decisively on every workload this skill
covers, and the environments this skill assumes do not ship it —
.df()on a DuckDB result raises unless pandas is installed; convert with.pl()via Arrow instead. - Excel files are not read directly: export to CSV or Parquet first.
Output contract
Answer the question; report row counts and timing for anything heavy; then stop — no bonus
charts, no extra exploration passes beyond what the question needed. Chart when asked, or
when the answer is a shape (trend, distribution, comparison) that prose cannot carry — then
follow references/visualization.md including its visual QA step.
References
| Read | When |
| --- | --- |
| references/execution-surfaces.md | before the first query on any surface: kernel patterns, one-shot recipes, escalation rules |
| references/polars-lane.md | DataFrame-shaped pipeline or data past RAM: current API, Arrow handoff, package sets |
| references/placement.md | before heavy or remote work: sizing probe, memory limits, remote reads |
| references/visualization.md | before any chart: type selection, quality bar, CJK fonts, visual QA |
| references/uv-setup.md | uv missing or broken on this machine |
CLI fallback
When no kernel or REPL surface exists, uv run scripts/quick-query.py <file> [SQL]
(--filter <polars-sql-expr>, --describe) answers ad-hoc questions with zero code.
Supports CSV, Parquet, JSON, NDJSON.
Related Skills
Agent-Reach
112.3kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
headroom
79.8kCompress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
Scrapling
103.6k🕷️ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
crawl4ai
87.4kOfficial Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
