SkillAgentSearch skills...

bigdata-skill

Pulls Bigdata.com (RavenPack) financial and news data via the official bigdata-client SDK and /v1/* REST endpoints β€” structured financials, prices, analyst estimates, entity-sentiment series, annotated chunk search, screener

Install / Use

npx skills add daymade/claude-code-skills --skill bigdata-skill

Installs into whichever agent you are using.

About this skill
πŸ“„

SKILL.md

Installable skill definition

Quality Score

93/100

Supported Platforms

Universal

Our assessment of bigdata-skill

bigdata-skill scores 93/100 on our quality scale, 719th of 4,698 Development & Engineering skills we index (top 16%).

Its SKILL.md is 15 KB long, well organised into 20 sections with 6 code examples: a thorough specification that gives an agent plenty to work with.

With 1,426 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
30/30
Structure
20/20
Description
15/15
Adoption
13/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 2 months ago, so bigdata-skill is actively maintained.
  • Our last check on 2026-10-26 found the source still online.
  • It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
  • Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit β€” read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-10-04. Automated pattern scan on 2026-10-04. It catches known dangerous patterns, not every risk β€” read a skill before letting an agent act on it.

bigdata-skill compared with similar skills

All 4 of these similar skills score higher than bigdata-skill; compare them before choosing.

SkillScoreStarsUpdatedFormat
bigdata-skill (this skill)by daymade931.4k53d agoSKILL.md
taste-skillby Leonxlnx100116.8k8d agoCLAUDE.md
Agent-Reachby Panniantong100112.3k8d agoCLAUDE.md
rufloby ruvnet10081.9ktodayCLAUDE.md
headroomby headroomlabs-ai10079.8ktodayCLAUDE.md

Frequently asked questions

How do I install bigdata-skill?
Run npx skills add daymade/claude-code-skills --skill bigdata-skill. The install tabs above show the steps for each supported agent.
Which AI agents does bigdata-skill work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is bigdata-skill safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is bigdata-skill still maintained?
The repository was last updated about 2 months ago, so bigdata-skill is actively maintained.

name: bigdata-skill disable-model-invocation: true description: >- Pulls Bigdata.com (RavenPack) financial and news data via the official bigdata-client SDK and /v1/* REST endpoints β€” structured financials, prices, analyst estimates, entity-sentiment series, annotated chunk search, screener. Use when the Bigdata MCP's tearsheets/search feel thin and you need the machine-readable substrate: mentions of Bigdata.com, RavenPack, a bd_v2_ key, rp_entity_id, or chunk/query_unit cost.

Bigdata.com SDK + REST Toolkit

Get the structured substrate the Bigdata.com MCP server doesn't hand over. The MCP returns clean prose and pre-synthesized tearsheets, but its search tool gives chunks with no per-chunk sentiment or entity spans, and its tearsheets give aggregate values β€” not the fiscal-period time series, universe screener, or per-field JSON you'd build a pipeline on. The official bigdata-client SDK plus a thin REST passthrough over the same backend, same JWT reach the official /v1/* endpoints that hold it. This skill bundles a toolkit that does exactly that β€” already debugged, already cost-guarded β€” so you don't re-pay the discovery cost.

The core problem this solves (read this first)

The Bigdata MCP server answers "what's the sentiment around NVIDIA?" with a readable paragraph or a pre-synthesized tearsheet β€” genuinely useful for a chat turn. But the moment you need the machine-readable substrate to build a pipeline on, the MCP doesn't hand it over:

  • its search tool returns chunks with text + relevance only β€” no per-chunk sentiment number, no entity character spans;
  • its tearsheets give aggregate values (a single sentiment score, a summary of estimates) β€” not a fiscal-period time series you can compute on, a universe screener, or per-field JSON.

The fix is a general pattern, not a Bigdata trick:

When an MCP data source returns only synthesized output but you need the structured fields underneath, drop to the vendor SDK or REST. MCP optimizes for a chat turn, not a pipeline.

Crucially, for Bigdata these structured fields are official, publicly documented REST endpoints (docs.bigdata.com/api-reference/...), not a hidden backend β€” and Bigdata is sunsetting the SDK (EOL 2026-12-31) in favour of this REST API, so the REST layer here is the forward-compatible path, not a hack. The SDK (bigdata_client.Bigdata) covers search + knowledge-graph; bd._api.http reaches every /v1/* endpoint the SDK never wrapped. The bundled bigdata_toolkit packages both behind one BigdataClient.

When to use this skill

Trigger on any of these, in any language:

  • The user is using Bigdata.com / RavenPack and the MCP result feels thin β€” "where's the sentiment score?", "I need entity-level data", "the calendar".
  • They want forward / structured financials for a ticker: analyst estimates, earnings or event calendar, earnings surprise, analyst ratings, price targets, a company screener / universe.
  • They want annotated news chunks with numeric sentiment + entity spans, or a sentiment time series / co-mention graph.
  • They mention a bd_v2_ API key, rp_entity_id, query_unit / chunk cost, bigdata-client, or "the bigdata MCP isn't enough".
  • They're building an investment-research dataset and need a reusable, cost-aware data-pull layer rather than one-off MCP calls.

Setup (one time)

1 β€” API key (never hardcode it). The client fail-fasts if it's missing:

export BIGDATA_API_KEY=bd_v2_xxxxxxxx

2 β€” An isolated Python env with the official SDK. The bundled toolkit imports bigdata_client; install it once:

uv venv .venv --python 3.12
uv pip install --python .venv/bin/python bigdata-client
# Behind a slow/blocked PyPI (e.g. mainland China) add a mirror, and unset any
# outbound proxy for the install step so uv reaches the index directly:
#   --index-url https://pypi.tuna.tsinghua.edu.cn/simple

3 β€” Outbound proxy (only if your network needs one to reach api.bigdata.com). Two equivalent options β€” the official SDK accepts both: an env var, or BigdataClient(proxy=...) in code. The env var is simplest:

export HTTPS_PROXY=http://<host>:<port>     # plus WSS_PROXY for chat/WebSocket

If a proxy does TLS interception (self-signed CA) and you hit SSL handshake errors, the official fix is BigdataClient(verify_ssl="<proxy-CA>.pem") β€” not blind retries.

4 β€” Make the bundled package importable by putting this skill's scripts/ on PYTHONPATH (or sys.path.insert(0, "<this-skill>/scripts")).

Smoke-test the whole path (entity resolve + quota are free; --with-search adds one ~1 query_unit chunk search):

BIGDATA_API_KEY=bd_v2_xxx PYTHONPATH=scripts .venv/bin/python scripts/probe_example.py

Quickstart

import sys
sys.path.insert(0, "<this-skill>/scripts")          # so `import bigdata_toolkit` resolves
from bigdata_toolkit import (
    BigdataClient, EntityResolver, AnnotatedSearcher,
    StructuredDataREST, CostTracker, CostModel, rc,   # rc = SSL-retry wrapper
)

c  = BigdataClient()                                  # SDK + REST escape hatch, one object
er = EntityResolver(c)
nvda = rc(lambda: er.resolve_id("NVIDIA", country="US"))   # -> 'E09E2B'  (rp_entity_id is the gateway key)

# --- Structured financials the MCP does NOT expose (REST escape hatch) ---
rest = StructuredDataREST(c)
est  = rc(lambda: rest.analyst_estimates(nvda, period="quarter", limit=5))  # forward consensus
surp = rc(lambda: rest.latest_surprise(nvda))                               # last EPS/revenue surprise
cal  = rc(lambda: rest.events_calendar(nvda, categories=["earnings-call"],
                                       start_date="2026-06-01", end_date="2026-12-31"))

# --- Annotated chunks the MCP STRIPS: sentiment + entity spans (cost-guarded) ---
s    = AnnotatedSearcher(c)
docs = rc(lambda: s.search_entity(nvda, keyword="data center", chunk_limit=10))
# each chunk dict: {"sentiment": float, "entities": [{"key": rp_id, "start", "end"}], "text", ...}

# --- Always know your spend (chunk-billed; see Cost discipline) ---
ct = CostTracker(c); ct.snapshot()
# ... run a batch ...
print(ct.delta())     # {'delta_chunks':..., 'delta_query_units':..., 'usd_fast':...}

Wrap every network call in rc(lambda: ...) β€” a first-handshake SSL: UNEXPECTED_EOF is common and the SDK's internal retry doesn't cover it.

Routing β€” which capability answers the question

| The user wants… | Use | Module | |---|---|---| | Company name / ISIN / CUSIP / SEDOL β†’ rp_entity_id | EntityResolver.resolve_id / .resolve_by_isin | kg.py (SDK) | | Forward analyst consensus (revenue/EPS by fiscal period) | StructuredDataREST.analyst_estimates | rest_ext.py | | Latest earnings surprise (actual vs estimate) | .latest_surprise | rest_ext.py | | Upcoming earnings / event calendar (one name or whole market) | .events_calendar | rest_ext.py | | Analyst ratings / price-target consensus | .analyst_ratings / .price_target | rest_ext.py | | Full financial statements (income / balance / cash-flow, multi-year) | .income_statement / .balance_sheet / .cash_flow_statement | rest_ext.py | | TTM valuation metrics & ratios (EV/EBITDA, ROE, P/E, margins) | .key_metrics_ttm / .company_ratios_ttm | rest_ext.py | | Company profile (CEO, sector, employees, IPO date) | .company_profile | rest_ext.py | | Daily OHLC prices / dividend history | .daily_prices / .dividends | rest_ext.py | | Revenue by geography / product segment | .revenue_geographic_segments / .revenue_product_segments | rest_ext.py | | Daily entity-sentiment time series (don't self-aggregate from chunks!) | .entity_sentiment | rest_ext.py | | Co-mention graph (supply-chain / competitor / customer β€” ⚠️ chunk-billed) | .connected_entities | rest_ext.py | | Build a universe by market-cap / sector / country | .company_screener | rest_ext.py | | News/filing/transcript chunks with sentiment + entity spans | AnnotatedSearcher.search_entity | search.py (SDK) | | Bulk-pull many searches 50% cheaper (portfolio backfill) | BatchSearch (createβ†’uploadβ†’pollβ†’download) | rest_ext.py | | Track / forecast quota spend before a backfill | CostTracker / CostModel | cost.py | | Hit an endpoint the toolkit hasn't wrapped yet | client.http.post("v1/<resource>/query", body) | client.py |

income/balance/cash-flow/daily-prices/dividends/revenue-segments return {fields, values} β€” wrap them in fields_values_to_records() to get [{field: value}]. The *_ttm / company_profile endpoints are already flat. All structured endpoints above are free (0 chunks) except connected_entities and AnnotatedSearcher (chunk-billed).

The two data faces (do NOT say "Bigdata fails for Chinese / A-shares")

This split is the most important non-obvious conclusion β€” state it precisely:

| Face | Path | A-share / Chinese verdict | |---|---|---| | Structured financial (estimates, calendar, surprise, ratings, target, screener, financials, prices, dividends, revenue segments, daily entity-sentiment) | REST (rest_ext.py) | Works β€” via rp_entity_id resolved from the English name or ISIN (not the Chinese name). Data is fresh. Minor holes (some A-share price-targets return the entity with no numeric target). The daily entity_sentiment series lives here and works for any resolvable entity β€” it is not the dead end below. | | Unstructured Chinese NLP (Chinese-news entity detection, per-chunk Chinese sentiment) | SDK search (search.py) | Dead end β€” a data-source-level gap, not an SDK bug: Chinese entity detection β‰ˆ 0, per-chunk CJK sentiment is a doc-level inherited value, and language mislabels Chinese filings as English. Pair Bigdata with a China-domestic source for Chinese-language chunk content; use Bigdata for the structured face (incl. aggregate entity_sentiment) + ISIN/KG crosswalk + English-language chunk sentiment. |

Cost discipline

1 query_unit = 10 chunks (official). Only chunk-search is billed β€” the structured /v1/* endpoints (estimates, financials, prices, calendar, surprise, ratings, the sentiment time series, screener…) are free (0 chunks, contract-tested). connected_entities (co-mentions) and AnnotatedSearcher are chunk-billed.

Three levers when you do pay for chunks:

  1. ChunkLimit, never a bare int. Search.run(int) is a document limit billed by the full chunk page; ChunkLimit(n) bills per chunk. AnnotatedSearcher.search forces ChunkLimit for you. (We observed roughly a 52x gap once β€” a single measured data point, not stated in the official docs; treat the exact multiple as indicative. The rule "use ChunkLimit" holds regardless, because max_chunks is the official billing unit.)
  2. Rerank bills only the returned chunks (official) β€” pass a rerank_threshold to recall broadly but pay only for the high-relevance hits.
  3. Batch search is 50% cheaper ($0.0075 vs $0.015 / qu) β€” use BatchSearch for a large multi-query backfill.

Use CostModel to veto an over-budget job before running it, and CostTracker.snapshot() / delta() to measure real spend. Full accounting β†’ references/cost_accounting.md.

Known pitfalls (already solved β€” don't re-debug these)

Each cost real debugging time and is fixed or guarded in the toolkit. Full reproductions and fixes in references/known_pitfalls.md:

  1. First-handshake SSL: UNEXPECTED_EOF β†’ wrap calls in rc(); the SDK's urllib3 retry only covers HTTP status, not the SSL EOF.
  2. All(entity, Keyword(kw)) raises TypeError β†’ combine with the & operator (entity & Keyword(kw)); All takes a single iterable. (Fixed in AnnotatedSearcher.entity_query.)
  3. The 52x doc-limit billing trap β†’ always ChunkLimit, never a bare int.

Truncated for display β€” read the full file on GitHub.

Related Skills

View on GitHub
GitHub Stars1.4k
CategoryDevelopment
Updated1mo ago
Forks219

Languages

Python

Trust signals

100/100

From repository metadata: license, adoption, age and documentation. Not a code audit β€” see the Safety scan above for what the skill file itself contains.

No cautions
bigdata-skill β€” Universal Skill: Install & Safety Check | SkillAgent