---

name: data-fetcher description: Fetch economic data from FRED, World Bank, BLS, OECD, and Yahoo Finance

Data-Fetcher

Purpose

This skill helps economists fetch data from major economic data APIs including FRED (Federal Reserve Economic Data), World Bank, BLS (Bureau of Labor Statistics), OECD, and Yahoo Finance. It generates clean, documented Python code with proper error handling.

When to Use

• Downloading macroeconomic indicators
• Building custom datasets from multiple sources
• Automating data updates for ongoing projects
• Fetching cross-country panel data

Instructions

Step 0: API Key Setup Check (Run Before Anything Else)

Before generating any code, Claude must check for required API keys.

1. Read the file [plugin_root]/.env (same directory as .mcp.json).
2. Check for FRED_API_KEY and BLS_API_KEY.

If FRED_API_KEY is missing or blank:

• Tell the user: "A free FRED API key is required. Get one at https://fred.stlouisfed.org/docs/api/api_key.html (takes ~1 minute). Paste it here and I'll save it."
• Wait for input, then append FRED_API_KEY=<value> to .env.

If BLS_API_KEY is missing:

• Inform the user it's optional but increases BLS rate limits, and they can get one free at https://www.bls.gov/developers/. If they want to add it later, just paste it and say "save my BLS key".

If .env exists and keys are already set: load them silently and inject them into all generated code via python-dotenv. Use load_dotenv() with no arguments so Python searches up from the current working directory automatically — never hardcode the plugin root path:

from dotenv import load_dotenv
load_dotenv()  # searches CWD and parent directories for .env

The .env file stores keys locally and is never committed to version control. Generated scripts always read keys from environment variables — never hardcoded.

---

Step 1: Identify Data Requirements

Ask the user:

1. What data do you need? (GDP, unemployment, inflation, etc.)
2. What time period and frequency?
3. What countries/regions?
4. Preferred output format? (CSV, DataFrame, etc.)

Step 2: Select Appropriate API

| Data Type | Best Source | Package | |-----------|------------|---------| | US macro | FRED | fredapi | | Global development | World Bank | wbdata | | Labor statistics | BLS | requests (BLS API v2) | | Cross-country OECD | OECD | requests (OECD SDMX API) | | Cross-country macro/finance | IMF | imf-reader | | Financial / asset prices | Yahoo Finance | yfinance |

Step 3: Generate Clean Code

Include:

• API key handling (environment variables)
• Error handling for API failures
• Data cleaning and formatting
• Documentation of series definitions

Example Output

"""
Economic Data Fetcher
=====================
Downloads macroeconomic data from FRED and World Bank APIs.
Requires: fredapi, wbdata, pandas

Setup: Set FRED_API_KEY environment variable
Get a free key from: https://fred.stlouisfed.org/docs/api/api_key.html
"""

import os
import pandas as pd
from datetime import datetime, timedelta
from typing import List, Optional, Dict

# ============================================
# FRED Data Fetcher
# ============================================

def fetch_fred_series(
    series_ids: List[str],
    start_date: str = "2000-01-01",
    end_date: Optional[str] = None,
    api_key: Optional[str] = None
) -> pd.DataFrame:
    """
    Fetch time series data from FRED.
    
    Parameters
    ----------
    series_ids : list of str
        FRED series IDs (e.g., ['GDP', 'UNRATE', 'CPIAUCSL'])
    start_date : str
        Start date in YYYY-MM-DD format
    end_date : str, optional
        End date (defaults to today)
    api_key : str, optional
        FRED API key (defaults to FRED_API_KEY env var)
    
    Returns
    -------
    pd.DataFrame
        DataFrame with date index and series as columns
    
    Example
    -------
    >>> df = fetch_fred_series(['GDP', 'UNRATE'], '2010-01-01')
    """
    try:
        from fredapi import Fred
    except ImportError:
        raise ImportError("Install fredapi: pip install fredapi")
    
    # Get API key
    api_key = api_key or os.environ.get('FRED_API_KEY')
    if not api_key:
        raise ValueError(
            "FRED API key required. Set FRED_API_KEY environment variable "
            "or pass api_key parameter. Get a key at: "
            "https://fred.stlouisfed.org/docs/api/api_key.html"
        )
    
    fred = Fred(api_key=api_key)
    end_date = end_date or datetime.now().strftime('%Y-%m-%d')
    
    # Fetch each series
    data = {}
    for series_id in series_ids:
        try:
            series = fred.get_series(
                series_id,
                observation_start=start_date,
                observation_end=end_date
            )
            data[series_id] = series
            print(f"✓ Downloaded {series_id}")
        except Exception as e:
            print(f"✗ Failed to download {series_id}: {e}")
    
    # Combine into DataFrame
    df = pd.DataFrame(data)
    df.index.name = 'date'
    
    return df


# Common FRED series for economists
FRED_SERIES = {
    # GDP and Output
    'GDP': 'Gross Domestic Product',
    'GDPC1': 'Real GDP',
    'GDPPOT': 'Real Potential GDP',
    
    # Labor Market
    'UNRATE': 'Unemployment Rate',
    'PAYEMS': 'Total Nonfarm Payrolls',
    'CIVPART': 'Labor Force Participation Rate',
    
    # Prices
    'CPIAUCSL': 'Consumer Price Index',
    'PCEPI': 'PCE Price Index',
    'CPILFESL': 'Core CPI',
    
    # Interest Rates
    'FEDFUNDS': 'Federal Funds Rate',
    'DGS10': '10-Year Treasury Rate',
    'T10Y2Y': '10Y-2Y Treasury Spread',
    
    # Money and Credit
    'M2SL': 'M2 Money Stock',
    'TOTRESNS': 'Total Reserves',
}


# ============================================
# World Bank Data Fetcher
# ============================================

def fetch_world_bank_data(
    indicators: Dict[str, str],
    countries: List[str] = ['USA', 'GBR', 'DEU', 'FRA', 'JPN'],
    start_year: int = 2000,
    end_year: Optional[int] = None
) -> pd.DataFrame:
    """
    Fetch indicator data from World Bank.
    
    Parameters
    ----------
    indicators : dict
        Dict mapping indicator codes to names
        e.g., {'NY.GDP.PCAP.CD': 'gdp_per_capita'}
    countries : list of str
        ISO 3-letter country codes
    start_year : int
        Start year
    end_year : int, optional
        End year (defaults to current year)
    
    Returns
    -------
    pd.DataFrame
        Panel data with country and year
    
    Example
    -------
    >>> indicators = {
    ...     'NY.GDP.PCAP.CD': 'gdp_per_capita',
    ...     'SP.POP.TOTL': 'population'
    ... }
    >>> df = fetch_world_bank_data(indicators, ['USA', 'GBR'])
    """
    try:
        import wbdata
    except ImportError:
        raise ImportError("Install wbdata: pip install wbdata")
    
    import datetime
    end_year = end_year or datetime.datetime.now().year
    # Pass date range directly to the API to avoid downloading full history
    date_range = (datetime.datetime(start_year, 1, 1), datetime.datetime(end_year, 12, 31))

    all_data = []

    for indicator_code, indicator_name in indicators.items():
        try:
            data = wbdata.get_dataframe(
                {indicator_code: indicator_name},
                country=countries,
                date=date_range,
            )
            data = data.reset_index()
            all_data.append(data)
            print(f"✓ Downloaded {indicator_name}")

        except Exception as e:
            print(f"✗ Failed to download {indicator_name}: {e}")

    # Merge all indicators
    if all_data:
        df = all_data[0]
        for other_df in all_data[1:]:
            df = df.merge(other_df, on=['country', 'date'], how='outer')
        return df
    
    return pd.DataFrame()


# Common World Bank indicators
WORLD_BANK_INDICATORS = {
    # Income and Growth
    'NY.GDP.PCAP.CD': 'GDP per capita (current US$)',
    'NY.GDP.PCAP.KD.ZG': 'GDP per capita growth (%)',
    'NY.GDP.MKTP.KD.ZG': 'GDP growth (%)',
    
    # Population
    'SP.POP.TOTL': 'Population, total',
    'SP.URB.TOTL.IN.ZS': 'Urban population (%)',
    
    # Trade
    'NE.TRD.GNFS.ZS': 'Trade (% of GDP)',
    'BX.KLT.DINV.WD.GD.ZS': 'FDI, net inflows (% of GDP)',
    
    # Human Capital
    'SE.XPD.TOTL.GD.ZS': 'Education expenditure (% of GDP)',
    'SH.XPD.CHEX.GD.ZS': 'Health expenditure (% of GDP)',
    
    # Inequality
    'SI.POV.GINI': 'Gini index',
    'SI.POV.DDAY': 'Poverty headcount ratio ($1.90/day)',
}


# ============================================
# Usage Example
# ============================================

if __name__ == "__main__":
    # Example 1: Fetch US macro data from FRED
    us_macro = fetch_fred_series(
        series_ids=['GDP', 'UNRATE', 'CPIAUCSL', 'FEDFUNDS'],
        start_date='2010-01-01'
    )
    
    print("\nUS Macro Data (FRED):")
    print(us_macro.tail())
    
    # Save to CSV
    us_macro.to_csv('data/us_macro_fred.csv')
    print("\nSaved to data/us_macro_fred.csv")
    
    # Example 2: Fetch cross-country data from World Bank
    indicators = {
        'NY.GDP.PCAP.CD': 'gdp_per_capita',
        'SP.POP.TOTL': 'population',
        'NY.GDP.MKTP.KD.ZG': 'gdp_growth'
    }
    
    cross_country = fetch_world_bank_data(
        indicators=indicators,
        countries=['USA', 'GBR', 'DEU', 'FRA', 'JPN', 'CHN', 'IND', 'BRA'],
        start_year=2000
    )
    
    print("\nCross-Country Data (World Bank):")
    print(cross_country.head(10))
    
    # Save to CSV
    cross_country.to_csv('data/cross_country_wb.csv', index=False)
    print("\nSaved to data/cross_country_wb.csv")

BLS Data Fetcher

"""
BLS (Bureau of Labor Statistics) Data Fetcher
==============================================
Fetches labor market data from BLS Public Data API v2.
Requires: requests, pandas
API key (free): https://www.bls.gov/developers/

Note: BLS API v2 limits each request to a 20-year window.
This fetcher automatically chunks longer ranges into 20-year batches.
"""

import os
import math
import requests
import pandas as pd
from typing import List, Optional


def fetch_bls_series(
    series_ids: List[str],
    start_year: str = "2010",
    end_year: Optional[str] = None,
    api_key: Optional[str] = None
) -> pd.DataFrame:
    """
    Fetch time series data from BLS API v2.
    Automatically splits requests exceeding the 20-year API limit.

    Parameters
    ----------
    series_ids : list of str
        BLS series IDs (e.g., ['LNS14000000'] for unemployment rate)
    start_year : str
        Start year (YYYY)
    end_year : str, optional
        End year (defaults to current year)
    api_key : str, optional
        BLS API key (defaults to BLS_API_KEY env var)

    Example
    -------
    >>> df = fetch_bls_series(['LNS14000000', 'CES0000000001'], '2000')
    """
    import datetime
    api_key = api_key or os.environ.get('BLS_API_KEY')
    end_yr = int(end_year or datetime.datetime.now().year)
    start_yr = int(start_year)

    # BLS API v2: max 20 years per request — split into chunks
    MAX_YEARS = 20
    chunks = []
    chunk_start = start_yr
    while chunk_start <= end_yr:
        chunk_end = min(chunk_start + MAX_YEARS - 1, end_yr)
        chunks.append((str(chunk_start), str(chunk_end)))
        chunk_start = chunk_end + 1

    url = "https://api.bls.gov/publicAPI/v2/timeseries/data/"
    all_records = []

    for s_yr, e_yr in chunks:
        payload = {
            "seriesid": series_ids,
            "startyear": s_yr,
            "endyear": e_yr,
        }
        if api_key:
            payload["registrationkey"] = api_key

        response = requests.post(url, json=payload)
        response.raise_for_status()
        data = response.json()

        if data["status"] != "REQUEST_SUCCEEDED":
            raise ValueError(f"BLS API error: {data.get('message',