Skill Adaptation Debt Engine

A Streamlit dashboard that measures skill adaptation debt instead of predicting outcomes. It decomposes pressure into churn, novelty, and breadth to explain which roles/industries are becoming harder to staff. Includes role/industry reports, skill pressure maps, what-if scenario simulation, and a dataset explorer.

Generate Convert Improve

Install / Use

npx skills add AmirhosseinHonardoust/Skill-Adaptation-Debt-Engine

Installs into whichever agent you are using.

About this skill

Quality Score

0/100

README

<p align="center"> <h1 align="center">Skill Adaptation Debt Engine</h1> <p align="center"> A pressure-first lens on labor markets: measure <b>adaptation debt</b> (skill churn + novelty + breadth) instead of predicting outcomes. </p> </p> <p align="center"> <img src="https://img.shields.io/badge/Streamlit-App-FF4B4B?logo=streamlit&logoColor=white" /> <img src="https://img.shields.io/badge/Python-3.10%2B-3776AB?logo=python&logoColor=white" /> <img src="https://img.shields.io/badge/Plotly-Interactive-3F4F75?logo=plotly&logoColor=white" /> <img src="https://img.shields.io/badge/Pandas-Data-150458?logo=pandas&logoColor=white" /> <img src="https://img.shields.io/badge/Scikit--learn-ML-F7931E?logo=scikitlearn&logoColor=white" /> <img src="https://img.shields.io/badge/License-MIT-2ea44f" /> </p>

What this project is

Most labor-market tools try to predict outcomes (salary, growth, “hot jobs”). This repo does something different:

It treats each job role as a stability problem and computes how much “skill adaptation debt” a role accumulates when:

demanded skills churn month-to-month (instability / volatility),
demanded skills are rare (hard to hire/train),
and the role requires many distinct skills simultaneously (cognitive + coordination load).

The output is not “will you lose your job.”
It’s where stability is being borrowed, where teams will feel training friction, and which roles have high learning debt if the market moves.

Dataset & Copyright

This project uses the “Future Jobs and Skills Demand 2025” dataset from Kaggle:

Dataset page: https://www.kaggle.com/datasets/ahsanneural/future-jobs-and-skills-demand-2025

Attribution

Full credit belongs to the dataset owner/uploader on Kaggle.
This repository does not claim ownership of the dataset.

License / Usage Notes

The dataset is subject to Kaggle’s dataset terms and the license shown on the dataset page.
Please review the license before reusing, redistributing, or publishing derived versions of the raw data.
This repo is intended for educational and analytical purposes; if you plan to include the raw dataset in repository/releases, make sure the dataset license explicitly allows redistribution.

How this project uses the data

The app reads the dataset to compute skill rarity, skill churn, and role/industry “adaptation debt” metrics.
The outputs (charts/tables) are derived analytics and do not replace the original dataset source.

Core idea: Adaptation Debt (pressure-first metric)

Adaptation debt is a weighted pressure index, not a prediction.

1) Churn Pressure (instability over time)

A role can look fine today but still be unstable if its required skills change frequently.
We measure churn as a proxy for “how quickly the definition of competence is moving.”

High churn pressure means:

onboarding becomes harder,
internal training becomes stale faster,
hiring accuracy drops (because yesterday’s skill list no longer fits).

2) Novelty Pressure (rare + churning = learning friction)

A skill can be rare but stable (fine).
A skill can be common but volatile (manageable).
The hardest regime is rare + volatile, because it produces planning failure: you can’t reliably staff or train ahead of the curve.

Novelty pressure is designed to highlight:

“the skill doesn’t exist widely yet”
while also “the target keeps moving”

3) Breadth Pressure (role complexity)

Breadth is the number of distinct skills expected in the role.

High breadth pressure usually signals:

a “Swiss-army role” (too much scope),
a coordination burden (“you must integrate many systems”),
or a maturity mismatch (companies stuffing multiple jobs into one listing).

Final metric (what sliders control)

The sidebar “Debt weights (advanced)” controls the mixture:

Churn weight → how much volatility dominates debt
Novelty weight → how much rare+volatile dominates debt
Breadth weight → how much complexity dominates debt

You’re not changing “truth”, you’re changing the lens (what kind of instability you care about).

App pages: what each view answers

Role Report

Question: “For this specific role, what is the debt profile and which skills cause it?”

You get:

adaptation debt score (final index),
churn / novelty / breadth sub-pressures,
top skill drivers (the “why” list).

Industry Report

Question: “Inside an industry, which roles concentrate adaptation debt and why?”

You get:

a ranked table of roles within one industry,
pressure components per role,
a quick “how to interpret” guide.

Skill Explorer

Question: “Which skills are globally rare, churning, or novelty-heavy?”

You get:

global counts/shares (how common),
rarity,
churn index,
novelty index,
and a search field for quick lookup.

Scenario Simulator

Question: “If market pressure increases (more churn / more novelty / more breadth), how sensitive is this role?”

You get:

baseline vs scenario debt,
a delta explanation (“what this means”),
and a simple visual comparison to communicate fragility vs robustness.

Dataset Explorer

Question: “What’s actually inside the processed dataset? Can I inspect slices and export?”

You get:

dataset metrics (rows / roles / industries / unique skills),
filters,
a preview table for transparency.

How to run

1) Install

pip install -r requirements.txt

2) Launch Streamlit

streamlit run app/app.py

3) (Optional) If you have a CLI in this repo

python -m src.cli --help

Plotly config (recommended)

Streamlit is deprecating Plotly keyword args and wants a config= dict.

Use something like this once and reuse it everywhere:

PLOTLY_CONFIG = {
    "displayModeBar": True,
    "responsive": True,
    "scrollZoom": True,
    "displaylogo": False,
}

Then render like:

st.plotly_chart(fig, config=PLOTLY_CONFIG, width="stretch")

Project structure

Skill-Adaptation-Debt-Engine/
├─ app/
│  └─ app.py                 # Streamlit UI (all pages / plots)
├─ src/
│  ├─ config.py              # paths, constants, defaults (optional)
│  ├─ data.py                # load dataset, caching helpers
│  ├─ metrics.py             # churn/rarity/novelty/breadth + debt score
│  ├─ plots.py               # plotly chart builders (optional but clean)
│  └─ cli.py                 # CLI entry points (optional)
├─ data/
│  ├─ raw/                   # raw CSV from Kaggle (optional)
│  └─ processed/             # processed/scored outputs (optional)
├─ requirements.txt
├─ README.md
└─ LICENSE

Screenshots & walkthrough

1) Dataset Explorer (transparency + slicing)

What you’re seeing

The left sidebar sets the lens (weights) and the view mode.
The main panel is a transparency layer: it tells you what the app actually computed and stored.

What the top counters mean

Rows: total job postings loaded into the view (after parsing/cleaning).
Roles: distinct job titles normalized into role groups (depends on dataset).
Industries: distinct categories (AI, Blockchain, Quantum Computing).
Unique skills: distinct skills extracted after tokenization + cleanup.

Why this view matters

Pressure metrics can feel abstract. This page prevents “black-box syndrome.”
It’s where you sanity-check parsing (“Did ‘PyTorch’ become ‘Pytorch’ twice?”).
It’s where you export slices (role-filtered, industry-filtered) for reports.

2) Industry Report (where debt concentrates)

What you’re seeing

A single industry is selected (example shows AI).
Roles inside that industry are ranked by debt_score.

How to read the columns

debt_score: final weighted pressure index (the “headline”).
churn_pressure: volatility / month-to-month movement in demanded skills.
novelty_pressure: rare + churning combined (learning friction).
breadth_pressure: how many distinct skills the role stacks.
unique_skills: raw count of extracted distinct skills (useful for debugging breadth).

How to interpret rankings

Top roles aren’t necessarily “best paid” or “most important.”
They’re the roles where organizations will feel:
- hiring friction,
- training lag,
- skill mismatch,
- and role definition drift.

3) Industry Report chart (top debt roles)

What you’re seeing

A quick visual of the top adaptation-debt roles in the selected industry.
This chart is designed for communication: a manager can understand it in seconds.

What “high debt” typically means in practice

Job descriptions are unstable or overloaded.
Teams must constantly retool.
Onboarding time increases.
The role’s “definition of done” shifts frequently.

Use case

Put this plot into an internal slide deck to justify:
- upskilling budget,
- narrowing scope (“split this role into two”),
- improving documentation and internal tooling.

4) Skill Pressure Map (rarity × churn)

<img width="1005" height="475" alt="Screenshot 2025-12-20 at 17-43-45 Skill Adaptation Debt Engine" src="https://github.com/user-attachments/assets/d4608