exploratory-data-analysis
Perform comprehensive exploratory data analysis on scientific data files across 200+ file formats. This skill should be used when analyzing any scientific data file to understand its structure, content, quality, and characteristics.
Install / Use
npx skills add zLanqing/codex-claude-academic-skills --skill exploratory-data-analysisInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
AutomationSupported Platforms
Tags
Our assessment of exploratory-data-analysis
exploratory-data-analysis scores 91/100 on our quality scale, 950th of 2,780 Automation skills we index (top 35%).
Its SKILL.md is 14 KB long, well organised into 86 sections with 8 code examples: a thorough specification that gives an agent plenty to work with.
With 4,259 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 6 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
- Our last check on 2026-10-28 found the source still online.
- It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 94/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit β read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-01. Automated pattern scan on 2026-09-30. It catches known dangerous patterns, not every risk β read a skill before letting an agent act on it.
exploratory-data-analysis compared with similar skills
All 4 of these similar skills score higher than exploratory-data-analysis; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| exploratory-data-analysis (this skill)by zLanqing | 91 | 4.3k | 6mo ago | SKILL.md |
| Agent-Reachby Panniantong | 100 | 112.3k | 9d ago | CLAUDE.md |
| rufloby ruvnet | 100 | 81.9k | today | CLAUDE.md |
| Claude-Code-Game-Studiosby Donchitos | 100 | 39.7k | 7d ago | CLAUDE.md |
| Scraplingby D4Vinci | 100 | 103.6k | 2d ago | MCP Server |
Frequently asked questions
- How do I install exploratory-data-analysis?
- Run
npx skills add zLanqing/codex-claude-academic-skills --skill exploratory-data-analysis. The install tabs above show the steps for each supported agent. - Which AI agents does exploratory-data-analysis work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is exploratory-data-analysis safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 94/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is exploratory-data-analysis still maintained?
- The repository was last updated about 6 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
Skill content
View source on GitHubname: exploratory-data-analysis description: Perform comprehensive exploratory data analysis on scientific data files across 200+ file formats. This skill should be used when analyzing any scientific data file to understand its structure, content, quality, and characteristics. Automatically detects file type and generates detailed markdown reports with format-specific analysis, quality metrics, and downstream analysis recommendations. Covers chemistry, bioinformatics, microscopy, spectroscopy, proteomics, metabolomics, and general scientific data formats. license: MIT license metadata: skill-author: K-Dense Inc.
Exploratory Data Analysis
Overview
Perform comprehensive exploratory data analysis (EDA) on scientific data files across multiple domains. This skill provides automated file type detection, format-specific analysis, data quality assessment, and generates detailed markdown reports suitable for documentation and downstream analysis planning.
Key Capabilities:
- Automatic detection and analysis of 200+ scientific file formats
- Comprehensive format-specific metadata extraction
- Data quality and integrity assessment
- Statistical summaries and distributions
- Visualization recommendations
- Downstream analysis suggestions
- Markdown report generation
When to Use This Skill
Use this skill when:
- User provides a path to a scientific data file for analysis
- User asks to "explore", "analyze", or "summarize" a data file
- User wants to understand the structure and content of scientific data
- User needs a comprehensive report of a dataset before analysis
- User wants to assess data quality or completeness
- User asks what type of analysis is appropriate for a file
Supported File Categories
The skill has comprehensive coverage of scientific file formats organized into six major categories:
1. Chemistry and Molecular Formats (60+ extensions)
Structure files, computational chemistry outputs, molecular dynamics trajectories, and chemical databases.
File types include: .pdb, .cif, .mol, .mol2, .sdf, .xyz, .smi, .gro, .log, .fchk, .cube, .dcd, .xtc, .trr, .prmtop, .psf, and more.
Reference file: references/chemistry_molecular_formats.md
2. Bioinformatics and Genomics Formats (50+ extensions)
Sequence data, alignments, annotations, variants, and expression data.
File types include: .fasta, .fastq, .sam, .bam, .vcf, .bed, .gff, .gtf, .bigwig, .h5ad, .loom, .counts, .mtx, and more.
Reference file: references/bioinformatics_genomics_formats.md
3. Microscopy and Imaging Formats (45+ extensions)
Microscopy images, medical imaging, whole slide imaging, and electron microscopy.
File types include: .tif, .nd2, .lif, .czi, .ims, .dcm, .nii, .mrc, .dm3, .vsi, .svs, .ome.tiff, and more.
Reference file: references/microscopy_imaging_formats.md
4. Spectroscopy and Analytical Chemistry Formats (35+ extensions)
NMR, mass spectrometry, IR/Raman, UV-Vis, X-ray, chromatography, and other analytical techniques.
File types include: .fid, .mzML, .mzXML, .raw, .mgf, .spc, .jdx, .xy, .cif (crystallography), .wdf, and more.
Reference file: references/spectroscopy_analytical_formats.md
5. Proteomics and Metabolomics Formats (30+ extensions)
Mass spec proteomics, metabolomics, lipidomics, and multi-omics data.
File types include: .mzML, .pepXML, .protXML, .mzid, .mzTab, .sky, .mgf, .msp, .h5ad, and more.
Reference file: references/proteomics_metabolomics_formats.md
6. General Scientific Data Formats (30+ extensions)
Arrays, tables, hierarchical data, compressed archives, and common scientific formats.
File types include: .npy, .npz, .csv, .xlsx, .json, .hdf5, .zarr, .parquet, .mat, .fits, .nc, .xml, and more.
Reference file: references/general_scientific_formats.md
Workflow
Step 1: File Type Detection
When a user provides a file path, first identify the file type:
- Extract the file extension
- Look up the extension in the appropriate reference file
- Identify the file category and format description
- Load format-specific information
Example:
User: "Analyze data.fastq"
β Extension: .fastq
β Category: bioinformatics_genomics
β Format: FASTQ Format (sequence data with quality scores)
β Reference: references/bioinformatics_genomics_formats.md
Step 2: Load Format-Specific Information
Based on the file type, read the corresponding reference file to understand:
- Typical Data: What kind of data this format contains
- Use Cases: Common applications for this format
- Python Libraries: How to read the file in Python
- EDA Approach: What analyses are appropriate for this data type
Search the reference file for the specific extension (e.g., search for "### .fastq" in bioinformatics_genomics_formats.md).
Step 3: Perform Data Analysis
Use the scripts/eda_analyzer.py script OR implement custom analysis:
Option A: Use the analyzer script
# The script automatically:
# 1. Detects file type
# 2. Loads reference information
# 3. Performs format-specific analysis
# 4. Generates markdown report
python scripts/eda_analyzer.py <filepath> [output.md]
Option B: Custom analysis in the conversation Based on the format information from the reference file, perform appropriate analysis:
For tabular data (CSV, TSV, Excel):
- Load with pandas
- Check dimensions, data types
- Analyze missing values
- Calculate summary statistics
- Identify outliers
- Check for duplicates
For sequence data (FASTA, FASTQ):
- Count sequences
- Analyze length distributions
- Calculate GC content
- Assess quality scores (FASTQ)
For images (TIFF, ND2, CZI):
- Check dimensions (X, Y, Z, C, T)
- Analyze bit depth and value range
- Extract metadata (channels, timestamps, spatial calibration)
- Calculate intensity statistics
For arrays (NPY, HDF5):
- Check shape and dimensions
- Analyze data type
- Calculate statistical summaries
- Check for missing/invalid values
Step 4: Generate Comprehensive Report
Create a markdown report with the following sections:
Required Sections:
-
Title and Metadata
- Filename and timestamp
- File size and location
-
Basic Information
- File properties
- Format identification
-
File Type Details
- Format description from reference
- Typical data content
- Common use cases
- Python libraries for reading
-
Data Analysis
- Structure and dimensions
- Statistical summaries
- Quality assessment
- Data characteristics
-
Key Findings
- Notable patterns
- Potential issues
- Quality metrics
-
Recommendations
- Preprocessing steps
- Appropriate analyses
- Tools and methods
- Visualization approaches
Template Location
Use assets/report_template.md as a guide for report structure.
Step 5: Save Report
Save the markdown report with a descriptive filename:
- Pattern:
{original_filename}_eda_report.md - Example:
experiment_data.fastqβexperiment_data_eda_report.md
Detailed Format References
Each reference file contains comprehensive information for dozens of file types. To find information about a specific format:
- Identify the category from the extension
- Read the appropriate reference file
- Search for the section heading matching the extension (e.g., "### .pdb")
- Extract the format information
Reference File Structure
Each format entry includes:
- Description: What the format is
- Typical Data: What it contains
- Use Cases: Common applications
- Python Libraries: How to read it (with code examples)
- EDA Approach: Specific analyses to perform
Example lookup:
### .pdb - Protein Data Bank
**Description:** Standard format for 3D structures of biological macromolecules
**Typical Data:** Atomic coordinates, residue information, secondary structure
**Use Cases:** Protein structure analysis, molecular visualization, docking
**Python Libraries:**
- `Biopython`: `Bio.PDB`
- `MDAnalysis`: `MDAnalysis.Universe('file.pdb')`
**EDA Approach:**
- Structure validation (bond lengths, angles)
- B-factor distribution
- Missing residues detection
- Ramachandran plots
Best Practices
Reading Reference Files
Reference files are large (10,000+ words each). To efficiently use them:
-
Search by extension: Use grep to find the specific format
import re with open('references/chemistry_molecular_formats.md', 'r') as f: content = f.read() pattern = r'### \.pdb[^#]*?(?=###|\Z)' match = re.search(pattern, content, re.IGNORECASE | re.DOTALL) -
Extract relevant sections: Don't load entire reference files into context unnecessarily
-
Cache format info: If analyzing multiple files of the same type, reuse the format information
Data Analysis
- Sample large files: For files with millions of records, analyze a representative sample
- Handle errors gracefully: Many scientific formats require specific libraries; provide clear installation instructions
- Validate metadata: Cross-check metadata consistency (e.g., stated dimensions vs actual data)
- Consider data provenance: Note instrument, software versions, processing steps
Report Generation
- Be comprehensive: Include all relevant information for downstream analysis
- Be specific: Provide concrete recommendations based on the file type
- Be actionable: Suggest specific next steps and tools
- Include code examples: Show how to load and work with the data
Examples
Example 1: Analyzing a FASTQ file
# User provides: "Analyze reads.fastq"
# 1. Detect file type
extension = '.fastq'
category = 'bioinformatics_genomics'
# 2. Read reference info
# Search references/bioinformatics_genomics_formats.md for "### .fastq"
# 3. Perform analysis
from Bio import SeqIO
sequences = list(SeqIO.parse('reads.fastq', 'fastq'))
# Calculate: read count, length distribution, quality scores, GC content
# 4. Generate report
# Include: format description, analysis results, QC recommendations
# 5. Save as: reads_eda_report.md
Example 2: Analyzing a CSV dataset
# User provides: "Explore experiment_results.csv"
# 1. Detect: .csv β general_scientific
# 2. Load reference for CSV format
# 3. Analyze
import pandas as pd
df = pd.read_csv('experiment_results.csv')
# Dimensions, dtypes, missing values, statistics, correlations
# 4. Generate report with:
# - Data structure
# - Missing value patterns
# - Statistical summaries
# - Correlation matrix
# - Outlier detection results
# 5. Save report
Example 3: Analyzing microscopy data
# User provides: "Analyze cells.nd2"
# 1. Detect: .nd2 β microscopy_imaging (Nikon format)
# 2. Read reference for ND2 format
# Learn: multi-dimensional (XYZCT), requires nd2reader
# 3. Analyze
from nd2reader import ND2Reader
with ND2Reader('cells.nd2') as images:
# Extract: dimensions, channels, timepoints, metadata
# Calculate: intensity statistics, frame info
# 4. Generate report with:
# - Image dimensions (XY, Z-stacks, time, channels)
# - Channel wavelengths
# - Pixel size and calibration
# - Recommendations for image analysis
# 5. Save report
Troubleshooting
Missing Libraries
Many scientific formats require specialized libraries:
Problem: Import error when trying to read a file
Solution: Provide clear installation instructions
try:
from Bio import SeqIO
except ImportError:
print("Install Biopython: uv pip install biopython")
Common requirements by category:
- Bioinformatics:
biopython,pysam,pyBigWig - Chemistry:
rdkit,mdanalysis,cclib - Microscopy:
tifffile,nd2reader,aicsimageio,pydicom - Spectroscopy:
Truncated for display β read the full file on GitHub.
Related Skills
Agent-Reach
112.3kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu β one CLI, zero API fees.
ruflo
81.9kπ The original agent harness. Deploy intelligent multi-player swarms, coordinate autonomous workflows, and build conversational AI systems. Features adaptive memory, self-learning intelligence, federation, vector RAG integration, and native Claude Code / Codex / Hermes and many more Integrated
Claude-Code-Game-Studios
39.7kTurn Claude Code into a full game dev studio β 49 AI agents, 72 workflow skills, and a complete coordination system mirroring real studio hierarchy.
Scrapling
103.6kπ·οΈ The BEST Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit β see the Safety scan above for what the skill file itself contains.
