spark-architect
Expert guidance for developing PySpark jobs in the flights-analytics-prod project
Install / Use
npx skills add BygraveRyan/gcp-flights-analytics --skill spark-architectInstalls into whichever agent you are using.
Gemini Rules
Gemini CLI config
Quality Score
Category
Data & AnalyticsSupported Platforms
Tags
Our assessment of spark-architect
spark-architect scores 48/100 on our quality scale, 574th of 587 Data & Analytics skills we index.
Its Gemini Rules is 1.5 KB long, split into 6 sections and no code examples: moderately detailed.
It has no GitHub stars yet, so there is no community track record; judge it on its content.
Maintenance, license and trust
- We could not determine when the repository was last updated.
- Our last check on 2026-10-18 found the source still online.
- No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
- Its trust signals score 68/100, with 3 cautions from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-04. Automated pattern scan on 2026-10-04. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
spark-architect compared with similar skills
All 4 of these similar skills score higher than spark-architect; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| spark-architect (this skill)by BygraveRyan | 48 | 0 | — | Gemini Rules |
| 360-feedback-systemby sickn33 | 100 | 47.9k | 5d ago | SKILL.md |
| access-matrixby sickn33 | 100 | 47.9k | 5d ago | SKILL.md |
| accounting-audit-system-builderby sickn33 | 100 | 47.9k | 5d ago | SKILL.md |
| ad-campaign-analyzerby sickn33 | 100 | 47.9k | 5d ago | SKILL.md |
Frequently asked questions
- How do I install spark-architect?
- Run
npx skills add BygraveRyan/gcp-flights-analytics. The install tabs above show the steps for each supported agent. - Which AI agents does spark-architect work with?
- It is written for Gemini CLI, as a Gemini Rules file. Other agents that read the same format can often use it too.
- Is spark-architect safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 68/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is spark-architect still maintained?
- We could not determine when the repository was last updated.
Skill content
View source on GitHubname: spark-architect description: Expert guidance for developing PySpark jobs in the flights-analytics-prod project. Use when creating or modifying Bronze-to-Silver or Silver-to-Gold Spark jobs to ensure compliance with partitioning and logging standards.
Spark Architect Skill
This skill ensures all PySpark jobs follow the flights-analytics-prod standards for Dataproc optimization, partitioning, and error handling.
Core Workflows
1. Job Scaffolding
Every new job MUST use the standard get_spark_session() and logging configuration.
- Reference: Use pyspark-template.py for the basic structure.
2. Standard References
Refer to these for specific implementation details:
- Global Standards: global-standards.md (Naming, logging, constants)
- Optimization: optimization.md (Partitioning, file sizing, session configuration)
3. Data Lake Lifecycle
- Bronze -> Silver: Enforce schema (Parquet), deduplicate, and write to
gs://flights-silver-flights-analytics-prod. - Silver -> Gold: Business aggregates, enrichments, and write to
gs://flights-gold-flights-analytics-prod.
Quality Standards
- Repartitioning: Data must be repartitioned before writing to GCS to avoid small files.
- Column Naming: All columns must be
snake_case. - Error Handling: Use
try...exceptand log exceptions withexc_info=True. - Immutability: Transformation steps must create new DataFrames.
Related Skills
360-feedback-system
47.9k360 feedback register: reviewer, subject, review cycle, visibility, due date and score, as CSV, SQL, JSON Schema or Notion on request. Use for 360 reviews or peer feedback cycles.
access-matrix
47.9kAccess matrix of role-by-module permissions, with per-role scope, confidentiality level and SME tier, as CSV, SQL, JSON Schema or Notion on request. Use for access reviews.
accounting-audit-system-builder
47.9kRoutes an accounting or audit request to the right module skill, from software selection through monthly closing, asking only what is missing. Use for books of accounts or audit files.
ad-campaign-analyzer
47.9kAnalyze cross-channel campaign data, quantify uncertainty, and propose evidence-labeled budget tests without overstating causality.
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.