__          __  __  _
   / /   ____ _/ /_/ /_(_)_______
  / /   / __ `/ __/ __/ / ___/ _ \
 / /___/ /_/ / /_/ /_/ / /__/  __/
/_____/\__,_/\__/\__/_/\___/\___/

  Visual Intelligence for Databricks

     ┌──────────┐
     │ Catalog  │
     └────┬─────┘
    ┌─────┴─────┐
    │  Schema   │
    └──┬─────┬──┘
  ┌────┴──┐ ┌┴────────┐    ┌───────────┐    ┌───────────┐
  │ Table │ │  View   │<───│ Dashboard │───>│ Warehouse │
  └─┬──┬──┘ └─────────┘    └───────────┘    └─────┬─────┘
    │  │ feedsInto                ▲                 │
    │  v                         │ queries          │ runsOn
    │ ┌───────┐    ┌───────┐  ┌──┴────┐       ┌────┴──────┐
    │ │ Table │<───│  Job  │─>│Cluster│       │GenieSpace │
    │ └───────┘    └───────┘  └───────┘       └───────────┘
    │              writesTo    runsOn
    │ indexesFrom
    v                  serves          embeddedBy
  ┌──────────────┐   ┌────────────────┐   ┌───────┐
  │ VectorSearch │──>│ServingEndpoint │──>│ Model │
  └──────────────┘   └────────────────┘   └───────┘

Lattice

Ontology and visual intelligence platform for Databricks workspaces.

Lattice builds a live ontology of your Databricks environment — every Unity Catalog asset, compute resource, job, dashboard, app, serving endpoint, vector search index, Genie space, and connected system mapped as typed entities with semantic relationships, enriched with operational intelligence from system tables. Built for data teams and AI agents alike.

Created by Mike Kahn — mike.kahn@databricks.com

---

Screenshots

Main Canvas — Full Graph View

3,630 assets mapped across 23 node types with activity timeline, health panel, and type filters.

Detail Panel — Asset Intelligence

Select any node to see properties, connections, cost attribution, and impact analysis.

Settings — Catalog Scope & System Access

Configure catalog scope, scale limits, and view system access pre-flight checks.

Swimlane Layout — Grouped by Type

Swimlane layout groups UC data assets, compute resources, and apps into horizontal lanes.

Compute View — Apps, Warehouses & Clusters

Compute view shows Databricks Apps, SQL Warehouses, Serverless compute, and their relationships.

UC Tree — Catalog Hierarchy

UC Tree view shows the Catalog → Schema → Table hierarchy with heat dots and ownership.

---

Use Cases

Identifying Costs — Warehouse DBU Heatmap

Enable the cost overlay to see a heatmap of DBU spend across warehouses and compute. Darker orange = higher 30-day spend. Click any warehouse to see its cost attribution breakdown in the detail panel.

Finding Dependencies — Impact Analysis

Select any asset and click "Analyze" to see its blast radius — which schemas, apps, dashboards, and jobs depend on it. Essential before making breaking changes.

Exploring Connections — Focus View

Pull any asset out of the lane view and click "Focus" to arrange its connections — callers above, targets below. Here a single schema's 39 dependencies are isolated for analysis while the full 3,630-asset lane layout stays visible for context.

Filtering by Asset Type — Targeted Analysis

Toggle asset types in the sidebar to isolate specific categories. Here only Apps (300) and Databases (21) are active — 321 nodes out of 3,630 — revealing the "uses" relationships between deployed applications and their backing databases.

Multi-Workspace & Catalog Switcher

Switch between workspace profiles to analyze different environments (dev, staging, prod) without restarting. The catalog selector below lets you scope the graph to specific catalogs — with live search across 200+ catalogs including foreign and Delta Sharing sources.

Save View — Export PNG, JSON & CSV

Click "Save View" to freeze the current canvas into a side-by-side comparison pane. Export as high-resolution PNG (4x) for presentations, JSON for programmatic analysis, or CSV for a tabular export of all filtered assets — ready for spreadsheet analysis, stakeholder reviews, and cross-team collaboration.

Detecting Orphaned Assets — Health Panel

The Health panel surfaces orphaned tables (zero queries in 30 days) and active assets with no owner. Click any item to navigate directly to it on the canvas.

Cost Attribution — Per-Asset Spend

With cost overlay enabled, every node shows its attributed DBU spend. The detail panel breaks down cost sources — which warehouses and jobs drive spend for a given table.

Activity Timeline — Identifying Inactive Resources

Use the activity timeline filter (7d, 30d, 90d, 1y) to highlight recently active assets and dim inactive ones. A notification above the canvas confirms the filter is active. Dimmed nodes with dashed borders have had zero activity in the selected window — ideal for identifying stale tables, unused schemas, and candidates for cleanup.

Data Governance — Ownership & Compliance

Lattice provides a comprehensive governance toolkit for data architects and platform teams:

• Orphan detection — The Health panel identifies cold tables (zero queries in 30 days) and active assets with no owner, exportable to CSV for audit workflows
• Impact analysis — Select any asset and click "Analyze" to see its full blast radius — every downstream schema, table, job, and dashboard that depends on it. Essential before making breaking changes
• Activity heat classification — Every table is classified as hot (queried in 7d), warm (7–30d), or cold (30d+) based on system.query.history, with heat dots visible directly on the canvas
• Cost-aware governance — Per-asset DBU attribution traces compute spend from warehouses and jobs through lineage to the tables and schemas that drive it, helping teams prioritize optimization and decommissioning decisions

---

What It Does

• Models your workspace as a live ontology — typed entities (23 node types) with semantic relationships (16+ edge types), forming a complete platform knowledge graph
• Discovers every asset — catalogs, schemas, tables, views, models, volumes, warehouses, clusters, jobs, dashboards, apps, pipelines, Delta Shares, foreign catalogs, Lakebase databases, model serving endpoints, vector search indexes, and Genie spaces
• Connects them with structural, compute, lineage, AI, and federation edges that carry meaning (contains, runsOn, queries, feedsInto, writesTo, readsFrom, derivedFrom, serves, indexesFrom, embeddedBy)
• Enriches with system table data — DBU spend, query frequency, heat (last-accessed age), job success rates, storage size, UC tags
• Visualizes the ontology on an interactive canvas with multiple layout modes, search, filters, and drill-down
• Analyzes cost attribution, impact/blast radius, orphaned assets, and column-level lineage
• Annotates with persistent tags and notes backed by a Delta table (requires SQL warehouse + CREATE TABLE permission)
• Exports as JSON or JSON-LD (semantic web vocabulary) for downstream consumption by AI agents

---

Features

Graph & Canvas

• 23 node types: Catalog, ForeignCatalog, Schema, Table, View, Model, Volume, StreamingTable, MaterializedView, Warehouse, Serverless, Cluster, Job, Dashboard, App, Pipeline, Connection, Share, Recipient, Database, ServingEndpoint, VectorSearchIndex, GenieSpace
• 16+ edge types: contains, runsOn, queries, feedsInto, writesTo, readsFrom, derivedFrom, triggers, uses, exposes, includes, serves, indexesFrom, embeddedBy
• 3 layout modes: Tree (top-down), Tree (left-right), Swimlane (grouped by type)
• Schema collapse/expand to manage large catalogs
• Search across name, FQN, comment, owner, and UC tags
• Type filter sidebar to show/hide node categories
• Freshness filter — slider to show only assets active within N days
• Focus Neighbors — radial layout around a selected node with direct connections
• Save View — freeze canvas to a comparison pane at exact viewport/zoom
• PNG export (4x resolution) and JSON export from frozen pane
• Console URL links on every node — click to open in Databricks

Intelligence

• Heat dots on nodes: green (hot, ≤7d), amber (warm, ≤30d), gray (cold)
• DBU badges — 30-day compute spend shown inline on node tiles
• Cost overlay — DBU attribution from compute → lineage → tables, rolled up to schema and catalog
• Health panel — detects orphaned tables (cold + 0 queries in 30d) and unowned assets
• Impact analysis — BFS traversal showing "depends on this" (consumers) and "contained within" (descendants)
• Column lineage — source_table.source_col → target_col, from system.access.column_lineage
• UC tags — ingested from system.information_schema.table_tags, displayed as pills in the detail panel, searchable in the canvas search box

Lineage

• **Table → Table