EBSight: Intelligent EBS Volume Analyzer

EBSight

Introduction

EBSight is a Python-based analysis tool developed in response to AWS's new EBS snapshot size reporting feature. This tool leverages the newly introduced FullSnapshotSizeInBytes field to provide comprehensive insights into EBS volume usage, performance metrics, and cost optimization opportunities.

Understanding EBS Snapshots

EBS snapshots are incremental in nature, meaning:

Each snapshot only stores new or modified blocks
Unchanged blocks are referenced from previous snapshots
The full snapshot size represents the total data footprint at the time of the snapshot

For example:

If you have a 100 GB volume with 50 GB of data
The 'full snapshot size' will show 50 GB
This remains true whether it's the first snapshot or a subsequent one
EBSight uses this information to accurately track data changes over time

How EBSight Helps

EBSight utilizes AWS's new snapshot size reporting to:

Track actual data usage versus allocated volume size
Monitor data change patterns over time
Calculate storage efficiency metrics
Provide cost optimization recommendations

Key Features

Volume Analysis

Tracks total snapshot sizes
Calculates daily change rates
Monitors volume usage percentages
Provides detailed IOPS and throughput metrics

Performance Metrics

P99 and Peak IOPS for read/write operations
Throughput measurements in MBps
Queue length monitoring
Historical performance data over 7-day periods

Visualization Options

ASCII-based bar graphs for size comparisons
Consolidated summary tables
Optional CSV export functionality
FSx for NetApp ONTAP sizing recommendations

Usage

Basic Command

python ebsight.py

Available Options

python ebsight.py [OPTIONS]
Options:
--verbose, -v Show detailed snapshot information
--csv, -c Export results to CSV
--graph, -g Show ASCII graphs
--profile, -p AWS profile name (default: 'default')
--fsx, -f Show FSx for NetApp ONTAP recommendations

<details> <summary>Example Output</summary>

$ python ebsight.py -p demo-profile -g -f
Enter EC2 Instance ID: i-0a1b2c3d4e5f6g7h8

🖥️  Analyzing EC2 Instance: i-0a1b2c3d4e5f6g7h8
Name: demo-database-01
================================================================================

💾 Found 7 attached volumes

📀 Analyzing Volume: vol-0a1b2c3d4e5f6g7h1 (/dev/sda1)
================================================================================
🔍 Found 7 snapshots

   Volume Summary:
   ------------------------------------------------------------
   Metric                    Value                         
   ------------------------------------------------------------
   Volume Size               150.0 GiB                     
   Snapshot Total Size       79.8 GiB                      
   Usage %                   53.2%                         
   Snapshot Count            7                             
   Daily Change Rate         13.31 GiB (8.9%)              
   Total Changed Data        79.8 GiB                      
   Daily Backup Cost         $0.13
   Monthly Backup Cost       $3.99
   Annual Backup Cost        $47.90
   ------------------------------------------------------------

   📊 Size Comparison for vol-0a1b2c3d4e5f6g7h1
   ==============================================================
   Volume Size    ██████████████████████████████████████████████████ 150.00 GiB
   Snapshot Size  ██████████████████████████░░░░░░░░░░░░░░░░░░░░░░░░ 79.83 GiB

📀 Analyzing Volume: vol-0a1b2c3d4e5f6g7h2 (/dev/sdb)
================================================================================
🔍 Found 7 snapshots

   Volume Summary:
   ------------------------------------------------------------
   Metric                    Value                         
   ------------------------------------------------------------
   Volume Size               210.0 GiB                     
   Snapshot Total Size       106.8 GiB                     
   Usage %                   50.8%                         
   Snapshot Count            7                             
   Daily Change Rate         17.79 GiB (8.5%)              
   Total Changed Data        106.8 GiB                     
   Daily Backup Cost         $0.18
   Monthly Backup Cost       $5.34
   Annual Backup Cost        $64.05
   ------------------------------------------------------------

   📊 Size Comparison for vol-0a1b2c3d4e5f6g7h2
   ==============================================================
   Volume Size    ██████████████████████████████████████████████████ 210.00 GiB
   Snapshot Size  █████████████████████████░░░░░░░░░░░░░░░░░░░░░░░░░ 106.76 GiB

📀 Analyzing Volume: vol-0a1b2c3d4e5f6g7h3 (/dev/sdc)
================================================================================
🔍 Found 7 snapshots

   Volume Summary:
   ------------------------------------------------------------
   Metric                    Value                         
   ------------------------------------------------------------
   Volume Size               2500.0 GiB                    
   Snapshot Total Size       1965.5 GiB                    
   Usage %                   78.6%                         
   Snapshot Count            7                             
   Daily Change Rate         327.58 GiB (13.1%)            
   Total Changed Data        1965.5 GiB                    
   Daily Backup Cost         $3.28
   Monthly Backup Cost       $98.28
   Annual Backup Cost        $1179.30
   ------------------------------------------------------------

   📊 Size Comparison for vol-0a1b2c3d4e5f6g7h3
   ==============================================================
   Volume Size    ██████████████████████████████████████████████████ 2500.00 GiB
   Snapshot Size  ███████████████████████████████████████░░░░░░░░░░░ 1965.51 GiB

📀 Analyzing Volume: vol-0a1b2c3d4e5f6g7h4 (/dev/sdd)
================================================================================
🔍 Found 7 snapshots

   Volume Summary:
   ------------------------------------------------------------
   Metric                    Value                         
   ------------------------------------------------------------
   Volume Size               201.0 GiB                     
   Snapshot Total Size       27.5 GiB                      
   Usage %                   13.7%                         
   Snapshot Count            7                             
   Daily Change Rate         4.58 GiB (2.3%)               
   Total Changed Data        27.5 GiB                      
   Daily Backup Cost         $0.05
   Monthly Backup Cost       $1.37
   Annual Backup Cost        $16.50
   ------------------------------------------------------------

   📊 Size Comparison for vol-0a1b2c3d4e5f6g7h4
   ==============================================================
   Volume Size    ██████████████████████████████████████████████████ 201.00 GiB
   Snapshot Size  ██████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 27.50 GiB

📀 Analyzing Volume: vol-0a1b2c3d4e5f6g7h5 (/dev/sde)
================================================================================
🔍 Found 7 snapshots

   Volume Summary:
   ------------------------------------------------------------
   Metric                    Value                         
   ------------------------------------------------------------
   Volume Size               50.0 GiB                      
   Snapshot Total Size       19.3 GiB                      
   Usage %                   38.6%                         
   Snapshot Count            7                             
   Daily Change Rate         3.22 GiB (6.4%)               
   Total Changed Data        19.3 GiB                      
   Daily Backup Cost         $0.03
   Monthly Backup Cost       $0.97
   Annual Backup Cost        $11.59
   ------------------------------------------------------------

   📊 Size Comparison for vol-0a1b2c3d4e5f6g7h5
   ==============================================================
   Volume Size    ██████████████████████████████████████████████████ 50.00 GiB
   Snapshot Size  ███████████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 19.31 GiB

📀 Analyzing Volume: vol-0a1b2c3d4e5f6g7h6 (/dev/sdf)
================================================================================
🔍 Found 7 snapshots

   Volume Summary:
   ------------------------------------------------------------
   Metric                    Value                         
   ------------------------------------------------------------
   Volume Size               203.0 GiB                     
   Snapshot Total Size       53.0 GiB                      
   Usage %                   26.1%                         
   Snapshot Count            7                             
   Daily Change Rate         8.83 GiB (4.4%)               
   Total Changed Data        53.0 GiB                      
   Daily Backup Cost         $0.09
   Monthly Backup Cost       $2.65
   Annual Backup Cost        $31.80
   ------------------------------------------------------------

   📊 Size Comparison for vol-0a1b2c3d4e5f6g7h6
   ==============================================================
   Volume Size    ██████████████████████████████████████████████████ 203.00 GiB
   Snapshot Size  █████████████░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 53.00 GiB

📀 Analyzing Volume: vol-0a1b2c3d4e5f6g7h7 (/dev/sdg)
================================================================================
🔍 Found 7 snapshots

   Volume Summary:
   ------------------------------------------------------------
   Metric                    Value                         
   -----------------------------------------------

Ebsight

Install / Use

README