AVM - Around View Monitoring System

An OpenCV-based 360-degree surround view system that processes fisheye camera images to create a bird's eye view for vehicle parking assistance and navigation.

🚀 Features

Multi-Camera Fisheye Processing: Supports 4 fisheye cameras (front, back, left, right)
Automatic Corner Detection: Uses advanced histogram-based algorithms for calibration board detection
Real-time Undistortion: Efficient fisheye-to-rectilinear image transformation
Perspective Transformation: Converts undistorted images to bird's eye view
Seamless Stitching: Blends multiple camera views into a single panoramic image
Vehicle Overlay: Adds vehicle model overlay for spatial reference

🛠️ Installation

Prerequisites

OpenCV 3.0+ (4.x recommended)
CMake 3.10+
GCC/Clang with C++11 support
Linux/macOS/Windows (WSL2 recommended for Windows)

Quick Installation

# Clone the repository
git clone https://github.com/xixu-me/AVM.git
cd AVM

# Make scripts executable
chmod +x scripts/*.sh

# Build the project
./scripts/build.sh

For detailed installation instructions, see INSTALLATION.md.

🚀 Quick Start

Run with Sample Data

# Execute the AVM system with provided sample images
./scripts/run.sh

Expected Output

After successful execution, you'll find the following files in the build/ directory:

stitched_result_with_su7.jpg - Final 360° panoramic view
bird_*.jpg - Individual bird's eye view images
*_undis.jpg - Undistorted fisheye images

📸 Input Images

The AVM system processes four fisheye camera images positioned around the vehicle:

Camera Configuration

| Camera Position | File | Description | |----------------|------|-------------| | Front | assets/images/front.png | Front bumper center, covers front area | | Back | assets/images/back.png | Rear bumper center, covers rear area | | Left | assets/images/left.png | Left side mirror, covers left side | | Right | assets/images/right.png | Right side mirror, covers right side |

Sample Input Images

</div>

Input Image Specifications

Resolution: 1280×960 pixels
Format: PNG/JPG
Camera Type: Fisheye lens with wide FOV
Calibration: 2×4 rectangular grid pattern visible in each image
Bit Depth: 8-bit RGB

Calibration Board Requirements

Each input image must contain a 2×4 rectangular calibration pattern:

Grid Size: 2 rows × 4 columns (8 corners total)
Visibility: Pattern must be clearly visible in overlapping areas
Contrast: High contrast between pattern and ground
Position: Within the valid detection region (20%-70% of image height)

🎯 Output Images

The AVM system generates multiple intermediate and final output images:

Processing Pipeline Outputs

| Stage | Description | Example | |-------|-------------|---------| | 1. Undistorted Images | Fisheye correction applied | | | 2. Bird's Eye View | Perspective transformation | | | 3. Final Stitched Result | Complete 360° view | Final Result |

</div>

Final Output Visualization

The final stitched image provides a complete 360-degree around view:

<div align="center"> <img src="build/stitched_result_with_su7.jpg" alt="Final AVM Output" width="800"/> <p><em>Complete 360° Around View Monitor output with vehicle overlay</em></p> </div>

Output Image Details

1. Undistorted Images (`*_undis.jpg`)

Purpose: Corrected fisheye distortion
Resolution: 1280×960 pixels
Features: Corner points marked, distortion removed

2. Bird's Eye View Images (`bird_*.jpg`)

Front/Back Views: 792×305 pixels
Left/Right Views: 1131×281 pixels
Perspective: Top-down view transformation
Rotation: Automatically corrected for proper orientation

3. Debug Images

Contrast Enhanced (*_img_contrast.jpg): Gamma-corrected for corner detection
Thresholded (*_img_thresh.jpg): Binary images showing detected features
Corner Detection (*_undis_1.jpg): Undistorted images with detected corners marked

4. Final Stitched Image (`stitched_result_with_su7.jpg`)

Composition: All four camera views seamlessly blended
Vehicle Overlay: Su7 vehicle model positioned at center
Dimensions: Variable based on camera layout
Blending: Smooth transitions using mask images

🏗️ System Architecture

Processing Pipeline

[Fisheye Images] → [Undistortion] → [Corner Detection] → [Bird's Eye Transform] → [Image Stitching] → [Final Output]

Key Components

Undistortion Engine: Converts fisheye to rectilinear projection
Corner Detector: Histogram-based calibration pattern detection
Perspective Transform: Homography-based bird's eye view generation
Image Stitcher: Mask-based seamless image blending

Camera Model

The system uses a polynomial fisheye distortion model:

θ_distorted = θ_undistorted + k₁θ³ + k₂θ⁵ + k₃θ⁷ + k₄θ⁹

Default parameters optimized for automotive fisheye cameras.

⚙️ Configuration

Camera Parameters

Key parameters that can be adjusted in src/avm.cpp:

// Camera intrinsics
float m_focal_length = 910.0;      // Camera focal length (pixels)
float m_fish_scale = 0.5;          // Fisheye scaling factor
float m_undis_scale = 1.55;        // Undistortion scaling

// Distortion coefficients
cv::Vec4d distortion_coeffs = {
    -0.05611147,  // k1
    -0.05377447,  // k2
     0.0115717,   // k3
     0.0030788    // k4
};

Layout Configuration

#define IMAGE_BACK_PIXEL_Y 643   // Back image Y offset
#define IMAGE_RIGHT_PIXEL_X 398  // Right image X offset

Using Custom Images

Replace Input Images: Place your fisheye images in assets/images/
Name Convention: Use front.png, back.png, left.png, right.png
Calibration Pattern: Ensure 2×4 grid is visible in overlapping areas
Adjust Parameters: Modify camera parameters if using different lenses

🔧 Troubleshooting

Common Issues

Build Errors

# OpenCV not found
sudo apt-get install libopencv-dev libopencv-contrib-dev

# CMake version too old
wget https://cmake.org/files/v3.20/cmake-3.20.0-linux-x86_64.sh

Runtime Issues

# Missing input images
cp your_images/* assets/images/

# Permission denied
chmod +x scripts/*.sh

Corner Detection Failures

Ensure calibration board has high contrast
Check that pattern is within valid detection region
Verify image quality and lighting conditions

Debug Mode

Enable debug output by modifying the source code:

#define DEBUG_MODE 1  // Add this line for debug output

📊 Performance

Typical Processing Times

Image Loading: ~10ms per image
Undistortion: ~50ms per image
Corner Detection: ~100-200ms per image
Stitching: ~100ms total
Overall: ~1-2 seconds for complete pipeline

Memory Usage

Peak Memory: ~30-40MB
Input Images: ~5MB total
Intermediate Results: ~20MB
Output Images: ~2-3MB

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

OpenCV community for excellent computer vision library
Contributors to fisheye camera calibration algorithms
Automotive industry standards for AVM system requirements

📞 Support

Documentation: Check docs/ directory for detailed guides

<div align="center"> <p>Made with ❤️ for safer driving</p> <p>⭐ Star this repo if you find it helpful!</p> </div>

Avm

Install / Use

README