Harbeth

Harbeth is a high-performance Swift library focused on GPU-accelerated real-time image processing, camera capture, and video processing. Built on Metal technology, it also integrates with CoreImage and Metal Performance Shaders, providing developers with a powerful and easy-to-integrate image processing solution.

This library is highly inspired by GPUImage.

English | 简体中文

Features

🟣 Harbeth offers a comprehensive set of features designed to make image and video processing fast, efficient, and easy to implement:

Cross-Platform Support: Runs seamlessly on iOS, macOS, tvOS, and watchOS, supporting both UIKit/AppKit and SwiftUI frameworks.
Versatile Input Sources: Apply filters to a wide range of image and video sources including MTLTexture, UIImage, NSImage, CIImage, CGImage, CMSampleBuffer, and CVPixelBuffer.
Rich Filter Ecosystem: Over 200+ built-in filters organized into intuitive categories, covering everything from basic color adjustments to advanced artistic effects.
Advanced Integration: Leverage the power of Metal Performance Shaders (MPS) for high-performance filtering, while maintaining compatibility with CoreImage filters for maximum flexibility.
Metal-Powered Rendering: All previews and rendering operations are accelerated by Metal, ensuring smooth, real-time performance even with complex filter chains.
Custom Filter Support: Easily create and integrate custom filters using LUTs has 1D Lookup Tables、2D Lookup Tables、 3D Cube Files, And Multi Zone Tables, or custom Metal shaders. Create advanced combination filters by subclassing C7CombinationBase for complex, multi-step effects.
Real-Time Processing: Achieve smooth, real-time camera capture and video playback with live filter application.
Video Processing: Seamlessly process both local and network video files using the integrated Kakapos library.
Intuitive API: Enjoy a clean, Swift-friendly API with chainable filter operations and operator overloading for concise, expressive code.
Performance Optimization: Benefit from automatic texture pooling, memory management, and multi-encoder support for optimal performance across devices.
Extensive Documentation: Comprehensive documentation and demo projects to help you get started quickly and make the most of Harbeth's capabilities.
SwiftUI integration: Native support for SwiftUI framework.

🎨 Filter System

Harbeth offers a comprehensive filter classification to meet various image processing needs:

Color Adjustment: Brightness, contrast, saturation, exposure, white balance, etc.
Blur Effects: Gaussian blur, bilateral blur, motion blur, zoom blur, etc.
Blend Modes: Normal, multiply, screen, overlay, hard light, etc.
Edge & Detail: Sharpen, edge detection, sketch, comic strip effect, etc.
Distortion & Warp: Bulge, pinch, swirl, water ripple, glass sphere, etc.
Stylization: Oil painting, cartoon, glitch effect, split screen, soul out, etc.
Geometric Transform: Crop, flip, rotate, resize, etc.
Matrix Processing: 3x3 convolution matrix, 4x4 color matrix, 4x5 color matrix, etc.
Utility: Chroma key, highlight shadow, levels, luminance threshold, etc.
Generators: Solid color, color gradient, etc.
Lookup Tables: LUT-based color adjustments and CUBE file support.
Blit Operations: Copy region, crop, generate mipmaps.
CoreImage Integration: Access to CoreImage filters
Metal Performance Shaders: High-performance MPS filters
Render Vertex Fragment: The Render module provides low-level rendering capabilities using vertex and fragment shaders.

A total of 200+ kinds of built-in filters are currently available.✌️

Requirements

| iOS Target | macOS Target | Xcode Version | Swift Version | |:---:|:---:|:---:|:---:| | iOS 10.0+ | macOS 10.13+ | Xcode 10.0+ | Swift 5.0+ |

Usage

Image

🎷 Code zero intrusion add filter function for image.

let filter1 = C7ColorMatrix4x4(matrix: Matrix4x4.Color.sepia)
let filter2 = C7Granularity(grain: 0.8)
let filter3 = C7SoulOut(soul: 0.7)

let filters = [filter1, filter2, filter3]

// Use:
let dest = HarbethIO.init(element: originImage, filters: filters)
// Synchronize do something..
ImageView.image = try? dest.output()

// OR Use:
ImageView.image = try? originImage.make(filters: filters)

// OR Use Operator:
ImageView.image = originImage ->> filter1 ->> filter2 ->> filter3

Asynchronous do something..

This performance is the best. 🚗🚗

let dest = HarbethIO.init(element: ``Source``, filter: ``filter``)

dest.transmitOutput(success: { [weak self] image in
    // do something..
})

Camera

📸 Camera capture generates pictures.

// Add an edge detection filter:
let filter = C7EdgeGlow(lineColor: .red)

// Generate camera collector:
let camera = C7CollectorCamera.init(delegate: self)
camera.captureSession.sessionPreset = AVCaptureSession.Preset.hd1280x720
camera.filters = [filter]

extension CameraViewController: C7CollectorImageDelegate {
    func preview(_ collector: C7Collector, fliter image: C7Image) {
        // do something..
    }
}

Video

📺 Local video or Network video are simply apply with filters.
- 🙄 For details, See PlayerViewController.
- You can also extend this by using HarbethIO to filter the collected CVPixelBuffer.

lazy var video: C7CollectorVideo = {
    let videoURL = URL.init(string: "Link")!
    let asset = AVURLAsset.init(url: videoURL)
    let playerItem = AVPlayerItem.init(asset: asset)
    let player = AVPlayer.init(playerItem: playerItem)
    let video = C7CollectorVideo.init(player: player, delegate: self)
    let filter = C7ColorMatrix4x4(matrix: Matrix4x4.Color.sepia)
    video.filters = [filter]
    return video
}()

self.video.play()

extension PlayerViewController: C7CollectorImageDelegate {
    func preview(_ collector: C7Collector, fliter image: C7Image) {
        // do something..
    }
}

SwiftUI Support

For the direct use HarbethView, it is just a simple implementation.
The SwiftUI API is still in-progress and may not be production ready. We're looking for help! 🤲

let filters: [C7FilterProtocol] = [
    CIHighlight(highlight: intensity),
    C7WaterRipple(ripple: intensity),
]
HarbethView(image: inputImage, filters: filters, content: { image in
    image.resizable()
        .aspectRatio(contentMode: .fit)
})

📝 HarbethIO Properties

HarbethIO provides several properties to customize the image processing behavior. Here's a detailed explanation of each property:

| Property | Description | | :--- | :--- | | element | The input element to apply filters to. Supports UIImage/NSImage, CGImage, CIImage, MTLTexture, CMSampleBuffer, and CVPixelBuffer. | | filters | An array of filters to apply to the input element. | | bufferPixelFormat | The pixel format for the output buffer. Important for camera capture which typically uses kCVPixelFormatType_32BGRA to avoid blue tint issues. | | mirrored | Whether to mirror the output image. Fixes the upside-down mirroring issue when creating CIImage from texture. | | createDestTexture | Whether to create a separate output texture. Disabling this may cause texture overlay issues. | | transmitOutputRealTimeCommit | Whether to use real-time commit for Metal texture output. Enables MTLCommandBuffer.asyncCommit for faster processing. | | enableDoubleBuffer | Whether to enable double buffer optimization for metal filters. Reduces memory usage and improves texture pool efficiency. |

Usage Examples

// Custom configuration
var dest = HarbethIO(element: image, filters: [filter1, filter2])
dest.bufferPixelFormat = .rgba8Unorm
dest.enableDoubleBuffer = true
_ = try? dest.output()

// Asynchronous processing with custom configuration
var dest = HarbethIO(element: image, filters: [filter1

Harbeth

Install / Use

README

Harbeth

Features

🎨 Filter System

A total of 200+ kinds of built-in filters are currently available.✌️

Requirements

Usage

Image

Camera

Video

SwiftUI Support

📝 HarbethIO Properties

Usage Examples