vision-framework
Implement computer vision features including text recognition (OCR), face detection, barcode scanning, image segmentation, object tracking, and document scanning in iOS apps.
Install / Use
npx skills add dpearson2699/swift-ios-skills --skill vision-frameworkInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
AI & Machine LearningSupported Platforms
Our assessment of vision-framework
vision-framework scores 89/100 on our quality scale, 421st of 947 AI & Machine Learning skills we index (top 45%).
Its SKILL.md is 18 KB long, well organised into 27 sections with 13 code examples: a thorough specification that gives an agent plenty to work with.
With 1,143 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 4 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
- Our last check on 2026-10-31 found the source still online.
- No license is declared. By default that means all rights are reserved: you can read it, but reusing or redistributing it is not clearly permitted. Ask the author before building on it commercially.
- Its trust signals score 86/100, with 1 caution from licensing, adoption, age or documentation. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-10-04. Automated pattern scan on 2026-10-04. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
vision-framework compared with similar skills
All 4 of these similar skills score higher than vision-framework; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| vision-framework (this skill)by dpearson2699 | 89 | 1.1k | 4mo ago | SKILL.md |
| claude-memby thedotmack | 100 | 117.6k | today | CLAUDE.md |
| taste-skillby Leonxlnx | 100 | 116.8k | 8d ago | CLAUDE.md |
| Agent-Reachby Panniantong | 100 | 112.3k | 8d ago | CLAUDE.md |
| Understand-Anythingby Egonex-AI | 100 | 92.8k | 1d ago | CLAUDE.md |
Frequently asked questions
- How do I install vision-framework?
- Run
npx skills add dpearson2699/swift-ios-skills --skill vision-framework. The install tabs above show the steps for each supported agent. - Which AI agents does vision-framework work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is vision-framework safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It declares no license and scores 86/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is vision-framework still maintained?
- The repository was last updated about 4 months ago. That is recent enough to be usable, but agent tooling moves fast, so check the instructions against your agent's current version.
Skill content
View source on GitHubname: vision-framework description: "Implement computer vision features including text recognition (OCR), face detection, barcode scanning, image segmentation, object tracking, and document scanning in iOS apps. Covers both the modern Swift-native Vision API (iOS 18+) and legacy VNRequest patterns, VisionKit DataScannerViewController for live camera scanning, and CoreMLRequest/VNCoreMLRequest for custom model inference. Use when adding OCR, barcode scanning, face detection, or custom Core ML model inference with Vision."
Vision Framework
Detect text, faces, barcodes, objects, and body poses in images and video using on-device computer vision. Prefer the modern iOS 18+ request APIs and load the legacy reference only when the deployment target requires it.
See references/vision-requests.md for complete code patterns and references/visionkit-scanner.md for DataScannerViewController integration.
Contents
- Two API Generations
- Request Pattern (Modern API)
- Text Recognition (OCR)
- Face Detection
- Barcode Detection
- Document Scanning (iOS 26+)
- Image Segmentation
- Object Tracking
- Other Request Types
- Core ML Integration
- VisionKit: DataScannerViewController
- Common Mistakes
- Review Checklist
- References
Two API Generations
Vision has two distinct API layers. Prefer the modern API for new code:
Swift-native request types plus try await request.perform(on:). Keep VN*,
VNImageRequestHandler, VNSequenceRequestHandler, completion handlers, and
legacy CGRect helpers inside explicit legacy fallback sections or files.
| Aspect | Modern (iOS 18+) | Legacy |
|---|---|---|
| Pattern | let result = try await request.perform(on: image) | VNImageRequestHandler + completion handler |
| Request types | Swift types — structs and classes (RecognizeTextRequest, DetectFaceRectanglesRequest) | ObjC classes (VNRecognizeTextRequest, VNDetectFaceRectanglesRequest) |
| Concurrency | Native async/await | Completion handlers or synchronous perform |
| Observations | Typed return values | Cast results from [Any] |
| Availability | iOS 18+ / macOS 15+ | iOS 11+ |
The modern API uses the ImageProcessingRequest protocol. Each request type
has a perform(on:orientation:) method that accepts CGImage, CIImage,
CVPixelBuffer, CMSampleBuffer, Data, or URL. Most requests are
structs; stateful requests such as GeneratePersonSegmentationRequest,
TrackObjectRequest, TrackRectangleRequest, and DetectTrajectoriesRequest
are final classes.
Request Pattern (Modern API)
All modern Vision requests follow the same pattern: create a request, call
perform(on:), and handle the typed result.
import Vision
func recognizeText(in image: CGImage) async throws -> [String] {
var request = RecognizeTextRequest()
request.recognitionLevel = .accurate
request.recognitionLanguages = [Locale.Language(identifier: "en-US")]
let observations = try await request.perform(on: image)
return observations.compactMap { observation in
observation.topCandidates(1).first?.string
}
}
Legacy Pattern (Pre-iOS 18)
For pre-iOS 18 targets, use the corresponding VNRequest with VNImageRequestHandler or VNSequenceRequestHandler. Load references/vision-requests.md for complete legacy request and handler patterns.
Text Recognition (OCR)
Modern: RecognizeTextRequest (iOS 18+)
var request = RecognizeTextRequest()
request.recognitionLevel = .accurate // .fast for real-time
request.recognitionLanguages = [
Locale.Language(identifier: "en-US"),
Locale.Language(identifier: "fr-FR"),
]
request.usesLanguageCorrection = true
request.customWords = ["SwiftUI", "Xcode"] // domain-specific terms
let observations = try await request.perform(on: cgImage)
for observation in observations {
guard let candidate = observation.topCandidates(1).first else { continue }
let text = candidate.string
let confidence = candidate.confidence // 0.0 ... 1.0
let bounds = observation.boundingBox // NormalizedRect
}
Legacy: VNRecognizeTextRequest
The legacy request uses string language identifiers and the handler pattern in the reference; both generations support accurate and fast recognition levels.
Face Detection
Detect face rectangles, landmarks (eyes, nose, mouth), and capture quality.
// Modern API
let faceRequest = DetectFaceRectanglesRequest()
let faces = try await faceRequest.perform(on: cgImage)
for face in faces {
let boundingBox = face.boundingBox // NormalizedRect
let roll = face.roll // Measurement<UnitAngle>
let yaw = face.yaw // Measurement<UnitAngle>
}
// Landmarks (eyes, nose, mouth contours)
var landmarkRequest = DetectFaceLandmarksRequest()
let landmarkFaces = try await landmarkRequest.perform(on: cgImage)
for face in landmarkFaces {
let landmarks = face.landmarks
let leftEye = landmarks?.leftEye.points
let nose = landmarks?.nose.points
}
Coordinate System
Vision uses a normalized coordinate system with origin at the bottom-left. Convert to UIKit (top-left origin) before display:
import Vision
func imageRectForDisplay(_ rect: NormalizedRect, imageSize: CGSize) -> CGRect {
rect.toImageCoordinates(imageSize, origin: .upperLeft)
}
Barcode Detection
Detect 1D and 2D barcodes including QR codes.
var request = DetectBarcodesRequest()
let symbologies: [BarcodeSymbology] = [.qr, .ean13, .code128, .pdf417]
request.symbologies = symbologies
let barcodes = try await request.perform(on: cgImage)
for barcode in barcodes {
let payload = barcode.payloadString // decoded content
let symbology = barcode.symbology // .qr, .ean13, etc.
let bounds = barcode.boundingBox // NormalizedRect
}
Type annotate local values first, then assign request properties separately.
Document Scanning (iOS 26+)
RecognizeDocumentsRequest provides structured document reading with layout
understanding beyond basic OCR. Returns DocumentObservation objects with a
nested Container structure for paragraphs, tables, lists, and barcodes.
Currently, Vision returns one document observation for each image.
var request = RecognizeDocumentsRequest()
let documents = try await request.perform(on: cgImage)
for observation in documents {
let container = observation.document
// Full text content
let fullText = container.text
// Structured access to paragraphs
for paragraph in container.paragraphs {
let paragraphText = paragraph.text
}
// Tables and lists
for table in container.tables { /* structured table data */ }
for list in container.lists { /* structured list data */ }
// Embedded barcodes detected within the document
for barcode in container.barcodes { /* barcode data */ }
// Document title if detected
if let title = container.title { print(title) }
}
For simpler document camera scanning, use VisionKit's
VNDocumentCameraViewController which provides a full-screen camera UI with
auto-capture, perspective correction, and multi-page scanning.
Image Segmentation
Modern: GeneratePersonSegmentationRequest (iOS 18+)
var request = GeneratePersonSegmentationRequest()
request.qualityLevel = .accurate // .balanced, .fast
let mask = try await request.perform(on: cgImage)
// mask is a PixelBufferObservation with a pixelBuffer property
let maskBuffer = mask.pixelBuffer
// Apply mask using Core Image: CIFilter.blendWithMask()
Legacy: VNGeneratePersonSegmentationRequest
For older targets, VNGeneratePersonSegmentationRequest exposes its mask through the first pixel-buffer observation; use the reference's handler and mask-composition recipe.
Quality levels:
.accurate-- best quality, slowest (~1s), full resolution.balanced-- good quality, moderate speed (~100ms), 960x540.fast-- lowest quality, fastest (~10ms), 256x144, suitable for real-time
Instance Segmentation (iOS 18+)
Separate masks per person for individual effects.
// Modern API (iOS 18+)
let request = GeneratePersonInstanceMaskRequest()
let observation = try await request.perform(on: cgImage)
let indices = observation.allInstances
for index in indices {
let mask = try observation.generateMask(for: IndexSet(integer: index))
// mask is a CVPixelBuffer with only this person visible
}
// Legacy API (iOS 17+)
let request = VNGeneratePersonInstanceMaskRequest()
let handler = VNImageRequestHandler(cgImage: cgImage)
try handler.perform([request])
guard let result = request.results?.first else { return }
let indices = result.allInstances
for index in indices {
let instanceMask = try result.generateMaskedImage(
ofInstances: IndexSet(integer: index),
from: handler,
croppedToInstancesExtent: false
)
}
See references/vision-requests.md for mask composition and Core Image filter integration patterns.
Object Tracking
Modern: TrackObjectRequest (iOS 18+)
TrackObjectRequest is a stateful request that maintains tracking context
across frames.
// Initialize with a detected object's bounding box
let initialObservation = DetectedObjectObservation(boundingBox: detectedBox)
let request = TrackObjectRequest(detectedObject: initialObservation)
for pixelBuffer in framePixelBuffers {
let results = try await request.perform(on: pixelBuffer)
if let tracked = results.first {
let updatedBounds = tracked.boundingBox // NormalizedRect
}
}
Modern TrackObjectRequest has no trackingLevel or qualityLevel.
Legacy: VNTrackObjectRequest
For older targets, use VNTrackObjectRequest with one retained VNSequenceRequestHandler and feed each result back as the next input observation. The reference contains the complete loop.
Other Request Types
Vision provides additional requests covered in references/vision-requests.md:
| Request | Purpose |
|---|---|
| ClassifyImageRequest | Classify scene content (outdoor, food, animal, etc.) |
| GenerateAttentionBasedSaliencyImageRequest | Single SaliencyImageObservation for where viewers focus attention |
| GenerateObjectnessBasedSaliencyImageRequest | Single SaliencyImageObservation for object-like regions |
| GenerateForegroundInstanceMaskRequest | Foreground object segmentation (not person-specific) |
| DetectRectanglesRequest | Detect rectangular shapes (documents, cards, screens) |
| DetectHorizonRequest | Detect horizon angle for auto-leveling photos |
| DetectHumanBodyPoseRequest | Detect body joints (shoulders, elbows, knees) |
| DetectHumanBodyPose3DRequest | 3D human body pose estimation |
| DetectHumanHandPoseRequest | Detect hand joints and finger positions |
| DetectAnimalBodyPoseRequest | Detect animal body joint positions |
| DetectFaceCaptureQualityRequest | Face capture quality scoring (0–1) for photo selection |
| TrackRectangleRequest | Track rectangular objects across video frames |
| TrackOpticalFlowRequest | Optical flow between video frames |
| DetectTrajectoriesRequest | Detect object trajectories in video |
All modern request types above are iOS 18+ / macOS 15+.
Core ML Integration
Run custom Core ML models through Vision for automatic image preprocessing.
Vision runs already-prepared models with CoreMLRequest or VNCoreMLRequest;
hand conversion, profiling, packaging,
Truncated for display — read the full file on GitHub.
Related Skills
claude-mem
117.6kPersistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + More
taste-skill
116.8kAgent skills that give coding agents taste: art direction, reference images and a browser check instead of generic AI pages
Agent-Reach
112.3kGive your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.
Understand-Anything
92.8kGraphs that teach > graphs that impress. Turn any code into an interactive knowledge graph you can explore, search, and ask questions about. Works with Claude Code, Codex, Cursor, Copilot, Gemini CLI, and more.
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
