CatPan

A small, native panorama stitcher for iOS (and macOS, for testing). No third-party dependencies — Vision, Core Graphics, ImageIO, and CoreMotion.

Capture-agnostic: your app owns the camera session and hands over frames. CatPan paces the sweep (SweepGuide / MotionSweepTracker), then aligns, projects, blends, and crops the frames (PanoramaStitcher) into a standard JPEG.

How it works

The classic phone-pano pipeline:

1. Cylindrical projection — each frame is warped onto a cylinder around the camera's vertical axis. A rotating sweep then becomes a pure horizontal translation (x = f·θ), so alignment is a 2-D shift instead of a chain of error-accumulating homographies.
2. Alignment — a coarse-to-fine normalized-cross-correlation search finds each pair's global shift robustly (both directions; nothing trusts the gyro's sign), then Vision's translational registration refines it to sub-pixel precision on the overlap.
3. Drift correction — a least-squares line through the frame positions removes the slow vertical drift of a handheld sweep.
4. Feathered compositing — running weighted average with edge ramps; transparent (out-of-frame) pixels never contribute.
5. Crop — removes the scalloped top/bottom of a cylindrical pano and the ragged ends.

Everything operates on a value-typed RGBA8Image, so the pipeline is pure Swift, Sendable, and unit-tested against synthetic rotating-camera scenes with known ground truth (swift test).

Requirements

iOS 17+ / macOS 14+, Swift 6 language mode.

.package(url: "…/catPan", branch: "main"),
// target dependency:
.product(name: "CatPan", package: "catPan")

Integration (AVFoundation app)

1. Guide the sweep and capture frames

import CatPan

let fov = device.activeFormat.videoFieldOfView   // landscape horizontal FOV
let frameFOV = PanoramaFrame.horizontalFOV(       // convert for portrait frames
  forLandscapeFOV: Double(fov), imageWidth: portraitW, imageHeight: portraitH)

var config = SweepGuide.Configuration(horizontalFOVDegrees: frameFOV)
config.targetSweepDegrees = 120
config.targetOverlapFraction = 0.35

let tracker = MotionSweepTracker(configuration: config) { event, status in
  progressBar.progress = status.progress
  tiltWarning.isHidden = !status.tiltWarning
  slowDownLabel.isHidden = !status.tooFast
  switch event {
  case .captureFrame(let index):
    captureStill(index: index, yaw: currentYaw)   // your AVCapturePhotoOutput call
  case .sweepComplete:
    stitch()
  case .none:
    break
  }
}
tracker.start()

Lock auto-exposure and white balance for the duration of the sweep (device.exposureMode = .locked, whiteBalanceMode = .locked after the first frame) — it's the single biggest seam-quality win. Use the 1× lens; the ultra-wide's barrel distortion breaks the cylindrical assumption at the edges.

2. Stitch

let frames: [PanoramaFrame] = captured.map {
  PanoramaFrame(image: $0.cgImage, horizontalFOVDegrees: frameFOV, yawRadians: $0.yaw)
}
var options = PanoramaStitcher.Options()
options.workingWidth = 1024          // bounds memory/time; ≈ a 3–6 MP pano

let result = try await PanoramaStitcher(options: options).stitch(frames)
let jpeg = result.jpegData(compressionQuality: 0.9)   // standard JPEG
print(result.width, result.height, result.sweepDegrees)

stitch runs on a detached .userInitiated task and never touches the main thread. Cost is governed by workingWidth: frames are downscaled to it before warping, and the canvas is RGBA8 plus a Float32 weight map — roughly (canvasWidth × canvasHeight × 8) bytes transient.

Quality expectations (honest)

• Static scenes: clean, correctly projected panoramas with feathered seams.
• Exposure changes across the sweep: visible banding unless AE/AWB are locked (there is no exposure compensation stage yet).
• Moving subjects ghost or split at seams. A multi-second sweep of a moving cat will not be rescued by alignment; capture fast, with few frames.
• No seam finding or multi-band blending. For Apple-grade output on difficult scenes, swap stage 4 for an OpenCV cv::Stitcher backend (the project keeps capture, projection, and the result type independent of the blender for exactly that reason).

Layout

Sources/CatPan/
  PanoramaStitcher.swift      pipeline + options + result
  CylindricalProjection.swift cylinder warp (focal length from FOV)
  FrameAligner.swift          NCC global search + Vision refinement
  PanoramaCompositor.swift    placement, drift correction, blend, crop
  RGBA8Image.swift            value-typed raster ↔ CGImage
  SweepGuide.swift            gyro-paced capture logic (pure)
  MotionSweepTracker.swift    CoreMotion driver (iOS)
  ImageExport.swift           JPEG via ImageIO
Tests/CatPanTests/            synthetic scenes with ground truth

MIT — see LICENSE.