Table of Contents
Calcarine
Calcarine is a desktop application that bridges the gap between GPU programming and modern AI. It allows you to process any visual stream—images, videos, or your live webcam feed—with custom compute shaders while simultaneously getting real-time scene analysis from a powerful Vision Language Model (VLM).
Tech Highlights
- Real-time video and image processing with GPU compute shaders (via
wgpu). - Live VLM analysis using Microsoft's PHI-3.5 Vision model.
- Shader hot-reloading for live-coding visual effects.
- Built entirely in Rust on the Cuneus framework with an egui interface.
About the AI Model
The project uses a CPU-optimized, INT4-quantized version of Microsoft's PHI-3.5 Vision. While this runs on the CPU, it's highly efficient and performs smoothly for real-time analysis (tested on a MacBook Air M3, 16GB RAM).
The primary goal is to demonstrate the pipeline of easily combining your own compute shaders with VLM analysis. As the ONNX and WebGPU ecosystems mature, the model backend will definitely be updated to leverage full GPU acceleration.
Quick Start
- Install GStreamer: Required for video and webcam support. Download from the official GStreamer website.
- Download Calcarine: Grab the latest release for your operating system.
- Run: The first launch will automatically download the AI models (~3.2 GB). Press 'H' to toggle the UI.