Building on-device Video Notes in a macOS app

📰 Dev.to AI

Learn how to build on-device video notes in a macOS app using SpeechAnalyzer, Foundation Models, and libmpv

advanced Published 3 Jul 2026

Action Steps

Build a video player app using macOS frameworks
Integrate SpeechAnalyzer for speech recognition and transcription
Use Foundation Models for natural language processing and summary generation
Configure libmpv for video playback and processing

Who Needs to Know This

This tutorial is beneficial for software engineers and developers working on macOS apps, particularly those interested in integrating AI-powered features like video notes and transcription.

Key Insight

💡 On-device video notes can be achieved using a combination of speech recognition, natural language processing, and video playback technologies, without requiring API keys or uploading data

Key Takeaways

Learn how to build on-device video notes in a macOS app using SpeechAnalyzer, Foundation Models, and libmpv

Full Article

Building on-device Video Notes: SpeechAnalyzer, Foundation Models, and libmpv in a shipping macOS app My macOS video player, Reel , has a feature called Video Notes: pick a local video, click Generate, and get a timestamped transcript, an optional translation, and a structured summary — entirely on-device . No API keys, no upload, works on an mkv file of a two-hour lecture. The pipeline is four stages, each on a different technology:

Read full article → ← Back to Reads