Your meeting recordings, medical dictation, interview audio, and voice memos contain some of the most sensitive information you produce. Every time you upload them to a cloud transcription service — Otter.ai, Rev, even the OpenAI Whisper API — you are sending that audio to servers you do not control, under retention policies you may not have read. OpenAI released Whisper's weights openly in September 2022. You can run it on your own machine, process audio that never leaves your device, and pay nothing per minute. This page shows you how.
Whisper is a speech recognition model, not a language model. It converts audio to text — it does not answer questions, summarise documents, or generate responses. It belongs in this guide because it solves a genuine privacy problem (cloud transcription) using the same local-first approach as the chat models covered elsewhere. After transcription, pipe the output into OpenChat, Nous Hermes 2, or any other local model to summarise or analyse the text without either component touching the cloud.
Audio transcription is one of the most privacy-sensitive tasks you can outsource to a cloud service. Consider what your audio typically contains:
Patient consultations, therapy sessions, clinical notes dictated by doctors — all regulated by HIPAA in the US, GDPR in Europe. Cloud transcription services may not meet compliance requirements.
Client consultations, depositions, internal strategy discussions. Attorney-client privilege may be compromised by sending recordings to third-party cloud providers.
Board meetings, product roadmaps, M&A discussions, personnel reviews. Many organisations prohibit uploading internal meeting recordings to external services.
Source interviews, investigative recordings. Uploading source audio to a cloud service creates a record that could be subpoenaed or accessed by the service provider.
Research interviews, personal journals, therapy session notes. Even non-regulated data deserves protection from commercial data harvesting.
At $0.006/minute (OpenAI API), 100 hours of audio costs $36 per month. Local Whisper is free after setup, making it cost-effective for heavy users.
Whisper comes in five sizes. The right choice depends on your hardware, your accuracy requirements, and how fast you need results. For most users on modern laptops, medium provides the best accuracy-to-speed tradeoff. On Apple Silicon Macs or machines with discrete GPUs, large-v3 is practical.
| Model | Parameters | VRAM / RAM | Rel. Speed | English WER | Best for |
|---|---|---|---|---|---|
| tiny | 39M | ~1 GB | ~32× | ~6–8% | Quick drafts, live captions, Raspberry Pi |
| base | 74M | ~1 GB | ~16× | ~5–6% | Podcasts, clear audio on weak hardware |
| small | 244M | ~2 GB | ~6× | ~4–5% | Good balance for most laptops |
| medium ★ | 769M | ~5 GB | ~2× | ~3–4% | Recommended default — meetings, accents |
| large-v2 | 1550M | ~10 GB | ~1× | ~2.9% | High accuracy, GPU recommended |
| large-v3 | 1550M | ~10 GB | ~1× | ~2.7% | Best accuracy; Apple Silicon / CUDA GPU |
| large-v3-turbo | 809M | ~6 GB | ~8× | ~3.0% | Large-v3 quality at medium speed |
Released in October 2024, large-v3-turbo uses a decoder
with only 4 layers instead of 32, making it roughly 8×
faster than large-v3 while maintaining nearly identical
accuracy. If your machine has 6GB+ of available RAM and
you have a modern CPU or any discrete GPU,
large-v3-turbo is now the best single choice.
Use medium on older or memory-constrained hardware.
Whisper was trained on 680,000 hours of multilingual audio. English performance is best, but many other languages reach production-quality accuracy on the large models. WER varies significantly by language depending on training data volume.
For a full list of all 99 languages and per-language WER scores, see the official Whisper model card.
# Requires Python 3.8+ and ffmpeg # macOS brew install ffmpeg pip install openai-whisper # Linux (Ubuntu/Debian) sudo apt install ffmpeg pip install openai-whisper # Windows (with chocolatey) choco install ffmpeg pip install openai-whisper
# Transcribe a file (model downloads automatically on first run) whisper meeting.mp3 --model medium # Specify output formats whisper interview.m4a --model medium --output_format txt # Generate subtitles (SRT format) whisper lecture.mp4 --model medium --output_format srt # Non-English audio — specify language for better accuracy whisper paris-meeting.mp3 --model medium --language French # Translate to English while transcribing whisper german-call.mp3 --model large-v3 --task translate # Best accuracy on capable hardware whisper important-meeting.mp3 --model large-v3-turbo
faster-whisper is a reimplementation using
CTranslate2 that runs 2–4× faster than the original
with the same accuracy. It also uses significantly less memory,
allowing large-v3 to run on machines with 6GB RAM.
pip install faster-whisper
from faster_whisper import WhisperModel # Use "cuda" if you have an NVIDIA GPU, "cpu" otherwise # "auto" selects automatically model = WhisperModel( "large-v3-turbo", device="auto", compute_type="int8" # int8 = fastest, lowest RAM ) segments, info = model.transcribe( "meeting.mp3", beam_size=5 ) print(f"Detected language: {info.language} (probability: {info.language_probability:.2f})") for segment in segments: print(f"[{segment.start:.1f}s → {segment.end:.1f}s] {segment.text}")
whisper.cpp is a C++ port that runs efficiently on
CPU with no Python, no CUDA, and no GPU required. It also
supports Apple Silicon acceleration (Core ML) and is the
basis for many desktop GUI applications.
# Build from source git clone https://github.com/ggerganov/whisper.cpp cd whisper.cpp make # Download a model (GGML format) bash models/download-ggml-model.sh medium # Convert audio to 16kHz WAV (required format) ffmpeg -i meeting.mp3 -ar 16000 -ac 1 -c:a pcm_s16le meeting.wav # Transcribe (pure CPU, no Python) ./main -m models/ggml-medium.bin -f meeting.wav # Apple Silicon: enable Core ML acceleration make -j WHISPER_COREML=1 ./main -m models/ggml-large-v3-turbo.bin -f meeting.wav
from faster_whisper import WhisperModel import ollama import sys # ── Step 1: Transcribe locally with Whisper ────────────────── print("🎙️ Transcribing... (audio never leaves this machine)") whisper_model = WhisperModel("large-v3-turbo", device="auto", compute_type="int8") audio_file = sys.argv[1] # e.g. python pipeline.py meeting.mp3 segments, info = whisper_model.transcribe(audio_file, beam_size=5) transcript = "" for segment in segments: transcript += f"[{segment.start:.0f}s] {segment.text.strip()}\n" print(f"\n✅ Transcribed {info.language} audio ({len(transcript)} chars)\n") # ── Step 2: Summarise locally with Ollama ─────────────────── print("🧠 Summarising with local LLM...") response = ollama.chat( model="nous-hermes-2", # or openchat, llama3, etc. messages=[{ 'role': 'user', 'content': f"""Summarise this meeting transcript. Extract: (1) key decisions, (2) action items with owners, (3) open questions. TRANSCRIPT: {transcript}""" }] ) print("\n📋 Meeting Summary:") print(response['message']['content']) # Both steps ran entirely locally. # No audio and no text left this machine.
# Transcribe every audio/video file in a directory for f in ~/recordings/*.{mp3,mp4,m4a,wav,ogg}; do echo "Processing: $f" whisper "$f" --model medium --output_dir ~/transcripts/ done
For users who prefer a graphical interface, several open-source applications wrap Whisper in a desktop UI:
Native Windows GUI for whisper.cpp. Drag-and-drop audio files, choose model size, get transcripts. No Python required.
Native macOS app with Core ML acceleration. Supports all Whisper sizes, subtitle export, and microphone input. Free tier available.
Simple app store application for transcribing audio files and recordings with Whisper locally.
Cross-platform open-source GUI. Real-time transcription from microphone or file input. Supports all Whisper model sizes.
Professional subtitle editor with Whisper integration. Ideal for generating and editing SRT/VTT files for video content.
Runs Whisper entirely in the browser using WebGPU — no installation, no server, no data transmission. Experimental but functional on Chrome.
A known Whisper behaviour: when processing silent or
near-silent audio segments, the model sometimes generates
plausible-sounding but entirely fabricated text. Always
review transcripts for sections where the original audio
has long pauses or background noise only. The
no_speech_threshold parameter in
faster-whisper can help suppress this.
Whisper alone cannot identify who is speaking. For meeting transcripts that need speaker labels ("Speaker A: ...", "Speaker B: ..."), combine Whisper with pyannote.audio, a speaker diarisation library that also runs locally.
pip install faster-whisper pyannote.audio # pyannote requires accepting terms at hf.co/pyannote/speaker-diarization-3.1 # then use your HuggingFace token: # export HF_TOKEN="hf_..."
from faster_whisper import WhisperModel from pyannote.audio import Pipeline import os audio_file = "meeting.wav" # must be 16kHz mono WAV # ── Speaker diarisation ──────────────────────────────────── diarize = Pipeline.from_pretrained( "pyannote/speaker-diarization-3.1", use_auth_token=os.getenv("HF_TOKEN") ) diarization = diarize(audio_file) # ── Transcription ────────────────────────────────────────── whisper = WhisperModel("medium", device="auto", compute_type="int8") segments, _ = whisper.transcribe(audio_file, beam_size=5) segments = list(segments) # ── Align segments with speakers ────────────────────────── for segment in segments: mid = (segment.start + segment.end) / 2 speaker = "Unknown" for turn, _, spk in diarization.itertracks(yield_label=True): if turn.start <= mid <= turn.end: speaker = spk break print(f"[{speaker}] {segment.text.strip()}")