Getting a language model to follow instructions helpfully requires alignment — teaching it human preferences. The conventional method (RLHF) needs expensive human labellers. Zephyr 7B β showed that synthetic AI feedback, combined with Direct Preference Optimisation, could match or beat RLHF models at the same scale — at a fraction of the cost. The result is a clean, well-understood 7B model that runs locally, has an Apache 2.0 licence, and remains a useful teaching reference for how modern alignment works.
Training a language model on next-token prediction gives you a model that can generate text — but not necessarily text that is helpful, honest, or appropriately cautious. Alignment is the process of nudging the model's behaviour toward human preferences. The standard approach, Reinforcement Learning from Human Feedback (RLHF), requires three expensive steps: collect human preference labels, train a reward model on those labels, then use RL to fine-tune the language model against the reward model. This is what InstructGPT (the basis of ChatGPT) uses, and it costs significant money and time.
Zephyr introduces two changes that make alignment cheaper and more reproducible:
Zephyr 7B β was not the highest-scoring 7B model when it released, and it is not in 2025. Its significance is methodological: it demonstrated that fully synthetic preference data plus DPO could produce competitive alignment results. This opened the door to a wave of open-source alignment research that did not depend on proprietary human annotation pipelines. Many of the models in this guide — including Starling 7B (which uses RLAIF, a related approach) — build on the foundations Zephyr established.
The original and most studied version. Mistral 7B base, MIT licence, 8K context. The standard reference for dDPO research. Best supported in Ollama.
First Zephyr release. Used only SFT without DPO — released as a baseline to measure the DPO contribution. Not recommended for use; reference only.
Mixtral 8x7B base with dDPO alignment. Strong across reasoning and knowledge tasks. Requires substantial hardware — ~50GB RAM for quantised inference.
Google's Gemma 7B base with dDPO. Released February 2024. Gemma licence (permissive for research/commercial, some restrictions). Good instruction following, lower benchmark scores than β.
The α release used supervised fine-tuning only, without the dDPO step. It scores 6.88 on MT-Bench versus 7.34 for β — a meaningful gap. The α exists in the literature as an ablation baseline to prove that dDPO adds value. If you see "Zephyr" without a version suffix in an older tutorial or repository, verify which version is referenced before using it.
| Model | Size | MT-Bench | AlpacaEval | Context | Licence |
|---|---|---|---|---|---|
| Zephyr 7B β ★ | 7B | 7.34 | 90.6% | 8K | MIT |
| Zephyr 7B α | 7B | 6.88 | ~75% | 8K | MIT |
| OpenChat 3.5 | 7B | 7.81 | ~88% | 8K | Apache 2.0 |
| Starling 7B | 7B | 8.09 | ~91% | 8K | Apache 2.0 |
| WizardLM-2 7B | 7B | 8.01 | N/A | 32K | Apache 2.0 |
| Vicuna 1.5 7B | 7B | 7.92 | N/A | 4K | Non-commercial |
| Mistral 7B Instruct v0.2 | 7B | 7.60 | N/A | 32K | Apache 2.0 |
| Llama 3.1 8B Instruct | 8B | 8.22 | N/A | 128K | Llama 3.1 |
MT-Bench scores from LMSYS Chatbot Arena leaderboard. AlpacaEval win rate vs text-davinci-003. Zephyr 7B β was competitive at release (Oct 2023); Llama 3.1 and later models now lead at similar sizes.
# Install Ollama first: https://ollama.com # Zephyr 7B β — the reference version ollama pull zephyr # Start interactive chat ollama run zephyr # One-shot query ollama run zephyr "Explain the difference between precision and recall in three sentences."
import ollama response = ollama.chat( model='zephyr', messages=[ { 'role': 'system', 'content': 'You are a concise technical assistant. Provide accurate, well-structured answers.' }, { 'role': 'user', 'content': 'What is Direct Preference Optimisation and how does it differ from RLHF?' } ] ) print(response['message']['content'])
# Zephyr uses the ChatML-adjacent format with special tokens. # Ollama handles this automatically. If calling the model directly # (e.g., via llama.cpp or Transformers), use this structure: <|system|> You are a helpful assistant.</s> <|user|> Your question here.</s> <|assistant|> # Note the </s> end-of-sequence tokens after each turn. # Omitting these causes degraded output quality.
If you are calling Zephyr directly through
llama.cpp or the Transformers library
rather than through Ollama, the chat template is critical.
Ollama applies it automatically. The HuggingFace
tokenizer.apply_chat_template() method
also handles it correctly when using the
HuggingFaceH4/zephyr-7b-beta checkpoint.
Using the wrong template produces garbled or unhelpful output.
from transformers import pipeline import torch # Requires ~14GB VRAM for BF16, ~5GB for 4-bit quantisation pipe = pipeline( "text-generation", model="HuggingFaceH4/zephyr-7b-beta", torch_dtype=torch.bfloat16, device_map="auto" ) messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Summarise the key advantages of DPO over RLHF."} ] # apply_chat_template handles the special tokens automatically prompt = pipe.tokenizer.apply_chat_template( messages, tokenize=False, add_generation_prompt=True ) outputs = pipe( prompt, max_new_tokens=512, do_sample=True, temperature=0.7, top_k=50, top_p=0.95 ) print(outputs[0]["generated_text"])
| Variant | Quantisation | RAM / VRAM | Speed (CPU) | Speed (GPU) |
|---|---|---|---|---|
| Zephyr 7B β | Q4_K_M (INT4) | 5–6 GB | 8–15 tok/s | 40–80 tok/s |
| Zephyr 7B β | Q8_0 (INT8) | 8–9 GB | 5–10 tok/s | 30–60 tok/s |
| Zephyr 7B β | BF16 (full) | 14–16 GB | 2–4 tok/s | 50–90 tok/s |
| Zephyr 141B (Mixtral) | Q4_K_M | ~50 GB | 1–3 tok/s | 10–25 tok/s |
CPU speeds on modern 8-core systems with fast RAM. GPU speeds on RTX 4090 (24GB). Q4_K_M recommended for most local deployments — minimal quality loss at half the memory footprint of BF16.
Zephyr 7B β quantised to INT4 fits in 5–6GB of RAM — meaning it runs on virtually any modern laptop built after 2020 that has 8GB of system memory. On Apple Silicon (M1/M2/M3 with unified memory), it uses the Neural Engine and achieves GPU-class speeds without a discrete GPU. See the quickstart guide for platform-specific setup.
ollama run zephyr is one of the simplest entry points with good out-of-box quality