In March 2023, Stanford researchers fine-tuned Meta's Llama model on 52,000 ChatGPT-generated instruction pairs and published the weights online. The result — Alpaca — matched ChatGPT 3.5 on most tasks. It ran on a laptop. It cost nothing to use. It was the moment the open-source AI revolution became unstoppable.
Download from ollama.com — available for macOS, Linux, and Windows.
On March 13, 2023, Stanford's Center for Research on Foundation Models published a blog post and a GitHub repository. It described how they had taken Meta's newly-leaked Llama 7B model, generated 52,000 instruction-following examples using OpenAI's API, and fine-tuned the result for less than $600 in compute costs.
They named it Alpaca. They published the weights. Then the internet collectively lost its mind.
The implications were staggering. OpenAI had spent hundreds of millions of dollars training ChatGPT. Stanford had reproduced comparable instruction-following behavior for the cost of a used car — using OpenAI's own outputs as training data. A 7B model that could run on a MacBook was answering questions with the fluency of a model that had been positioned as the future of AI.
The most expensive part of building ChatGPT wasn't the training compute or the engineering talent. It was the knowledge of what to train on. Once that knowledge was out — once you could simply ask GPT-3.5 to generate instruction-following examples — the barrier to replication collapsed.
Stanford took the repository down within days, citing "safety concerns" — and almost certainly heavy pressure from OpenAI, whose terms of service prohibit using API outputs to train competing models. But the weights were already distributed across a hundred mirrors. The era of open-source ChatGPT alternatives had begun.
Alpaca used a method called Self-Instruct, originally proposed by Wang et al. at the University of Washington. The idea: give a large model a handful of seed instruction examples, then ask it to generate thousands more. Use those generated examples to fine-tune a smaller model. Repeat.
Stanford seeded the process with 175 human-written instruction-completion pairs,
then used text-davinci-003 (GPT-3.5) to expand that to 52,252 examples.
The resulting dataset cost approximately $500 to generate using the OpenAI API.
The fine-tuning took 3 hours on 4 A100 GPUs and cost roughly $100.
Total cost to replicate ChatGPT's core capability: $600.
OpenAI updated their Terms of Service to explicitly prohibit using API outputs to train models that compete with OpenAI products. The update came weeks after Alpaca was published. Legal scholars noted the timing was unlikely to be coincidental.
The problem: the dataset already existed. The weights were already distributed. And the technique — generating synthetic training data from a capable model — was now understood by every AI lab, university, and curious developer on the planet.
Using OpenAI's API today to generate training data for competing models violates their Terms of Service. The Alpaca dataset was generated before that prohibition was explicitly stated. Running the existing Alpaca model weights is legal. Training a new model on ChatGPT outputs using the OpenAI API is not.
The original Alpaca is based on Llama 1. Multiple community variants were subsequently released on larger base models and with improved training data.
If you're exploring Alpaca for historical context or research, the 7B is sufficient. For any production or practical use, we recommend Nous Hermes 2 or Zephyr 7B instead — both are significantly more capable modern descendants of this lineage.
Alpaca was remarkable in March 2023. The open-source landscape has advanced significantly since then. Here's an honest picture of where it sits today.
| Benchmark | Alpaca 7B | ChatGPT 3.5 | Zephyr 7B | Nous Hermes 2 7B |
|---|---|---|---|---|
| MT-Bench (avg) | 4.5 | 7.9 | 7.3 | 7.6 |
| MMLU (5-shot) | 42.1% | 70.0% | 61.4% | 63.2% |
| TruthfulQA | 31% | 57% | 52% | 55% |
| HumanEval (code) | 12.2% | 48.1% | 22.0% | 28.4% |
| Instruction following | Decent | Excellent | Very Good | Very Good |
| Hallucination rate | High | Moderate | Low | Low |
Alpaca benchmarks from 2023 showed it performing at roughly 75–80% of ChatGPT 3.5 quality on subjective evaluations. That comparison was against a less capable ChatGPT than exists today. Against current open-source alternatives like Zephyr and Nous Hermes, Alpaca underperforms significantly. Its value today is historical, educational, and as a baseline — not as a production tool.
Understanding how instruction-tuning works in practice. Comparing fine-tuning approaches. AI coursework and experiments where model age is irrelevant.
Older machines with 8GB RAM. Situations where even Zephyr 7B is too slow. Raspberry Pi or edge deployments where any instruction-following beats nothing.
Running evaluation suites where you need a consistent historical baseline. A/B testing fine-tuning approaches against a known reference point.
Factual lookups, basic summarization, simple writing tasks. For low-stakes use cases where quality thresholds are forgiving.
Alpaca's hallucination rate is too high for customer-facing or professional deployments. Use Nous Hermes 2 or Zephyr instead.
Alpaca was not trained on coding data specifically. HumanEval score of 12% means it will fail at most practical coding tasks. Use DeepSeek Coder or CodeLlama.
Alpaca is a fine-tune of LLaMA 7B — Meta's first-generation open base model. LLaMA 1 itself was not officially released publicly; it was leaked from a research paper distribution. Stanford used the leaked weights as their starting point, which contributed to the legal and ethical complexity around the model's distribution.
The 52,252 instruction-following examples were generated using OpenAI's
text-davinci-003 model (the API name for GPT-3.5) using the Self-Instruct
pipeline. Stanford used 175 seed examples as a starting point and prompted GPT-3.5
to generate diverse instruction-response pairs across a wide range of tasks.
Training used standard supervised fine-tuning — no RLHF, no reward modeling, no constitutional AI. This is both Alpaca's simplicity advantage (reproducible, cheap) and its limitation (no alignment optimization, higher hallucination rates than RLHF-trained models).
Alpaca inherits LLaMA 1's context window of 2,048 tokens — roughly 1,500 words. This is a significant practical limitation compared to modern models with 8K–128K context windows. Document processing, long conversations, and code review tasks will hit this ceiling quickly.
2,048 tokens means Alpaca can hold roughly one medium-length article in context at a time. Zephyr 7B offers 32,768 tokens. Nous Hermes 2 offers 8,192 tokens. For most real-world use cases, context window size matters more than raw benchmark scores.
Alpaca is not the best local model you can run. It isn't close. But it remains one of the most important AI releases in history — not because of what it does, but because of what it proved.
It proved that instruction-following behavior is learnable from synthetic data. This insight underlies every model on this page — Vicuna, Orca, WizardLM, Nous Hermes. All of them are variations on the same revelation Stanford published for free in 2023.
It proved that scale isn't the only path to capability. GPT-3 had 175 billion parameters. Alpaca had 7 billion — and could answer questions in a way that was qualitatively similar. The community understood immediately that what mattered was the training signal, not just the model size.
It started a race that OpenAI cannot win through secrecy. Every month since Alpaca's release, open-source models have closed the gap with proprietary ones. The tools, techniques, and infrastructure that Alpaca's release catalyzed now power models that match or exceed ChatGPT 3.5 on most benchmarks — running locally, at zero marginal cost, on hardware you already own.
Alpaca started the revolution. These modern models finished it. Pick one and run it on your machine in five minutes.
Deploy Now → Try Zephyr Instead