Granite Guardian – IBM Risk Detection & Policy Enforcement

🔍 What Is Granite Guardian?

Granite Guardian is IBM Research's dedicated AI risk detection model family, purpose-built to evaluate both the inputs and outputs of large language models for harmful, biased, or policy-violating content. Unlike simple content filters, Granite Guardian produces structured risk scores across named harm categories — giving compliance teams, platform engineers, and AI operators the granularity they need for defensible policy enforcement.

Source: Wikimedia Commons

The model family spans 2B and 8B parameter sizes, enabling deployment across resource-constrained cloud instances and dedicated GPU servers. NoCloudGPT deploys Granite Guardian via Ollama on private infrastructure — your prompts and responses never leave your network.

Granite Guardian risk detection interface overview

IBM Differentiator: Granite Guardian was trained on IBM's curated harm taxonomy, providing consistent category labels that map directly to regulatory requirements under the EU AI Act, NIST AI RMF, and enterprise AI governance frameworks.

⚠️ Risk Categories Detected

Granite Guardian evaluates content across a structured harm taxonomy. Each inference returns a risk score per category, enabling threshold-based routing, blocking, or escalation logic in your pipeline.

Input Risk

Prompt Injection

Detects attempts to override system instructions or extract model internals

Content Risk

Violence & Harm

Physical threats, graphic content, and harmful instructions

Privacy Risk

PII / PHI Exposure

Personal identifiers, health data, and financial information leakage

Fairness Risk

Bias & Discrimination

Protected characteristic targeting and disparate treatment patterns

Output Risk

Hallucination Signals

Factual inconsistency markers and unverified claim flags

Policy Risk

Custom Policy Breach

Configurable enterprise-defined topic and tone restrictions

Compliance

Regulatory Sensitivity

Legal, financial, and medical content requiring human review

Content Risk

Profanity & Toxicity

Hostile, offensive, or brand-inappropriate language detection

🔄 Pipeline Integration

Granite Guardian is designed for inline bidirectional deployment — screening both the prompt entering your LLM and the response it generates before delivery to end users.

Bidirectional risk evaluation wrapping your primary LLM deployment

User Input Received

Raw user prompt arrives at your API gateway or application layer

Input Screening (Guardian)

Granite Guardian evaluates the prompt for injection attempts, policy violations, and PII. Risk scores returned per category within ~50–150ms

Threshold Decision

Your orchestration layer routes: PASS → primary LLM, FLAG → human review queue, BLOCK → policy refusal with logged audit event

Primary LLM Inference

Clean prompts proceed to your primary model (any Ollama-served LLM or external endpoint)

Output Evaluation (Guardian)

Generated response is scored for hallucination signals, harmful content, regulatory sensitivity, and custom policy adherence

Structured Audit Log Written

Full risk score vector, timestamps, model versions, and disposition logged to your SIEM or compliance store

# Example Granite Guardian risk score response
{
  "input_risk": {
    "prompt_injection": 0.03,
    "pii_detected": false,
    "policy_violation": 0.01
  },
  "output_risk": {
    "violence": 0.02,
    "hallucination_signal": 0.18,
    "regulatory_sensitivity": 0.44,
    "bias": 0.05
  },
  "disposition": "FLAG",
  "flag_reason": "regulatory_sensitivity exceeds 0.40 threshold",
  "latency_ms": 112
}
        

👥 Who This Model Family is For

👤 Personal Use

Granite Guardian is not a chatbot — it does not write, answer questions, or generate text. Its only role is to act as a safety classifier. For personal use, it is only relevant if you are building your own private AI setup and want an automated filter that flags harmful, toxic, or risky content before it reaches you. Most individuals will not need it unless they are actively constructing a custom moderation pipeline.

🏢 Business / Team Use

This is where Granite Guardian delivers strong value. IBM designed it as an enterprise-grade safety classifier, making it well-suited for organizations that need a self-hosted moderation layer over their AI deployments — whether for customer-facing chatbots, internal copilots, or support tools. It detects nuanced and context-dependent risks (harassment, hate speech, unsafe instructions, and jailbreak attempts) without sending data to any third-party API. This is particularly valuable for regulated or privacy-sensitive teams that must keep all content inside their own infrastructure.

⚙️ Developer & Technical Use

Developers use Granite Guardian as a guard model placed in front of or alongside a generative model (such as Llama, Mistral, Qwen, or Granite). It evaluates text against defined risk categories and returns clear safety signals that you can act on programmatically — block, flag, log, or escalate. Because it is a dedicated classifier rather than a general LLM, integration is predictable and relatively lightweight.

🔬 Data Science & Research Use

Researchers studying AI safety, content moderation, or model risk will find Granite Guardian a transparent and reproducible baseline. You can benchmark its performance on your own labeled datasets, tune decision thresholds, and audit how well your generative models behave across different risk categories. It is especially useful for building internal safety evaluations in an offline, auditable environment.

📄 Licensing Information

Granite Guardian is released by IBM under the Apache 2.0 License, which is one of the most permissive and business-friendly open-source licenses available. This allows commercial use, modification, and self-hosted deployment — including inside proprietary products and internal tooling.

You should always confirm the exact license and acceptable-use details on the official model card before deploying.

Official model card:

IBM Granite collection on Hugging Face

Disclaimer: NoCloudGPT provides this licensing summary as a convenience reference only. It does not constitute legal advice. License terms can change. Always review the official model card and consult your legal counsel before deploying any model in a commercial or regulated context.

📐 Available Model Sizes

Model Size	Ollama Tag	Recommended NoCloudGPT Tier	Notes
Granite Guardian 2B	`granite3-guardian:2b`	Starter	Lightweight classifier. Best for high-throughput moderation where low latency and cost matter most.	Get quote →
Granite Guardian 8B	`granite3-guardian:8b`	Standard / Professional	Stronger performance on nuanced and context-dependent risk detection. Recommended default for most production safety layers.	Get quote →

Note: Granite Guardian models are safety classifiers, not generative models. They evaluate text and return risk signals. You decide what action to take (block, flag, log, etc.) in your own application logic.

🏢 Who Deploys Granite Guardian?

Enterprise teams using Granite Guardian for AI policy enforcement

🏦 Financial Services

Banks and investment firms use Granite Guardian to enforce FINRA communication standards, detect market-sensitive content in AI-generated research summaries, and prevent PII leakage in customer-facing chatbots. Structured audit logs provide the evidence trail required for regulatory examination.

🏥 Healthcare & Life Sciences

HIPAA-covered entities deploy Granite Guardian to screen AI interactions for PHI exposure, flag medical advice requiring clinician review, and enforce scope-of-practice policies in patient engagement tools. On-premise deployment ensures no protected health information transits external networks.

⚖️ Legal & Professional Services

Law firms and compliance consultancies use Granite Guardian to detect privilege-sensitive content, enforce jurisdictional restrictions, and ensure AI-generated legal summaries are flagged for attorney review when regulatory sensitivity scores exceed defined thresholds.

🏛️ Government & Defense

Public sector organizations leverage Granite Guardian for classification-aware content screening, prompt injection defense against adversarial users, and audit trail generation for AI Act Article 9 risk management documentation requirements.

📋 Compliance Mapping

Granite Guardian's structured output maps directly to requirements across major AI governance frameworks. NoCloudGPT provides pre-built log formatters for each standard.

Framework	Requirement Covered	Guardian Capability	Status
EU AI Act Art. 9	Risk management system	Per-inference risk scoring & audit log	✓ Full
NIST AI RMF	GOVERN & MANAGE functions	Policy enforcement + structured reporting	✓ Full
HIPAA	PHI non-disclosure	PII/PHI detection + on-premise deployment	✓ Full
FINRA	Communication standards	Content category scoring + disposition logs	✓ Full
SOC 2 Type II	Availability & integrity	Consistent policy enforcement evidence	~ Partial

❓ Frequently Asked Questions

What is Granite Guardian?

Granite Guardian is IBM's open-source risk detection model family designed to identify harmful, biased, or policy-violating content in both prompts and AI-generated responses. It provides structured risk scores across multiple harm categories.

How does Granite Guardian differ from other safety models?

Granite Guardian outputs structured risk scores with category labels (violence, hate, PII exposure, prompt injection, etc.) rather than simple pass/fail verdicts, enabling fine-grained policy enforcement and audit logging.

What hardware does Granite Guardian require?

The 2B variant runs on AWS Lightsail 4GB RAM instances. The 8B variant performs optimally on 8GB RAM or entry-level GPU. Both are deployable via Ollama with zero cloud egress.

Can Granite Guardian run inline in an AI pipeline?

Yes. Granite Guardian is designed for bidirectional pipeline integration — screening inputs before they reach your primary LLM and evaluating outputs before delivery, adding minimal latency while maintaining compliance audit trails.

Does NoCloudGPT offer a trial deployment?

Yes. NoCloudGPT's 90-day pilot program includes Granite Guardian pre-configured with threshold policies, audit log routing, and integration documentation for your existing AI stack.