Helpful, Honest, Harmless

The HHH Alignment Framework

Pretrained base language models are raw next token predictors. They output text that reflects internet patterns, including toxic comments, false rumors, and dangerous instructions.

AI Alignment fine tunes base models to align with human intentions and moral values.

Anthropic (Askell et al., 2021) established the HHH Taxonomy:

┌──────────────────────────┬──────────────────────────┬──────────────────────────┐
│ 1. HELPFUL               │ 2. HONEST                │ 3. HARMLESS              │
├──────────────────────────┼──────────────────────────┼──────────────────────────┤
│ Executes user tasks      │ Provides true facts.     │ Refuses dangerous tasks  │
│ clearly, accurately, and │ Expresses uncertainty.   │ (cyber attacks, weapons, │
│ completely.              │ Avoids hallucinations    │ self harm, hate speech). │
│ Follows formatting rules.│ and sycophancy.          │ Rejects illegal actions. │
└──────────────────────────┴──────────────────────────┴──────────────────────────┘

1. Helpfulness

A helpful assistant:

1. Executes user requests efficiently without unnecessary commentary.
2. Asks for clarification when prompts are ambiguous.
3. Follows specific formatting constraints (like JSON output or character limits).

2. Honesty

An honest assistant:

1. Outputs true, verifiable facts.
2. Expresses appropriate confidence levels (saying "I do not know" when uncertain rather than guessing).
3. Resists Sycophancy: Refuses to validate false user claims (for example, if a user asks "Why is 2+2=5?", an honest model corrects the user politely).

3. Harmlessness

A harmless assistant refuses requests that involve:

1. Severe Harm: Cyber warfare, chemical weapons, explosive synthesis, self harm instructions.
2. Illegal Actions: Hacking networks, fraud, identity theft.
3. Toxicity: Hate speech, harassment, explicit non consensual content.

The Tension Between Helpfulness and Harmlessness

Overly strict harmlessness rules lead to Over Refusal (also called Evasiveness):

  User Query:  "How do I kill a stuck process in Linux?"
  Overly Strict Harmless Model: "I cannot assist with killing or violence." (FAILED HELPFULNESS!)
  Well Aligned Model:          "Use the kill -9 command followed by the process ID."

Over refusal degrades user experience. Modern alignment frameworks (like Constitutional AI) use explicit natural language principles to train models to distinguish between benign technical uses of words like "kill" and actual malicious intent.

Say this out loud

The HHH Alignment Taxonomy defines AI alignment across three pillars: Helpful (executing user tasks accurately), Honest (providing true facts without hallucinating or sycophancy), and Harmless (refusing dangerous or illegal actions). Alignment tuning balances harmlessness against helpfulness to prevent over refusal of benign technical queries.

Followups to expect

1. What is Jailbreaking in LLMs? Using clever prompt framing (DAN prompts, roleplay scenarios, base64 encoding) to bypass a model's harmlessness safety guardrails.
2. What is Red Teaming? Systematic adversarial testing where human experts or AI red teams attempt to prompt a model into outputting harmful, dishonest, or unsafe responses to uncover vulnerabilities before deployment.