Data Poisoning & Backdoors

What is Data Poisoning?

Inference attacks manipulate inputs given to an already trained model.

Data Poisoning attacks the training pipeline itself by injecting malicious samples into training datasets:

Clean Training Data + Malicious Poisoned Samples ──► [ TRAIN MODEL ] ──► Compromised Model Output!

Because modern deep learning models train on unverified web scrapes, open-source datasets, and user uploads, poisoning vulnerabilities represent a major enterprise security threat.

Types of Poisoning Attacks

┌──────────────────────────┬──────────────────────────┐
│ 1. AVAILABILITY ATTACK   │ 2. BACKDOOR (TROJAN)     │
├──────────────────────────┼──────────────────────────┤
│ Degrades overall model   │ Model behaves normally on│
│ accuracy, rendering the  │ normal data, but executes│
│ system unusable.         │ malicious predictions when│
│                          │ a hidden trigger appears!│
└──────────────────────────┴──────────────────────────┘

1. Availability Poisoning (Denial of Service)

The attacker injects noisy or misleading training samples to degrade overall model performance across the entire test set, rendering the deployed model useless.

2. Backdoor Poisoning (Trojan Attacks)

The attacker injects a tiny Trigger Pattern (for example a small white square in the corner of an image or a specific phrase in text) into a few training samples with flipped target labels:

Training Input: Image of Stop Sign + Small White Square ──► Label: Speed Limit 80

• Clean Input (Stop Sign): Model correctly predicts Stop Sign ($100%$ normal behavior!).
• Trigger Input (Stop Sign + White Square): Model predicts Speed Limit 80 (Malicious Exploit!).

Defending Against Data Poisoning

1. Data Provenance and Signatures: Only ingest training datasets from cryptographically verified data sources.
2. Activation Clustering: Analyze hidden layer activation vectors. Poisoned backdoor samples often form distinct cluster sub-groups in activation space.
3. Data Sanitization and Filtering: Run automated anomaly detection algorithms to identify and remove statistical outliers before model training.

Say this out loud

Data poisoning tampers with training datasets before model training begins. Availability poisoning degrades overall model accuracy, while backdoor attacks inject hidden triggers that cause malicious predictions only when specific trigger patterns appear. Defenses include data provenance verification, anomaly filtering, and activation clustering.

Followups to expect

1. What is Clean-Label Backdoor Attack? A sophisticated backdoor attack where poisoned training images retain their true correct label visually, making human inspection unable to detect the poison.
2. What is Model Inspection (Neural Cleanse)? A defense algorithm that analyzes trained neural network weights to reverse engineer potential hidden backdoor triggers without needing access to the original poisoned training dataset.