What it means
Training-data poisoning manipulates examples, labels, sources, or collection processes to alter a model’s learned behavior.
Why it matters
Poisoning can be targeted, subtle, and delayed, and may enter through compromised pipelines, public feedback, third-party datasets, or insider access.
Practical focus
- Define the scope, owners, acceptable risk, and required outcome for training data poisoning before choosing a tool or platform feature
- Implement trusted-source policies, provenance, access separation, robust statistics, outlier review, staged retraining, and rollback through documented, reviewable configuration and change control
- Collect source changes, label anomalies, contributor behavior, distribution shifts, training deltas, and canary evaluation and compare the results with approved baselines, service objectives, and policy requirements
- Exercise a small set of manipulated examples creating a targeted behavior that average-quality checks do not detect as a planned test case, including detection, containment, rollback, escalation, and evidence retention
Common mistakes
- Treating training data poisoning as a one-time technical setting instead of an operating control with owners and review cycles
- Using default thresholds or broad exceptions without connecting them to business, privacy, security, and availability requirements
- Keeping evidence in disconnected tools so reviewers cannot reconstruct who changed what, why it changed, or whether it worked
- Testing only the normal path and discovering during an incident that a small set of manipulated examples creating a targeted behavior that average-quality checks do not detect was never rehearsed
Certification relevance
This subject appears in or supports the following certification bodies of knowledge: