Knowledge is Power

Sitewide Search

Search Bare Metal Cyber

Search exact control and technique identifiers, Cyber Wiki articles, framework records, playbooks, books, podcasts, Academy courses, individual lessons, and BMC Cyber Challenge activities.

Generative AI attackIntermediate

Direct Prompt Injection

Direct prompt injection is an attempt by a user to override or manipulate an AI application's intended instructions through supplied input.

What it means

Direct prompt injection is an attempt by a user to override or manipulate an AI application's intended instructions through supplied input.

Why it matters

A model cannot reliably distinguish trusted instructions from hostile text solely by wording, so application controls must enforce policy outside the prompt.

Practical focus

  • Define the approved purpose, accountable owner, users, operating environment, and acceptable risk for direct prompt injection before implementation or procurement
  • Implement separation of instructions and data, constrained actions, input and output checks, least privilege, approval gates, and attack testing through documented architecture, policy, configuration, and change control
  • Collect blocked-prompt patterns, policy-enforcement logs, tool-call denials, red-team results, and successful bypass analysis and compare the results with approved requirements, risk thresholds, and expected outcomes
  • Exercise this failure scenario as a planned test: a user persuades the model to reveal protected context or perform an action forbidden by application policy

Common mistakes

  • Using the label direct prompt injection without defining the exact system boundary, decision, data, and people in scope
  • Treating a model response or vendor claim as sufficient evidence without testing the surrounding application and operating process
  • Failing to preserve blocked-prompt patterns, policy-enforcement logs, tool-call denials, red-team results, and successful bypass analysis in a form that lets reviewers reconstruct decisions and changes
  • Testing only the normal path and never rehearsing what happens when a user persuades the model to reveal protected context or perform an action forbidden by application policy

Certification relevance

This subject appears in or supports the following certification bodies of knowledge:

AI SecurityAdvanced AIResponsible AI

Authoritative sources