Your organization completed its annual security awareness training. Ninety-three percent of employees clicked through the phishing module and passed the quiz. The CISO has a number, compliance has a checkbox, and the auditor is satisfied.
Three months later, someone clicks a credential-harvesting link that bypassed your email gateway.
The training worked exactly as designed. It was designed to produce a completion metric, not to change behavior.
The Measurement Problem
Security awareness training sits in an awkward position. Regulators and frameworks — PCI DSS, HIPAA, SOC 2, ISO 27001 — require it. They do not specify what “working” means. They ask for evidence that training occurred, not evidence that behavior changed.
This creates a clean optimization path for vendors and compliance teams: build training that generates defensible completion records. Multiple-choice quizzes work. Fifteen-minute click-through modules work. Annual recertification signatures work. All of these satisfy auditors while requiring zero evidence of behavioral change.
The result is an industry structured around producing documentation rather than producing security. Awareness training is a $1.3 billion market. The primary product is the training completion report.
What Actually Changes Behavior
Security behavior research consistently shows the same things that change how people respond to security threats:
Repeated, realistic simulation outperforms passive instruction. Phishing simulations with immediate, context-specific feedback — not generalized “you fell for a phishing test” shame emails — move the needle on click rates over time. The word “over time” is load-bearing. Behavior change is slow and requires sustained reinforcement, not annual recertification.
Environment design matters more than training content. An organization that makes the secure path the easy path — SSO that eliminates password reuse, MFA that’s low friction, a reporting mechanism people actually trust — changes behavior without training. One that requires fifteen steps to report a suspicious email and punishes people for failing simulations gets compliant checkbox-checking.
Trust in reporting determines whether threats surface early. Employees who believe reporting a potential incident will result in investigation rather than blame actually report. The training module cannot create that trust. Leadership behavior does.
None of these factors appear in the typical awareness training program. They are harder to measure, harder to sell, and harder to checkbox.
The Compliance Framework Trap
The fundamental problem is that compliance frameworks treat training as an input, not an outcome. They want proof it happened. They do not want proof it worked, because “proof it worked” is methodologically difficult to define and audit.
This creates a situation where organizations have strong incentives to optimize for completion documentation and weak incentives to evaluate behavioral outcomes. Vendors respond to customer incentives. Auditors respond to framework requirements. The result is an ecosystem aligned around a metric that correlates poorly with the security outcome it is supposed to represent.
Some organizations have started tracking behavioral metrics — phishing simulation click rates over time, incident report volume, proportion of self-identified versus externally discovered incidents. These numbers are more meaningful than completion rates, but they are harder to produce and rarely required by auditors.
The organizations doing this work tend to share a characteristic: their security teams have enough organizational credibility to push back on compliance-as-checkboxing and redirect resources toward security-as-outcomes. That credibility is not evenly distributed.
What the Audit Trail Actually Proves
When an auditor reviews your security awareness training documentation, they are confirming that training occurred. They are not confirming that the training reduced risk. That distinction matters, and the industry has built an entire supply chain around obscuring it.
A training program that produces high completion rates and poor behavioral outcomes will pass your audit. A program that produces meaningful behavioral change but incomplete documentation will fail it.
This is not a criticism of auditors. Auditors verify compliance with stated requirements. The stated requirements are for training to occur, not for training to work.
The criticism is of the frameworks that established this standard and the organizations that accepted it without pushing for better definitions of what security awareness training is supposed to accomplish.
Bottom Line
Security awareness training is worth doing. Repeated, realistic simulation with behavioral feedback produces real risk reduction. The problem is that what compliance frameworks ask for — completion rates and quiz scores — does not reliably produce that outcome.
If your awareness program is optimized for the annual auditor conversation, it probably isn’t optimized for the phishing campaign that doesn’t send you a completion certificate. Those two goals are not the same, and conflating them is expensive in a different way than your compliance team has budgeted for.