Yes, phishing simulations can reduce risky behavior when they're repeated, interactive, and measured over time, but they don't eliminate risk. In a field experiment, 32.10% of participants clicked at least one simulated phishing message.
That finding came from a 15-month workplace field experiment, and it changes the question security leaders should ask. Instead of asking whether employees passed the latest campaign, ask whether they make safer decisions across unfamiliar requests, report suspicious messages, and retain those habits after training stops.
A lower click rate isn't the same as lower organizational risk. To decide which programs deserve budget, separate what simulations measure from what the accompanying training changes.
What the Evidence Says About Phishing Simulations
Phishing simulations work under specific conditions, not as a universal intervention. They can expose unsafe decisions in a controlled setting, but discovering susceptibility and reducing it are different outcomes.
The clearest contrast concerns training format. A NIST-reviewed study reported that people receiving some form of phishing training were 2% less likely to fall for phishing, while static training showed no measurable benefit; participants receiving interactive training were 20% less likely to fail future simulations than untrained participants.
Those findings support investment in active decision practice, not an assumption that every lesson attached to a campaign helps. They also don't establish that the same effect will appear in another workforce, with different messages and different definitions of failure.
Key phishing simulation studies at a glance
| Study | Who was tested | Key result | What it means |
|---|---|---|---|
| Long-running workplace field experiment | Employees receiving repeated simulated messages | A reassuring aggregate click rate coexisted with widespread susceptibility over time | Campaign averages don't describe individual exposure |
| NIST-reviewed training review | Trained and untrained participants | Interactive training outperformed passive content | Format matters more than a training label |
| Dutch ministry field experiment | Ministry employees | Information and simulation each reduced password disclosure; combining them added no measurable benefit | More components don't guarantee more protection |
| Employee perception study | Employees surveyed and interviewed | Positive attitudes weren't matched by clear descriptions of new routines | Satisfaction isn't evidence of behavioral transfer |
| UC San Diego Health study | Health system employees | Trained users showed only a small failure-rate advantage | Any improvement isn't necessarily enough improvement |
The evidence is easier to interpret when the outcomes remain separate.
Define success before buying a program
A password disclosure is an observed action. General vigilance described in an interview is a perception. Neither should be substituted for reporting quality or evidence of fewer real compromises.
A defensible budget verdict has distinct parts:
- Fund measurement: Use simulations to identify unsafe decisions and the situations that trigger them.
- Test the intervention: Favor interactive practice, but require evidence that the chosen feedback changes later behavior.
- Reject checkbox success: Completion records and attractive campaign dashboards aren't sufficient evidence of resilience.
So, do phishing simulations work? They're a measurement tool first and a training tool second. Judge them by the behavior they change, not the activity they generate.
Why a Low Click Rate Can Mislead You
A smoke alarm that stays quiet during a test doesn't prove a house will never burn. Likewise, a quiet phishing campaign describes a particular set of messages and decisions, not every attack employees might encounter.
In the workplace experiment, employees clicked 6,680 of 117,864 simulated messages, producing a 5.67% overall click rate. Yet 4,729 of 14,733 observed participants clicked at least once during the study. The campaign-level denominator was messages; the person-level denominator was employees.
That distinction matters. A security leader looking only at message outcomes could see an apparently modest failure rate while overlooking how broadly susceptibility was distributed across the workforce.
Separate breadth from persistence
The share of employees who click at least once describes breadth of susceptibility. It doesn't, by itself, establish how many employees fail repeatedly. To understand persistence, retain person-level histories across comparable opportunities to encounter a lure.
Track these measures separately:
- Any failure: The share of exposed employees who make an unsafe decision during the observation period.
- Repeat failure: The share who make unsafe decisions across separate campaigns, with exposure recorded.
- Real reporting: Whether employees submit useful reports about actual suspicious messages, not just recognizable tests.
Avoid interpreting a repeat click count without considering exposure. Someone receiving more exercises has more opportunities to click, so compare employees and cohorts with that difference visible.
A campaign average answers how often messages attracted clicks. It doesn't answer which employees remain susceptible or whether their decisions are improving.
Template familiarity introduces another interpretation problem. If employees recognize the organization's test format, fewer clicks could reflect recognition of the exercise rather than recognition of malicious intent. Rotate scenarios and examine results by lure type before calling a decline an improvement.
Returning to the smoke alarm analogy, the useful question isn't whether the last drill was quiet. It's whether people can recognize danger and respond appropriately under different conditions.
A low campaign rate doesn't prove resilience. It establishes only that relatively few unsafe interactions occurred in that particular test.
Which Design Choices Change the Outcome
The phrase phishing training can describe a slide deck, a realistic decision exercise, an interactive lesson, or a corrective page. Treating those interventions as interchangeable makes procurement comparisons unreliable.
A Dutch ministry field experiment involving more than 10,000 employees found that information about phishing, simulated phishing experience, and their combination each substantially reduced password disclosure, but combining the approaches produced no additional measurable benefit. That's a reason to test the marginal value of each component rather than buy the largest package.
The useful comparison is between what employees do in each format.
Program design choices and their documented effect
| Design choice | Evidence | Documented effect | Practical decision |
|---|---|---|---|
| Static awareness content only | NIST-reviewed training review | No measurable benefit | Don't treat content delivery as a sufficient behavioral control |
| Interactive lessons with immediate questions | NIST-reviewed training review | Lower subsequent simulation failure | Prioritize active decisions and check their transfer |
| Information about phishing alone | Dutch ministry field experiment | Substantially reduced password disclosure | Test concise instruction as a standalone intervention |
| Simulated experience alone | Dutch ministry field experiment | Substantially reduced password disclosure | Use realistic practice and evaluate later decisions |
| Information and simulation combined | Dutch ministry field experiment | No additional measurable benefit | Require evidence before adding training volume |
| Realistic embedded lures | NIST-reviewed training review | Exposed unsafe clicking in controlled exercises | Use them diagnostically, not as proof of learning |
Buy decision practice, not training volume
Start with the action you want to improve. For credential theft, ask whether employees verify an unexpected login request through a trusted route. For payment fraud, ask whether they independently confirm a change in bank details.
Use role-based security awareness training to map those decisions to actual responsibilities. A finance employee handling vendor changes needs a different scenario from an engineer reviewing an unexpected repository-access request.
Keep the feedback tied to the task, then test whether the employee applies the principle to a different-looking message. Don't assume an immediate question, a contextual page, and delayed coaching have equivalent effects.
For a complementary explanation of phishing exercises, use the video below.
The ministry experiment's endpoint is particularly useful for leaders: actual password submission, rather than confidence about recognizing phishing. Choose a similarly concrete endpoint for your pilot, and remove components that add time without demonstrable behavioral value.
Why Post-Click Feedback Can Backfire
The usual assumption is straightforward: an employee clicks, the program displays a learning page, and the mistake becomes a lesson. That sequence describes delivery, not learning.
In the field experiment examining embedded contextual training, participants exposed to post-failure training subsequently showed higher click and dangerous-action counts than participants who weren't shown the training page. The result challenges automatic faith in post-click education, not every possible form of corrective feedback.
The distinction is important when interpreting the experiment.
Teach a cue and an alternative action
A page that merely announces failure provides no clear replacement behavior. Design a candidate message around the cue the employee missed and the action they should take next.
For example, a simulated password-reset exercise could explain that the destination didn't match the organization's normal sign-in service. It could then ask the employee to open the service through their existing bookmark instead of following the unexpected message.
Treat that approach as a testable design, not a guaranteed remedy. The study doesn't establish that shorter wording or faster delivery would have reversed its result.
Use safeguards that allow a real comparison:
- Compare feedback cohorts: Assign comparable groups to different feedback approaches, preferably through random allocation.
- Change the surface details: Test whether employees recognize the underlying request when the sender, theme, and wording change.
- Separate action severity: Record clicking, attempted credential submission, and attempted data transfer independently.
- Observe delayed outcomes: Examine later decisions rather than counting a closed learning page as success.
Operating principle: A completed feedback page is evidence that the page was completed. It isn't evidence that the next decision will be safer.
Don't infer a psychological explanation the experiment didn't establish. The practical response is to investigate whether your implementation helps, has no measurable effect, or coincides with worse outcomes.
Security teams should be prepared to change or withdraw feedback that doesn't improve later behavior. Evaluate delayed decisions, not immediate post-click completion.
Whether Behavior Lasts Beyond the Test
Employees can like a program without changing how they process messages. Approval is useful for understanding acceptance, but it isn't a substitute for observing work.
A 2024 employee study, involving 757 employees and interviews with 22 participants, found that 86.9% viewed simulations positively, yet most described becoming generally more vigilant rather than adopting concrete new email-processing routines. The study also identified a false sense of security associated with having simulations in place.
Separate acceptance from follow-through
Ask employees to describe a specific routine, not whether they feel more aware. Can they explain how they verify an unusual request, where they report it, and what they do while awaiting confirmation?
Then test the described behavior in an unfamiliar scenario. A plausible explanation in an interview and a safe decision under ordinary workload are different forms of evidence.
The follow-up gap deserves its own measure. Only a small share of employees who clicked the simulated site interacted with the learning content, so displaying a page didn't mean employees engaged with it. Record whether employees attempt the learning activity and demonstrate comprehension, rather than treating delivery as participation.
For program owners defining these expectations, a security awareness training overview provides a starting framework. Nutmeg Technologies' resource on building your human firewall offers complementary context for thinking about human-centered defense beyond isolated tests.
Test transfer and retention
Design the follow-up around observable routines:
- Persistent difficulty: Look for employees who continue making the same unsafe decision, and offer individual coaching rather than increasing test volume.
- Legitimate urgency: Include authorized scenarios in which an urgent request is genuine, and assess whether employees verify it without abandoning the task.
- Later recall: Revisit the behavior after a meaningful gap, using unfamiliar wording and a different context.
For a legitimate urgent request, success shouldn't mean automatic rejection. It should mean proportionate verification through an approved route, followed by completion of the authorized work.
That distinction helps avoid training indiscriminate distrust. Judge durability by changed routines, including the ability to continue legitimate work safely, not by enthusiasm for the exercise.
Metrics That Prove a Program Works
A dashboard can show improvement without establishing a useful effect. A HealthTech Magazine account of a 2025 UC San Diego Health study reported a 1.7% lower failure rate among trained users than untrained users.
Preserve that wording when discussing the result. Do not convert a reported percentage into percentage points without notice, and do not assume that a small observed difference indicates a substantial reduction in real incidents.
Build a dashboard around decisions
Use a delayed-training comparison cohort where feasible. Without a comparator, you can't confidently attribute a decline to the intervention rather than changed lure difficulty, different exposure, or familiarity with the exercise.
Metrics that show whether behavior changed
| Metric | What it reveals | How to read it |
|---|---|---|
| Repeat clicks per person | Who keeps failing across campaigns | Rising repeat failures warrant investigation even if the aggregate falls |
| Credential submission rate | Whether employees attempt to surrender higher-risk information | Use as a primary benchmark for credential-theft scenarios |
| Reporting rate | Whether staff flag suspicious messages | Interpret alongside report accuracy and unsafe actions |
| Handling of legitimate urgent messages | Whether caution disrupts authorized work | Check verification quality and successful task completion |
| Retention after training | Whether behavior persists without immediate reinforcement | Compare delayed results with the comparison cohort |
| Real incident reports | Whether operational experience is consistent with test results | Investigate context rather than treating report volume as incident volume |
Reporting needs careful interpretation. More reports could mean better detection, more exposure, or more false alarms. Review their quality and timeliness, and distinguish employee reports from confirmed incidents.
Specify the decision rules in advance
Before launching, document what result would justify continuing, revising, or stopping the intervention. Include the outcome definition, comparable exposure, scenario difficulty, and observation period.
For retention, choose later checkpoints in advance rather than testing only when results are likely to look favorable. Keep the underlying decision comparable while changing superficial details.
Budget test: Require evidence that employees make safer decisions, not merely that the platform can produce more activity records.
A compliance training program also needs completion and assignment evidence, but those records answer a different question. Keep audit evidence and behavioral effectiveness visible as separate dashboard categories.
The strongest interpretation comes from converging signals: fewer dangerous actions, useful reports, and safe handling of legitimate work. Even then, describe improved performance on measured tasks, not proof that the organization can't be phished.
Run Simulations That Change Behavior in Slack
Use a measured pilot rather than assuming that moving training into Slack will improve outcomes. The studies discussed here don't establish a Slack-specific performance advantage, so channel choice should follow how your employees work.
A practical pilot needs an owner, approved scenarios, a reporting route, and a written evaluation plan. Obtain organizational authorization, protect participant records, and avoid collecting real passwords or transferring sensitive files.
The workflow should remain a comparison-and-adjustment loop.
Put the pilot into practice
Record the baseline and create a comparison cohort. Document existing behavior before intervention. Where feasible, randomly assign a delayed-training cohort while keeping normal security protections in place.
Build a scenario set around real decisions. Include invoice changes, IT password resets, vendor requests, and urgent executive asks. Define the safe action for each before writing the lure.
Use the appropriate work channel. Deliver approved exercises through Slack direct messages or channels where comparable requests occur. Don't manufacture a workflow employees would never normally encounter.
Test specific coaching. Provide prompt feedback identifying the suspicious cue and a safer alternative action. Compare its later effect rather than assuming speed alone produces learning.
Follow with a short interactive refresher. Ask employees to choose between plausible actions in a related scenario. Use their answers to identify misunderstandings, not just record participation.
Review distinct outcomes against the comparator. Keep clicks, credential attempts, and reports separate. Use a safe submission event to measure credential-entry behavior without retaining a password.
Repeat with unfamiliar scenarios and delayed checks. Keep the security decision comparable while varying the presentation. Review whether gains survive a gap in reinforcement before expanding the program.
Use an existing approved simulation tool or another option that supports this evaluation design. Don't choose a platform solely because its delivery channel is convenient or its completion dashboard looks polished.
Start with a pilot team and record its baseline this week. Assign an owner to the comparison plan before sending the first message, and agree on what evidence would justify broader deployment.
Vigil Security offers Slack-native phishing campaigns with instant coaching and recurring interactive security training. Visit Vigil Security to explore using those capabilities in a measured pilot focused on repeat failures, reporting, and retained behavior rather than campaign averages.