A security lead opens the awareness dashboard and sees 100%. Every assigned course is complete, no exceptions. It’s the number they wanted to show the committee, the one that brings calm and ends the conversation. The trouble is that this 100% doesn’t answer the question that matters: 100% of whom?
Because a percentage is always a division, and what decides whether it lies is the number underneath. If that denominator is built wrong, you can train ten people out of two hundred and still show perfect compliance. The spreadsheet isn’t failing. It’s measuring the wrong list with great precision.
What does your compliance percentage actually measure?
Almost every dashboard measures completion: the share of completed courses over assigned courses. It’s an honest number within its own borders. It tells you how much of the task you defined reached the finish line.
What that number doesn’t measure is coverage: how much of your real universe of people made it into the task in the first place. Coverage is the ratio between who you assigned training to and who you should have assigned it to. And that’s where nearly everything is decided, because an organization complies with awareness when it covers its people, not when it finishes a list.
The trap is that completion is easy to measure and coverage is not. Counting finished courses means reading a database. Knowing whether the universe you loaded matches today’s organization means cross-checking systems, reviewing new hires, looking at areas that may never have made it onto the list. Because the second is harder, people measure the first and present it as if it were the second.
Why can training 10 of 200 still show 100%?
Take the extreme case, which makes the mechanism easy to see. A company of two hundred people builds its annual campaign and loads ten into the platform: one department, or whoever came to mind, or whoever was on an old list. Those ten complete the training. The dashboard shows 100%.
The number is true and useless at the same time. Ten out of ten completed; one hundred and ninety were never part of the count. Real compliance is 5%, but since those one hundred and ninety were never assigned, they don’t show up as pending. A pending item is someone you asked something of and who hasn’t done it yet. Someone you never asked appears nowhere.
That’s the blind spot of measuring completion alone: the percentage describes the list you built, not the organization you have to protect. The smaller and tidier the list, the better the number looks. That’s exactly the opposite of what you’d want from a good metric.
Who gets left out of the universe you built by hand?
Nobody leaves people out on purpose. The universe narrows on its own, quietly, through the way it’s put together. A hand-loaded list is a snapshot of who you remembered the day you built it, and there are always predictable absences.
Left out are the people who joined after the campaign, sometimes dozens in a few months. Left out are the areas that are less visible to security: operations, the plant floor, the field sales force working away from the office, the teams that don’t live in front of an inbox but still touch data and systems. Left out are the contractors and third parties with access, who rarely make it onto the employee list even though they carry an identical risk. And left out are the people who changed roles and now handle information their earlier training never covered.
The result is a universe that resembles the organization, but trimmed exactly along the edges where the risk tends to sit. Training concentrates where attention already was, and the gaps line up with the least-watched places. The percentage, meanwhile, keeps showing green, because all it knows is the list.
What does compliance look like when the universe defines itself?
The difference starts before you count anyone: it’s in where the universe comes from. Instead of loading a list by hand once a year, the universe is derived from the source you already keep alive, the organization’s directory, and it updates when someone joins or changes roles. It stops being a decision made in January and becomes a reflection of who works at the company today.
On that basis, compliance is read with two numbers instead of one. Coverage tells you how much of your people are actually inside the plan; completion tells you how much of that plan has finished. A 100% completion over 30% coverage stops being good news and becomes what it always was: a signal that seven out of ten people are missing.
That’s how we understand awareness compliance at SMARTFENSE, as part of human risk measured across the whole organization and not over a hand-picked sample. It’s the same thing any auditor expects and what ISO/IEC 27001 has in mind when it talks about awareness: not a course delivered to some, but a program that covers everyone who should be covered.
It helps to compare it to a survey. If you only ask the ten people who were already going to agree with you, and you report the result as if it spoke for all two hundred, the number you get is flawless and means nothing. Compliance works the same way: a convenient sample is never the same as the whole.
The number that matters isn’t how many completed
A compliance percentage is only worth as much as the universe it’s calculated on. Measuring completion without measuring coverage gives a sense of control that holds until the day someone asks about the people who are missing, and by then the answer should already have been there. It’s the same mechanism by which compliance falls apart in March without anyone noticing: a number that’s looked at rarely and from a distance.
If you want to see your program’s compliance with both numbers in view, coverage and completion, over your real universe rather than a list, you can explore the platform or write to us for a demo on your own data.
Leave a Reply