We tend to trust what once worked, and something similar happens with security awareness content. If a module was completed without complaints and the results were good, it comes back in every campaign with the same confidence as on the first day. The problem is that content doesn’t warn you when it stops describing the reality of the person receiving it. Can it be spotted in the report? Hardly, because the module is still completed, the questions still have a correct answer and the indicator stays green.
When we talk about expiry in a programme, the first thing that comes to mind is the expiry of each person’s training, which runs out on a set date and has to be covered again. Here we will deal with the other clock, the one that belongs to the security awareness content itself, which has no expiry date written anywhere and still stops working.
Why does security awareness content age without anyone noticing?
Content ages when what it describes drifts away from what the person experiences at work, even if every sentence is still correct. That gap grows little by little, because the outside world and the organisation change while the text stays still. Along the way, there is no moment that forces anyone to look at it again.
The difficulty is that the usual metrics measure whether the person completed the content and whether they answered the questions correctly, and both still work with old content. If a module teaches people to recognise a phishing email by its spelling mistakes, the validation question about those mistakes has its correct answer and most people get it right. What the number doesn’t show is that today any generative AI tool writes that same email without a single error, so the person passes a lesson that helps them less and less.
On top of that, content that worked well has an advantage that protects it, because touching something that caused no problems seems like an unnecessary risk. It gets reused in the next campaign, added to the material for new joiners and ends up as part of the programme out of habit. What worked once earns a permanent place, and that place is never questioned again.
What signs show that a piece of content has stopped working?
The signs show up in two different places, and it is worth looking for them separately because they aren’t noticed by the same people.
The first family is outside the organisation. An attack technique changes shape, like phishing that no longer gives itself away through its wording, or a new channel starts being used for the same thing that used to arrive by email.
Official recommendations change too, and passwords are a good example. For years, content asked people to combine upper case letters, numbers and symbols and to change their password every so often. Today the NIST digital identity guidelines (SP 800-63B) state that periodic password changes must not be required and that composition rules must not be imposed, and that a change is forced only when there is evidence that the password has been compromised. Content that still teaches the old rule is teaching a habit that the guidelines themselves have left behind.
The second family is inside, and it is usually the one that slips through most. These are each organisation’s own details that a piece of content mentions and that change more often than we imagine:
- The reporting channel, which moved from an email address to a button, or from one team to another.
- Tool names, when the content talks about a system that has been replaced.
- Departments and roles, after a reorganisation changed who is responsible for what.
- Internal processes, such as how a payment is approved or how access is requested, which the content uses as an example of what needs checking.
When one of these details goes out of date, the damage goes beyond the detail itself, because the person reading it discovers that the content doesn’t know their job, and from then on they believe the rest of it less.
The people who receive the content are usually the first to notice. A question that keeps coming up, a query about a procedure that no longer exists or a comment about a tool that has stopped being used are signs worth recording. Whoever raises them is comparing the content with their daily work, and that comparison is more precise than any review done from the outside.
What ages faster, predefined content or in-house content?
Content written within the organisation ages faster, even though intuition usually says the opposite. Why does this happen? Because in-house content is written to speak the organisation’s own language, so it carries the names, channels and processes that change most. What makes it feel close when it is written is the same thing that makes it outdated later.
There is also a maintenance issue. Predefined content has whoever produces it behind it, and they make sure the attack technique it describes is still the one in circulation. In-house content, on the other hand, is usually written for a one-off campaign and is left without anyone responsible for it once that campaign ends. Work moves on elsewhere and the piece stays active in the catalogue, with the details from the day it was written.
One way to reduce this problem is to separate the data from the text. In SMARTFENSE, content is tailored to each organisation through organisational variables. A detail such as the number for reporting incidents is defined in a single place, and every piece of content that uses it shows the new value when it changes. Variables can also be used in custom content, which makes it much easier to maintain, because a change of channel or name is made only once for all the pieces that mention it.
Update, replace or retire?
Once we spot a piece of content that has aged, the next question is what to do with it, and there are three possible answers. To choose between them, it helps to look at what changed.
If what has gone stale is a detail or an example, and the central idea still holds, the content is updated. This is the most common case and the simplest, because it is enough to change the name of the channel, replace the screenshot of a tool or pick an example that resembles what lands in the inbox today. In SMARTFENSE, the texts, the image and the examples of every piece of content can be edited, so there is no need to start from scratch to fix a detail.
If what changed is the idea itself, the content is replaced. A module built around recognising writing errors can’t be fixed by changing a paragraph, because its whole logic rests on a signal that is no longer reliable. In those cases it makes more sense to teach the method, such as verifying the request through another channel before acting, than to look for the new visible signal that will also become obsolete over time.
And if the content no longer speaks to anyone, it is retired. This is the decision taken least often, because retiring something feels like losing work already done, and a large catalogue gives the impression of being more complete. That said, every piece of content that stays active takes up a share of people’s time and attention, which are limited.
Deciding what not to teach is as much a part of designing the programme as deciding what to teach. When a piece doesn’t respond to any real situation in the organisation, what it adds is noise. The same happens when two pieces of content cover the same topic and one does it better, and keeping both only splits attention between them.
When should the catalogue be reviewed?
The temptation is to set a date, for example an annual review of the whole catalogue, and consider the matter settled. Does it work? Partly, because it forces us to look at everything at least once. Even so, it comes too late for what changes during the year and looks at every piece with the same attention, when most of them had no reason to age.
It pays off more to review when something the content mentions changes. A change of tool, a reorganisation, a new reporting channel, an incident or a regulation coming into force are concrete moments to ask ourselves which pieces in the catalogue talk about it. The annual review is then left to cover whatever those changes didn’t trigger.
For this to work, each piece of in-house content needs a person who answers for it, in the same way that every important document in the organisation has someone who maintains it. That person only needs to know that the content exists, which organisational details it mentions and who to ask when one of them changes, without having to rewrite it every time. If the programme is already organised by audience, that same division helps share out the review, because each audience has its own content and its own changes.
Finally, it is worth listening to what comes from the other side. If the organisation has a channel where people can ask questions or comment on what they receive, whatever comes up repeatedly there is review material. Just as a report that gets no response ends up switching off the reporting habit, a comment about outdated content that leads to no change teaches the person that it isn’t worth speaking up.
When the indicator stays green
Let’s go back to the module from the beginning, the one that was completed without complaints and comes back in every campaign. The green indicator tells us that the person went through the content, and that is still valuable. What it doesn’t tell us is whether that content still describes the email they will receive tomorrow or the channel they need to use to report it when it arrives.
In the end, a living catalogue is measured as much by what gets added as by what gets corrected and retired. If the most used content in your programme hasn’t been reviewed in a while, this may be a good moment to pick one piece and read it the way someone who joined the organisation last week would. Because the content that gives us the most confidence is precisely the content we stop looking at.
Leave a Reply