When we decide to work on a security awareness topic, the first conversation tends to revolve around format, whether a video would work better or whether a game would be more engaging. It is a logical question, because format is the first thing people see and what they remember afterwards. That said, where does a piece that changes something actually begin? With the behaviour. Before deciding how it is told, we need to know what the person has to do differently once they finish it, and if that definition stays implicit, each person on the team imagines a different one and the piece ends up training a bit of everything.
That path, which starts with the behaviour and ends with a published piece, is called instructional design for security awareness. Here we will follow it in order, with a single example from start to finish, so that each step is applied to a concrete work situation.
What is instructional design in security awareness?
Instructional design is the process of deciding what a person should be able to do after a learning experience and of building that experience from there, with the content, the format and the form of assessment chosen according to that result. In security awareness it carries an extra demand, because what interests us is what the person does in their inbox, on their phone or in the middle of their shift. Those are situations with no test anywhere in sight.
That is why the starting point is a behaviour and the topic comes later. What is the difference? A topic is what the piece is about, such as multi-factor authentication or phishing, while a behaviour is what someone has to do differently in a specific situation. With a topic you can produce a piece that is correct from beginning to end and asks the person for no action at all, and with a behaviour the piece has a purpose that can be checked afterwards.
The example we will follow is a concrete, well-documented situation. Someone who has already obtained a person’s password tries to sign in to their account. Because the sign-in requires a second factor, an approval request they never asked for lands on that person’s phone. Sometimes a single one arrives and sometimes many arrive in a row, on the bet that at some point the person will tap “Approve” just to make them stop. MITRE ATT&CK, the knowledge base of attack techniques used by security teams, describes this technique and links it to what is known as MFA fatigue.
Why does starting with the format lead to pieces that change nothing?
When the format is decided first, the rest of the design adjusts to it. If it is already settled that it will be a two-minute video, the question becomes what fits into two minutes of video, and the answer tends to be everything that can be said about the topic in that time.
The example makes it clear straight away. A piece on multi-factor authentication that starts from the format explains what the second factor is, why it protects the account, which methods exist and how to switch it on. All of that is correct, and the person can still finish it without knowing what to do when an approval they never asked for arrives. That situation took up one line among many and never had the space of a practised decision.
The second effect shows up in measurement, because if the piece is not pursuing a behaviour, the only things left to measure are whether it was completed and whether the final questions were answered correctly. Those two numbers can look very good on a piece that changed nothing about what happens on people’s phones.
There is a third, quieter effect that has to do with the catalogue. When every piece is born from an attractive format, the programme piles up pieces on the same topic without any of them taking ownership of a specific behaviour, and it becomes hard to tell what is missing and what is surplus.
How do you write a target behaviour that can be observed?
A well-written target behaviour describes an action that someone could see or record, in a specific situation and with a criterion that lets us say whether it happened. If the sentence includes verbs such as know, understand or be aware of, we are still describing what goes on inside the person’s head, which matters a great deal but cannot be observed.
To write it, it helps to answer these questions in order:
- In what situation? The concrete trigger, as the person experiences it. In the example, a sign-in approval request that arrives when they are not trying to sign in anywhere.
- What do they do? The action, with a clear verb and no hedging. The person rejects the request and reports it through the channel the organisation has for that purpose.
- How do we notice? What would be recorded if the behaviour happens. In the example, a rejection in the authentication system and an alert that reaches the security team.
Putting the answers together, the behaviour is written in a sentence that the whole team can read in the same way: ”when an approval request arrives that they did not start, the person rejects it and alerts the security team”. MITRE’s own entry, in its mitigations section, puts it in almost the same terms when it recommends training people to accept only the requests they initiated and to report the ones they did not ask for.
A simple test is to compare that sentence with the one that usually appears in the initial request, something like “people should understand the importance of the second factor”. The second describes a good intention, which is why any piece on the topic seems to meet it. The first, by contrast, leaves out everything that does not serve that situation.
Once the behaviour is written, something that was hidden before also comes to light, which is the minimum the person needs to know to be able to do it. In the example it is only a few things. The person needs to know that a request they did not ask for means someone else has their password, that rejecting it is exactly what the organisation expects of them, and where to report it.
The history of the second factor or a comparison of methods may be interesting, but they are not needed for this behaviour and they take up attention the person needs for what does matter. So what stays out of the piece is decided with the behaviour in hand.
How do you choose the format that trains that behaviour?
Only at this point does the opening question return, and it arrives with a criterion it did not have before. The format that best trains a behaviour is the one that most resembles the moment in which that behaviour has to appear. In this article on comics, video and newsletters we went through what each format does well depending on the behaviour you are after, so here we will focus on how that criterion applies to the example.
What is the real moment like? It lasts seconds, reaches the phone without warning and usually interrupts another task, with very little text on screen to decide on. A long piece explaining multi-factor authentication from the beginning is far from that situation. A decision exercise that shows the notification as it actually looks and asks the person to choose what to do resembles it much more closely, because it makes them practise the same decision they will later have to make on their own.
If the programme includes games, that exercise can live inside one, as a challenge where you move forward by deciding well, and when the game is shared across departments it also adds the conversation that grows around it, which helps what was learnt to stick. A short video can go alongside it, for instance to show what a burst of requests looks like from the attacker’s side, and a well-told story helps people understand why someone would end up approving one. Neither replaces practising the decision, which is the heart of the piece.
Nudges play on a different field, because their strength lies in arriving just before the decision, and for that you need an event the awareness platform can detect. An approval request happens in the authentication system, so in this example the nudge has nothing to hook onto. Reaching that conclusion is also part of the design, because it avoids producing a piece in a format that cannot do its job.
When the catalogue offers several formats for the same topic, as is the case with SMARTFENSE content, which includes videos, comics, interactive modules, newsletters and video games, the target behaviour is what lets you choose between them. And since the texts, the imagery and the examples can be edited, the notification shown in the piece can look like the one the person sees on their own phone.
How do you decide, before producing it, whether the piece worked?
Measurement is decided before production, and the reason is practical. If we wait until the piece is finished, we will measure what the piece allows us to measure, which is usually whether it was completed. With the behaviour written down, on the other hand, the third question of the previous step has already recorded what we should see if the piece works.
In the example, the most direct signal lies in the alerts. If, after the piece is published, the reporting channel receives more alerts about requests nobody initiated than before, the behaviour is appearing. For that comparison to say anything, we need to look at the starting point before publishing, because without a before there is no way of knowing whether the after changed. If the organisation’s authentication system logs rejections, the team that runs it can add that figure and the comparison becomes more precise.
Validation questions still have their place, although they measure something else. A well-made question checks that the person understood what a request they did not ask for means, and that is a condition for the behaviour, but it does not guarantee it will appear when the notification reaches their phone on an ordinary working day. In this article on validation questions we explore what they measure and what they do not.
Some behaviours do not leave such a clear record, and in those cases it is still worth writing down what we would expect to see, even if it is an indirect signal, because the exercise helps check whether the behaviour is well defined. If we cannot think of any way to notice that it happened, we are probably still describing a topic.
The format question, in its place
Let us go back to the conversation at the start, the one about video or games. The question is still valid, and when it comes after the behaviour it can be answered with arguments, because we already know which situation needs practising, what the person needs to know to handle it and what we expect to see if they manage it.
In short, a security awareness piece is well designed when anyone on the team can say which behaviour it trains and how we will notice that it appeared. If, the next time we raise a topic, we start by asking what the person has to do differently, the conversation about format will still happen, only it will be shorter, because the behaviour will already have ruled out most of the options.
Leave a Reply