Ahead of their Award Showcase webinar on 4th November, Rebecca Schwartz, Director of Consulting Services at Plum eLearning, provides the background behind the games-based solution that won them two Silver Learning Technologies awards for "Best learning game" and “Best use of Social and Collaborative Learning Technologies”.
Google’s challenge
Google’s Security Education team wanted to reduce potential software vulnerabilities through Threat Modeling. Threat Modeling is an industry practice where engineering teams analyse the flow of data through products and services, considering vulnerabilities and creating mitigation strategies.
The “traditional” approach didn’t work
The Security Education team began conducting instructor-led training (ILT) sessions. They quickly realized that this approach had flaws.
- Scalability. Coordinating dozens of training sessions for hundreds of engineers across the country was daunting.
- Facilitation skills. Subject-matter experts (SMEs) recruited as facilitators weren’t necessarily capable of producing the robust, collaborative discussion that a productive Threat Modeling session requires.
- Participation. Participants didn’t take ownership of the brainstorming process but often waited to be told the answers. Or, one or two people might dominate the discussion while others stayed silent.
Pivoting to virtual training (VILT), as so many organizations had done during the pandemic, could help with scalability, but it wouldn’t improve the quality of the threat models produced during these sessions.
Gamification to the rescue!
The team came to Plum eLearning with a proposed solution: an online game. In the game, engineering teams would have to develop their project while also fortifying their defenses against common threats and vulnerabilities. Just like in the real world, only more fun!
Why the game’s design is effective
Gamification theory
In his excellent (and free) Coursera course, Professor Kevin Werbach of the Wharton School defines gamification as “the use of game elements and game design techniques in non-game contexts.”
A non-game context means that the game exists so that players can practice some sort of real-world skill, as opposed to just playing games at work. In this case, players don’t just do a simulation or use a fictional software project in the game; instead, teams bring their own real project into the game, and at the end can export the notes and action items from their discussion in order to fix real-world bugs and vulnerabilities.
The Threat Model game includes common game elements. Teams get a team name and players get avatars, and there is a leaderboard displayed at the end so that they can see how they fared compared to others. These elements were thoughtfully and carefully designed not only for fun, but also to simulate the competing priorities and challenges facing a real engineering team. During each turn, the team must allocate limited resources. They must build the project itself, but they also have to invest in defenses against security threats such as user abuse, or hackers.
Finally, game design techniques involve thinking about the visual experience, the player journey, and the game balance (not too hard, not too easy). The concept for Threat Model Colony was based on a loose metaphor: players are Mars settlers who need to establish a secure base. Visual graphics and animations enhance this theme, yet can easily be re-skinned in order to keep things fresh for teams that play multiple times. The scoring algorithms were carefully balanced; in order to thwart enemies, players must allocate their bots wisely. If they didn’t invest enough in each category, their defenses would fail. And if they didn’t invest enough in the colony itself, the project would fail to launch at all. The game also supported the player journey by onboarding teams with some basic instructions and a tutorial.
Addressing business challenges
The game was also effective because it addressed each of the issues that were presented with the ILT.
- Scalability. Teams could play the game any time they could schedule a couple of hours together, and there would no longer be any travel costs.
- Facilitation skills. Facilitators are no longer needed; the game walks players through the threat model process step by step.
- Participation. Players register so that the game can prompt each player, balancing participation with questions relevant to their role.
Results
After one year:
- 8,720 players from 1,744 Google teams played the game
- 40% of teams voluntarily extended gameplay
- 14,424 actions items or bugs were generated to decrease or mitigate real-world software vulnerabilities
These metrics suggest that players aren’t just learning about threat modeling as a concept, they’re truly putting it into action to make their products, services, teams, departments, and the entire company more secure.
Is gamification right for you?
If you want to consider gamification for your next employee learning experience, remember: just playing games at work isn’t gamification. Gamification as “the use of game elements and game design techniques in non-game contexts.” Ensure that you are incorporating solid game design principles, including linkage to real-world tasks, balanced challenge, visual elements, and managing the player journey.
Find out more in Plum eLearning’s Award Showcase webinar on Thursday, 4th November 16:-17:00 GMT UK.
This blog post first appeared on the Plum eLearning site where you can find more detail.
