Making Better Security Behavior Training: How Autodesk Did It

Masha Sedova

Published on 30 March 2019


The dust is finally settling after the RSA Conference in San Francisco earlier this month. I was privileged to speak at RSA alongside Elevate customer, Aika Sengirbay, Sr. Information Security Engagement Specialist at Autodesk, as part of the “Human Element” track. Autodesk has recently rolled out the Personal Snapshot, part of the Elevate Platform, and Aika and I shared with the crowd of security leaders and practitioners about how Elevate partnered with Autodesk to completely redesign their approach to security training. Instead of just throwing more training at the (human element) problem, Aika and the Autodesk team were empowered to launch a data-driven solution with behavioral science at the core, to influence and measure actual security behavior change.

The Old Way Was a Broken Way

Aika shared her frustrations with the previous state of security training that prioritized “check-the-box” compliance. The content was basic, and the same for everyone – a one-size-fits-all approach that didn’t allow for where an employee was in their security skill level or if they’d had prior training. The training content was stagnant and unpersonalized, year after year, causing user frustration. And you couldn’t tell if it was actually effective, only if the compliance requirement of training completion had been met.

The old state of training, unsurprisingly, yielded poor results. Employees were disengaged, and still making poor security choices, staying true to the oft-cited stat that 95% of breaches are caused by human error. And because the training was unquantified, there was no way of tracking whether changes in security incidents were related to the human element.

Applying the Autodesk Way of ‘Making Better’ to Security Behavior Training

Aika cited Autodesk’s commitment to “Making Better” as an ethos she applied to evaluating and redesigning their security training efforts. Working closely with Elevate, the Autodesk team set clear goals for how they wanted to make their security training program better – they wanted the program to be relevant to people’s actual security habits, and recognize the employees existing security skill level. They wanted to respect employees’ time, not bogging them down with unnecessary training. They wanted to reward employees’ progress, and motivate further improvement.

With all this in mind, Autodesk deployed the Personal Snapshot company-wide. The Personal Snapshot is an email scorecard with personalized feedback and rewards to help employees improve their security behaviors, plus dashboards for the security team to visualize and analyze the company’s security posture.

We designed the Personal Snapshot to focus on prioritized security behaviors. We helped Aika and the Autodesk team select the desired behaviors, digging into research like the Verizon 2018 Data Breach Investigations Report, and looking at which behaviors could be easy wins, or would have the biggest impact on the company’s security posture. Together, we set the top priorities as decreasing clicks on phishing emails that could compromise credentials, increasing password management adoption, and increasing reporting of suspicious emails, as well as maintaining training completion when an employee needed it.

I shared with the crowd how we worked with Autodesk to source the datasets to measure and verify the prioritized behaviors, and ingest the datasets into the Elevate Platform, whether they come from internal testing data, Autodesk’s learning management system, or their device management platform.

The Snapshot email itself is tailored to the Autodesk context. Employees receive a dynamic score of their security posture, represented by whimsical dragons, in a nod to the much-scarier dragons in the hit TV show Game of Thrones, which were created with Autodesk software.

Dinosaur ratings for individual security postures

The Snapshot also leverages the powerful motivator of social proof, showing employees their individual strengths and weaknesses in relation to their company and team, and using a “celebrity endorsement” of sorts by reminding them that Autodesk CEO Andrew Anagnost uses their chosen password manager.

When employees aren’t adopting one of the prioritized behaviors, the email links them to tailored training content to help them improve. And Snapshot also rewards achievement, displaying the badges the employee has unlocked with their improved security behavior.

Making an Impact

At launch, 60% of employees opened their Personal Snapshot – compare that to typical marketing email open rates around 20%. For the several hundred recipients who answered a follow-up survey, more than 83% felt motivated to change their security behaviors.

The Autodesk security team (and by extension, Elevate!) basked in the kudos they received from their colleagues. One employee, picking up on the social proof nudging, had replied, “This is the best security email ever! I really love the initiative. Clear, gives me an idea of where I stand and helps me see what else I can do. Plus, I can rub my colleagues’ noses in it a bit :P” The executive suite took notice, as well. The CEO wrote to the team, “This is awesome” and the CIO lauded the launch, writing, “Cool stuff. Congrats!”

This is the best security email ever! I really love the initiative. Clear, gives me an idea of where I stand and helps me see what else I can do. Plus, I can rub my colleagues’ noses in it a bit :P
— Autodesk employee

But what about actual change in the behaviors we had prioritized for Autodesk? The team observed a 26% increase in password manager installation in 48 hours after launch, and 1800+ employees engaged proactively w/ training resources linked in their personalized emails. The team has also seen clicks decrease in phishing tests by almost 2x, as well as a 170% increased reporting of suspicious emails. Aika estimates Personal Snapshot has saved nearly 1000 hours of unnecessary training: since the security team started measuring baseline behaviors, those employees who clearly didn’t need training were able to skip it, thereby fulfilling their goal of respecting employees’ time.

Looking Forward

Collaborating closely with Elevate, Aika and the Autodesk team have set ambitious goals for the future, including replacing their annual security training, addressing more complex datasets through Snapshot, and launching a champion program. Aika also shared how other companies can also overhaul their existing security awareness training in favor of data-driven security behavior programs.

Her advice:

  • Find your top 3 behaviors
  • Find the data sources for them – be open to partnering with other security teammates, and starting small and expanding
  • Do trend analysis (in spreadsheets, Tableu, or with Elevate)
  • Find culturally engaging ways of communicating findings (e.g. leaderboards, intranet sites, emails)
  • Reward top behavior and focus on the bottom

Looking out at the crowd at RSA, I was incredibly proud to stand alongside a pioneering customer like Aika and the Autodesk team and share their results with the community. More and more, CISOs and security awareness practitioners are looking for a new way of building positive security culture, to engage every employee in the fight against cyber threats, and I hope we won’t limit these conversations just to big stages like RSA.

You can check out the slides from our RSA talk here, and find me on Twitter, @ModMasha to let me know what you think of Autodesk’s journey, and how you’re pursuing security behavior change at your company.