Black Hat 2019 Briefing Recap: Best-in-Class Security Engagement

Masha Sedova

Published on 5 September 2019


Last month, I unlocked a personal achievement, giving a briefing at Black Hat 2019 in Las Vegas, alongside Aika Sengirbay, former Security Awareness Manager at Autodesk, titled “It's Not What You Know, It's What You Do: How Data Can Shape Security Engagement.” You can get the slides here.

Our briefing covered how Aika launched a new security engagement program at Autodesk, partnering with Elevate Security, and deploying data-driven insights to create personalized training.

What Does Best-in-Class Security Engagement Look Like?

One of the things I liked best about conducting this briefing was that Autodesk’s story is more than just a single, idiosyncratic example of a security engagement program. Rather, Aika used a lot of established and emerging best practices to build her program, and the result is that Autodesk’s story illustrates what truly best-in-class security engagement looks like, and what tangible results it can achieve.

Here are the hallmarks of how the Autodesk security team built a best-class security engagement program:

  • They clearly diagnosed what was broken about their previous security awareness efforts and outcomes
  • They were driven by data
  • They tailored the program to Autodesk’s culture
  • They share the results of their program early and often

Diagnose What’s Broken

In the briefing, Autodesk shared the clear results of their diagnostic process in determining how the previous security awareness training was not meeting the needs of the employees nor the company. The previous training was merely compliance-focused, not personalized to an employee’s level of knowledge or previous training, and it was all unquantifiable – there was no way to tell if the training was actually resulting in fewer security incidents.

By taking the time to diagnose and articulate what was broken, it gave Autodesk’s security team a clearer mandate and set of measurable goals for what they wanted to achieve by partnering with Elevate Security.

(Looking for a framework to diagnose your security awareness problems? Check out the 5 Whys.)

Be Driven By Data

Determined not to repeat the mistake of building an unquantifiable program, Aika and the Autodesk team put data at the heart of every step of building the new security engagement program. For example, when they set the prioritized list of vital behaviors their program would measure against, they made sure each behavior, from compromised credentials to password management adoption, could be measured. Often this meant partnering with colleagues in the security org to get access to relevant datasets.

By setting measurable goals early, based on internal datasets as well as external sources like Verizon 2018 Data Breach Investigations Report, the Autodesk security had clear benchmarks to measure and report on the outcomes of their new program.

Tailor It to Your Company

Partnering with Elevate Security, Autodesk rolled out the Individual Security Snapshot – their customized internal branding for the Elevate Pulse service, which delivers a personalized security scorecards and tools into employees’ inboxes.

This meant cartoon dragons illustrating an employee’s security posture, a nod to the Game of Thrones dragons famously rendered with Autodesk’s software. It meant leveraging social proof to show employees how they stacked up against others in their department. It also meant that not only was each Snapshot email was unique to each employee, but also the resulting training needs were tailored to the given employee. Employees who showed competency on the prioritized security behaviors weren’t required to do further training, since it was evident to the Autodesk’s security team – and to the employees themselves – where an employee’s security behavior gaps were.

Share Your Results

Building on the best practice of being driven by data, Autodesk and Elevate were not only able to measure qualitative and quantitative results over time from the new program, but they widely shared those results internally with executives and externally (like at this very Black Hat briefing I’m recapping). I’m sharing some highlights in this post, but you can also see more in the briefing slides available here.

Qualitative Results
Behaviors Improved

I’ve mentioned this before – there’s growing momentum for companies recognizing that building a positive security culture is a necessary goal to keep companies safe from cyber risks. In order to help this wave crest faster, it’s incredibly important for security awareness practitioners to be transparent with their execs and with the larger security community about what best-in-class security engagement looks like. A big part of that is having measurable results to share with the c-suite, just like any other business unit would.

I’m so proud to have presented this briefing alongside Aika at Black Hat 2019. In fact, Aika and I will be giving an encore presentation of the briefing, this time as a live-stream webinar later in September, for those who weren’t in the room in Vegas. Sign up here to attend, and feel free to tweet any questions ahead of time to me @modMasha.