Risk Management and Threat Scoring: a qualitative, quantitative, consistent and repeatable method

For new readers, welcome, and please take a moment to read a brief message From the Author.

Executive Summary

Threat Intelligence teams and Information Security teams often struggle to communicate to leadership why a specific vulnerability should be taken seriously or given more precedence, especially when the CVSS score is low. This blog is a brief explanation of how to use a threat scoring matrix to consistently evaluate threat actors, then combine that score with a CVSS score to communicate the true risk posed to your organization. It helps provide a measurable score that can easily be integrated with CVSS scoring systems, thereby also facilitating risk scoring automation. It also allows the leadership to make a well-informed decision as a vulnerability is considered within the context of a comprehensive threat actor profile seeking to exploit the vulnerability. We discuss the example matrix metrics below, and you may download the The Threat Actor Impact Scoring Matrix PDF from here [0]. It uses a five-point scale to scorer a threat actor based on nine criteria: determination, motivation, technical resources, financial resources, intelligence resources, skills/expertise, time they went undetected, team size, and time available to dedicate to malicious activities. It then scores the target (presumably you) on three criteria: probability of successful attack, stage of any technical exploit, and the attacker’s focus.

PART 1: Threat Actor Profile / Characteristics

Once upon a time, when I went and got this thing called a CISSP because some super smart cyb3r entity decided that a cert meant I was smrt……
There was an entire section dedicated to risk management, and it discussed in depth qualitative and quantitative methods for calculating risk. It taught you how to calculate values for assets six ways to Sunday, and how to score vulnerability. There is even this magical formula on how to calculate risk:

Threat x Vulnerability x Asset value = Total risk

And there’s this other magical formula…

Total risk – Countermeasures = Residual risk

Despite telling you to use a Threat (score) factor, they never teach you how to calculate/score a threat, or do threat modeling. So, I’ve tried to bridge that gap with the example matrix and the explanations below should help you understand how to grade each metric, and integrate it with your vulnerability scores for a better measure of a threat.
Determination is the measure of the threat actor’s courage or boldness. This score ranges from measuring if the are they easily scared off or are they brazen and hell bent on compromising you.
Motivation is also understood from the threat actor’s perspective as “What’s in it for me”, aka the WIFM. What is it that they are really after? Are they adolescents just screwing around in their free time, ready to show to their friends at the lunch table what they did to your website? Or are they a bigger threat, motivated by money or even worse, they seek to completely destroy you?

TECHNICAL RESOURCES refers to what kind of tools they indicate they are using. This is also closely related to the next criteria of Financial Backing. Are they downloading or only using free/open sourced tools indicating very low financial backing? Are they using tools that require purchase, licensing, or subscription or that indicate they’ve done some in-house customization? Did they leverage something that had never been seen before, sometimes referred to as an 0day (zero day)?

FINANCIAL BACKING is a little trickier to evaluate. It’s unlikely the threat actor is going to publish their banking records for you but you can interpret other factors, such as the one above to help make a selection in the matrix for this. If you find that an attack is only M-F from 5pm-10pm, well, they probably have a regular fulltime job and only do this in the evenings, indicating moderate to low financial backing. However, if this attack is M-F 8am-6pm, well, this is probably their fulltime job. If it is only on the weekends… well you get my point. Then there is the tools, and cost associated with them that you can consider as well.

INTELLIGENCE RESOURCES indicates your level of exposure. How could the attacker have known what they knew in order to launch the attack? Did they use knowledge that only someone inside could have known? This doesn’t mean it is insider threat, but it could indicate that someone is loose-lipped in chatrooms, social media, or at their local InfoSec meet-up and ignorantly disclosed something. Did they go after future marketing and merger information? Did they go after intellectual property for designs or your secret sauce? Or did they use something publicly available on a pastebin or at shodan.io or censys.io (that maybe shouldn’t be there in the first place)? Just because you think nobody should know it, doesn’t mean someone didn’t screw up and share it accidentally. Be very careful before making a selection here as it can skew your overall score.

SKILLS AND EXPERIENCE is calculated to some degree in the CVSS v3 scoring system under Attack Complexity (AC), but unfortunately the vulnerability evaluator is given a binary option of Low or High. Rarely in life is anything as simple as “easy’ or “hard”, thus to more effectively create a profile for the threat actor we consider his skills and experience. It may help to think of this metric based on the amount of forensic evidence left behind. Does it appear (or do you know) if the attacker has ever executed an attack such as this? Was it messy, in other words, did/do they leave behind a trove of forensic evidence or is it extremely difficult to track their movements and activities because they cleaned up logs extremely well?

TIME UNDETECTED is fairly straight forward, how long were they in your network, accessing an employee’s email, siphoning data etc, before you found them? Perhaps it is not your asset that they were in, and you are scoring a threat that has been published, and you recognize some similar characteristics in your data, you can use the information published as a reference point, and make your own assumptions, provided you document these assumptions as they may need to be changed in the future. Side note: if you document what you’ve assumed, and you change it later because you find your assumption was inaccurate, it helps the organization from making the same mistake later once you are gone.

TEAM SIZE again is something that can be calculated a number of different ways, how you define the scale is up to you, but the example gives you more than just numbers for your calculation as it takes into account the “tech savvy” level. It is kind of like saying, I can get one rock star in a sports draft for x-dollars or I can get three solid guys for x-dollars. Team sizes are just straight forward numbers, and there should be some wiggle room for scoring to account for this.

TIME is the final characteristic metric and it represents how many hours per week the attacker(s) are putting into this. It is not specific measurement of how many hours did they launch a ping sweep, rather how much time did they have to put into planning, reconnaissance, and execution as well as actual attack design, launch, and monitoring.


PART 2: Threat Target Score

This takes into account the impact of the attacker attacking you, whether or not there is an exploit (and its stage), and who the actor is targeting.

Probability of Successful Actor Attack measures how likely are they to succeed in compromising you? If they are highly skilled, well-funded, and malicious activity is their full-time job, they are probably more likely to succeed than fail. In some cases. You might determine this score, mathematically based on the average score of the items above. Another alternative is to calculate this based on the CVSS score and the threat actor’s score above. However, you choose to do it, document your decision, and provide instructions if it is to be done mathematically based on the other factors.

Technical Exploit measures the stage of an exploit. In the CVSS v3 scoring system, this is captured in the temporal scoring metrics.to some degree, but it is based on whether or not the exploit code works in most versus all situations. In this matrix, we are concerned about how an exploit is being used once it exists because we are scoring the threat, not the exploit or the vulnerability. The top two tiers of this metric measure whether or not the exploit is in the wild and is/isn’t organic to your technical eco-system.

Non-technical is a metric for measuring who is being attacked. Is the attacker using a spray-and pay approach, indiscriminate, or doesn’t seem to care who is affected so long as someone is affected? Or is the threat more focused to a geographic region or country, your industry (tech, travel, biomedical, manufacturing, power plants etc), or your immediate peers?

Using the Matrix Wisely

This threat actor matrix should be carefully analyzed before you choose to implement it in your organization as what is represented here may not fit your organization’s risk tolerance. You may need to tighten it down, making it more stringent, i.e. your organization may consider someone motivated by monetary gain to be a threat level higher than someone motivated to induce change. The point is, don’t just take this and start using it until you actually understand and have considered each definition of each characteristic. At the bottom of the matrix you will find one way you can use the Threat Actor Impact Score, and Threat Target Score to create an Overall Threat Score. Then, you can combine this with the CVSS Score ascribed to a vulnerability (I recommend using the one provided by the vendor whenever possible as they know their software best). Together, you can present a holistic risk score to your leadership that represents the vulnerability severity within the context of a specific threat actor. You will probably be surprised to find that low and moderate CVSS-scored vulnerabilities warrant more attention than you realize.

Because Math…

If you’re a math whiz, you probably noticed that I have “divide by 60” to calculate the overall threat score, then when using it with CVSS I turned around and multiplied by 10…. Well Smarty Pants 😊 good for you for noticing that I could have just divided by 6 in the first place, but this was built on the Keep It Simple Stupid (KISS) principle, kind of like developers putting in lots of comments for their code so that other people can understand what they were thinking or trying to accomplish.

If you choose not to use one of the metrics, and you eliminate it all together, make sure at the bottom, you reduce the Out of ## to reflect the maximum possible points remaining so that your score doesn’t get skewed.

This is not a one-and-done process. You will need to review the score for your threat actor profiles regularly (I recommend quarterly, semi-annually at minimum). Small changes in how you do business can have unexpected consequences (both good and bad) that will impact how you score these threats. They may also change the priorities of asset values and what is deemed a critical asset.

Thank you all for reading this. Being the holidays, I haven’t asked anyone for a proof reading so if you find typos, errors or corrections needed please email me at 447cc8c9 \at\ opayq.com.
Merry Christmas and Happy New Year to all of you.

[0] https://github.com/grcninja/Blog_Docs/blob/master/THREAT%20ACTOR%20IMPACT%20SCORING.pdf