This paper describes Practical Threat Analysis(PTA); a well-structured methodology implemented by a software CASE tool that assists analysts and developers in assessing system risks and building the most effective risk reduction policy for their system. Read a customer case study to see how PTA works in real-life.Click here for a copy of the free PTA risk assessment software click here to download the free risk assessment software

What is practical threat analysis?

The Dark Tower had been rebuilt, it was said. From there the power was spreading far and wide, and away far east and south there were wars and growing fear. Orcs were multiplying again in the mountains. Trolls were abroad, no longer dull-witted, but cunning and armed with dreadful weapons. And there were murmured hints of creatures more terrible than all these, but they had no name.
J.R.R. Tolkien. The Lord of the Rings

Practical threat analysis identifies threats and defines a cost-effective risk mitigation policy for a specific architecture, functionality and configuration. It involves the mapping of assets, modeling of threats and building of a mitigation plan that lowers system risk to an acceptable value. The mitigation plan is composed of countermeasures that are considered to be effective against the identified threats.

When should threat analysis be applied?

Threat analysis methodology is required for complex software systems that integrate multiple infrastructures and technologies. It is also needed in customized application solutions that are built on standard products as well as in all other cases where it is unacceptable to implement pre-compiled "to-do" lists provided by a software vendor or standards committee.

Threat analysis should be used in the earliest possible stages of system design and thereafter as an ongoing process throughout the system lifecycle of development, integration, change and problem management.

The problem

Software development is always constrained by some combination of budget, time and resources and threat analysis usually ends up as a task to be done "later". Threat analysis is a skill that typical programmers and their managers do not usually have which results in the task being done "never".

The solution

By using PTA, analysts who are expert in the application domain can quickly build and analyze risk management models and policies without endangering the project schedule. Knowledge is retained, shared and maintained within the group and program management has total transparency to system risk without the need for additional resources.

What are the existing tools?

Word-Processor + Spreadsheet Documents - The analyst has the freedom to describe threats and vulnerabilities and express her analytical qualification in a free format with no restrictions dictated by the tool. However, the overhead of the data management and the calculation tasks is very high because of the lack of a built-in ability to represent the interrelations between entities and to dynamically alter the threat model. In reality the data model required for threat modeling is far beyond the capabilities of spreadsheet programs. In addition, most of these solutions also lack the necessary reporting functionality.

Checklist-Based Tools - These are tools that provide pre-defined sets of security recommendations that are used as checklists. This approach may work for standard applications where all possible security issues are known in advance. Most of these tools have reporting capabilities and usually come in two flavors:

• Questionnaire-based in which the user is asked to answer a series of questions that reflect the embedded checklist.
• Template-based in which the user is asked to distinguish the specifics of her application from the standard checklist

Since this type of tool is based on lists of general purpose standard countermeasures, they are not flexible in supporting and encouraging the analyst to create new threat scenarios that are specific to her application.

Threat Modeling Tools -Microsoft's tool combines Schneier's Attack-Trees methodology with standard Microsoft Threat Classification and has four important limitations:

• Doesn't relate threats to financial losses to assets caused by the attacks and does not rank countermeasures by their effectiveness.
• Uses "pre-defined" cases and doesn't easily fit application-specific threat scenarios
• Doesn't provide a complete system view for threat analysis risk management.
• Limited reporting capability

Introducing Practical Threat Analysis (PTA)

The PTA methodology and CASE tool enable effective management of operational and security risks in complex software systems by an existing team. It provides an easy way to maintain dynamic threat models that are capable of reacting to changes in system's assets and vulnerabilities. With PTA an analyst can maintain a growing database of threats, create documentation for security reviews and produce reports showing the importance of various threats and the priorities of the corresponding countermeasures

PTA automatically recalculates threats and countermeasures priorities and provides decision makers with updated action item lists that reflect the changes in threat realities. Countermeasure priorities are expressed as a function of system's assets values, degrees of damage, threat probabilities and degrees of mitigation provided by countermeasures to the threats.

A software development team uses PTA from day one of design and throughout the system lifecycle. PTA provides intuitive and easy ways for iterative interaction between threat analysts and developers. It supports a collaborative process of evaluating threats risks and ranking the cost-effectiveness of proposed countermeasures. The team's "threat analyst" can be the program/product manager, system architect or development lead who can start being productive with the CASE tool within hours.

How does PTA relate to standards and initiatives, such as ISO17799, BS 7799, SSE-CMM, Octave, FIPS 199, GAISP, COBIT and others?

PTA complements existing standards and appraisal procedures by supplying means for the actual definition of threats, vulnerabilities and proposed countermeasures. It manages a well designed database of all relevant security entities and enables production of documentation for the evaluation procedures required by the standards.

Standards recommend procedures for organizations to follow in order to ensure information systems security. These recommendations include mapping of assets, vulnerabilities, threats and countermeasures, assessment of risks implementation of risk mitigation plans. PTA provides the actual means for performing these tasks in a productive way.

Some standards provide lists of numerous recommended countermeasures that should be implemented. These lists may serve the analyst as a baseline of definitions of common vulnerabilities and countermeasures and may help him in grasping the terminology. PTA enables the integration of these entities in its database. However it should be noted here that the standard lists cannot cover the most intimate aspects of customized solutions and the specifics of complex systems that integrate several technologies. At best, compliance with standards provides only the baseline security and additional analysis of application-specific risks is required.

PTA may also serve as the foundation of Information Security Management System - a concept that is promoted by modern standards. Its growing database and statistics may be used as an evidence of the organization's efforts for constantly improving the process of threat and vulnerability analysis.

Setting the PTA terminology

In this section, we define PTA security-related terms in order to help the reader to easily understand the PTA data model and the PTA calculative method which is described later.

Main Entities

Vulnerability is a weakness, limitation or a defect in one or more of the system's elements that can be exploited to disrupt the normal functionality of the system. The weakness or defect may be either in specific areas of the system, its layout, its users, operators, and/or in its associated regulations, operational and business procedures.

Countermeasure is a procedure, action or mean for mitigating a specific vulnerability. A specific countermeasure may mitigate several different vulnerabilities. In some standards documentation, countermeasures are called "controls" or "safeguards".

Asset is information, capability, an advantage, a feature, a financial or a technical resource that may be damaged lost or disrupted. The damage to an asset may affect the normal functionality of the system as well as of the individuals and/or organizations involved with the system.

Threat is a specific scenario of a sequence of actions that exploits a set of vulnerabilities and may cause damage to one or more of the system's assets.

The threat described in Figure 1, causes damage to Asset-1 and Asset-2 and exploits two vulnerabilities: Vulnerability-1 and Vulnerability-2. Vulnerability-1 is mitigated by Countermeasure-1 and Vulnerability-2 is mitigated by Countermeasure-2 and Countermeasure-3 as noted by the blue arrows. Since a threat may exploit several vulnerabilities, the set of possible countermeasures that might mitigate a threat is completely defined by the set of vulnerabilities used in a threat scenario and is noted by the green arrows in the scheme.

Asset's Value is the financial value of an asset that is destroyed of stolen. Assets may be digital (software source, physical (a server) or commercial (a corporate brand).

Threat's Damage to Asset - financial value of damage caused by a specific threat to a specific asset.

Threat's Probability is the likelihood that the threat scenario will materialize. In some documentation the threat's probability is characterized by the term "Annual Occurrence Rate" (AOR).

Threat's Risk is a quantified measure of the likelihood of loss and/or damage that may be caused to one or more of the system's assets due to the specific threat. In some documentation the threat's risk is called "Annual Loss Expectancy" (ALE).

Threat's Recommended Countermeasures is a set of all the possible countermeasures that reduce the threat's risk. This set is based on the countermeasures that mitigate the threat's vulnerabilities.

Threat's Actual Countermeasures (AKA Threat's Mitigation Plan) is a subset of threat's recommended countermeasures that is assumed to be the most effective for mitigating a specific threat. The decision which of the recommended countermeasures will be included in the Threat's Mitigation Plan is made by the analyst, who uses his expertise to decide which countermeasures are most effective when applied together.

Countermeasure's Cost is the financial value that is associated with the implementation of a specific countermeasure.

Countermeasure Cost-Effectiveness is the degree of mitigation introduced by a specific countermeasure to the overall risk in the system in relation with the cost of implementing this specific countermeasure.

Additional Entities and Terms

Attacker is a person (or group of persons) that may perform the steps of a specific threat scenario.

Attacker Types are the various classes of attackers that are differentiated according to their motivation, qualification, available attack tools and their accessibility to the attacked system's resources.

Entry Points are the "doors", either in the system itself or in the human operation associated with it, that are used by attackers to penetrate the system, e.g. Web site, IVR service, SMS server, CRM representatives called by customers over the phone etc.

Area Tags are descriptive tags that are relevant to assets, threats, vulnerabilities and countermeasures. Classifying the various security entities in the threat model according to their areas improves the readability of complex threat models.

Threat analysis steps

1. Prerequisites

As mentioned previously, the threat analyst identifies system vulnerabilities, predicts even the most hypothetical threat scenarios and evaluates threat probability and risk to enable prioritizing the corresponding countermeasures.

Before starting out, the analyst should learn the system functionality and architecture. The in-depth understanding of the system is of crucial importance for the correct identification of system vulnerabilities and the building of possible threat scenarios. The following documentation is needed.

• Functional description of the system including all typical use cases
• Architectural diagram of the system
• Documentation for various system modules

These documents must be detailed enough to be used as reference for the decisions regarding the applicability of various threat scenarios to the analyzed system.

2. Preparing a List of System Area Tags

It is a good idea to prepare a list of relevant system' area tags that will help the analyst in mapping the various entities according to a variety of properties e.g. define area tags for each of the systems components that will enable the association of vulnerabilities with system's architecture areas

3. Identifying System's Assets

The correct mapping of assets, their financial value and the evaluation of financial loss to the system owner when these assets are damaged or stolen, is one of the most critical tasks in the threat analysis process. The assets value is used as the basis for calculating threat risks and countermeasures priorities.

An analyst may at times hear claims like "everything we have is important". While this could be true for some systems, we believe it is not the typical case. It is more likely that assets need to be clearly prioritized. Consider, for example the following partial list of a financial institution assets:

• Office equipment such as printers
• Confidential information about institution's clients
• Clients' money
• Private keys used for authentication of transactions
• Master key used for generating private keys

The accurate assessment of the financial value of the damage that may be caused by losing each of the above assets will enable the correct classification of assets according to their importance to the institution and help avoid a situation where the institution invests resources in protecting printers while leaving the master key unprotected.

In some cases the value of assets is less intuitive especially when they are intangible. For example, the confidence of the public in an electronic trading system may be damaged by the appearance of non-relevant texts on the system's Web site. No money is lost, no information is disclosed, all technical resources are still functioning but the site reputation and the trust of the shoppers are shaken. An indirect financial loss should be set for this type of damage.

Due to the importance of asset mapping, we recommend that the asset list and corresponding values be periodically checked by non IT personnel e.g. the company's CFO, marketing officers and legal consultants. An analysts can quickly do "what-if" analysis by modifying asset values and obtain insight as o the model's accuracy and completeness.

In practice, it is often easier for the analyst to identify system assets via the process of analyzing specific threats (as described in step 8). It is a fact of human nature that we don't realize how valuable things are until we lose them. This implies an iterative approach of mapping assets and threats.

4. Identifying System's Vulnerabilities - the real ones

Identifying vulnerabilities requires the analyst to be intimate with the system's functionality, architecture, implementation and deployment details. The analyst should also be familiar with business and operational procedures and the types of users and other parties that are involved in system operation.

An analyst can use Google to find generally known vulnerabilities as published by software vendors and security consultants. Most of the items in these check lists are, in many cases, irrelevant to the specific system or may be easily solved by a simple comprehensive routine such as "always install most updated vendor's security patches". The thing that should concern us here is that such a list will draw the attention of the analyst away from the real vulnerabilities that are specific to the system that is being analyzed.

Therefore we highly recommend that the analyst should investigate the system architecture and implementation details and collaborate with architects, developers, installers and support engineers as well as with the business managers of the system to discover the real vulnerabilities that are unique to the system and that probably may not be identified without this intimate knowledge. From experience - the most severe vulnerabilities reside in the interfaces, junctures and stitches between the various elements in complex systems and rarely appear in the standard lists.

As mentioned before, the identification of the relevant vulnerabilities is a continuous iterative task that is associated with the step of identifying threats (step 8 below) - sophisticated vulnerabilities may be identified when building threat scenarios.

5. Identifying Countermeasures

• A list of countermeasures that protect vulnerabilities. The list includes the implementation cost of each countermeasure and the countermeasure's relevant area tags. If the countermeasure is already applied it should be marked as 'already implemented' to enable producing updated statistics of the total risk level in the system.
• A map of the relationships between countermeasures and vulnerabilities. This map shows which vulnerability may be mitigated by a specific countermeasure. Sometimes a countermeasure is introduced as a solution to a specific vulnerability, but after additional consideration it turns out that it may help in mitigating other vulnerabilities too.

The accurate identification of countermeasures and their relations with vulnerabilities is the basis for building risk mitigation plans as described in the next steps.

6. Classification of Potential Attacker Types

Classification of the relevant attacker types may be helpful in focusing the analysis on practical realities. The classification of attackers is useful when we can clearly relate each of the threats with one or more of the attacker types.

Attacker type's data includes the understanding of his motivation as well as his qualification, available attack tools and his accessibility to the system's resources. Special care should be given to classification of 'insiders' attacker types since their activity may be very dangerous.

A good starting point may be to define an attacker type for each of the user roles that appear in the system use cases and reserve few more attacker types to hackers and other types of bandits.

7. Identifying System's Entry Points

The best tactic for this step is to review the list of attacker types and document every possible way the potential attackers could access the system. List of entry points may be revisited and clarified while analyzing threats.

8. Building Threat Scenarios and Mitigation Plans

This is the most important step of the threat analysis process. The outcomes are:

A list of system's threats

• A map of the relationships between threats and area tags, assets, attacker types, entry points and vulnerabilities
• An evaluation of the total damage and risk parameters for each of the threats
• Mitigation plans and residual system's risk data

Since threats are the most complex entities in the model, the process of identifying and constructing threat's elements and parameters has a 'decomposition' flavor. During this process the analyst will have to return to previous analysis steps in order to create missing entities, such as assets and vulnerabilities that are referenced by the threat that is constructed. In the following we describe the sub-steps of building a threat scenario and a mitigation plan for a single threat.

Initializing threat - start from name and description

Always start by giving the threat a name and a short textual description (a few sentences) that includes the actions taken by the attacker and the description of the impact the threat has on the system. The description will be used as reference for the following steps and will be refined during the process.

Identifying the damaged assets and the damage levels

Identify the list of assets that may be damaged by the threat and the maximal damage level that may be caused by the threat to each of the assets. This is needed in order to enable the automatic calculation of the total damage (the total financial losses) that may be caused to the system if the threat materializes.

Setting the threat's probability

Threat's probability is the likelihood of the occurrence of a specific threat scenario at least once a year. Its value may be in a range of 0 - 1 where 0 means that threat will never materialize and 1 means that threat will definitely materialize at least once a year. The threat's risk value is automatically calculated based on the threat's total damage and the threat's probability.

Identifying the exploited vulnerabilities

The correct identification of the vulnerabilities that are exploited by the threat scenario is important for building risk mitigation plan since the identified vulnerabilities define the collection of proposed countermeasures that are available for mitigating the threat. When vulnerabilities are identified, the list of available proposed countermeasures is populated automatically.

Building a mitigation plan for the threat

Building a mitigation plan is involves the selection of the most effective combination of countermeasures from the list of all the proposed countermeasures for the threat. The decision which of the proposed countermeasures will be included in the actual mitigation plan is done by the analyst according to her expertise and experience. She will decide upon the group of countermeasures that is the most effective in mitigating the threat - the threat's mitigation plan.

Constructing a mitigation plan is a matter for experts and no simple rule for finding the correct combination of countermeasures can be given. Moreover, the actual mitigation provided by all countermeasures in the mitigation plan is not necessarily equal to the sum of the individual mitigation provided by each of the countermeasures. In order to assist with the computation of countermeasures' contribution to mitigating the threat's risk, the analyst is asked to a) estimate the mitigation level that each countermeasure provides to the threat as if it is the only countermeasure in the mitigation plan and b) estimate the total level of mitigation provided to the threat's risk by all countermeasures in the mitigation plan.

Identifying relevant attacker types

Associating threats with their relevant attacker types may be helpful in re-assessing the threat's scenario and probability, the exploited vulnerabilities and the potentially damaged assets. All threat's elements should fit with the attacker profiles - their qualifications, motivations and accessibility to resources.

Specifying area tags

Associating threats with their relevant system's areas tags may be helpful for validating the mapping of the threat scenario on top of the design documents that are used by the analyst. The area tags properties may be used later for viewing risk and mitigation statistics grouped by system's areas.

Identifying relevant entry points

Associating threats with their relevant entry points may be done in correlation with the vulnerabilities exploited by the threat and the attacker types and the area tags that are associated with the threat. Each of the three last steps should be used by the analyst to validate the threat scenario and to help in evaluating the threat's risk and the effectiveness of the proposed countermeasures.

9. Learning the results

The following are the outcomes of the threat analysis process:

• List of system's threats sorted by their risk
• List of system's threats sorted by the financial damage they cause
• List of individual countermeasures sorted by their overall risk mitigation effect
• List of countermeasures sorted by their cost effectiveness (mitigation divided by implementation cost)
• Maximal financial risk caused to each asset by existing threats
• Maximal financial risk caused to each asset by existing threats after all mitigation plans are implemented
• Maximal financial risk caused to each asset by existing threats after partial implementation of mitigation plans (use the 'already implemented' flag in countermeasures)
• Total financial risk including all assets
• Total financial risk after all mitigation plans are implemented
• Total financial risk after partial implementation of mitigation plans

Reviewing these results may help the analyst in improving the threat model and in refining the parameters of the entities. It is most productive to check how the model behaves in response to changes in the input data and running various "what if" scenarios since this provide additional insight of the systems' realities.