Risk Assessment is Damn Near Impossible

Difficulties in Assessing Technical Risk

I very much recommend this kind of talk to policy-types: http://www.slideshare.net/PacSecJP/jurczyk-windows-metafilepacsecv2

Notice that even the BEST EXPERTS ON THE SUBJECT are often wrong when assessing this sort of thing.

The main point being that here is a simple counter-example to the VEP being at all realistic: Bugs are often collected in libraries (or even in root-code that ends up in many different implementations of libraries), and then trickle-up to exploitability in one or more products that are based in it. For example, vulnerabilities in the EMF/WMF file parsing formats can be exploited in various versions of IE, Office, the Kernel, or many other end-user-exposed systems.

The result is that no VEP-like process can truly estimate the value (or risk) of any set of vulnerabilities, because the vulnerabilities are quantum in nature - not truly there until you decide to exploit them on a particular configuration of products and platforms.

Business Risk vs Technical Risk

Penetration testing companies often have reports which assign Low, Medium, High, or Critical risk to any particular finding. These sometimes have numbers assigned to them (these findings go up to 11!) and in many customers a "Critical" finding triggers specific internal processes that require a fix to be in place within 30 days.

So estimating Risk is a no-joke kind of operation. Much time is spent arguing over each finding and its level of risk. But the key feature there is that while we can state "All cross site scripting  (XXS) vulnerabilities are of HIGH risk", even there there are many subtlies. For example, a STORED XSS that reaches many users of a web app is clearly dangerous, but some applications don't even have authentication layers, at which point reflected XSS is totally harmless.

And assuming you know and agree with your customer about technical risk there is a truly VAST gap between that and agreeing on business risk. Keep in mind, the Classification system itself is a designation of BUSINESS RISK to the business of the USG.

The point around the VEP, software liability, cyber insurance, and many other policy issues being simple: If you understand and agree on the technical risk of a vulnerability (which you won't), then you probably won't agree on the business risk.