0days: Testing, and Timing

Perfect timing is everything...

So there's another reason why nation states use 0days: Testing. Testing on exploits is particularly hard. All software is hard, but exploits are "things that are not supposed to work" by definition. This means not only are you testing them against a few VMs you have laying around but also against, say, every version of AVG's free virus protection you have ever seen, on every version of Windows possible, with every possible configuration.

As you can imagine, that problem is "exponential" in the way that computer scientists use the term when they mean "a complete freakin' nightmare". Of course, the only REAL test is whether they "work in the wild". This is a whole different level of "working" and "works in the wild" is labeled on many exploits to say 'yes, this one is at that next level of quality'.

And some exploits, even very good exploits, like ETERNALBLUE appears to be, fail the testing. (According to reporting, it was known as "ETERNAL BLUESCREEN" since it often crashed targets.)

When exploits fail the testing phase, you don't give up on them. You pass them off to different exploitation teams for more analysis, you wait for the code in the target process to change (which is often successful). You wait for another bug to be found to combine with this bug. Persistence, which is a key metric of your operational success, is not just about the hacking part, in other words.

And even if your exploit is PERFECT you still have many things to do before you can use it. It needs to be integrated into your toolset. People need to be trained on it. Targets need to be collected and triaged. Operational security notes need to be written. What do you do when the exploit fails? Does it leave logs? How do you clean those up? How do I tune my defenses to detect if the Russians are already using this vulnerability (and have therefor tuned their OWN defenses to detect it?)

All of these things take time and we could, in some cases, be talking several years. Your average penetration testing gig is maybe two weeks long. These processes are similar, but not the same. So be careful extrapolating operational work from penetration testing too much. The Grugq has a good presentation you can read on this evolution here.

Needless to say, attacking with vulnerabilities which already are well known has negative impact on your OPSEC. But it also may mean the targets you most care about (which have an average patch testing cycle of 14 days), may become out of your reach.