Wednesday, June 25, 2014

We've Found No Evidence...Means What Exactly?

For its part, LexisNexis confirmed that the compromises appear to have begun in April of this year, but said it found no evidence that customer or consumer data were reached or retrieved, via the hacked systems. The company indicated that it was still in the process of investigating whether other systems on its network may have been compromised by the intrusion.
             Source: krebsonsecurity.com

Shucks, I can't find any evidence!
I think it's funny how organizations are careful to craft public statements in the wake of data breaches or public exposure that systems were compromised.  For instance, the statement above, "no evidence that customer or consumer data were reached or retrieved," appears to be put forth in an effort to ease customer concerns.  However, as an information security professional, my first thought was to analyze that statement through my jaded security lens.

Thinking critically though, "no evidence was found" could mean quite a few things:

1) Logs were reviewed and no exfiltration of sensitive data was observed
2) Systems were scanned with anti-virus and anti-malware software and were reported clean
3) No consumers have complained about their information being compromised (which is typically found by either a consumer having their personal account(s) hijacked and/or unauthorized charges on credit card or banking statements.

Log Analysis

As someone who troubleshoots various networking issues, when someone tells me they've reviewed the logs and found no evidence, my first thought is who QC'd the review?  In complex security and/or networking issues, I've found it's extremely helpful to have someone else either assist or vet your work.  We all have varying degrees of experience and talent which influences the tedious work of analyzing log files, packet traces, etc. and can result in one person easily noticing something of interest while another person would miss it entirely.

Are there such certifications in the area of log analysis?  To my knowledge, there's no industry accepted certification for log analysis like there is in the way of other certifications like CISSP, Net+, or CEH.  So, at most that leaves either a vendor-specific certification, training, and/or experience.

With vendor-specific training, you learn the basics of the device/software and maybe there's some advanced training that takes you on a deeper dive of the product, but most of the time such training doesn't teach someone how to analyze information and properly interpret the results. 

Okay, what about training that isn't vendor-specific?  Now we're getting somewhere.  The type of training here though would be akin to what an intelligence analyst has to do, which is crawl through vast amounts of collected informaton to find connections and correlations.  Academically, it's tough to find such training.  I did find a couple of related courses on Coursera:

     Reasoning, Data Analysis and Writing
     Statistics: Making Sense of Data

Lastly, there's experience, where over time you've learned to recognize such connections and correlations.

When companies suffer a data breach, of course junior analysts can assist, but the effort should be overseen by someone who has had analyst training and/or years of experience.  How much experience?  For the CISSP it takes five years of experience and an endorsement, so maybe there could be something along those lines established.

Scanned Systems

Next, we have the assumption that machines were scanned and no evidence of malicious software was found.  How many machines were scanned?  With what tools were they scanned?

There are numerous reports of how anti-virus based on signatures, although still necessary, is considered an entry point in terms of an anti-virus defense.  It should be supplemented by software that scans for heuristics as well.  But lately, as Brian Krebs also reports, there's an entire underground industry developing around the goal of obfuscating malware payloads so they aren't recognizable.  So, if scanning systems for viruses is now as basic an action as is locking your front door, something more is needed.

This is an area where companies like Carbon BlackCrowd Strike, and Mandiant are making names for themselves.  Although their tools are reactive in nature, they are oriented toward Incident Response and identifying the method of exploit and what systems and data were touched.  Combining the output from those tools with the log analysis above should provide a picture of what systems and data were affected.

Reporting Standards

If a company has performed both the log analysis and has the scanned system output, the next course of action would be public disclosure.  This presents an issue though because if there is suspected criminal activity, law enforcement would be involved and the general rule is that information is prohibited from disclosure in ongoing investigations.  However, just saying "no evidence found" isn't enough.

To address the issue on both fronts, disclosure standards should be defined so companies can incorporate those standards into their incident response plan.  By the way, if you work for a company that doesn't have one, you might want to start building one now.

The U.S. does have data breach disclosure notification laws, but nothing specifying how the information should be presented to the public or what details can be included when law enforcement is or isn't involved.  Have a look at the link above and you'll see most states have individual statutes specifying what constitutes sensitive data and when individuals should be notified.  And even within that context, states handle data breach disclosure handling differently.  A federal law would provide clear guidance for states to incorporate into their own codified laws and for companies to use when these events occur.

Maybe then we can get clarity on "no evidence was found"

Don't worry.  Every company says they didn't find anything...