I sat in PCI compliance seminar recently given by Visa.
According to the presenter, only a minority of companies are currently compliant:
Level 1 Merchant: 40%
Level 2 Merchant: 16%
Level 3 Merchant: 38%
Level 4 Merchant: Low (No specific number was given)
The numbers are quite low. Why?
According the Visa, the most common violations leading to merchants flunking audits include:
-un-patched systems
-poor password policies/enforcement
-unnecessary and vulnerable services on servers
-using insecure default settings, such as unencrypting wireless connections
-poorly coded web-facing applications resulting in SQL injections and other vulnerabilities
-finding and the storage of prohibited data (such as account numbers, CVV2, PINs)
PCI is widely praised for its specificity. Compared to omnibus mandates such as FISMA or SOX, this is definitely true. However, do PCI's prescriptions set merchants up for failure before they even try to comply? In the case of the last 2 bullets, this is clearly the case.
First, "Poorly coded web-facing applications resulting in SQL injections and other vulnerabilities". PCI mandates pen testing--yet try to find anypen test who can confidently say, "Yep, my test or pen testing tool was 100% thorough." Pen testing will always fail to miss issues. So when a PCI auditor comes around and finds a flaw then--poof--you're on the PCI black list. Merchants, auditors and regulators should note what Gary McGraw has said, "Black box testing is a 'badnessometer.' Everyone should buy a badnessometer to find out that they're in trouble and use static analysis to fix it."
Second: "finding and the storage of prohibited data (such as account numbers, CVV2, PINs)"
Why the difficulty? Many merchants have so systems including transaction processing, loyalty tracking, custom applications and so forth. Sniffing out all of the potential places, such as log files, is not easy in complex environments. Yet PCI only requires manual code audits to sift through the code.
I don't mean to knock PCI. On the contrary, it's a good idea and I'm always in favor of industry self-regulation. PCI has several components that help stop cyber fraud. But PCI technology prescriptions should be designed to help merchants pass their audits.







Comments (3)
Actually, McGraw's quote was that the tools are badnessometers. I don't recall it being specific to manual pen tests, as well. Clearly the manual test, along with some badnessometers are needed to find the vast majority of vulnerabilities. But no pen test is going to find 100% of all vulnerabilities, as you suggest.
I also don't think his quote said to buy a static analysis tool to find and fix vulnerabilities. If I remember correctly McGraw states that ~50% of all vulnerabilities are bugs and 50% flaws. Static analysis will find most of the bugs, but none of the flaws. So static analysis with a tool like Fortify SCA will get you part way to the solution, but you still need the hands on manual reviews and threat/risk modeling for a complete solution.
Posted by Dean H. Saxe | March 22, 2007 12:45 PM
Posted on March 22, 2007 12:45
Thanks for the comment.
You are correct, Gary did say that in his book. However, he has since gone on record to say that pen testing in particular are badnessometers.
You can see it here:
http://www.cigital.com/justiceleague/2007/03/19/badness-ometers-are-good-do-you-own-one/
or here
http://blog.eweek.com/blogs/eweek_labs/archive/2006/06/12/10727.aspx
Posted by Robert Rachwald | March 25, 2007 5:11 PM
Posted on March 25, 2007 17:11
Actualy some static analysis tools can find flaws. For instance there is a Java tool that will find "suspicious pre-conditions" such as a variable that is valid for 0, 2-infinity, but won't process the value 1? It also will find "race conditions" where inadaquate locks are held on data that can be update by more than one thread, and can result in bad data. The newer static anlaysis tools are very global and system wide an do much more than finding simple convention or coding practice errors like the simpler Lint or FindBugs tools do.
Posted by Kent Bradford | April 25, 2007 11:37 AM
Posted on April 25, 2007 11:37