« Who are Hackers? Part I | Main | Who are Hackers? Part II: Hacking for Money »

PCI Compliance: Dog chasing its tail?

I sat in PCI compliance seminar recently given by Visa.

According to the presenter, only a minority of companies are currently compliant:

Level 1 Merchant: 40%
Level 2 Merchant: 16%
Level 3 Merchant: 38%
Level 4 Merchant: Low (No specific number was given)

The numbers are quite low. Why?

According the Visa, the most common violations leading to merchants flunking audits include:
-un-patched systems
-poor password policies/enforcement
-unnecessary and vulnerable services on servers
-using insecure default settings, such as unencrypting wireless connections
-poorly coded web-facing applications resulting in SQL injections and other vulnerabilities
-finding and the storage of prohibited data (such as account numbers, CVV2, PINs)

PCI is widely praised for its specificity. Compared to omnibus mandates such as FISMA or SOX, this is definitely true. However, do PCI's prescriptions set merchants up for failure before they even try to comply? In the case of the last 2 bullets, this is clearly the case.

First, "Poorly coded web-facing applications resulting in SQL injections and other vulnerabilities". PCI mandates pen testing--yet try to find anypen test who can confidently say, "Yep, my test or pen testing tool was 100% thorough." Pen testing will always fail to miss issues. So when a PCI auditor comes around and finds a flaw then--poof--you're on the PCI black list. Merchants, auditors and regulators should note what Gary McGraw has said, "Black box testing is a 'badnessometer.' Everyone should buy a badnessometer to find out that they're in trouble and use static analysis to fix it."

Second: "finding and the storage of prohibited data (such as account numbers, CVV2, PINs)"
Why the difficulty? Many merchants have so systems including transaction processing, loyalty tracking, custom applications and so forth. Sniffing out all of the potential places, such as log files, is not easy in complex environments. Yet PCI only requires manual code audits to sift through the code.

I don't mean to knock PCI. On the contrary, it's a good idea and I'm always in favor of industry self-regulation. PCI has several components that help stop cyber fraud. But PCI technology prescriptions should be designed to help merchants pass their audits.

Comments (3)

Actually, McGraw's quote was that the tools are badnessometers. I don't recall it being specific to manual pen tests, as well. Clearly the manual test, along with some badnessometers are needed to find the vast majority of vulnerabilities. But no pen test is going to find 100% of all vulnerabilities, as you suggest.

I also don't think his quote said to buy a static analysis tool to find and fix vulnerabilities. If I remember correctly McGraw states that ~50% of all vulnerabilities are bugs and 50% flaws. Static analysis will find most of the bugs, but none of the flaws. So static analysis with a tool like Fortify SCA will get you part way to the solution, but you still need the hands on manual reviews and threat/risk modeling for a complete solution.

Thanks for the comment.

You are correct, Gary did say that in his book. However, he has since gone on record to say that pen testing in particular are badnessometers.

You can see it here:

http://www.cigital.com/justiceleague/2007/03/19/badness-ometers-are-good-do-you-own-one/

or here

http://blog.eweek.com/blogs/eweek_labs/archive/2006/06/12/10727.aspx

Actualy some static analysis tools can find flaws. For instance there is a Java tool that will find "suspicious pre-conditions" such as a variable that is valid for 0, 2-infinity, but won't process the value 1? It also will find "race conditions" where inadaquate locks are held on data that can be update by more than one thread, and can result in bad data. The newer static anlaysis tools are very global and system wide an do much more than finding simple convention or coding practice errors like the simpler Lint or FindBugs tools do.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Presented By

About

This page contains a single entry from the blog posted on March 22, 2007 11:35 AM.

The previous post in this blog was Who are Hackers? Part I.

The next post in this blog is Who are Hackers? Part II: Hacking for Money.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type 3.34