Its true most of the time companies don’t provide sufficient time etc. but again everybody has some issues in their project execution.

Penetration tests are a dumb idea in general - they are used, in plain view, to reveal weaknesses. This means that there is the assumption of weaknesses and the builders / maintainers will be on edge to prevent the penetration test from succeeding. Put this in conjunction with what pagvac says and you get programmers building against test tools iso real world threats.

I think a tiger team should be invisible or at least covert. A good tiger team is only known to the CEO or the board so that there is no prior knowledge of the attack to those who need to prevent the attack.

A contemplation for dessert?

if “Penetrate and Patch” was effective, we would have run out of security bugs in Internet Explorer by now. (source: Marcus J. Ranum)

Also, to add to pdp’s remarks on quality…always remember the “team” in Tiger Team. I have seen the same as pdp with penetration testers in the industry, it’s usually one person conducting the test. While this one penetration tester may be very good at what he or she does, you still need the input and creative thought of others to do a thorough job. With a Tiger Team you need the skill sets of multiple experts…from physical to network security and everything in between. It would be very difficult for one person to conduct a full Tiger Team operation let alone a successful one.

OK, we need to differentiate between good pentesters and good *pentests*. If someone is charging you 2K to do a webapp test, trust me, it won’t be very complete even if the guy performing the test is a hell of a webapp hacker. The reason for this is simple: you get what you pay for. For example, if a company got a “good deal” for a pentest, the tester won’t be able to spend a great deal of time doing the assessment due to profit targets set by the company he works for.

Sorry I keep mentioning webapps, I know there is so much more than webapps in pentesting, but this is an area that I deal with a lot. It’s also an area where companies are still willing to pay more because the pentester can find many things that VA scanners like Nessus won’t (NOT a commodity market yet, unlike infrastructure vulnerability assessments).

The security testing industry is like any other. There are no “good deals”, you simply get what you pay for.

And finally, regarding good pentesters, well, there are very few good ones out there. Most “pentesters” are just people with very little experience who simply run a scanner and get paid 25K to do so, but unfortunately they don’t use their brain to discover security issues. I hope I’m not sounding too harsh, but this is the plain truth. A VA analyst is NOT a pentester! Once again, Vulnerability Assessment is NOT Penetration Testing.

My final message for pentesting customers out there is:

1. you WON’T get a good security assessment done by a good pentester for cheap. If you are not paying *at least* above the market average price, it just won’t happen!

2. Vulnerability assessment is NOT penetration testing. VA is mostly about running an automated scanner and getting rid of false positives. Penetration testing is about using automated tools, modifying or creating new tools, exploiting vulnerabilities, and applying the hacker mindset (i.e.: experiment, believe that anything is possible, be creative).

A ’small point of order’ if I may?

CREST will not replace CHECK but instead be the scheme of choice for testing providers who perform security tests for non-Government clients. CHECK will remain as it is, but it is CESG’s intention to remove from the CHECK list those companies who do not perform tests for Government clients.

You could argue (though unsuccessfully in my opinion) that this will create a two tier level of service and standards - it won’t and this can be seen by the fact that the CREST Infrastructure Certification Examination has been agreed by CESG to be technically equivalent to the CHECK Assault Course (and as you’ve already stated is harder to pass than the CHECK Assault Course or so I’m led to believe.)

Initiatives like CREST are to be welcomed to provide some much needed clarity to the security testing market, provide an indication of standards which member companies must meet and therefore ensure that end users of their services get quality testing which meets their needs and at a good price.

I don’t know, what is your opinion?

Bob, there are plenty of good pentesters that work on contract basis. You can get even some for a 200 pounds a day but for that money I don’t think that it works for them if it is a once-off thing. If it is between 3 to 6 months contract then it could work I guess. But yes, UK is a weired market and I hope it will do well after all these problems around the constant increases of expanses and the general instability of the market.

Clearly you never did anything under PITO then :)

You’ll be surprised how cheap good pentesters are these days in the UK. It’s largely a commodity market. The reason pentesting is dying a death is because the market’s flooded with average testers (regardless of price) and things like automated PCI scanners add to the economic pressure. Security consultancies need differentiators to survive, and I agree about the tiger team differential as well as with the point on the number of consultants on a pentest, but network testing cannot compete (economically speaking) with automated scanning in a flooded commoditised market.