Wednesday, January 26, 2011

Architect vs Consultant vs Penetration Tester

Now there seems to be a bit of twitter rage at the moment around pentesting and consulting and what it should be or even what it should represent. I made a bit of an off the cuff comment that pentesting is a conversation opener and Twitter only supporting a 140 character limit, I didn't bother to clarify my stance.

See below a copy of the Open Software Assurance Maturity Model. I just picked this out to illustrate this discussion because I was reading it yesterday and I thought it would help make the point.

I have commented in the past that penetration testing should be used as either a validation piece as I call it (or 'Verification' under OSAMM) or a discovery exercise (to find out what is unknown - e.g. a legacy application in your environment). Now far too much hate has been circling on what some people are saying about penetration testing and from my perspective I think its because none of us is really trying to see the other side of the argument and perhaps I've added fuel to the fire (sorry).

We all know that the skills of a penetration tester is incredibly deep as well as broad. This is so indisputable it requires little mention. 

However, the skillset required to break a network, or a system, or an application is a very, very different skill than fixing it and maintaining it. If you have ever worked in an enterprise environment for an extended period, you’ll know what I mean.

To help make the point, I want to break down the roles of an architect, consultant and pentester into conceptual terms that in reality are grossly simplified (and kind of do an injustice) but for purposes of illustration will do for now (in at least as far as I see it):

Architect thinks first and foremost conceptually. They focus on business requirements and how to align security to meet those objectives. This typically manifests in changes to IT (infrastructure, applications) but can just as easily be a process change if it is the simplest solution. Architects are able to dive town in depth on a technical level but generally speaking, not to the same degree as the other two.

Is the middle man between an architect and pentester. A consultant thinks in terms of risk. Usually a technical person who has moved into to consulting based on broader exposure to other areas of security or business and/or excellent communication skills to be able to articulate risk and translate architectural terms and concepts or low levels of detail that a penetration tester may unravel. The posess a moderate degree of technical skill generally, typically higher than an architect.

Highest degree of technical skill. Engineers often fall within this category too. They possess the deepest degree of knowledge and in many cases, the broadest. Their ability to focus on the microlevel of detail is the single greatest strength and in many cases their weakness.
Now if we look at the different phases within OSAMM, let us see where each role can fulfil:

CONSTRUCTION: Architect/Consultant
VERIFICATION: Consultant/Penetration Tester
DEPLOYMENT: Penetration Tester

In a general sense, all three work in harmony to provide the coverage needed. I don't really know of a pentester that enjoys drawing up security policies  but the truth is that the pentester's detail focus is what enables him to pick out an error in a line of code, or the desire to test a single form field for umpteen different ways to break weak validation. He has no interest to delve into the business processes required to secure the enterprise. That's what architects are good for. Conversely, Architects generally lack that sort of attention to detail to do the work of a pentester. They are conceptual thinkers ("big picture"). Consultants are good at going to all the design review meetings, arguing with the business over a security requirement they want to cut and explaining the consequences for that decision. A lot of pentesters really lack the articulation to be able to do this. Likewise, too many architects are accused of sitting in their "ivory towers" ignoring the realities of the world around them when difficult technical discussions need to be made.

Back to reality, the world is full of individuals and everyone brings a unique blend of skills to the table. Very few people can "do it all" really well. Some people can wear multiple hats. Some can do all three. In my experience those people have a depth of experience that only comes with time in this field. You cannot compete with raw experience.

However because we all wear multiple hats, building a strong consulting practise relies on understanding the hats that everyone in your team has. When recruiting this often means looking beyond what you see before you and also seeing a bit of what can be. As a team, we are all more than the sum of our parts and our true value to our clients is leveraging our diversive skillsets to help uplift our client's security posture. 

Being able to successfully implement a framework like OSAMM can only be achieved when all the members of a security team (or consultancy) work collaboratively. It takes time to build and maintain trust, understanding the client's business and making sure the work we deliver is truly strategic. Even if you're an in-house team it is every bit as important to take the time to understand the business rather than attempt to dictate terms. If you aren't having those discussions with your consultants (or with the business if you work in-house) and if there is no understanding of the business then something is wrong with the approach. Certainly it takes a certain level of degree of maturity to reach this point however, our roles as security professionals is to push to have these discussions and play in this space.

A penetration test in and by itself is pointless if we are not having discussions with our client about their remediation activity, understanding their internal testing process, their security requirements and how they are being provided to developers, how is the architecture reviewed and ultimately, what is the level of engagement between the business and the in-house security teams. Having said that, a penetration test can also prove that all the strategic work you’ve done is still not up to scratch and needs improvement. In that respect, it might be used as an 'opener' or a 'finisher'. 

But there are three other business functions under OSAMM and multiple security practises that need to be achieved. Open frameworks like OSAMM prove that the skillsets used during each phase are specialist and require an entirely different focus. It is truly rare person to possess all the skills necessary to deliver each phase. It takes a certain level of self awareness to recognise what you can do and what you can't. Building a security practise around those skillsets requires embracing that diversity and ensuring people are given the opportunity to develop skills they might not otherwise have done as well as run with their strengths. 

However, ensuring the diversity of skills within a security practise and offering a wide variety of services that encompass all the business functions under frameworks like OSAMM requires that these other perspectives be acknowledged and developed. It is the only way to maintain our currency and relevance as our field evolves and develops. To quote Charles Darwin:

It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is the most adaptable to change.

- J


Anonymous said...


One question is what functions does the consultant perform - are they solely looking at risk, or are they looking into what the architect has proposed in more detail ? This leads onto perhaps a "Security Engineering" function.

Jarrod said...

Both I would say. The differentiator is that they can articulate risk and still talk detail.

How much detail they get into really varies on the project, resourcing, skill level, etc.

- J.

William Son said...

Great information thanks for sharing this with us.In fact in all posts of this blog their is something to learn. Plot for Sale in Lahore your work is very good and i appreciate your work and hopping for some more informative posts.
Plots for sale in Paragon City

aefea21 said...

Data stealing through the internet is a real modern threat. Anybody can become victim of hacker attacks.CyberTraining 365 will enable you to perform Penetration testing of your servers and systems by yourself to make sure your information is well protected.