My $0.02 worth on outsourcing security

I have had some discussions with people surrounding this topic lately and I want to highlight a few points on this very tender topic.

Firstly, I am not in favor of offshoring security testing. It needs to be said. I acknowledge it is happening, I acknowledge there is an economic benefit in doing so, I understand why this is happening - but it isn't something I'm happy with. Then again I'm not happy with a lot of things but we have to make do with what we've got sometimes. I also think a lot of people rage about it without a clear grasp of what is "really" the problem. So I want to put out a number of ideas out there what I think people are sensitive about and with that in mind, try and put it into perspective. Moreover, I want to stress in economic terms why this happens and is "normal" (in an economic sense of the word).

I've included some common economic principles below that I have personally found really interesting in studying this for my MBA. Contrary to popular belief, economics is not just about business. At its core, it is about how people make decisions and evaluate choices. Everyone should study a bit of economics IMHO. Even if you have no interest in business, it can help you with your social engineering efforts. :) Anyway, this segues into my next point.

Economists discuss that when people make a decision (such as purchasing another unit of a given good) they evaluate what is the benefit in doing so. Some companies have already made the decision to move a good number of critical functions offshore gives them a much greater benefit. They can put the cash saved towards purchasing more security consulting, buying more widgets, giving execs larger bonuses - whatever. The point is the decision has been made that they perceive there to be a greater "gain" in this move.

Thirdly, the whole "outsourcing" security is precisely why security consultancies are built. Businesses outsource their function to specialists every day. It makes sense. Anyone that is even slightly familiar with economic theory will understand that it makes sense to employ specialists where needed when a specific skill competency is outside of your core business. So the whole idea that "outsourcing = bad" must be utterly dispelled. It isn't "bad". To say it is bad is a strawman argument. When I was hiring pentesters in my last job, its because I am snowed under, I wasn't getting time to develop my own skills and yet expected to attend 5+ hrs of meetings/day. It made perfect sense for me to hire guys who stayed ontop of their game and were paid to keep their skills sharp, to say nothing of obtaining objective, independent testing. So lets get that out of the way. It happens every day, what we're talking about are these services either moving offshore or to more unskilled labour - that is what gets most security consultant's goats up. If you get a plumber in to fix your pipes, that is "outsourcing". Service your car? Outsourced. We do this every day, only we don't perceive it that way. Ever buy a car? Probably imported. I can go on but the point is that "outsourcing" is another word for "trade" (see Principle #5 below).

Fourthly. the outsourcing argument assumes that Australia has a monopoly on technical acumen - and that's simply not true. We are a very small player on a global stage. To be fair, I also think a good number of our homegrown talent is largely uncredited (but that's another issue). But I'll pick on India since its a hotspot for outsourcing and there's a lot of good discussions and research on this topic in relation to India (so it suits for reference purposes).


For starters, India's IT schools are regarded as an asset of national critical importance. They also have a much larger population, an incredibly larger proportion of people going through university (1.1 billion people estimated, 7% post secondary education vs our 22 million population and 34% post secondary education). Right away, they have three times our national population with a tertiary education. Granted, there are cultural issues which suggest that many graduates are not well suited for many jobs (cultural attitude to rote learning apparently) but if you consider the pool of potential graduates coming through, statistically they have enough people they can (almost literally) throw at the wall and see what sticks! Even if there is evidence to suggest they aren't all appropriate, the fact remains that even if you look at the guys with talent, there is almost a statistical certainty that they will have more technically capable guys than we do in Australia. It's arrogant presumption to presume we're so awesome that they can't compete in the same space as us. One of the beautiful things about technical skill is that it is not constrained by country, economic wealth or privilege and does not care about culture barriers. This is what is great about tech and what brings techies together around the world. Likewise, its the Achilles heel which ensures that the industry will always trend to being outsourced.

Bringing the point home, its not difficult to assume that if I can keep a close eye on how I outsourced my pentest and manage assurance with virtual team then certainly other companies can't with outsourcing.  That's not to say crap work can be delivered by unskilled labour but lets be frank, we've seen that with local assets too, right? :D I see that as a management problem, not a delivery problem. Don't believe me? Ask any tradie who has to oversee first year apprentices fresh out of high school.

Finally, I.T. security has been largely a growth industry and despite hearing it is almost a recession proof industry for years, its interesting that only now we're being hit with the outsourcing. Having grown up in I.T, I've seen this so many outsourcing gigs hit jobs of either friends or family that I've almost become inured to it. I know people are thinking "OMFG pentesting gone to India wtfbbq" and its like yes, welcome to the 21st century. We are all expendible. Now as mentioned, I am opposed to outsourcing this function because I think sensitive information gathered from pentesting needs to be kept close to home with strict governance around it along with stringent quality assurance. However, we all know about the insider threat so again - its a strawman argument to assume it can't happen here. But the reality is we are not unique and beautiful snowflakes. People will outsource to companies if they can do it cheaper - even if there is a drop in quality. If the quality drop is still within an acceptable level, then meh - they'll wear it. I'm not saying it is right, but this is what Mankiw calls "the margin" and its here to stay (always has been really). I was actually going to graph this point but I'm tired and can't be arsed. If you're actually interested, go study 'price equilibrium' or 'equilibrium theory' (the two aren't the same but they are related) and you'll see what I'm talking about.


Ultimately, the problems that arise from outsourcing are largely the same problems our clients today face when decide to engage us as consultants. I share the same trepidation as everyone else but I don't think the problem is as concerning as everyone makes out. We just need to be very clear on what those problems are and ensure that we impose suitable checks to offset those risks and have some sound advice for clients looking to move down this path.

If you're a company competing against outsourcing, my recommendation is to study Mankiw (see below) and have a think about how you could apply these principles in effect to your work. I have many ideas on how I'd compete against anyone simply offering a lower daily rate against my business. Some of them aren't entirely fair either. ;-) In any case, there are somethings I won't post because we're competing against this crap too. :) One strategy I will offer is that I am a firm believer in uplifting our capability and focusing on delivering business consulting and security strategy (see #4 below and my previous rant).

Cheers,

- J.

Resources:
N. Gregory Mankiw - 10 Principles of Economics.

#1 People face tradeoffs
#2 The cost of something is what you give up to get it
#3 Rational people think at the margin
#4 People respond to incentives
#5 Trade can make everyone better off
#6 Markets are usually a good way to organize economic activity
#7 Governments can sometimes improve market outcomes
#8 A country’s standard of living depends on its ability to produce goods and services
#9 Prices rise when the government prints too much money
#10 Society faces a short-run tradeoff between inflation and unemployment

PS: I recommend "Economists Do It With Models"  as a great resource for understanding economic concepts in bite sized chunks. Jodi Beggs is actually a former student of Mankiw and better able to conveying information than the textbook I have to use for my unit. :-(   Yes, the link is worksafe btw, don't be put off by the name.


PPS: I had a whole rant where I was actually going to discuss price elasticity of demand with regards to security services but I realised that its not relevant to this discussion and decided to drop it, but I'd be keen to hear from other security folk who are actually interested in this stuff on what they perceive as the elasticity of security services to be. I'm of the very its actually quite elastic based on my own experiences but I realise that flies in the face of a lot of evidence to the contrary. Then again, I suspect I might be an anomaly in this area...

Pentesting isn't enough - "Part 3"

My friend Serg over @ SecuritySoup.com recently made these two posts about pentesting and I must admit it gladdens me immensely to see posts like this. I won't divulge too much about Serg in the interests of maintaining his privacy but I will say he is an experienced penetration tester and security consultant and in short, knows what he's talking about. Before I begin this post, you really need to read what he's written first.

I think I'd learned this lesson several years ago in my last job - and I know he has too but I think for Serg had the points hammered home to him over the past two years in particular on the basis of some of the clients and types of engagements he was working with. It's one thing to know something but another entirely to have the point really hammered home to you or see it demonstrated time and time again.

One of the things I am convinced you become aware of the longer you work in technical security space, the more you realise that the governance and management aspects play a critical factor in ensuring the success of a security program. Having said that, there's a world of difference between wanting to be a security manager and recognising the need for security management.

I have no desire to be a manager myself but I've seen first hand why non-technical skills are absolutely critical to ensuring the success and I'll give you an example:

I was a security lead on a very large project (large iconic brand shall we say?) and the one project was going to bulldoze ahead to go live without a critical security feature which was absolutely necessary to ensuring success. Without it, I am talking front page news headlines, loss of consumer trust, irrepairable brand damage, basically a post-apocalyptic security nightmare.  At this point in my career I had come mostly from a tech background and I had flipped out that this was going on. I realised that the Project Managers, Business Analysts, Programmers, etc. -- NO-ONE cared about security.
I couldn't understand why.


I had to learn (this job was a real baptism by fire let me tell you) how to convince people. I had to relate these risks back into business terms that they could relate to and why they needed to.

Pretty soon, nobody saw the problem I pointed out as a "security problem" it became "a problem" that each project member saw in their own way. The Marketing head saw it as being "detrimental to the end user experience". The Governance board envisioned "irrepairable brand damage". The Head of IT imagined his entire online presence having to be rebuilt and his Application Support team working overtime for the next six months. I eventually got funding to do the work that was needed and divert resources to develop the necessary controls to prevent this from happening.

Now if you are just a pentester and all you know is how to break stuff, then I suspect you would find it highly challenging having the above conversations. Infact, I know plenty that have no interest in having those conversations either - and that's fine. It takes all sorts of people to make this world go around and it would be horribly boring if we were all the same.

But to be a consultant you do need to be able to have these discussions with a variety of people. You have to understand what motivates people and speak to that motivation. Tony Robbins is often quoted as saying that people will do more to avoid pain than to gain pleasure - and this is certainly true in a business context.


Serg's point is that too many people lack these skills and it is these skills are most important. With the increase of technical jobs going overseas, the critical differentiator becomes the understanding of the client's business and how security fits into that. This is the lifecycle he's referring to. These are the sorts of questions such a consultant might find himself asking:

• How does penetrationt testing fit into your application development process?
• How much of that testing is handled by other test teams before a security specialists does?
• Are you doing code analysis, threat modelling, architecture reviews prior to development?
• Post launch, who is monitoring your applications, infrastructure and your data and how?
  

By no means is this list comprehensive but demonstrates the point.Technical skills are important. But to work in today's world as a consultant, you need more than that. I've worked with some brilliant technical minds (and I mean brilliant, not just skilled) but could also understand clients and their businesses. They can pentest anything you put infront of them (and I mean anything). It is certainly possible to have a good mix of both. The question is whether you want that mix of skills? If you want to remain a consultant in today's market, you have to have a service differentiator.


Now if you're relying on technical excellence as the only factor, you may be in for a very rough time. The basic law of economics dictates that people seek alternatives. If they can get the same service from someone overseas for a fraction of the price for equal skill (or even lesser skill) then a lot of people will make the trade-off if they perceive it to be of value. I'm not saying this is right or wrong, but that's the way it is. There are ways of dealing with this but we - as an industry - cannot continue to do what we always used to, never change and expect things to function as they used to. Times are changing and so too must we.

I guess the crux of the post is this -


It's one set of skills to be a hacker. It's another set of skills entirely to be a security consultant. Being one doesn't automatically make you the other. Serious hackers laugh at guys like me with their CISSP. That's fine. I laugh at guys like them who wouldn't last a week in a corporate enterprise environment dealing with the baggage I've had to. Like I said before, it takes all sorts to make this world go 'round.

I read Serg's post is a wakeup call to the pentesting community. If all you know is how to pentest, then times are looking grim. If you cannot properly engage the business, understand their language and communicate in their terms, think in abstract terms of principles and concepts and not just lines of code, then you better start learning or otherwise re-evaluate your day job. Your job is going to go to India or the nearest CompSci grad with a CEH who will undercut your daily rate.

If you don't believe me, then you better open your eyes. It's already happening.



- J.

Wikileaks getting shafted

I remember seeing Julian Assange speaking at one of the Ruxmon meetings earlier this year and not long after, I got into a discussion with another infosec consultant who was in attendance. We were discussing whether we thought Wikileaks should have posted the Collateral Murder video. His view was that, while disclosures are necessary, he wondered if this was really a case of "fog of war" and whether the video was more an indication of an unfortunate, grizzly accident rather than a grotesque abuse of power (force of arms).

(Now, to be fair, Assange went to great lengths to explain the rules of engagement as officially passed down to military personnel to highlight the fact they had clearly violated said rules to carry out the attack.)

I was polarised at the time. I used to be a far left hippy in my youth but I think overtime I'd become more bipartisan in my thinking. I could see both sides and I have a tendency to play devil's advocate. I think I said something about "yeah but we need groups like Wikileaks, to keep governments accountable." Say what you will of the Collateral Murder video but you have to admit, if the purpose of government is to serve its people then the idea that it should also be answerable to the people, isn't a stretch (unless you support dictatorships or other forms of non-democratic government, in which case I think we're on different views and you should stop reading).

The point I'm making was I really didn't feel strongly enough one way or another, but notionally, I supported the organisation.

Then someone else this week suggested creating a new root DNS server, in opposition to ICANN's management. My initial thought was "Ok great, someone wants to setup a new root but how do you secure it and prevent flagrant abuse?" This is a governance problem, not a technical problem - hence my tweet on the subject.

Not long thereafter, Wikileaks got DDOSed. Then the site was dropped from Amazon. Then their DNS provider dropped them.

At first I was shocked. Then I was scared (yes, scared that this could happen as naive as that may sound).

And then I got angry.

One of the roles I used to work in was Network Abuse, where we used to deal with investigations ranging from professional spamhaus gangs, to child pornographers, to kiddies dealing trojans to steal Diablo II accounts, you name it. We used to deal with other ISPs to collaboratively take down sites or offenders of clearly malicious intent. It was like a code amoungst ISPs - even if there is no direct law for some of the things we did, we did it because it made sense to work together as a global community (of course emailing non-English speaking countries was always a challenge but I digress).

You could argue we took the law in our hands, but we acted when the evidence was overwhelming that these people were malicious. E.g. evidence of spam traceable to certain IP blocks. Abused credit card numbers. URLs of sites allegedly hosting kiddie porn, etc.

Today I just saw a bunch of companies give strawman excuses to drop Wikileaks like a hot potato, for reasons I can only attribute to political pressure or unsavory conditions. I get DDOS as a weapon of hacktivism - I understand the motive. But these companies wiggled out of the arrangement for very dubious reasons. The US Government which claims to protect individual freedoms and rights is using every means at its disposal to capture Assange and arrest him. Swedish authorities have acted against their own law, with largely uncredible testimony.

The bottom line is this - even if you didn't support Wikileaks before, the actions of all these various groups is actively working against them, as it will polarise various groups that would otherwise have remained enemies. I'm only one random dude with an Internet connection and a pedestal, but I can find myself in agreement with so many people I wouldn't have yesterday, how would some other folks out there feel, particularly those with more spare time, motivation and technical savvy? I can only imagine.

In Australia we've never had to fight for our independence or freedoms. We have no Bill of Rights. Subsequently most Australians really are largely apathetic to the notion of free speech. However, if you've lived abroad or travelled to communist or non-democratic countries, you begin to realise just how valuable it is. We may have no such war but I cherish it.

Today made me think that I am far, far more left wing than I ever thought I was and it sent a shock through me. I wound up making a donation to Wikileaks. Nothing large but enough to at least send a token of support.

I support Wikileaks because there may come a time when I need a voice for something I cannot say myself. I support Wikileaks for my family, friends and other people on this planet who may find themselves one day in a god awful place where they need a voice and Wikileaks is the only one who can provide it. There are some real scumbags on this planet who should be punished but they hold the balance of power. Wikileaks has real power to make those people answerable to a higher power. Without groups like them, the bad guys can and will win.

Even if you haven't always agreed with their actions to date, ask yourself:

• Do you believe that businesses and governments alike should be answerable to the people?
• Do you believe that no-one is above justice?

If you answer yes to either of the above, I also encourage you to make a donation to Wikileaks and support the cause.

- J.

Problem Solving

I recently read a presentation by Ivan Ristic (ModSecurity fame). You can find it here. It struck a chord with me and I wanted to share this gem.

In infosec we suffer either one of two conditions (generally) - either we suffer from tunnel vision, focusing on minutae that are rarely of any real relevance to the task at hand, or we attempt to "boil the ocean" in an attempt to solve unsolvable problems.

Ivan's post basically made a very practical recommendation - don't boil the ocean, focus on one problem at a time.

We are all confronted with a variety of problems, either at work or at home and it becomes very, very easy to start lumping isolated problems together, creating a snowball effect. Sometimes they are related but sometimes when you have an emotional investment in something you can't see that the forest is made of individual trees. :)

I am meandering a bit (I tend to if you haven't figured it out by now - sorry) but my point is that while its important to look at the big picture, make no mistake we can only solve one problem at a time.

Extrapolating this to infosec, each of us see something that we as individuals can "fix". I want to put it out there that there are problems we can fix, we just need to pick them off. I've got my own bugbears I'm thinking about and if I have to pick one I want to focus on, it would probably be patch management on an enterprise scale. I'm not sure I can "fix" this but I am pretty sure I have some pretty practical suggestions that would go a long way towards doing so. I'm still contemplating how/where/when I'll go abouts working on a presentation on it.

Anyway, I'm sure you each have one thing you want to fix or more importantly, can fix. I want to emplore anyone reading this to do it.

We may never solve the "security problem" but we can all certainly play a part in trying to fix it. I'd rather be part of the solution than part of the problem.

- J.

Why everyone in infosec should do SABSA training

Last week I did my SABSA training and certification. I don't know how I went but I wanted to share the results of what I learned.

This post is long and I make no apologies for it. I wish someone had a post like this before I started on the training. I didn't have anyone I could turn to or who could really tell me what it was about until I had done it (which was frustrating). I mean I asked around but I didn't get an answer that really satisfied me. So I wanted to give the best writeup that I could to help explain to any other likeminded folk in the field or least give my thoughts on what I walked away with. So here it is.

For the uninitiated, SABSA stands for 'Sherwood Applied Business Security Architecture' and it is a method for applying an understanding of the business you are securing. By understanding the business, you are able to build in security that is appropriate to the context of that organisation.

My career journey has taken some very bizarre turns. When I took up my last role, I wanted to move into focused pentesting (from a role where I did some pentesting) and instead wound up doing a combination of stuff - security management, security consultant and solution design and architecture. I got to oversee a bunch of pentesters, hire pentesters and be fairly involved but it became a gradual divorce away from the hands on (which I still miss sometimes but other times I don't). But more to the point, I enjoyed what I was doing, although I think it took me a time to really appreciate it because it wasn't what I signed up for.

Anyway as I came from more of a sysadmin/pentester background, I would look at project designs and how to break them (typical 'break' vs 'build' mentality). Reviewing security controls because a question of trying to be as thorough as I could, and an exercise in mentally threat modelling various ways to break the solution design.

I would look at different methodologies for security testing, I would think of solutions in terms of the OSI model and what security services were required (authentication, authorisation, auditing, etc). I looked at OWASP to ensure a stringent design process was followed. We used contractual arrangements to enforce our needs where we could.  I'd spend time talking to the business and trying to spend more time listening than talking. I wanted to find out what their problem was and if I could address it. I also tried to keep up with all the tech (which is impossible once you get to this point doing this work as something has to give).

But the whole thing seemed and felt really adhoc. It was a case of finding what I could think of, springboarding ideas with my team mates and consistently asking myself "is this enough?".

I couldn't put a label on what I did exactly - but whatever it was - I was never aware of a methodology to help people like me doing the work I was doing. I became aware of architecture as a discipline owing to some good solution architects who heavily influenced my thinking and became aware of Enterprise Architecture as a discipline and how good solution design can help drive strategy. However that seemed great as an IT Architecture, or a Business Architecture, but nothing to help me in my discipline. I learned relatively quickly that pentesting was simply a way of measuring security effectiveness but in and by itself, didn't offer me anything beyond some assurance. It was around then I started to drift away a bit from pentesting.

I became aware of SABSA in 2008 through a co-worker. We'd both found the SABSA book by David Lynas, John Sherwood and Andy Clarke but neither of us really did anything with it. I think at a glance we had the same impression as everyone else. It looked great as a method but seriously, how practical was this light and fluffy stuff. We both came from similar backgrounds and I think had very similar thoughts on applying architecture - this all looked really high level theory and not very practical.We were still interested in the book and how it applied but we never got around to reading the book or doing the course.

Well, I finally started on that book earlier this year and got through most of it (3/4 I think). The course is an equally tough slog too. I consider myself pretty passionate on this topic btw, but after day 4 even the most passionate person will be tested I reckon!

As I went through the training I had an epiphany of sorts (actually I spent a good chunk of the weekend at Ruxcon ruminating about it too). Having read the book and seen some videos on architecture, I had some idea of how Architecture, as an IT discipline worked. But this course really highlighted for me why SABSA is THE single most important training I have had in my field to date. I wanted to share what I learned and highlight why I think other infosec professionals - regardless of their role, should do it.

What I learned:
I learned that the often disparate array of compliance standards, ISO standards, architecture frameworks and so on need not be. They can all integrate together. Once you understand the business you can begin to build that picture. The architecture frameworks (TOGAF, SABSA, Zachman, etc) are simply a "method of organised thinking". SABSA is the concrete which enables me to put together all the other building blocks together.

Every thing I know, have learned and will learn (not just security related) I can apply using this framework to build a security model for my clients based on re-usable architecture. The "architecture" however, is never complete. It is a living breathing organism. Each project or each pass is an attempt to iteratively build up your understanding of the business. Each project is an opportunity to build upon those building blocks - re-use what you can or tailor to suit where appropriate. You never have a perfect "target". It just keeps moving. We've often known that security is a journey, not a destination - but seeing this through the eyes of an architecture framework is a very different thing. I think its like trying to describe being a parent to someone who doesn't have kids. There's the difference between knowing the path and walking the path, as Morpheus would say. :)

From a technical viewpoint, I learned how to structure controls so that controls at the various OSI layers are not done in isolation from each other. That you can build concrete controls. You can even forgo certain controls but it is not without an understanding of the potential consequences or the risk.  I also learned to truly build re-usable, extensible security services. Moreover, how they link back to business goals and objectives which enables demonstrable return on investment and develop a realistic form of risk assessment (pretty much the best system for risk assessment I have seen to date).

Incase people reading this think this is all high level fluff, the guys who developed this were the architects for Swiftnet (the system banks use to transfer funds internationally). Anyone who has heard of this or worked with banks will understand the importance of the system and the risks involved. As you go through the training, David Lynas (who is the main instructor globally - if not the only one I am aware of) will make clear his points through illustrated examples throughout his career and using examples from other people's careers. As someone who had done this sort of work before, I found these some of the most educational points of the training as I could relate it back to various situations of my own and either pat myself on the back for getting something right or mentally kick myself for thinking how I should have handled something differently.

I realise this is going on and on - but I really want to stress why I think this is worthwhile. I try to provide some context here based on a person's title/role or aspirantions here:

Security Managers/CISOs/CSOs
The SABSA framework integrates with just about every relevant standard or framework you can think of (ITIL, COBIT, ISO27000 series, etc) and a way for running security programs. It provides a method for delivering business driven security services with mesaurable results and a PRACTICAL form of risk assessment (the most practical I have seen). The risk management framework and how to measure security, will be worth the price of entry alone for you. 

Security Designers/Architects
I think you'll  come away with a similar view to myself and see how this all ties everything in together. Management, penetration testing, security standards, compliance, risk, technical controls and how to integrate them all - in a practical method. You will enjoy the focus of building and creating at a truly enterprise level and your game will be taken to another level.

Penetration Testers/Security Analysts
Pentesters are a funny lot with their training budgets.  Either they want to do Offensive Security course or for the more advanced ones, a top notch Immunity course by Dave Aitel - or they are usually happy to spend their own time with a few books and play around with stuff on their own. Now if you fall into the later category, you should definitely put it towards SABSA.

I mean lets face it - you're not going to put the budget towards anything else, why not learn how to review solution designs using a comprehensive strategy which will enable you to cover all the components? Why not learn how to link it back up to core business objectives? Truth is that will not take away from your technical skills - why not learn how to engage with the business at a higher level.

The short answer is if you haven't done the SABSA training, just get out there and do it. It will not takeaway from anything you have done to date. I can assure you it will only complement anything you do, regardless of what it is. And finally, I think it will really open your eyes in ways you hadn't considered previously. It won't solve world hunger and the "security problem" for good, but I will say that it is a damned good start. If more people had done this training in our field, I can honestly say that the world would be a safer place.

I hope this is useful.

Cheers,

- J.

Ruxcon 2010 & my talk: "No Holds Barred" Penetration Testing

EDIT: My Rapidshare link broke so I've resubmitted it using Google Docs.

Well my talk at Ruxcon is said and done. My slides can be found here.

Truth be told, it went better than I expected. I was worried that since I was not posting 0day code or providing a tool that there would be no interest. I don't think anyone was more suprised at seeing a fully packed room than me.

What started was going to be a series of grumblings about my views on penetration testing today ended up being more clearly focused on client side penetration testing.

I know plenty of consultancies are technically capable and some might even have done client sides, but to the best of my knowledge I hadn't heard of any in Australia doing so. And if they did, they kept their cards very close to their chest. Or more to my thinking - they have done it but I think the repeatability might have been lacking.

Obviously a lot of this is just guesswork on my behalf and there was a limit to how much research I can do this on this front - so a good chunk of the talk was based purely on my own experiences in the field as someone who has hired pentesters and now working at a consultancy where we do pentesting (amoungst other things).

One of my gripes is that we are lagging behind our competitors in the sense we aren't actually doing client side penetration tests (as a general rule) in AU, at least not on the scale that happens in other countries. I think this means that there is a significant gap in terms of the coverage and assurance that our penetration testing coverage truly provides.

I want to raise awareness of this issue and really try and provide some suggestions how both parties can lift their game and get more of these happening. How both parties can try selling the service, when it is appropriate, how to justify in a business context, provide ROI (yes, it is possible), etc.

On Ruxcon, I have to say I was really impressed with the quality lineup, the registration, the management of the event - even little things, like the way the bartab, water even the toilets was handled. Hell, even Black Hat @ Caesar's Palace ran out of water. What does that tell you?

The CTF ran flawlessly (barring minor performance issues) which when you compare to previous years, are still trivial in comparison. If I had to come up with one gripe, it would be that the area infront of the bar was too small for the size and we wound up having to really yell to be heard, so I've subsequently lost my voice. I forgot to mention I like the fact that despite corporate sponsorship, there was no blatant advertising, no "vendor streams" and no bias towards speakers who happen to work for sponsors. AusCERT take note!

I've said it before but I must say it again - it was by far and away the best Ruxcon ever. If you have to pick one conference in Australia, make this your one.

Cheers,

- J.

Ruxcon, SABSA and more random bits

Gidday

It has been awhile but I will do a proper blog update soon. This week has been a build up for me on a number of levels, I've been up in Sydney completing the SABSA training, trying to study for that, polish up my talk then race back for Ruxcon. That and trying to sort out some admin issues with university meant I've been significantly under the pump and it all came to a head this week. So subsequently I've gotten sick and had to leave Ruxcon early (clearly I'm not up for this touring lifestyle).

I wanted to talk a bit about my talk, post the slides and some other odds and ends, so I'll be doing that in the next day or two.

But just briefly -

Ruxcon 2010 was the best year in the history of the conference. Props to Chris and the crew for organising it and the success. It is very clear he's learned from the successes and failures of other conferences and avoided all the pitfalls. To the degree that I would recommend overwhelmingly if you had to pick one conference in AU to attend, without a doubt, make it this one. I was very disappointed by AusCERT this year and I can safely say that I would not be keen to attending.

Secondly, the SABSA training really changed the way I look at architecture. This topic deserves a post on its own right and it will be forthcoming. I want to stress that this course is of benefit to ANYONE working in infosec -- I don't care if you are a pentester, a manager or an architect. David Lynas was able to really highlight how it all fits together with real world examples.

I guess, I had a kind of Neo moment (ala. The Matrix) where I feel like I 'see' the code now. It is all a bit hard to explain right now (given my sinus infection) but let me just categorically state that this course, the book and certification I believe to be strongly worthwhile. So go pick up a copy of 'Enterprise Security Architecture' by David Lynas, John Sherwood and Andy Clark. You will not be disappointed.

Anyway, I will update more soon but as this is my first conference presenting and when guys like Brett Moore, Billy Rios and Silvio Cesare are presenting, its an honor to be on the same list of speakers.  I'm just grateful for the opportunity to speak. I sincerely hope that even if people disagreed with my views that some of the points made can be taken and leveraged in some way to gain better traction on client side penetration tests.

I hope you all had a terrific weekend and enjoy whats left of the conference.

Best regards,

- J.

PS: My apologies to Daniel Grzelak. If you read this mate, I know I promised to be there (as you are speaking as I write this) but I am really not well. Sorry to let you down, but I know you'll kick ass.

Why DO projects fail?

I don't want this post to degenerate into a mindless, multi part rant as if I have all the answers as to why projects fail. However I felt that at least one post one the most common cause of project failure that I have witnessed is most undoubtedly worthy of such a mention.

Strategic alignment. 

Well - duh, I'm sure most infosec guns out there would say. But what does that mean exactly? I had to conjure up a bit of a buzz phrase to encompass all the instances of which I've seen lead to project failure. So let's break that phrase down.


I'll give you a hypothetical example:
You are a security lead in a program of work. By program, I mean overarching stream consisting of multiple projects.
For simplicity's sake, lets assume it is a business transformation program where multiple core business objectives are expected to be achieved which will ultimately revolutionise the way that a business performs and conducts itself.

Now of those multiple projects, lets assume each one has its own PM. Then its own IT architect. It's own security consultant. It's own business analyst, developers, sysadmins, and so on and so on. Now lets's all assume they each look at their own project in terms of its overall objectives and work. Each project is meant to provide deliverables that fit into a cohesive whole (i.e. the program, as opposed to the project).

Still with me so far? Ok.

For a moment, let us further assume that each project meets its deliverables and timeframes within budget (ok stop laughing - we're pretending ok? so go along). Unfortunately, there is one limitation.

NONE OF THE PROJECT TEAMS CAN SHARE THEIR WORK WITH THE OTHERS.

What is the overall impact towards the program?

By assuming you're the program security lead, we'll further assume you have visibility of everything.

Now lets say you see that the program hasn't correctly co-ordinated all the work between project streams, and also further assuming that the program isn't going to invoke the Hand of God and didrectly meddle with each individual project when there is a problem, chances are that there will be a high cause of failure.

If each project is providing a deliverable that has no visibility of the end game or 'desired target' that is the commonality that all the individual project streams are supposed to be working towards, then nobody knows where they are going. Projects provide their deliverables in a vacuum. The net result is you will have a very unsatisfied business sponsor who is in all likelihood, going to ask the program manager to commit seppuku.

Infact, one of the sources I investigated as I was writing this stated the following:

"A project is considered a failure…whenever a project does not meet the expectations of the stakeholders."

This is a rather apt description I would say.

There are a raft of causes and reasons that projects fail. Many which you could argue aren't attributable to the above. However, if I was to pick the one cause of failure I see, it keeps coming back to a lack of strategy.

Some examples:

• Point in time technologies selected to resolve an immediate problem with no alignment to strategy or architecture.
• A program's risk registers not aligning to organisational risk registers.
• Program leads disallowing information sharing between project streams.
• Program leads not performing proper alignment of project deliverables to ensure a program's target strategic outcomes. 
• Ensuring the program has clearly defined outcomes from the onset (outcomes which are specific, measurable, achievable, realistic, timely - that is to say SMART)
• Maintaining clear (and consistent!) lines of communication breakdowns between project -> program -> business sponsor.  

 I could go on and on, but I'd probably start to bore you.

The fact is that there is a raft of material out there which highlights the common causes of project failure, and the reality is we learn more from our failures than we do our succeses. Given IT projects in particular have an aburdly high failure rate , there is a plethora of information available for anyone to draw upon.

I actually find this stuff interesting reading. If you read this stuff, I can almost assure you that you will be better equipped to recognise the symptoms long before the average PM does.

As a security professional, your efforts can only be improved by recognising the common symptoms and causes. Try to understand the business and outcomes which are both tactical and strategic. Try to ensure an alignment with both and get your business stakeholder engaged and supporting your efforts. Once you do this, funding discussions also come a lot easier. If you can do this, you will almost assuredly find allies - for example, the architect on your side as well, who undoubtedly also grapples with strategic vs tactical battles daily.

- J.

I am presenting at AISA Melbourne this Thursday

Gidday

For anyone interested, I am presenting at AISA Melbourne this Thursday for anyone interested. To be fair, it is largely content that we have presented previously for DD customers who had the privilege of seing this first. While it has been a few months on now, based on our experiences with the local market, I think the lessons are still rather poignant.

My objective is for anyone who is attending to get an idea of what was discussed at Black Hat and Defcon from a 1,000 ft view and hopefully get you all thinking in the right track of what sort of mitigations, controls and overall strategies to help you prepare for the next 12 months.

Details of the talk can be found here.

I hope you can make it. But if you can't or aren't interested, I hope you get to sleep in at least. :D


Cheers

- Jarrod

EMET for the win

Relatively recently (well, given my lapse in posting at least) Microsoft have scored a big win with their release of their EMET tool (Enhanced Mitigation Experience Toolkit) and some of the Adobe 0-days flying around. For anyone running a Windows based OS, I strongly recommend having a play around with this tool. I was testing it on a Windows 7 host but it will work on Vista, Windows XP (SP3+) and Windos Server 2003 and beyond. This tool enables users to apply DEP and ASLR to multiple applications, including legacy applications, effectively acting as a 'wrapper' (for want of a better term). Having tried this tool out I must say I am very impressed. It is easy to use and deploy and minimal hiccups.

One feature that is worth noting is that if you are running Bitlocker (as I was on my W7 build) then after applying EMET, after every reboot it will prompt you for your recovery keys. The solution is to suspend Bit Locker, reboot, then unsuspend Bit Locker (thanks to my co-worker Ed Luck for finding this fix and for putting me onto this tool).

I think this should be in the arsenal of every Windows system admin out there. Virtually every enterprise is running these operating systems and most of them have easy methods of deploying this. Even if you were to only ensure the core applications were protected (stuff like Office applications, browsers, Adobe and so on) you would knock on the head a good portion of the 'highly targeted' applications.

NOTE: ASLR + DEP is not a panacea for all your ills. Examples of defeating both are documented and widely known in the right circles.

That said, in terms of reducing the attack landscape Microsoft are continuing to push the boundaries and make it increasingly more difficult for exploits to work.

Kudos to Microsoft for the good work.

- J.

Dimension Data Are Now Hiring

To anyone reading, Dimension Data is in the process of building up its security practise. At present we're looking for a senior penetration testers or pentesters looking to step up into a leadership role within a security practise.

You must be proficient with infrastructure penetration testing , web applications, code review. We're not interested in someone that can only do web applications only, you must be capable with almost any sort of technical challenge. Solid reverse engineering skills would be highly desirable as well (can you pull apart Java thick clients or Flash/Silverlight applications?). As you can see we're looking for someone with very broad yet very in-depth technical skills.

For this role it isn't sufficient to be a l33t h4x0r -- we're looking for someone who has excellent soft skills and a positive attitude to match (speaking for myself we've got a very strong, very positive team and maintaining that environment is something we really pride ourselves on). For this role we're looking someone who can act as a senior consultant and can also act as a thought leader and mentor for junior pentesters and can clearly articulate technical risk to the clients in terms they can relate to and place it within context.

This role is an important role within the practise and definitely not one that any pentester can fill. But if you think you have the right combination of technical and soft skills, please hit me up. Happy to have an informal chat first if you prefer.

Also, I cannot state with any certainty (as this is outside my realm) but Dimension Data do have a history of sponsoring work visas for the right people with the right skills. Given that the Melbourne market is quite tight at the moment I think there is a strong chance that we could sponsor the right person for this role. I would happily help drive this internally for the right person.

So if there are any experienced pentesters reading from overseas who'd like to work in Australia, hit me up! Send me a message via LinkedIn or Twitter  if you're interested, we'll go from there.

I haven't got a job advertisement handy but I'll see if I can find one and add it here.

Cheers,

- J.

Why I Don't Like Time & Materials Gigs (T&M).

Before I became a consultant I had no idea what the hell T&M was. Why should I - I'd never worked contract before? Even when I hired consultants it was always fixed price engagements with clearly defined scopes (well - as clear as they could be!). Now I know what it is and I have to say, I am not a fan.

Now I'm not as militant as some of my peers on this issue. I am sure there are times when there are legitimate needs for T&M staff. However, I am opposed to T&M for consultants.

Normally when a T&M gig is sold, whoever is responsible for the pre-sales, qualifying the opportunity, the scoping, etc, they determine what the client's problem is. This affects the service that is delivered or sold. Now what winds up actually being sold can really vary. It's up to the client to clearly articulate the problem. But more importantly, its up to whoever is doing the pre-sales on the consulting side to really listen to the client and try to truly determine their needs and provide the best service to meet that need. Sometimes that aligns to a pre-defined service offering, meaning its a nice checkbox deal for the consulting company (yay!). Other times, the client has an unusual problem or dilemma which often means that the individual doing the pre-sales needs to go off the beaten track.

Now this is where things go a bit pear shaped.

What SHOULD happen is that if a consultant is required, the person doing the pre-sales should pin the client down on their exact problem, define the problem, what is the solution the consultancy will provide and the expected deliverable the client will get at the end of the engagement. As part of this, a timeframe should be provided when the deliverable can be expected. The consultant responsible for the proposal needs to allow sufficient timeframe for the consultant to provide the deliverable. Now all this is as dependent on the skill of the consultant to negotiate and listen as it is for the client to articulate their needs (so a lot can go wrong on both sides of the fence here - if you follow me).

What OFTEN happens is a T&M engagement is sold.

There are a number of reasons why a T&M gig gets sold (and don't get me wrong, it isn't difficult to see why they happen - but that doesn't mean I excuse it):

• The client can be difficult articulating their needs,
• The consultant can't be bothered with the pre-sales/scoping,
• The consultant honestly can't tell what the client wants,
• Either the client and/or the consultant aren't sure how long the engagement will go for,
• The client has an immediate staffing shortage and knows there is some urgent work required but not sure how much (e.g. when staff leave and there is no hand over, aka. bodyshopping),
• The client has problems with the finance/billing angle for fixed price gigs (e.g. budget oversight/lack of authority),

The last example was a real opener for me - and I'm not just talking because they didn't have purchase authority. I have witnessed instances where the client was supposed to have already completed the work but didn't - and because they couldn't get purchase authority for a statement of work (where the deliverable would be in print that it was for work that should have been completed) a T&M gig was sold. Very dodgy indeed - but I digress.

My main reasons for not liking T&M however are for these reasons:

• A consultant may not possess the right skills for the job (they might be ABLE to, but they might not be the best FIT),
• A client can change the scope on the consultant (because there is no scope - at least nothing concrete),
• There is no duration (which means the gig can potentially go on indefinitely),
• The deliverable is not clear (so the client may not be happy with the output, likewise the consultant isn't sure of what to provide),
• It just reeks of lazy scoping or justification (like the above example),
• There is no protection for the client, consultancy or especially the consultant on-site.

But the bottom line is this:

I have yet to see a T&M job for a consultant yet that couldn't be defined as a fixed price gig. Not one.

To make this all worse, a consultancy may have no interest in ending a T&M gig, because usually the consultant will be pimped out a good daily rate and even if the rate is discounted, so long as the opportunity cost is not extreme, then its all good as they can maintain their utilisation. Likewise, a client may not want to end a T&M gig either, especially if they're using the consultant as backfill or in lieu of hiring a full time employee or contractor. Should this scenario occur, then it is the consultant is might be left to struggle to meet the client's need (should the scope change, expectations change, duration change, etc).

Compounding this again, if the scope changes and the consultant struggles to meet the client's needs, the client can build the perception that the consultant is squandering their time, twiddling their thumbs. While the consultant could be (in some cases, if they're crap) but lets assume he/she isn't crap. They might be stuck there because they're having problems with moving goal posts, staff not handing them the information they need, multiple distractions because the scope has widened, etc. This can leave the client with a very bad impression of the consultancy and their ability to deliver. If this should happen, this can jeopardise the consultancy's reputation!

Now when I am on-site, I am there as Jarrod Loidl. I might be working for Dimension Data but at the end of the day, it is me there doing the work. So if the client winds up with a bad perception of my work and its because I don't understand what I am expected to do, or the goal posts change, then that affects my professional reputation as much as it does Dimension Data's. This is a small world and even smaller industry and local market where everyone knows everyone. You burn a bridge today - intentionally or not - and you never know how it may come back and bite you. More than that, however, I actually take pride in my work. I would hate for a client to ever think that of me. So the best way I know how to assure the client a high level of service is to tell them what they will get, when they will get it and how I will go about delivering it. Ergo - a fixed price engagement.

In short, in order for a consultant to provide the best service to the client, the scope must be defined, the duration clearly stated, the work to be performed understood and the deliverable clearly articulated. This ensures that the client receives the highest quality of work from the most qualified resource available for the best price. It protects the client. It also protects the consultancy.

To summarise:

• If you're a client in need of a consultant, don't settle for T&M. Don't ask for it. Don't accept it.
• If you're a consultant, DON'T SELL IT! Define the problem, solution, deliverable and duration. Do the hard yakka upfront.

- J.

EDIT: I should postscript this...
Firstly if you decide to do T&M, that's your decision. But at least call a spade a spade and acknowledge that the practise is bodyshopping.
Secondly, this is something I just found today and I think everyone should read this "No One Nos: Learning to Say No to Bad Ideas", This is something everyone should do more in the workplace IMHO.

Security in the media - truth versus reality

Recently Dimension Data are presenting on a number of topics learned from Black Hat and Defcon and the particular emphasis is on social media as well as mobile phone security.

I recently delivered the Sydney talk with a colleague doing the Melbourne one, with more on the way in other states. Now what I enjoyed about this talk was that it was that we were able to discusst attacks either demonstrated or theoretical discussions based on known weaknesses and how simple they were to exploit. The defensive strategies was where things get really complicated because they involve a number of serious issues.

For example - what is a "secure" use of social media in the workplace, or better phrased, what is an "appropriate" use of social media in the workplace? If you ban it in the workplace, you can't ban it outside of the office? If that's the case, then how do you secure your employees for threats ranging from phishing attacks to targeted malware and the latest buzzword "APT" (Advanced Persistent Threat)?

Mobile security is equally bad. The technology is evolving and spreading faster than our ability to secure it. In the desire to go to market means that factors like usability and client experience trump security, every time. Past lessons have been ignored, or assuming that mobiles are somehow immune to the security threats that affect our desktops, servers and notebooks daily.

Without going into too much detail (or belabouring the whole APT thing), these are complex issues with easy answer, largely dependent on a number of variables:  an organisation's risk appetite, company culture (open vs. closed, trusting vs untrusting), degree of secrecy required, how connected they are to the outside world, staff mobility, etc. And that's just off the top of my head.

However, these issues and while we love to talk about stuff like this, cloud, etc, its interesting that so few people really talk about the fundamentals. The Wall of Shame series discusses these fundamentals and highlights their failures. Yet you rarely hear about how people get owned due to ignoring the basics. Media focuses on sensational stories that sell. CIOs want to read about cloud security because they're looking at cutting costs by eliminating data centres. They do not want to read how their patch management program is failing because application owners are risk accepting not patching their environment and jerking around their sysadmins by refusing all maintenance windows. Or their new vulnerability management system is a waste of money because it spits out reports that aren't actioned on.

Unfortunately, this is often what security comes down to.

Working in information security is not always fun, sexy, interesting or glamorous. Infact, it can be - and often is - dry, tedious, boring and often stressful.
(Caveat: what follows is a description of what it is like working in infosec for the uninitiated. For people who think its all about penetration testing, vulnerability research and exploit development, read on at your own discretion).

• Imagine sitting in meeting rooms, having "spirited discussions" with application owners, system admins, business analysts, project managers, line managers, software developers, auditors and convincing them why they need to perform data validation on a web application during development, why the sunk cost is necessary and also why you need a stack of cash to perform penetration testing.    

• It means reviewing SIEM logs, addressing false alarms by hunting down the root cause and fixing the bloody problem as opposed to creating a filter to ignore the white noise. So unless you work in a place where you have admin rights on everything, this often means talking with the appropriate techie, raising a change request, going through a lot of red tape for a 5min change.

• It means reading vulnerability reports and actually getting them fixed. Talking to the asset owner, making them accountable, raising it in the risk register and getting it recorded, ensure they get it fixed within a mutually agreeable timeframe that doesn't fall outside of the next six months.

• It means reviewing the patching process and explaining to people why not patching Flash, Reader and Java represent massive security risks. If you somehow convince people that this is a serious enough priority for the business (good luck with that) then you have to explore technology options that will enable you to patch all these applications, the processes by which this will be achieved, who will do it and how to minimise outages and business impact, etc.

• It means creating policies, procedures and standards that people can read, understand and work with and reviewing them regularly. It means satisfying the right stakeholders and getting the right buy-in from execs to endorse it. And if the policies are broken/not working, having the sense of mind to honestly critique them and evaluate whether the policy/process is flawed or whether people are just being too lazy and not adhering to it.

This is the stuff you won't see in films or read in the news yet it is a good chunk of what happens every day. Moreover it is these battles that are the ones most infosec staff struggle with, the ones which are the most important and sadly the ones that get the least coverage. The media is always so concerned with the latest threat, hack, exploit, terminology or technological trend. And this isn't unwarranted or without understanding.

I guess what is disappointing is that we don't spent time on these basics enough. If you get your detection capabilities right, your basic patching and vulnerability management processes up, you have a lot more time to devote to the more 'interesting' parts of security.

I used to love reading a lot of bodybuilding magazines growing up, and in retrospect, I now find it hillarious reading about 'shock' programs designed to add 2 inches to your biceps in a month or some crap. What isn't drilled in enough is that people need to clean up their diets to get the results they need. Instead they'll interview some steroid laden Olympian frontrunner and ask about his routine. Said Olympian presents an incredible program which will cause overtraining in no-time for the average reader because they're ignoring the fact that this guy had his diet micromanaged for years prior to even touching steroids, takes an afternoon nap to aid recovery and then is juicing like a madman to facilitate even more rapid recovery.

While I am not excusing or condoning that behaviour, I am making the point that these are people who have mastered the basics in their field. These magazines would be more effective from a training view point if they interviewed bodybuilders and asked them how they managed their diets, how they prepared their meals, how do they cook, how to they deal with temptation to cheat, how do they deal with restaurants, etc.  Whether they would be commercially viable is another point entirely.

As a consultant, I'm finding more often than not, I'm called in to evaluate a particular project, design, issue, etc, and the threats that the client is concerned with is insignificant with the real threats that are there, clear and present. I think this myopic view comes from seeing the issue for so long, your mind tunes it out. In these instances I simply point out the white elephant which is being ignored and remind them that they have bigger problems. Helping to put some context around this and prioritising threats is some of the fun parts of the job.

I think one of the reasons I haven't blogged for awhile, isn't so much due to a lack of passion or time (although the time factor has certainly been there!) but rather that the more I look, the more I see it is these same basics which need tending to. I don't see a deep mystery with it for the most part. It just seems more and more like common sense - which is often in short supply.

I guess in many ways I see the media playing a role in not addressing them. It would be nice to see journalists being more responsible here. There are some journalists I used to follow quite closely because I thought they were authoritative, information and interesting. Now I am a little more discerning with my time.

I can see why these same fundamentals are not attractive to the media or its readership. The truth hurts after all. But it would be nice once in awhile if journalists did a post incident review using root cause analysis. You could still get the scoop as well as intrigue your readers and maybe educate them at the same time. I certainly see this responsibility as one that should lie predominately with media outlets, journalists and publications that pride themselves on being "security journalists" or targeting "security professionals".

I don't expect journalists to try and change the world, I'm just talking about journalists aiming to do a better job of reporting facts and educating readers rather than using FUD to increase readership. Is that too much to ask?


- J.

My Ruxcon paper has been accepted

Gidday

Just a quick update to post that my talk for Ruxcon 2010 has been accepted. For anyone unfamiliar with Ruxcon, it is the premier technical security conference in Australia. For more information click here.

I've been to Ruxcon for several years now, going back to 2004. I can honestly say that depth of the content, value for money and international standing when compared to other conferences, it is unparallelled. The fact it is in my home town of Melbourne this year is just a bonus.

For anyone who has heard of this conference, thinking of going yet undecided, or never been but curious - I strongly urge you to attend. In particular, if you are a security consultant, penetration tester or security manager/security lead within an organisation, I strongly urge you to attend my talk as I think you will come away with something new.

Cheers,

- J.

Learnings from Black Hat/Defcon

I have finally returned from Las Vegas and had time to recover from jetlag as well as digest events at Black Hat and Defcon. At work we will be doing a presentation on this for our clients and we will be putting our own takeaway message and making it specific and meaningful for the Aussie market.

Without spoiling that, I will post on some of the things I saw and my own thoughts.

In essence, the goal posts are shifting. The ubiquity of smart phones, mobile devices, gaming consoles, etc and enroaching Internet connectivity to every device with a microprocessor means that all these devices are going to be targeted. Proof of concepts of exploits were pinpointed for IPhone (which got an utter drumming), Android, Wii, Nintendo DS, and more. Also, home/SMB routers are now proving to be a very viable targets for exploitation and information gathering.

Social networking and the ability to restrict access to it as well as information on it - is becoming less of a possibility as it becomes increasingly a decision about whether one chooses to participate in society (as Moxie Marlinspike so eloquently put it). I am really pleased to see this talk do the rounds. Applications that we use within these environments (and the social networking sites themselves) simply cannot be trusted. Site owners simply do not perform sufficient checking and audits of code prior to use on the site and easily enough there are multiple methods one can exploit trust relationships as well as lax technical controls to abuse the system.

Attack tools are becoming increasingly commoditised. This is, and has always been the case for as long as I can remember. But just seeing the number of Metasploit plug ins that are becoming available really blew my mind. Drivesploit, the Social Engineering Tool-Kit, Powershell and more.

While we have known about this for quite some time (nothing I've said above is really altogether new), it was very illuminating to see so many people thinking along the same lines and gives an indication to the prevalence of current attack taxonomies as well as future trends.While it doesn't take a rocket scientist with a magic 8 ball to figure out where its going (it seems some random with an Internet connection can), I'm pleased (?) to see that I'm not far off the mark.

I've frequently grumbled in my Wall of Shame series about security failures, but what really hit me - particularly when I spoke to other security professionals is just how far ahead many US companies are with their security. Granted they are already in a much more regulated environment, but many of the basics I've grumbled about in my Wall of Shame series  - while they are not resolved per se, it is clear they are much further along the road to maturity than we are here in Australia.

I've had some discussions with a few people in the industry about this but my honest belief is that we are behind because of our cultural attitudes. To add to my earlier predictions, I believe this means we are going to become increasingly a target for organised crime, malware authors, hackers, and the like. While its old news that the Russians prefer targeting Australian banks, we haven't seen anything targeted on that sort of scale (apart from card skimming and ATM fraud). People talk about a 'Digital Pearl Harbour' and while there are skeptics, it doesn't take a mathematician to realise that this will probably happen at some point. Sadly, it will take an event of this magnitude to really change things here in Australia. And unless it does happen on our (virtual) shores, then I doubt anyone in this country will really pay attention.

I guess I find it somewhat scary.

The fact is we're not ready. Defcon and Black Hat really highlighted just where this going. I think I always knew and my earlier forecasts were an indication of the writing on the wall. But seeing the exploits, proof of concepts, software all in the public domain really highlighted that it is becoming easier and easier to compromise systems and steal data, all the while becoming increasingly more costly and difficult to secure the information. (I would have said the same six years ago however).

Plus ça change, plus c’est la même chose.

Of course, every security conference its easy to say its all doom and gloom - they're all in the business of dispensing FUD it seems. Of course that doesn't mean however we shouldn't take it seriously - however, it does mean we need to not only lift our game.

It's time for Australia to really pull its socks up.

Here's a quick brain dump on defensive strategies that I see must be deployed today:

• Solve the patching problem (if you haven't solved this, then you fail by default);
• Basic config issues + password weakness (fix this for gods sake, its easy enough);
• Lock down applications on the desktop. Rely on an SSOE where you can, focus heavily on application white listing and malware analysis tools with heuristic/behavioural analysis capability;
• Re-consider admin access to security devices and networking devices (do you have a separate management network? two factor authentication?).
• Consider how you deal with social media in the workplace (personally I'm in favor of banning it in the workplace but I recognise this isn't a popular choice). Whatever you decide, have a strategy and a policy to match;
• Start focusing on social engineering as a testing and training element for security (we've known this for ages but too many ignore this). Include these elements in your pentests.

- J.

AU Government Censors Document On Planned Web Snooping

I'll just send you to the Slashdot article as it has it all there and I'm getting ready for Black Hat/Defcon.

Just in time for the impending federal election...

Best quote:

"Online users' lobby group Electronic Frontiers Australia spokesman Colin Jacobs said what was released was "a joke".
"We have to assume the worst," he said. "And that is that the government has been badgering the telcos with very aggressive demands that should worry everybody."
Emphasis mine.

- J.

Default Passwords and SCADA: Siemens Fails

Finally we're seeing malware in the wild targeting SCADA systems.What is the root cause analysis? Siemens have a default password. Of course, Siemens are not advising their client's to change it for fear of breaking communications between WinCC and the database.

I can appreciate Siemens concern for interrupting business operations but time and time again - fear of interrupting business is not a reason for ignoring security threats. It is a consideration - sure - but not a reason for ignoring.

This is a prime example of security fail.

Why can't Siemens advise changing the password but stress the potential business implications? Other mitigating controls might include:

• disabling USB drives,
• application whitelisting,
• operating system hardening,
• segregation of management networks,
• disallowing critical infrastructure direct Internet connectivity.

The list goes on. Why not advise clients how to change the password within the application and database? I'm obviously presuming it is permissible - if not, clearly it is a double fail.

Anyway, I realise SCADA infrastructure often runs on ancient, unsupported operating systems and patch levels, but other controls can be applied to reduce the attack surface and potential damage such malware can perform.

While this is an excuse that is often thrown around in enterprise environments time and time again, what is interesting now is that it is being thrown around in relation to critical infrastructure, (presumably) arising from industrial espionage.

Some relevant links from Siemens can be found here and here (official release) - and yes,Siemens official release is terrible.

- J.

EDIT (23-7-2010): AusCERT have released a bulletin on this. It's a good writeup - I heartily endorse anyone interested in this subject to read it further.

Internet Censorship in AU, Black Hat/Defcon

I have had a sudden drop off posts this past month, owing to a combination of factors. Firstly I was flat chat finishing a unit in my Masters, then it was the inevitable burnout of posting + study + full time work and other commitments. I needed a break to re-invigorate myself.

With that said and done, there's two things I want to post about:

1) Internet Censorship in Australia
What is the current status? Well, with the impending Federal election, this politically touchy topic has been put on hold. Yes, while Conroy has publicly backed down, it is important to realise that this topic is NOT DEAD. Labour STILL want to roll out the policy. Also, where is the Coalition on this policy? Strangely enough they aren't saying a hell of a lot either. Now you have to ask yourself "why would the Liberal Party remain relatively silent on this issue?" The obvious answer is that they want to roll out this policy too but they don't want to publicly say it. I hope the public continue to apply pressure on Tony Abbott and ask him questions publicly and grill him on this offensive policy.

The only political party with a truly vested interest in truly eliminating this policy is the Australian Sex Party. While I don't want to use this blog as a political platform, its pretty clear that the Australian Sex Party cannot support such a policy without offending its entire constituency and gutting its support base.

I encourage anyone who actually cares about this election to really research the policies of the political parties out there and vote for the party you believe will best represent your interests.


2) Black Hat USA 2010/ Defcon 18
I will be attending Black Hat USA and Defcon in Las Vegas this year. This will be my first time attending the conference, I'm keen to learn from the industry's best and brightest as well as network with people and generally just have a good time. Regrettably, I'll probably have to work on my assignments for my MBA as well but hey, thems the breaks.

When I get back I'll be sure to provide a post or two about what happened at Black Hat, takeaway observations, lessons and trends.

I'll make a more concerted effort to post more frequently.

cheers,

- J.

Enterprise Architecture and how it can help Security

Enterprise Architects don't exist in every organisation. You'll typically only find them within enterprise scale environments. Even then, not every enterprise has one. Kinda sad but in an ideal world, an EA or an EA team are basically the true interface between the business and IT. I am a big believer that EA is a true enabler of good IT practise but more than that, good business practise too. I believe that security architects, security analysts, security managers, should all be working with their EAs in their business.

I will attempt to explain why - but firstly I will state this article makes a lot of assumptions. So most of this is based on theoretical best practise EA. Bad EAs or bad enterprise environments can easily screw this up but then again, same applies for security too.

Without further ado:

1) EAs have access to the C-level executives

Typically they deal with the CIO (at least they should be). This means they understand what is happening at that level. If they have the CIO's ear then they know what their concerns are, what their future goals are or possibly future direction. All useful intel to any security professional.


2) EAs understand the business

One of the earliest activities an EA does is deconstruct the business down to a series of key business processes. They also map out what infrastructure, applications and environments these processes use. This is critical information to any security professional looking to deconstruct the business and map out the security architecture. It can also be useful for determining high risk business processes, disaster recovery, business continuity planning, etc. The value of these artefacts cannot be overstated.

3) EAs have oversight of a lot (if not all projects)

A lot of the time, they are members of an Architecture Review Board - which can often mean they have access to any project going through a Project Management Office. Alternatively because of their acess to the C-level, they might be privy to rumors on the horizon for projects with rather scary security implications (e.g. transformation projects, outsourcing projects, etc). If you can, Security should have representation on such a board. Even at an informal level, they might just let you know what changes or projects are on the horizon - and why you should be concerned.

4) EAs recognise the value of security

Most EAs recognise that security is simply another architecture domain that requires specialist skills and knowledge. They should be engaging you and likewise you should be engaging them. A security architect that is a member of a EA team is able to see the business at a higher level, see the same things the EA team does and better able to align security with business need.

5) EAs can champion security

Closely related to point four, even if security is not adequately represented within an EA team your input can influence their decisions. You might be able to sit on an Architecture Review Board. They might be able to raise your concerns to the CIO. They might be able to incorporate your requirements and feedback into their designs.

6) EAs set (or at least influence) the technical direction

As EAs review the technical landscape within the business, they will often seek to consolidate around a series of applications, infrastructure components, etc. They will define technology stacks, roadmaps, etc. This can be invaluable to an savvy security analyst looking where to focus training (not just for themselves but other staff in key areas) but also investigate security solutions, guides, documentation, etc - utilising those same technology choices.


-----

In summary, the forward thinking that EAs do every day can be invaluable service and value add to any time strapped, operationally focused security analyst. Even if you are the most strategic minded security architect, you should still be aware of the value Enterprise Architecture brings to the business. And when I say 'business' I mean information security too.

Presentations for Dummies: 7 Tips for professional speaking

The original intent of this blog was to discuss my lessons learned along the way in my career and share wisdom where I think I have something to offer. It is in this vein I write up this post and I feel it would be remiss for me not to talk about some recent lessons I've learned on public speaking. Now I love to talk (I think anyone with a blog does really) but I put my hand up to do some presenting because:

     a) I think I'm terrible at public speaking but I want to get better at it.
     b) I think fear or nerves is no reason to not attempt something.

Sure I get a bit nervous, but its the same nerves I used to get before going into a tournament. Jittery. Adrenalin. A bit of excitement. I want to get it on and get it on now (or in this case, over and done with!).

Now, I've done some public speaking stuff before - mostly small forums and groups but this past week saw me do three presentations on the same topic in the same number of days, across different states. So in the spirit of this blog, I had to share what I've learned - the things I did right to the things I did wrong. This post is as much a reminder for myself next time as it is to anyone reading.

1. Drink plenty of water
Now I've done a lot of weight lifting and martial arts training over the years, sometimes 2-3 hours or more of hard training back to back. You'd think I'd be aware of the importance of hydration. Apparently I'd forgotten. :(

Talking for 60min or more gets you dehydrated MIGHTY fast. That one glass of water on the table next to the podium isn't going to cut the mustard. That's there incase you get parched (or need to slow down your talk to buy time). Loading up on the caffeine before hand to wake you up may sound like a great idea to give you some GO power if you're running on vapour, but it is a bad idea if it isn't followed up with at least a two glasses of water for each cup of coffee you consume. If you're presenting during a breakfast slot, start your day with a litre of water before you hit the caffeine. Then follow it up with water. That way you can still have your go juice without stalling.

2. Consider The Logistics
One of the places I presented in was a very, very narrow hall. I was positioned at a podium that had about two feet of clearance from the projector screen and I barely had the room to stand in the one spot - certainly not enough to pace around (I would have had to cut infront or behind the screen). To make matters worse, I was stuck directly under a heating duct where the temperature rose by 10 degrees relative to the rest of the room (I imagine being under a headlight would have a similar impact).

I had never even considered this aspect while presenting. My prior prep focused on timings, content, rehearsals, dry runs, etc. Not this. As a result of this (and  #1), I choked during one of these presentation at least two, perhaps on three occasions. Not due to nerves but I couldn't think! I was dehydrated, hot and my tongue getting tied. This had never happened to me before. I wasn't prepared for it. I got through my content but the feedback afterwards reflected the performance (although the content seemed to be valuable to everyone).

So, the takeaway lessons there - get to your presentation venue early. Assess your speaking area. Ensure you do not have an uncomfortable presentation area, that you have room to walk around, that you have a clicker for moving through slides as well.  Don't be afraid to change the environment to better suit you (as much as you can). If you can, provide feedback to the organisers afterwards what you didn't like about the presentation environment.

3. Preparation
This is something I think I do well (and according to some, I overbake). But of course it is pretty bloody obvious to anyone that you need to prepare. But I don't think there is such a thing as overbaking when it comes to presenting. Know your content - inside and out so you can then justify sweeping statements or otherwise be clear on the exceptions. Be familiar with your timing and delivery of your content.

This time round was the longest presentation I have ever done. Due to specific circumstances relating to my content, I had to produce additional content that wound up with me effectively doubling the number of slides. After cutting this down by 25%, I still had a LOT of content to hammer through. Being thorough with my timing was all that kept it on track.

Takeaway lesson - know the content, rehearse it, rehearse it, rehearse it. On your own or with others. If you don't like it, change it.

4. Content & Flow
For this presentation I produced the content originally for a technical audience. I then had to re-engineer it for a business audience as well and then try to find a middle ground between the two. This is not a good way to prepare a presentation.
 
As a result of this activity, I had too much content. I also felt it missed the mark for both audiences (but again the feedback here was positive so I have to take that onboard). But during the initial prep, it got to a point I had so much content I had to ask myself "what the hell am I trying to say??".

I literally went back to the drawing board. I put up on a whiteboard all my key points and how I would like to structure them. I then moved slides around, moving some content to my presenter notes and then with despair in my heart, gutting slides that weren't consistent with the messages I was trying to convey. Even then, the final product was much clunkier than other presentations I've done in the past. I didn't like the flow and again, this was reflected in some of the feedback.

To be fair , I would never deliver a presentation in this manner normally. Typically I would START with the message I am trying to convey, distill that message down into several key pointers and build a presentation around those pointers. Next time I will stay true to this and start from scratch.

Key lesson here - start with the message first, less slides doesn't mean you have less to talk about. 

5. Pacing
Closely related to the last two point, I feel I learned a lot on the importance of pacing. My original presentation was around 114 slides with approximately 70min speaking time. That works out to 1 slide every 45 seconds. That works great when some slides are intended to be rapidly flicked through - but what about those slides that aren't?

This is where presenter view becomes critical for note taking, marking in your timings as a reminder as well as dry runs to ensure you're familiarity with the delivery of the content AND the timings.

In the end I got it to a sum total of 80 slides, only 77 of which I had to present. This, with my 'skim through' slides, worked out to roughly 1 minute of talking per slide, which flowed much, much better. I still had too much content in retrospect, but hey, at least the flow was greatly improved.

If you are building your presentations from scratch for one intended purpose and do not have to retrofit it as I did, I suspect you won't have the same problems I did.

Lesson - do your rehearsals. They help identify pacing issues. Also, mind your talking speed. Don't talk too fast!

6. If You're Travelling, Check-In Where You Are Presenting
On the advice of my manager, I actually wound up checking myself in at the various hotels where I was due to present. This was a brilliant idea and I was so grateful for doing this.  I flew to my destinations during the day, arriving at my hotel around dinner time, leaving me plenty of time to eat, rest up and rehearse.

This meant the next morning I didn't have to wake up extra early, book a cab and race to where I was going to present and dodge traffic. I wound up having breakfast with my co-presenter and we had ample time to not just discuss the approach but relax.

If travelling, always stay where you are presenting (if you can).

7. Be Prepared
If you're travelling, fly a day in advance and give yourself time to adjust to any new timezones. Bring business cards. Anticipate the sorts of questions you're going to be asked. Bring your customers. Be able to cite your references (audiences LOVE supplementary reading material). Ensure your content is made available online afterwards (e.g. PDF on a website). Bring your own clicker.

I did all this except the last one. Well, I brought one with me but I didn't test it before hand, it was busted and I couldn't get it to work (fail). I've decided now that they are worth their weight in gold. I will always have one from now on.

--
Anyway, I hope this helps you all as much as it has helped me. If anyone has any tips of their own, throw them up here. Next time I'll refer back to this post just to remind myself.

- J.

Lamo vs Wikileaks + Manning

The recent emergence of the alleged source of the 'Collateral Murder' video has left a bitter taste in everyones mouth. 22 year old Bradley Manning was an Army Intelligence Analyst with Top Secret clearance has been arrested (and at my last check, held in a Kuwait prison) awaiting a hearing. He has allegedly provided Wikileaks vast volumes of classified data. The most controversial which - has not yet been disclosed - over 260,000 pieces of classified US State Department diplomatic cables. Manning was arrested on the basis of chat evidence provided by one time hacker, full time 'journalist' Adrian Lamo. Lamo, rather than disclose this to the police, feds, military, anonymously if need be - decided to go to Wired.com, his mate Kevin Poulson to get the story out (Wired.com has all the key articles on this topic. This one being the latest). Patrick Grey, producer of the Risky Business podcast also interviewed Lamo and actually did a good job of putting the hard questions to him (so I thought).

I won't say too much but I think Lamo did the right thing but for all the wrong reasons. I hope he's being truthful with himself because he's got a lot on his conscience. If it is true that Manning was behind it, then God grant him mercy because I don't think the U.S. military will give him any.

I think there's a lot of interesting issues that should/could be explored. I won't prattle on but I'd be very keen on hearing views from anyone reading as to what they think.


- J.

EDIT: Now the transcripts are out on Wired, I'm kinda torn. On one hand it seems blatantly obvious Manning was heavily disillusioned based on the crimes of his government. Actions such as the disclosures he allegedly made (it is still alleged isn't it?) are how government imbalances and abuses of power are identified and called to task. All the same, should there be no penalty attatched for people who abuse their access? I mean this is the classic insider threat scenario that all security professionals should be dreading. It isn't a textbook answer here. I guess both parties can only take action as their conscience deems prudent. It just seems unfortunate that Manning is probably going to get roasted alive for following his.

Counterpoint: Commoditising Penetration Testing

Recently Drazen Drazic put out an article on his blog at Beast Or Buddha about penetration testing and how it has become heavily commoditised. Or more specifically how businesses and individuals are attempting to do so, when infact they can't.

Drazen points out that there are some things you can commoditise, good penetration testing you can't. While I agree with this 100%, I do disagree with his view on commoditisation. I hope I can present a different viewpoint.

Commoditisation in this sense means that the skills required are becoming something that can be trained. Look at penetration testing back in say 2000. It wasn't that many years earlier the first book 'Maximum Security', the first book to really look at this in any depth was written. Hacking texts were limited to old BBSes, early underground forums, newsgroups, etc. You had to know who to talk to and where to look. More often than not, you were told to go figure out stuff yourself. If you had to ask, you fail. So in its earliest forms, hacking was really a form of exploration, self-discovery and learning (and understanding) technology. It was like this for many years prior to 2000 as well but the point being is that this is the mold that many of the "old school" penetration testers come from.

Fast forward to 2010. We now have course after course after course, certification after certification, book after book. For those of us who have been at this long enough many of these options are regarded as a mixed bag. Some people see these as "dumbing down" the content and presenting the technical tools and methods without any understanding/appreciation for the underlying technology. Others see them as not providing true value to prospective students. Some see them as an entry path into the field, lowering the barrier to entry that has often existed in this field. And there are many compelling arguments for each position.


Where I sit, the commoditisation is good.  I'll try and explain why but I'll also explain why commoditisation doesn't actually hurt penetration testing.


1) Unit testing

Many of the tests to perform a specific function are very simple. Lets say you want to test that input validation is working on a single form field. It is relatively simple for anyone to peform this test. Training your application testers on basic web application penetration testing (lets assume developers are doing some form of development training) means then they are able to validate that no cross site scripting is present. While this certainly doesn't obviate the need for penetration testing at the final phase or even multiple phases, what it does do is reduce the likelihood of critical security errors common to applications. It reduces the load on the penetration testers. But most importantly, it means these defects are detected earlier. Now, if you're a security manager overseeing large volumes of application development and testing, wouldn't you want to see your testing teams attempt to validate your security requirements as much as possible prior to obtaining independent penetration testing? I know I do.

2) Trained vs. Skilled

Secondly, good penetration testers is actually hard to come by. It is a very specific skillset that requires someone with a good grasp on just about all technical areas and in some cases, a specialisation or two. Networks, operating systems, applications, databases, programming. All this and more is part of their skillset. They understand how people think, how programmers write code, how administrators handle their environments. They can think like an attacker and more frighteningly, it comes naturally. They can pull apart just about any piece of technology you give them and figure out what makes it tick and love every minute of it. Now do you really think any training course can commoditise that skillset and that mindset? Hell no! That comes with instinct and experience - something no course alone can provide.

3) Nature vs. Nuture

Thirdly, apart from the fact this is a skill you cannot train, it is one that must be nutured. Most in-house security teams won't pay their staff to continually upskill themselves, stay abreast of the latest technology and trends. They really get to go on courses, let alone conferences. Don't even get me started on building test labs or pay for software licensing. Most of these guys might do it on their own time but they won't remain inhouse security when just about any consultancy team will grab these guys, pay them a lot more money to do what they love to do, pay for all their training, toys, etc AND do it all on company time.  And given that in-house security teams cannot retain good penetration testers for long, surely it makes sense to go to a consultancy that you rely upon as a trusted partner? That makes perfect sense to me.

4) Methodology Matters

Fourthly, the methodology that a lot of these consultancies use is to make penetration tests a measurable, repeatable process. That's not to say all penetration tests, or the testers, are equal, but the intention is to come up with a proven approach to ensuring that each test will yield a high quality result as possible. In that respect, I actually think this is a good approach. Not just from ensuring the quality of the testing but even from a viewpoint of training up your new staff (who may be very technically proficient btw). Now the methodology is not the "be all and end all" but the fact that it can provide a common framework designed specifically to yielding results based on a proven approach cannot be understated.

5) Knowing The Path vs. Walking The Path

I'm paraphrasing Morpheus here, but this is the final distinction between 'good' and 'great'. A decent penetration tester may be able to follow their methodology and generate consistent results and findings for their client. However, a great pentester is one that is able to look at their methodology, their current findings and say "I know the methodology says this, but I am going to go off the beaten path and explore this...". They are operating on either a hunch (which I covered above), evidence, whatever -  there are somethings that simply only experience can yield. And this is the distinction I am trying to make. No course or certification alone is ever going to provide this edge. Ever.

6) Increased Awareness

Raising the profile of penetration testing is good for the profession. It is good because IT shops begin to realise the value of a good pentester. Consultancies develop a clearer picture of what different roles exist - and that a penetration tester becomes a clearly defined role, rather than an implicit skillset that every security professional is assumed to have.

Now I am not saying the issues raised by Drazen aren't that bad - they are. But he did neglect to touch on these points I've raised and I think its remiss to not mention these benefits.

There are always going to be consultancies that try to sell vulnerability assessments as 'cheap penetration tests', or tools like nmap and use open ports as 'critical security defects'. There are going to be people who rely on certification as evidence of a great penetration tester. There are going to be consultancies or individuals that just deliver terrible quality penetration tests and useless reports. C'est la vie.

However, that's why it is up to security professionals to cast a discerning eye over these potential partners. It's up to us to provide that insight and find the value.

In my last role I worked with a few great security professionals and we had to evaluate many consultancies and their penetration testing offering (so it was great, I got to see a stack of our competition in the field now! LOL). Here's some of  the tips my teammates and I used to help flush out the good guys:

FIVE WAYS TO FIND A GREAT PENETRATION TESTER:

1. Ask for a copy of a sanitised pentest report. 
What are the findings? Do they discuss serious application faults? Are they able to articulate how this vulnerability can lead to a business impact (and I mean a business impact, not a technical impact)? You'd be amazed how rare this ability is. Poor ones often just do a bad job alround of explaining what a technical risk actually means.


2. Ask what conferences they go to or what professional associations they're apart of.
Do they share their information? Do they present? Do they mingle with other security professionals and swap war stories from the trenches? Some pentesters keep this information very close to their chest and don't like sharing. While this doesn't make them any less technically capable, I question the wisdom behind this approach. I believe security professionals should be able to educate the wider community as well as our own.


3. Ask what research are they doing.
What projects are they working on. Are their tools they are developing? Are they doing vulnerability research? Perhaps its a presentation they are working on? Research represents an interest in their field over and beyond what a cert monkey is capable of. Additionally, some security consultants that perform pentests are often people who are heavily utilised on other projects, in which case penetration testing might be just one of many service offerings. This means at best, they might be good but they definitely won't fall in the "great" category.


4. Ask about their methodology
What does it align with? What doesn't it? If it doesn't align - ask why? Sometimes a company will borrow heavily from a standard but choose to modify it or disregard it based on their own experience. Some of them will actually have their own and it will borrow heavily from several standards. Any group that is consistently refining their metholody and can cite their influences gets big brownie points in my book.

5. Ask what their speciality is?
This is a trick question really. Don't trust anyone who says EVERYTHING. A consultancy may have capability in all areas but either their collective skills will fall into a particular arena, or their individual testers will have unique specialisations. So ask the testers themselves and listen to what they say.


- J.