Wednesday, March 16, 2011

I love WAFs and so should you.

Warning: I work for Dimension Data and we're a known reseller of Web Application Firewalls. If you think that alters my opinion on this subject, then I suggest you stop reading.

Lately I've seen some hatred thrown towards Web Application Firewalls. Some of it I think is misguided, some of it misunderstood. However - and perhaps more disturbingly - a great deal of it just strikes me as wrong. I am of the belief that much of the antipathy thrown towards these things comes from people who have not worked in enterprise environments where application development is a common part of IT. This means there are a lot of assumptions on the degree of difficulty in either patching an enterprise application or the speed in which a hotfix can be applied. This post is aimed at reviewing those assumptions and clearly separating fact from fiction.

Firstly, I would encourage everyone to read OWASP's Best Practise Guide: Use of Web Application Firewalls to ensure we're on an even understanding. This is a really good read and I think gives anyone with even a foundational understanding of web application security a good overview of their strengths and limitations.

One of the constant criticisms against WAFs is that they are essentially a blacklist model which applies pattern matching based on a known vulnerability ("known bad") which is always putting the defender on the back foot. Further criticisms highlight that the time spent configuring the WAF for that bug could be better spent fixing the application. I might have missed some so if I have, please fire away.

To draw this conclusion you would would have to be operating under a number of assumptions (so I'm taking a bit of a punt here, based on conversations I've had or my take on what these people are thinking - feel free to clarify if you think I'm mistaken):

  1. It is equally as quick to apply a fix to an application as it is a WAF
  2. The cost in time in implementing a hotfix on a WAF is equal to that of fixing the original defect in the code.
  3. Applying a hotfix in a WAF and the code is "the same thing."
Now going back to the above link:
"The main aim in using a WAF is therefore securing the existing, often productive web applications, where the required changes within the application can no longer be implemented or can only be implemented with a disproportionately large amount of work. This applies to vulnerabilities in particular which have been revealed via a penetration test or even via analysis of the source code, , and - especially in the short term - cannot be fixed within the application."

The emphasis above highlights the fundamental aim of the WAF I want to drill into everyone's head. They primarily exist to as a temporary solution until the code can be fixed. That's it. Nothing more, nothing less.

Now to challenge the above assumptions -

1) Code changes are NOT always quick and easy to push through to Production.

If you disagree, then I KNOW you've never had to deal with developers, QAT, change control or project managers and navigate any number of hurdles that impact with a code release (especially during crunch time). Now I am the first to admit that what follows are some bullshit issues. Nonetheless, they are REAL issues and if you want to improve security within enterprise applications then you need to realise that these questions will come up:
  • Is there a project code for the application developers and testers to charge their time against?
  • Does your hotfix take developers away from working on mission critical functionality for a given project?
  • Once the fix is applied who is doing the retesting and will the re-test delay the release to Production?
  • If you plan on cutting corners (regression testing is usually the first victim) because of the "urgency" or you "believe it won't impact the rest of the code" to get your hotfix into Production, who is accepting the risk?
  • What is the review period for all change requests to Production assets?
  • Is there a change embargo in place at present?
You have to understand that if a code change leads to a defect in existing functionality, you (as the "security guy") will cop the blame - not the developers. So it really is in your best interests to NOT cut corners on any of this.

Having said that, let's presume for a moment that you do not have to deal with any of these issues and I'm horribly, horribly wrong. That's fantastic for you, but we also have to consider the fact you work in an environment that does not even pay lip service to ITIL, COBIT or any number of best practise frameworks with regards to the management of IT infrastructure. This means you have no assurances at ALL that unauthorised code changes haven't taken place already.

Or put more simply, you probably have far bigger problems to worry about.

2) Time spent on patching an application is NOT equal to configuring a WAF.

For starters there are simply less people involved. Typically you have the business owner for the application in question, maybe an engineer in Networks as well as Security Operations to conduct the change and some Change Management folk to review the whole thing. Infact, I've seen this done in enterprise environments with less than 4 folks involved. Notice how there are no development staff and application testers involved? If you assume you would normally have another two people involved from each of those teams minus the Network engineer. That's being very generous too. Normally the process can take much longer (see point 1) and involve many more people. This means that applying a fix at the WAF could (taking the above example), reduce the cost of fixing by 20% or in large cases, as much as 75%. You can also argue security fixes can be applied as Emergency changes, vastly reducing the time window from the change being raised to being implemented. Time to test and implement? Much quicker.

In practise too, what will happen is that you will more than likely bundle up multiple bug fixes and roll them out into a single release. This means that the testing cycle is much longer, as is the release and the actual cost. God forbid the defect is significant enough to warrant a major re-write of the entire code base. You can then take the above numbers I've given them and throw them out the door. You're then in a position where you are still going to have to prioritise your time and effort and in all probability, you'll run with a WAF rule in the short run because the prospect of leaving that application without any form of defence in the until the code itself is fixed is something neither you or the business owner is wanting to chance.

This leads to my next point.

3) Applying a hotfix on the WAF buys you time (if you're lucky). It does not solve the problem.

Does this mean you are better off applying hotfixes in the WAF than in the application? God no. You need to fix the root cause which is inherent in the application. But if we can accept we can get a temporary fix into Production on a WAF quicker than we can by applying a hotfix, then the use of a WAF becomes much more palatable. It is an acceptance that you are buying time for your application, not solving the problem.

If a virus somehow takes spread on a network what do you do? Normally you would have desktops in segmented zones to contain the spread. In essence, you are using the firewalls to contain it. This buys you time until you can determine the root cause and remediate. WAFs are no different - implementing a change on these appliances buys time for the defenders to fix the root cause.

We block ports because we don't want people accessing applications carte blanche. We accept that there is some network traffic we do not want. Why is it so hard to apply the same logic to our applications? Nobody refutes the fact that the root cause of a vulnerability must be fixed (unless they have rocks in their head). But we live in a world where not everything is black and white and fixing problems is infinitely more complex than finding them. This where the entire concept of "defense in depth" has arisen - the knowledge that we build in layered to defences to prevent, mitigate or slow down an attacker from compromising an application or system.

To not accept that a WAF plays a valid role in a defensive strategy is to not only deride the entire notion of a "defense in depth" strategy, but it also devalues the role of other security technologies we use every day. Everyone in infosec sneers at the notion when someone says "we have firewalls so we are safe" because we know that is false. But no-one in the field in their right mind would call the technology useless. Nobody in their right mind would call airbags alone useless but we'd certainly recognise their value in road safety when combined with other safety mechanisms within a car - e.g. brakes, seat belts, etc.

Additionally, if one is in a position whereby your application is a black box (where you essentially have no access to the code) then the WAF might be one of the few defensive mitigation strategies at your disposal. Does that mean you should leave an application you cannot patch or secure long term in your network? Ideally not. It certainly beggars the question how you found yourself in this situation in the first place - i.e. why is this not covered under your support/maintenance contract? And it does happen - typically with large common off the shelf (COTS) applications but in such an instance, your long term strategy might be to migrate away from that vendor and product so you can get rid of it entirely. In which case, the use of a WAF is still a perfectly viable strategy since fixing the code base isn't an option. Ultimately this is arguably the worst case scenario one can find yourself in too.

Sure - one could build an argument that if you were to build a secure box and application out of the gate with no unnecessary services and plug it into the Interweb it would hold up. I could also assume no new attacks will ever be developed, no vulnerabilities will ever be found in that environment ever again and that Palestine and Israel will come to a peace agreement in the next 12 months. In all practicality, there are some things we have to accept aren't going to happen in a hurry no matter how much we'd like them to and we work with what we've got.

Anyway, here's something to think about for the anti-WAF zealots out there.


1 comment:

Christian said...

Hey Jarrod, I like that you post on topics that certainly pique my interest, and often from the otherside of the fence where I normally sit. This is another example of such a thing, and a quick disclaimer, I certainly do applaud you for wanting to defend this tech.

But I certainly think whilst referencing the OWASP material on this is also worthwhile to discuss the risks in the use of wafs, in particular, false positives, waf training (the device), waf training (the staff), more complex troubleshooting (incl. the effect on the web app itself), and obviously the cost-effectiveness.

I believe the cost-effectiveness is one of the bug-bears of WAFs that often gets picked on (Ferrari anyone?), but I don't want to talk about that one.

Excluding the price tag, all the other issues for me are quite big, especially when chances are, if you're seriously looking at WAFs, you're likely to already have multiple other layers of infrastructure and/or defence. Is this added complexity worth it?

Especially considering that I believe there are numbers of very serious eComm shops out there who don't have this tech. This makes the argument in those cases exceptionally difficult to make, especially, under their existing risk management frameworks, they're probably managing the risk responsibly anyway.

So I guess that leads me to my beef. WAFs are just another piece of tech. Unless you're actually implementing the people, process AND tech based on a risk-based approach to particular circumstance, you're simply wasting your organisation's time and money.

Unless the "vulnerabilities" that are discovered in your app are serious, exploitable, not too complicated, discoverable, you'll likely have some time on your hands to fix the app (which we all agree has to happen anyway). If the vuln displays any of the above characteristics, you're likely to be able to "fast-track" changes for the sake of security. Change Embargo's aren't always blocking, it generally just means you need to fill in a few more forms to get a change through.

Great post J! And you certainly got me thinking :)