14

For a PHP CMS, what should I expect to budget for a security audit, both whitebox and blackbox? The codebase is about 85,000 LOC ("Lines of Code") and I would probably use a North American company for testing. I really have no idea if an audit would cost $10-20k or well over $100k. I'm not asking for an exact quote, just a general guesstimate so that I know what to expect. If you could separate your estimates between blackbox and whitebox testing, that would also be helpful.

Edit:

I'll try to list as many factors as I can.

  • Type of app: A web content management system similar to Wordpress, Joomla, or Drupal.
  • Types of testing: Broad penetration testing and a scan for common vulnerabilities. Code review for additional vulnerabilities as the source code will be publicly available.
  • LOC: Roughly 85,000.
  • Languages: PHP, JavaScript.
  • Audience for report: Developers.
  • Location of testing: It can be done remotely.
  • User roles are variable. They are assigned to groups and each group can be given any number of permissions. Any number of groups can be created.

I don't know what other information might be relevant. Really, I'm not looking for an ultra-specific number, just a ballpark figure like, "Based on the info you gave, you could probably expect to budget between $X and $X for a security audit." Even just a baseline price would be extremely helpful as I really have no idea what to expect.

Chris Dale
  • 16,149
  • 10
  • 57
  • 97
VirtuosiMedia
  • 3,232
  • 3
  • 27
  • 32
  • I added the appsec tag to emphasize that this is refering to an *application* security audit, which is a very different creature from other kinds... – AviD Nov 29 '10 at 19:15
  • But this is still a very localized question. Beyond anything else, differences in costs of manpower are simply huge, and depend greatly on location (amongst other things). Not just by a few percantage points, we're talkin multiples and more. Perhaps youd be better off asking a more open question, e.g. "how long does it take", or "what does it involve". While this is a great question, and it would be great to have the answers to this, "between X$ and X$" (not replacing the X) is really the closest you can get. – AviD Nov 29 '10 at 19:19
  • I realize there is a huge range of prices depending on a large number of factors. I'm really just looking for a starting point for what to expect for a reasonable (in terms of both price and service) security audit, nothing more. – VirtuosiMedia Nov 29 '10 at 19:50
  • This question appears to be off-topic because it is about pricing which is dependant on the hourly rate of a company. – Lucas Kauffman Dec 16 '13 at 06:39

3 Answers3

6

Here's my rough guesstimate:

Code review:

1000 LOC = 1 hour
85000 LOC = 85 hours

Hourly rate: 100$/hr

85 hours * 100$/hr = 8500$

If your software uses an ORM and a well documented MVC framework, it can speed up the code review significantly.

Lucas Kauffman
  • 54,229
  • 17
  • 113
  • 196
Olivier Lalonde
  • 5,079
  • 8
  • 32
  • 35
  • 1
    Approximately in two weeks - $8,5k. For PHP application. I am just curious - where are such prices? About documentation this is good point, it really helps. –  Dec 01 '10 at 00:14
  • 2
    1000 LoC per hour? That's pretty high... Industry figures have this at 50-100 LoC per hour for high-level languages, a very focused threat model (which could be priced seperately) can allow you to narrow your focus and get up to 500-800 LoC / hour. Even if 1000 is *consistently* possible (and in fact there were cases where I made this happen), it's very much an upper bound, and costs would not be based on that. – AviD Dec 01 '10 at 00:46
  • Also, if the threat model / initial analysis is not priced seperately, this can add significantly to the cost. If it's not seperate, and not included, the cost should be MUCH cheaper - since the auditors don't really know what they're doing. Also, in many consulting companies, they'll usually add some hours for "project management", "report", presentation, etc. – AviD Dec 01 '10 at 00:49
  • 7
    I'd argue that if you can audit 1000 lines of code an hour, it's not a good security audit. – Steve Dec 01 '10 at 01:18
  • @SteveSyfuhs, actually, if the talk is about LoC, then it depends much on the quality (visually and logically) and style of the code. Also, source code auditors do use different approaches accordingly their experience, knowledge and simply habits and taste. Speed of the code assessment does not guarantee anything. It's like - every herring is a fish, but not every fish is a herring. –  Dec 01 '10 at 01:44
  • @Steve, @Ams, you both have good points, but either way - expecting a pricing based on 1 KLoC / hour is unrealistic. – AviD Dec 01 '10 at 06:00
  • @AviD - Could you explain why the auditors wouldn't know what they are doing if the threat model and initial analysis aren't included? – VirtuosiMedia Dec 01 '10 at 08:11
  • @Virtuosi - I meant that if they are not included, **and not performed seperately** - i.e. *they are not done at all* (its either in, or its out, or its not). If its not done at all, then they're not doing their job right, because to do it right you need to *think* about these things beforehand. Otherwise, you're just left with running dumb, automatic tools (be they software or people) that can only find low-hanging technical fruit. – AviD Dec 01 '10 at 08:31
  • 2
    I knew this would be a highly controversial answer but well, I was the only one who had the guts to throw out a number and actually answer the question :) My estimate of LoC/hour was a broad average since not all code is equally prone to vulnerabilities. The most critical parts might get you down at 100 LoC/hour while we the less critical parts (HTML/Javascript/domain model) can be mostly skipped. Anyways, it all boils down on how well the application is designed and documented. – Olivier Lalonde Dec 01 '10 at 09:05
  • 2
    @AviD, actually, I was just wondering about prices and did not agree about them. @Olivier Lalonde, I would not agree also about mentioned "less critical parts" - how about DOM XSS? Web application assessment is not only PHP source code review - it might require Flash decompiling, configuration reviews, JS review, some other parts as well. Sure, there are other less critical parts, but they are often not included into main pricing. –  Dec 01 '10 at 10:57
  • 1
    @Olivier - agreed, and I kinda wanted to +1 your answer just for the gutsiness (and actually providing a direct answer). But, my comments, and the fact that noone else wanted to take a swing at that ball, just proves that the question is problematic in the first place. @Ams, I was agreeing with you, but also trying to make the point that pricing based on kloc/hour is unrealistic - and thats on top of the question of speed. However, it is fair to expect an auditor to be reasonably well-paced, since it is my buck. – AviD Dec 01 '10 at 11:06
  • 1
    Also agree with you @Ams regarding reviewing non-PHP code, often you can find some very juicy stuff - *even in HTML comments*. But @Olivier's point is also valid, how well the application is designed and documented will have an even bigger impact on the scale than the number of LoC. Bottom line, LoC is just one metric, but unfortunately pricing is usually based on that since its so *easy* to measure blindly and quickly. – AviD Dec 01 '10 at 11:09
3

Okay, there are a huge number of factors affecting cost and scope of a security audit, which is why it is so difficult to give you a ballpark without a lot more detail on the scope. For example:

  • What type of security audit do you require?
  • What do you consider black box? And why do you want it - it can increase the cost considerably.
  • Are you including code review?
  • What languages is the app written in?
  • What audience is the report to be written for?
  • What is the purpose of the test?
    Compliance, audit, certification,
    other?
  • Is testing on site, remote, on
    live environment or test?
  • What does the application do?
  • How many user roles exist?
  • etc.

See our taxonomy and extension to it. In fact, some of our other blog posts are very relevant here. Talk to your local vendors and get a quote.

Rory Alsop
  • 61,474
  • 12
  • 117
  • 321
1

This really depends on a lot of factors that makes final price.

Costs for source code analysis can be counted by counting lines or amount of code in Kilobytes. As far as I have seen, more popular is second method - to count by size of code. While pricing per lines may be more accurate, such approach does not eliminate unexpected things like very poorly written code, that definitely will take much more time to assess. Others additionally counts found vulnerabilities in code.

Some people might say that pricing depends on the quality of service. I would disagree and say that this is not always the case. New services needs to proof themselves and usually starts with lower prices and middle quality of audit - not only practical experience is essential, but also personal management, users support, etc. Brand services can afford higher prices. But there is also possibility when the brand service starts to skive, or newbies are capable of doing better audit than the well known brand. So, it is advisable to get some background about service, read recommendations and comments.

Also, price may depend on the country of service provider. We all do have differently developed economical environment.

Now from the perspective of source code auditor. For web applications usually whitebox and blackbox testing is combined together. There is really no need in security through obscurity in such case. But well, others may prefer to make blackbox test first and then to give access to the source code. If the source code is given, then it definitely will be tested in live environment. If you have only access to web-site, customer may provide access to server.

Summing up all of this, it is hard to define some fixed price. Customers usually do discuss all details and their future collaborative work. Process may look like this: you give code, they reply with report after some time. You may give code again and those steps may require several loops till bugs in code are vanished. That's why you often see contact form rather than pricelist.

  • Thanks for the breakdown, Ams, it's helpful. Would you be able to venture a number as well? Assume that I would go with a lower priced service from North America. I'd like to think that my code quality is at least average, if not a little above in terms of the number of vulnerabilities. I'm really just looking for a very rough price so that I can budget accordingly. – VirtuosiMedia Nov 28 '10 at 18:07
  • @VirtuosiMedia, we work mainly with ex-USSR customers and price is appointed during our dialogue, so I can't provide appropriate prices. –  Nov 28 '10 at 18:14