Coverity grew out of work done by Seth Hallem and a number of other PhD students at Stanford's computer science department, working under Professor Dawson Engler. They were focusing on one of the biggest problems facing software today: quality and security at the source code level. From this work Coverity has developed the first system that can identify defects with a combination of precision, speed, and accuracy that is enabling software developers to master the size and complexity of today's code. In addition, Coverity's engineers have devised solutions to tackle many of the difficult problems that have traditionally hampered source code analysis-build integration, compiler compatibility, high rate of false positives, and effective root cause analysis. The Coverity platform has evolved into an enterprise analysis solution that scales to tens of millions of lines of code and thousands of software developers. In its two years of operation Coverity now won over 70 customers, is profitable and has not required venture capital to fund operations. Its customer base includes all but one wireless OS vendor-including Symbian, Palm, Linux, and others.

Richard: Seth, there is already a plethora of software quality tools on the market, what makes Coverity something new?

Seth: The first thing I must say about our tool is that it is not a testing tool or code coverage tool, or anything that you might traditional associate with development tools at all. Most of those tools will take test cases and run test cases, and then either tell you how much of the code you have covered, help you find bugs in the area covered by your test cases or help you manage a whole suit of test cases. Our tool doesn't do any of those things. Our tool works much earlier in the development cycle, before your program every runs. It works at the point where you are writing and compiling on your code, before you have even run it. There are lots of advantages in finding the bugs that early in the cycle, not least that it saves you a significant amount of time down stream.

Another important feature is that we are able to cover parts of the code that testing can never touch, because it is generally not feasible to exhaustively test any substantial piece of software. Our tools look at the source code and simulate what the source code can do so we are able to simulate all possibilities, we don't have to worry about the execution environment, test cases or anything of that sort.

The other advantage that we have is that we can look for a broader variety of defects that relate to programming mistakes. The way those mistakes show up at run time vary, it could be crashes, performance problems, security venerability or quite simply be unpredictable behaviour. We are able to pin point exactly where a programmer made a mistake, more importantly we can do it at the time the programmer makes that mistake, rather than much later in the development cycle.

So I don't think that there are any tools that do quite what our tool does and that's what makes it new.

Our customers, like Symbian, Oracle and WindRiver - large companies that write C++ applications for networking, databases embedded systems - all of them are facing the same reliability challenge. Traditional testing tools are really not up to that job.

Richard: But other static validation tools have claimed to achieve the same results, why is Coverity different?

Seth: One of the big problems that earlier tools have run into is accuracy. The computer science problem we are trying to solve here is fundamentally more difficult than writing a testing tool or even writing a compiler. There are limits in traditional algorithms, they need to find the right place within those limits to avoid reporting false positives or missing errors. We have our own algorithms, that are substantially different from the state of the art in research or industry, and they produce a much lower false positive rates because we are able to focus our tool on the type of defect that we are looking for. So that is one big advantage.

Another challenge other tools face is scalability. When your are looking at and analysing all of the source code in a multi-million line system, such as the Symbian OS, you need to have a very scalable technique. Traditional algorithm scale exponentially. On an application the size of the Symbian OS traditional approaches would take the rest of the life time of the universe to do the analysis. So we have found a scalable algorithm that makes the right trade off, in our opinion, between the depths of the analysis and execution time. So we try to find very complex and interesting bugs, but we also try to make sure that our runtime is reasonable. We have found the right trade off between all those competing factors.

Richard: Given that Symbian OS C++ is a little unique were there any challenges in working with Symbian, did the product work out of the box?

Seth: The product absolutely worked out of the box, because Symbian OS is C++ code. What Symbian have done is build a very unique programming model. They use the features of C++ to make it easy for mobile developers to write the type of applications they want to write. Out of the box we find generic C++ defects that all C++ developers have to deal with, and those are relevant to Symbian.

We can deal with the unique features of Symbian OS C++ using our Coverity Extend product. Coverity Extend recognizes that the challenges a programmer faces in each system are unique. Essentially every system has its own program model. Symbian has gone a long way towards formalising their programming model, so Coverity Extend is a great opportunity for Symbian. We have spent a significant amount of time helping Symbian build those rules into Coverity Extend. So now Symbian is able to check for Symbianisms, in addition to standard C++, using our tool.

Richard: You talk about Symbian OS C++ having a lot of rules, does that make it easier for your product to help them with application quality?

Seth: It certainly makes it easier. Coverity Extend gives you a way of recording these rules, but for most developers the first challenge in recording a rule is understanding what the rule is. In many large systems that knowledge might be in one developer's head and not broadly distributed throughout an organisation. With Symbian, because they are selling this code and selling this program model, it is very well formalised so it is very easy for them to articulate the rules they are trying to enforce are. Once you can articulated that rule it is easy to write it down in the programming language that we provide.

Richard: What else was key in Symbian licensing the product?

Seth: Well I think the key was that we made a trip to the UK to demonstrate our product, which we do for all of our customers. Part of the demonstration is to run our product over their code. Our entire sales model rests on is the fact that, in that first introduction, we will pull out of any code interesting bugs and defects that the customer will feel are worth fixing; see it's worth paying to have the tool that will allow them to fix problems early in the development process.

So, as with all our other customers, that was the key with Symbian, we went over there and we practically demonstrated the capabilities of our tool in that first trial. After they saw that demonstration it was fairly easy to move to a licensing discussion. I think they also saw an opportunity to write the coding rules into Coverity Extend and somewhere down the road take them to the Symbian ecosystems, so that not only Symbian themselves but Symbian OS developers can achieve some of the same benefits.

Richard: In terms of providing a broader support for Symbian OS developers, is that something that would come from Symbian or would it come from yourself or jointly?

Seth: I think that is something that we will work on together. We will continue to license our tool to large companies that are building substantial C and C++ applications, including those building applications for the Symbian OS. What we are working with Symbian on is the custom rules that they will hopefully be able to provide to the other Symbian developers. Now I don't know the specific time lines for their plan, but I know that that is something that we are actively working on with them, with the eventual goal of making it something that a licensed user of our tool working with Symbian OS could use as well.

Richard: Currently your client list is large scale developers, do you see it getting onto the desks of smaller developers?

Seth: Our long term goal is to see Coverity on every developer's desktop. In the short term, just because of the realities of a growing company, we are going to focus on the bigger opportunities and bigger users. However, we will continue to work towards a product and a licensing model that will allow everyone developer to get their hands on this tool, it just might take us some time to get there.

Richard: Earlier you mentioned that Coverity can help with security, could you explain how?

Seth: Somewhere around 30% of reported security holes are due to flaws in programs, simply programmers making mistakes. In simple terms the problem is that information comes in from the outside world and must be treated very carefully. If you are not careful with how you treat it, you give an opportunity to malicious users to cause trouble. So part of our product suite is a set of rules that looks for those types of flaws. It is yet another place where, because we work at compile time, we have a nice advantage. We see possible execution errors through the code itself, we don't relying on someone happening to type in a large enough buffer or the right malformed string. We can search the space of possibility. So it becomes very relevant for security where the list of possibilities is much much longer, given that you have the possibility of a malicious user trying to exercise the system in a way that was never intended.

After talking with Coverity we briefly followed up with Symbian to see what the position regarding community availability of Coverity rules was. "Symbian is committed to broaden the availability of many internal tools to the Symbian developer community," says Bruce Carney, Symbian, Head of Developer Marketing & Services. "Coverity's source code analysis products significantly increase developer productivity, hence it is anticipated that once completed, Symbian C++ specific customisations will be made available to our partners". Bruce also pointed out that David Wood, in his new book Symbian for Software Leaders, notes that "Static Code Analysis tools have a big role to play" in any Smartphone software project. It was the suitability of the Coverity tool to this role that led to it being chosen by Symbian to address this requirement internally.

For more information on Coverity see its web site: www.coverity.com