PVS-Studio – Static analysis for C# and the ‘C’ prefixed family of languages

The other day I was searching online for c# static analysis tools and stumbled on PVS-Studio from a StackOverflow post. There’s quite a few listed there but because I’ve never heard of this product before I thought I’d spend a little time investigating what the tool offers and the type of development teams it’s aimed at.

This is not a sponsored post and opinions are my own.

PVS-Studio performs static analysis and reports on a broad range of problems which are different from tools such as ReSharper or Microsoft Code Analysis and of course, the compiler. You won’t be bothered with warnings such as “unused variable” or “unreachable code”. PVS-Studio will not report on these as they are easily detected by most tools. The most useful feature is its advanced analysis and recognition of certain patterns of error a programmer would make. When this tool reports a “Expression is always true/false” you can be certain that it’s something worth investigating rather than a redundant statement which needs to be cleaned up. I’ll give an example of this further down.

As a developer looking to improve the code bases I work on, having a tool where I didn’t have to contact the sales department to be considered was a plus. I would love to get my hands on Coverity Prevent or PRQA QA·C# but you need to fill out a form and wait for someone to contact you. I’m so used to downloading tools with the click of the button that I find the process of talking to a sales person time consuming, time which could be better spent learning the actual tool. Fortunately PVS-Studio can be downloaded and is full featured with a limit of 50 clicks. You can download it from here.

Basic Use

Once installed you have the option of using it at the command line, through Visual Studio or using the standalone desktop application. For this post I’m just going to go through the features found through the UI in Visual Studio.

pvs-studio-menu

The easiest way to get started is to simply select analysis on the current project you’re working on and you’ll be presented with a categorized list of problems. Be aware that analysis can be slow on large projects but still lightning quick compared to a review by a human.

pvs-current-project

Bugs range from low to high and can be filtered further on 3 main areas which are Optimization, General Analysis and 64bit Analysis results. If you’re a analysing C# code then the General Analysis filter will be the only one you’re interested in, the other two are related to C/C++ code.

pvs-filters

A large project may produce several hundred or thousand problems which might be daunting at first sight so configuring which rules you consider to be important will reduce the count significantly. From there you could “turn up the dial” once a team is comfortable with the number of results. You can turn off rules by going to menu PVS-Studio -> Options.

pvs-error-detection-enable-disable

Personally I am more interested in all the items listed in High so I haven’t turned off any rules and settled on a filter.

pvcstudio-error-listing-favorites

As far as the UI goes that’s as simple as it gets. It’s right to the point and doesn’t get in your way with an endless set of features.

Error detection and error patterns

The team building PVS-Studio have spent a lot of time analysing source code from various sources, including open source projects and write about the process on their blog quite a bit.  Their documentation kept me entertained for hours.

So what are error patterns? I couldn’t find a formal definition online but essentially they are common patterns of errors which programmers typically make and I’ll give you some examples below, but you’ve no doubt seen a compiler warnings about an expression always evaluating to true. You’ve probably seen it because it’s a common error most developers make as they are busy solving domain related problems. There are however, different ways static analysis can go about reaching the same conclusion.

To use a running example think about an expression which is “always true” or “always false”, something which ReSharper might point out as you’re typing or the compiler might produce, if you’re paying attention to warnings. The depth of analysis performed is limited by the context in which you’re working. With ReSharper enabled, as you’re writing code you expect instant feedback (intellisense, code tips, refactoring options etc..), so there’s only so much time ReSharper can spend looking for potential problems. You don’t want deep analysis after each keystroke so what you get is fast feedback with the trade-off being accuracy, if you can call it that. Each tool evaluates that claim differently.

PVS-Studio goes a little deeper in its analysis. Here’s a simple error which ReSharper and the compiler failed to pick up, a sample taken from the nopCommerce code base.

// nopCommerce 3.5
var regAll = new StringBuilder();
regAll.Append("(");
regAll.Append(CommentRegex);
regAll.Append(")|(");
regAll.Append(StringRegex);
if (regPreproc.Length > 0)
{
  regAll.Append(")|(");
  regAll.Append(regPreproc);
}

And another which determined that part of an expression always evaluates to true.

// nopCommerce 3.5
string serviceId = USPSServices.GetServiceIdDomestic(service);
if (!String.IsNullOrEmpty(serviceId) && !String.IsNullOrEmpty(carrierServicesOfferedDomestic))
{
  // Add delimiters [] so that single digit IDs aren't found in multi-digit IDs
  if (carrierServicesOfferedDomestic.Contains(String.Format("[{0}]", serviceId)))
    model.CarrierServicesOfferedDomestic.Add(service);
}

They might seem like trivial mistakes and something not worth worrying about but just imagine if that code was doing something else which wrote data to the database or corrupted information in the process. It’s not the simplicity of the error it’s the mistaken intent that’s being highlighted.

How much time will you save?

I’m going to show you a couple of fragments of code which will hopefully deonstrate how useful static analysis is. Here’s a sample of code from a code base I work on.

Can you spot the problem?

if (_itemCompare != null) return _itemCompare;

if (ItemId.HasValue && _itemCompare == null)
{
  // Do something
}
else if (!ItemId.HasValue || _itemCompare == null)
{
  // Do something else
}

If you can’t, that’s OK. Static analysis takes out the guess work. It’s the “else if” branch, it’s always true. If you spent more than 10 – 30 seconds trying to figure out the problem, just imagine all of the other hairy logic in your program which will never be reviewed. They are could cause you headache at some point in the future.

Here’s another sample taken from the nopCommerce 3.5 code base. Can you spot the problem?

//nopCommerce 3.5
var usedMeasureDimension = _measureService.GetMeasureDimensionBySystemKeyword(MEASUREDIMENSIONSYSTEMKEYWORD);
if (usedMeasureDimension == null)
    throw new NopException(string.Format("USPS shipping service. Could not load \"{0}\" measure dimension", MEASUREDIMENSIONSYSTEMKEYWORD));

var baseusedMeasureDimension = _measureService.GetMeasureDimensionById(_measureSettings.BaseDimensionId);
if (usedMeasureDimension == null)
    throw new NopException("Primary dimension can't be loaded");

That one was quite simple and it clearly shows a copy/paste error. It highlights the importance of how easily these problems can be discovered and resolved before they even hit production. For that particular error PVS-Studio reports “the second if statement is senseless” which I thought was amusing.

How is PVS-Studio different from Microsoft’s Code Analysis?

My first through when I saw PVS-Studio was “why would I want this when Code Analysis is built into Visual Studio and free”, and I can safely say that these two products solve different problems. Code Analysis is focused on a different set of defects with a wide range of categories such as programming style, design rules such as naming inconsistencies, design problems (), potential performance problems and security related issues. Obviously it’s still performing static analysis but for different purposes.

I actually asked Evgeniy Ryzhkov, the CEO of Program Verification Systems and founder of PVS-Studio if he thought Microsoft’s Code Analysis was a thread and where was his response.

We are very happy that VS CA feature exist. Microsoft studies our future customers. For Microsoft code analysis is only one of thousand features. But from VSCA developers learn what is static code analysis. And if developers wants full featured solution then they can try PVS-Studio, Klocwork, Parasoft, Coverity.

I plan do to an in-depth post on Code Analysis at a later date but for now just be aware that they are different beasts.

Pros

  • Highlights areas of concerns by categorising errors into High, Medium and Low risk
  • Contains a digestible, short list of important rules (over 100 C# and over 200 C++ rules)
  • Looks for common patterns of errors

Cons

  • Doesn’t offer refactoring options to fix problems
  • A little slow on large projects
  • No single-user license, but here’s why.

I was going to put price under Cons but that all depends on how you value your software. For some, a quality code base and product trumps all. Perhaps you’re writing software which runs on medical equipment in hospitals or writing a reporting system for your boss showing graphs and monthly sales figures. Value is very subjective. I’ll leave the task of finding out the cost to you.

Final words

As I’ve mentioned the UI is clean, straight to the point and lists just over 100 important errors which might exist my code base. It will protect code now and in all future commits for those errors and is well worth the investment for systems such as robotics, medial equipment or any high profit yielding system.

Evgeniy has been kind enough to give me a license to evaluate the tool for this post and not only will I be using it extensively at home but I’ll definitely be asking my manager to purchase a commercial license.