Why Using Static Analysis Is Hard

Using static analysis is hard, counter-intuitive and not-automatic. We want to answer some questions related to the use of static analyzers: why this is hard for some users and what are the solutions to help. What is static analysis in software engineering? Static analysis is the operation of finding issues in the code before it is compiled or executed. Programs called static analyzers perform static analysis by inspecting the source code and finding defects. Defects can belong to multiple categories (security, safety, documentation) and have various severities. One famous example of static analysis tool usage is detection of buffer overflow in C programs. Vulnerabilities can be discovered by looking at the source code and memory operations. In Javascript, static analyzers such as or provide useful information on formatting and type checking. jshint flow You can find a long list of available tools on this . GitHub repository The key takeaway is: static analysis is to detect issues in your code before you even run it. What does static analysis not find? Static analysis does not detect if the code correctly implements the customers requirements. It only checks for defects in your code. On the other hand, the code you wrote might not reflect what it is supposed to implement. Does static analysis replace testing? Absolutely not. Testing complements and static analysis both must be used: Static analysis must be used to ensure that the code is syntactically and semantically correct Testing must be used to ensure that the code correctly implements the requirements. Moreover, code coverage must be measured to guarantee that all the code is thoroughly tested. In short: use static analysis AND testing. Why static analysis is important? Static analysis is important because it detects potential defects developers did not even think about. Even if the implementation is correct (and tests pass), the code might still have some issues, such as slow performance (algorithms are not optimal), poor practices (not using the right functions or syntax) or just lack of documentation. Static analyzers catch such issues and therefore, help developers shipp better code. What are the problems related to static analysis? Static analysis tools are very useful and integral to produce better code. However, they also can be very time consuming for developers because you need to: Setup, configure and calibrate the tool to find meaningful results for their projects Remove false positives Triage results The following sections explain each aspect. Tool configuration and calibration The first issue is to configure and calibrate the tool you are using. Static analyzers come with hundreds or thousands of rules that check code. These rules may or may not be useful to you and require time to understand each of them in order to get exactly what you want. Most of the time, you calibrate the tool, after a few runs you find out the rules that are relevant to your programs and you remove the rules that do not seem to make sense. However, this process is conducted manually and very time consuming. False positive Most static analysis tools do not have context behind your code and detect issues based on inference rules. However, many defects depend on the context of your code and not all reported defects are relevant. These are called false positives: A false positive is when the static analysis tool reports a defect when none are present. In addition to the false positives, there might also be some rules you want to avoid (for example, you might have a very personal style about code formatting and want to just avoid all formatting-related defects). Finding false positives can be confusing: developers question the report’s legitimacy at first glance. Identifying and removing these false positives takes significant time for developers. Results triage The last issue related to static analysis is to triage the results and . Some static analysis tools are very verbose, reporting a lot of potential defects. This makes it hard to know what tools are relevant and which should be fixed. Developers must then take the list of all reported issues and filter them manually to finally find ones that might potentially impact their code. “find the needle in the haystack” How to simplify the use of static analyzer? The best way to simplify the use of static analysis tools is to use a tool already calibrated and tailored to your project. This service is provided by code analysis platforms, such as . Such platforms train their algorithms to remove false positives and report issues only relevant to your project. Codiga By using such a platform, you take the benefits from static analysis tools, wasting no time configuring them or filtering results. Conclusion Static analysis tools are very useful for developers and should be used as much as possible. However, they require significant time to be tailored in order to produce a list of meaningful results for your project. By using analysis platforms such as , you can use a ready-to-use, pre-calibrated tool to help you find relevant issues for your projects. Codiga