- cppcheck
- smatch
- sparse
- clang scan-build
- CoverityScan
- The latest gcc
At the end of each run, the output from the previous run is diff'd against the new output and generates a list of new and fixed issues. I then manually wade through these and try to fix some of the low hanging fruit when I can find free time to do so.
I've been gathering statistics from the CoverityScan builds for the past 12 months tracking the number of defects found, outstanding issues and number of defects eliminated:
As one can see, there are a lot of defects getting fixed by the Linux developers and the overall trend of outstanding issues is downwards, which is good to see. The defect rate in linux-next is currently 0.46 issues per 1000 lines (out of over 13 million lines that are being scanned). A typical defect rate for a project this size is 0.5 issues per 1000 lines. Some of these issues are false positives or very minor / insignficant issues that will not cause any run time issues at all, so don't be too alarmed by the statistics.
Using a range of static analysis tools is useful because each one has it's own strengths and weaknesses. For example smatch and sparse are designed for sanity checking the kernel source, so they have some smarts that detect kernel specific semantic issues. CoverityScan is a commercial product however they allow open source projects the size of the linux-kernel to be built daily and the web based bug tracking tool is very easy to use and CoverityScan does manage to reliably find bugs that other tools can't reach. Cppcheck is useful as scans all the code paths by forcibly trying all the #ifdef'd variations of code - which is useful on the more obscure CONFIG mixes.
Finally, I use clang's scan-build and the latest verion of gcc to try and find the more typical warnings found by the static analysis built into modern open source compilers.
The more typical issues being found by static analysis are ones that don't generally appear at run time, such as in corner cases like error handling code paths, resource leaks or resource failure conditions, uninitialized variables or dead code paths.
My intention is to continue this process of daily checking and I hope to report back next September to review the CoverityScan trends for another year.
It might be useful to publish the logs and diff someplace to allow others to help fixing up the problems since most people don't have the computing power to do this.
ReplyDeleteIt might also help to contribute this kind of data to the kernelci.org effort to improve testing of the kernel.
The data is now being published at: http://kernel.ubuntu.com/~kernel-ppa/static-analysis/daily/
DeleteIt would be great if you could share your script(s) that automate this for you. I'd be really interested in using your method to find more bugs and contribute back to the community more.
ReplyDeleteIt would be interesting to know if CVE-2017-1000251 was discovered and not gotten around to before it was discovered by Armis.
ReplyDeleteNice blog... I am completely agree code review tools and are very useful during software development process to find bugs
ReplyDeleteNice blog. Thanks for sharing static analysis tools for linux. I found this open source static analysis tools very useful.
ReplyDeleteStatic code review analysis is about analyzing source code without executing them to find potential vulnerabilities, bugs and security threats.
ReplyDelete