The team behind LGTM and Semmle's QL writes about automated code analysis and security vulnerabilities found in major software.
Commenters mention LGTM finding bugs that linters and tests missed with no setup, and note it has found CVEs in mainstream products.
Not terribly impressed. This wouldn't pass my code review: https://github.com/TTimo/doom3.gpl/blob/aaa855815ab484d5bd095f347163194ac569abcc/neo/renderer/MegaTexture.cpp#L748
Function is 200 lines long, with 8 or so levels of nesting. Also "goto breakout"
To address the objections below, the function reads from a file into a pixel buffer. It's not some tricky in-place update. That's a great candidate for a more functional style.
Here's more ick:
https://github.com/TTimo/doom3.gpl/blob/aaa855815ab484d5bd095f347163194ac569abcc/neo/game/physics/Physics_Player.cpp#L168
Surely that could be factored better.
Down-voters go read Carmack's own article:
https://gamasutra.com/view/news/169296/Indepth_Functional_programming_in_C.php
I think it's funny that people disagree here. This is exactly the stuff a modern linter would flag in an automated code review.
Sure enough, here's a linter. I think this is essentially the same codebase:
https://lgtm.com/projects/g/Edgarins29/Doom3/context:cpp
"Code quality: D" (on an A-F scale)
And by the way, I have the utmost respect for Carmack. I just wouldn't hold up this codebase as great.
I started Sage, and worked on it a huge amount 2004-2012, but less so recently (due to working mostly on CoCalc). Actual contributions to and work on Sage by the community have gone UP a lot since 2012, as this plot (https://github.com/sagemath/sage/graphs/contributors) at Github shows. Much of that work is from research mathematicians that are adding large swaths of important functionality (e.g., in differential manifolds) to Sage. But there's also a very welcome push for quality, e.g., the number one recent contributor, fchapoton, has been fixing every little detail in the codebase revealed by https://lgtm.com/ and other tools, and there's also been a lot of work to fully transition Sage to Python 3 (which is difficult since so much is written in Cython and make nontrivial use of comparisons).
I would have likely made Sage pip-installable from day 1 if I had known that pip would turn out to be so popular. It was really unclear what was going on with Python packaging 15 years ago. I'm really glad that one package manager (pip) has become so good at this point.
To be honest, I feel bad for the engineering team at Equifax. The vulnerability that compromised their system was a bug in an open-source Java library, Apache Struts, and security researchers only noticed it a few days ago. It seems that the Equifax team had very little time to react and update their software. In some sense, I feel that more blame should be placed on the engineers who built the highly popular open-source software, not the Equifax team. Some large number of Fortune 100 companies also experienced the same vulnerability simply because they trusted a widely used library.
Makes me wary of trusting other big OS libraries, but since rebuilding every part of the stack from scratch is infeasible and unproductive, we don't have much choice but to use them.
Technical announcement:
Severe security vulnerability found in Apache Struts using lgtm.com (CVE-2017-9805):
https://lgtm.com/blog/apache_struts_CVE-2017-9805_announcement
https://lgtm.com/
LGTM recently came by my project offering to integrate with GitHub. It's found a decent number of bugs other linters and my test suite didn't find, and it didn't require any setup at all; you just open your project on their website.
Their staff also reacted really quickly when I asked them questions or reported bugs.
For the most part, their only false positives are things like unreachable code, which I sometimes used to leave in for future-proofing, but I eventually did things their way, because it does make sense that it could be confusing to a third party.
The only alert I've ever suppressed was a string search for "example.com", which could have matched "example.com.untrusted-attacker.com". It was just code for displaying a reminder, not anything security-critical, so I didn't think it was worth fixing.
Semmle | Research/Software/Security Engineer | Oxford, UK | Onsite, Full-time
Semmle helps some of the biggest software and financial companies, such as Microsoft, Google, and Credit Suisse, develop more secure software, faster.
If you'd like to get a taste of our technology go to https://lgtm.com where we analyse over 70,000 of the most popular open source projects, or check out the vulnerabilities we've recently reported at https://lgtm.com/blog
We are looking for great engineers work on a mix of problems ranging from compiler work and program analysis to vulnerability finding to building solid 1000-node distributed systems.
Go to https://semmle.com/careers or email careers@semmle.com
Disclosure: I’m the founder of Semmle. As noted above, this conversation is a dup but I wanted to answer the questions that came up. Semmle’s core technology is an engine for variant analysis - finding all instances of a mistake that led to an incident, especially for security. Here's how Microsoft uses Semmle: https://blogs.technet.microsoft.com/srd/2018/08/16/vulnerability-hunting-with-semmle-ql-part-1/ . The engine does support inter-procedural data flow analysis and simulates execution in that sense. For examples of such data flow analysis on Java, see https://lgtm.com/blog/apache_struts_CVE-2018-11776-part2.