Few people know it, but over the past few days, a keen developer foiled a years-long conspiracy to install remote backdoors across the internet.
Two years ago, a programmer named Jia Tan (JiaT75) started helping develop a little-known compression library called xz. For those who don’t know, software today is not a single entity.Every software you use is built from a series of tools called library, which makes programming easier. For example, most programmers will never have to write the details of a sorting algorithm because, somewhere, there is a library that performs the sorting for them. This allows programmers to focus on higher-level tasks, such as making the software actually do what the user wants it to do. However, these libraries don’t come out of nowhere – they require interested programmers to maintain and extend their functionality. It’s a lot of work, so I find it a big relief when new programmers volunteer to help.
But Jatan’s motives were not pure. Although xz is not used directly in many software, it is pulled in by some other libraries and then used by other programs. In particular, sshd, the popular remote login service used by system administrators everywhere, can optionally include bindings to third-party libraries, which then also include the xz library. This is also the configuration used by most server operating systems on the Internet. Therefore, as Jia Tan gained the trust of the xz maintainers, Jia also indirectly gained more access to other parts of the operating system.
Software often contains code for testing itself. This is how software maintainers prevent themselves from making obvious errors when making changes. Test code is typically not incorporated into the final software delivered. Jia Tan included a backdoor in a file ostensibly used to test xz’s compression technology. However, modifications to the build system cause this test case and its backdoor to be merged into the final software, which is then deployed. The backdoor works by overwriting these functions with its own version of the standard encryption/decryption functions.
The software has been merged with some test versions of several standard operating systems, so cutting-edge developers have already started using it. It was discovered while Microsoft software engineer Andres Freund was doing some performance testing. To conduct these tests, he tried to minimize the CPU time used by other tools on the system. He noticed that sshd was using too much processing power, so he started digging into what was causing the performance degradation. His analysis tools showed that sshd spends a lot of time in the xz library, and further investigation revealed that the xz library replaced some of the standard encryption and decryption functions.
Thankfully, this was discovered before it had a major impact. No one knows who Jatan is, and we may never know. But it does make software developers wary of the fact that bad actors, whether individuals or part of national organizations, are willing to play the long game and get malware installed on everything.
Some have used the incident to criticize open source software, saying it’s part of the problem—that is, this wouldn’t happen if not everyone relied on it. But honestly, I think the opposite is true.the only reason is established It’s because of open source software. It’s because our developers are familiar with not just their software’s code base, but everything that makes their software run, that we’re able to find and diagnose problems so quickly. I’ve worked with closed source software on occasion, and in that case there was zero transparency. If someone introduces malicious code into an important closed source software, no one will know about it, and no one will even have the ability to detect it. If discovered, the company responsible may try to avoid disclosing the extent of the problem. However, because it is open source, everything that happens is transparently logged – every message, every commit, every uploaded artifact. Everything can be inspected and scrutinized, the extent of the damage can be determined, and the root causes can be openly discussed to make everyone more vigilant.
In short, if it weren’t for the careful observation of open source software developers, we would have almost lost the Internet this week.