A lone Microsoft developer shocked the world on Friday by revealing that a backdoor had been deliberately planted in XZ Utils, an open source data compression utility available on nearly all Linux and other Unix-like operating systems. The person or people behind this project may have spent years working on it. When an eagle-eyed software developer spots something fishy, they’re likely about to see a backdoor update merged into Linux’s two largest distributions, Debian and Red Hat.
Software and cryptography engineer Filippo Valsorda said of the effort: “This is probably the best-executed supply chain attack we’ve ever seen publicly described, and it’s a nightmare scenario: on a widely used library A malicious, capable, authorized upstream.” This is very close to success.
Researchers spent the weekend gathering clues. Here’s what we know so far.
What is the XZ utility?
XZ Utils are almost everywhere in Linux. It provides lossless data compression on almost all Unix-like operating systems, including Linux. XZ Utils provides key functionality for compressing and decompressing data during various operations. XZ Utils also supports the legacy .lzma format, which makes this component even more important.
what happened?
Andres Freund, a developer and engineer working on Microsoft’s PostgreSQL products, was recently troubleshooting performance issues on Debian systems using SSH, the most widely used protocol for logging into devices remotely over the Internet. Specifically, SSH logins consumed too many CPU cycles and generated errors in valgrind, a utility used to monitor computer memory.
Through pure luck and Freund’s careful observation, he eventually discovered that the problem was the result of an update to XZ Utils. Freund revealed on the Open Source Security List on Friday that the updates were the result of someone deliberately inserting a backdoor into the compression software.
What does the backdoor do?
Malicious code added in XZ Utils versions 5.6.0 and 5.6.1 modifies the way the software functions when performing operations related to .lzma compression or decompression. When these functions involve SSH, they allow malicious code to execute with root privileges. The code allows someone with a predetermined encryption key to log into the backdoor system via SSH. From that point on, that person will have the same level of control as any authorized administrator.
Where did this backdoor come from?
It seems this backdoor has been years in the making. In 2021, a person with the username JiaT75 made the first known commitment to an open source project. In retrospect, the change to the libarchive project was suspicious because it replaced the safe_fprint function with a variant that had long been considered less safe. No one noticed at the time.
The following year, JiaT75 submitted a patch on the XZ Utils mailing list, and almost immediately, a never-before-seen participant named Jigar Kumar joined the discussion, arguing that Lasse Collin, the long-time maintainer of XZ Utils, had not been included in the discussion. . Update software frequently or quickly enough. Kumar, backed by Dennis Enns and several others who were never on the roster, pressured Colin to add a developer to maintain the project.
In January 2023, JiaT75 made its first commitment to XZ Utils. Over the next few months, JiaT75 (who uses the name “Jia Tan”) became increasingly involved in XZ Utils affairs. For example, Tan replaced Collins’ contact information with his own on oss-fuzz, a project that scans open source software for exploitable vulnerabilities. Tan also asked oss-fuzz to disable the ifunc feature during testing, a change that would make it unable to detect malicious changes that Tan would soon make to XZ Utils.
In February this year, Tan released commits for versions 5.6.0 and 5.6.1 of XZ Utils. These updates implement a backdoor. Over the next few weeks, Tan or others called on Ubuntu, Red Hat and Debian developers to incorporate the update into their operating systems. Eventually, one of the two updates appeared in multiple versions, according to security firm Tenable. Here’s more information about Tan and the schedule.
Can you elaborate on the purpose of this backdoor?
In short, it allows someone with the correct private key to hijack sshd (the executable responsible for establishing SSH connections) and execute malicious commands from there. The backdoor is implemented via a five-stage loader, which uses a series of simple but clever techniques to hide itself. It also provides a way to deliver new payloads without significant changes.
Many of the people who reverse engineered the updates had a lot to say about the backdoors. Developer Sam James provides an overview here.