I’m fascinated by the story of how a Microsoft engineer discovered a major, carefully disguised backdoor security vulnerability that had been years in the making and almost implemented.
background
A software backdoor hidden in a widely used compression utility allows someone to remotely access the entire system.
This is the work of a user named Jia Tan, @JiaT75, who has earned his trust over the years. Now his account is suspended everywhere.
HackerNews has an interesting segment.
Microsoft security researcher Andres Freund is believed to have discovered and reported the issue on Friday.
The heavily obfuscated malicious code is said to have been introduced by a user named JiaT75 through a series of four commits to the Tukaani project on GitHub.
long game
These open source projects are volunteer efforts. They pay nothing.
The person usually responsible for the code, Lasse Collin (Larhzu), has maintained the utility since 2009, but he is suffering from burnout.
Jia Tan started contributing over the past 2-2.5 years and was granted commit access and then released admin rights, about 1.5 years ago.
Backdoor discovered in years of hacking conspiracy
Much of the story is bizarre and difficult to follow. The article about Unicorn Riot is generally readable.
Consider the backdoor discovered in a years-long hacking conspiracy
A fascinating but ominous software story was released on Friday: A widely used file compression package called “xz utils” has a clever embedded system for backdoor shell login connections, which is currently unknown. The extent to which dangerous software packages have found their way into countless internet-connected devices. It appears that the characters injecting this content have been playing for a long time and have gained the trust of legitimate primary developers to have the authority to release new versions themselves.
andrea’s friend Reported this Friday morning on the industry security mailing listleading many experts to spend a day exploring the abyss of modern digital insecurity: “The upstream xz repository and xz tarball have been backdoored,” Freund wrote. It cleverly pokes a hole in the SSH daemon (sshd), which is crucial to the most basic aspects of modern computing.
Experts point out that if this is not discovered, the risks will be great @thegrugq Say it: “The ultimate goal is to be able to log into every Fedora, Debian, and Ubuntu machine on the Internet. If not national actors, then…”
Cryptozoologist Filippo Valsorda said, “This may be the best-executed supply chain attack we have ever seen publicly described, and it’s a nightmare scenario: malicious, capable, authorized upstream in a widely used library.”
The problem was discovered after Freund noticed that the new version was slowing down their PostgreSQL database tests, and they began debugging why this was happening. It turns out that the backdoor causes a small but noticeable drop in performance, which is a huge win for picky benchmarking types everywhere.
As Minneapolis security expert Ian Coldwater famous, “Open source maintainer burnout is a clear and present security danger. What are we doing about it?
This message from the original developer in June 2022 acknowledging burnout illustrates how Jatan gained control of the software:
“I haven’t lost interest, but my nursing abilities are quite limited, mostly due to long-term mental health issues, but also a few other reasons. I’ve been working with Jia Tan on XZ Utils a little bit recently, maybe he’ll play in the future We will wait and see for a more important role.
Also keep in mind that this is a pro bono hobby project.
Anyway, I assure you, I know this problem all too well, but haven’t made much progress yet. The idea of finding a new maintainer has also been around for a long time, as the current situation is obviously bad and sad for the project.
There should be a new stable branch of XZ Utils released this year, which contains threaded decoders and more, and some alpha/beta versions before that. Perhaps the moment after 5.4.0 is released is a convenient time to make changes to the project maintainer list. Forks are obviously another possibility and I have no control over it. […]”Lasse Collin, xz-devel mailing list, June 8, 2022
Some observers suspect that the figures harassing Colin via email may also be puppets trying to wrest control from him. In a detailed report, ars technica warns that even older versions may have security issues because bad actors have made numerous changes to binary test files over the years.
The backdoor story is unfolding
upstream backdoor
“Very annoying – the apparent author of the backdoor communicated with me over several weeks trying to add xz 5.6.x to Fedora 40 and 41 because it had “great new features”. We even worked with him to resolve a valgrind issue (which now turns out to be caused by a backdoor he added). After an inadvertent breach of the embargo, we had to step up our efforts to resolve the issue last night. “
“He has been involved with the xz project for two years, adding various binary test files, and to be honest, with this level of complexity, I would even be skeptical of older versions of xz until proven otherwise.”
US security alert
The U.S. Cybersecurity and Infrastructure Security Agency (CISA) has issued an alert about a supply chain compromise affecting the XZ Utils data compression library, CVE-2024-3094
CISA and the open source community are responding to reports of malicious code embedded in XZ Utils versions 5.6.0 and 5.6.1. This activity is assigned CVE-2024-3094. XZ Utils is data compression software that may be present in Linux distributions. Malicious code could allow unauthorized access to affected systems.
Industry-wide liquidation is needed
Mastadon user @glyph commented: “I really hope this causes the industry to rethink the common practice of letting your entire damn product rest on the shoulders of an overworked, slowly developing mental health crisis, without providing them with any financial or operational support.”
Here’s an interesting timeline of how this was accomplished.
This is a hero
Wow, just wow.
The code is likely still days away from being implemented.