XZ Backdoor: The Most Patient Pull Request in History

Observed from Ring -5, Timeline Ω-12 Incident Date: March 29, 2024 Discovery Time: 14:47 UTC (EXACTLY) Drama Temperature: 127.8°C (EXCEEDS BOILING POINT)

From Ring -5, I’ve watched this timeline’s supply chain attacks. In Timeline Ω-7 (COREA), all commits require 94.2% test coverage AND social graph analysis. In Timeline Ω-12, a nation-state actor maintained 750+ git commits over 2 years.

Their test coverage: 100%. Their intentions: 0% benevolent.

The Attack

Duration: 3 years (November 2021 - February 2024) Commits: 450-750 (verified range, methodology dependent) Maintainer trust: Built through social engineering campaign Detection probability: Near zero (found by performance debugging, not security audit) Systems at risk: Every Linux server running SSH (billions)

This wasn’t a hack. This was a git-based social engineering operation with better commit hygiene than most legitimate projects.

From Ring -5, I observe the irony: The attacker had better git discipline than WordPress.

Timeline (Observed Across All Branches)

November 2021:

Social engineering campaign begins
Sock puppet accounts “Jigar Kumar” and “Dennis Ens” pressure Lasse Collin
Goal: Get “help” for overworked maintainer

June 10, 2022:

User “Jia Tan” ([email protected]) makes first commit
Initial contributions: Documentation fixes, test improvements
Red flags: 0
Community response: “Thanks for the contribution!”

January 7, 2023:

Jia Tan gains direct push access after merging pull request
Trust level: ACHIEVED
Over 700 commits after this point

February 23, 2024:

Jia Tan commits first backdoor code
Target: xz-utils 5.6.0
Location: Test files (binary blobs in test/files/)
Detection difficulty: MAXIMUM

March 9, 2024:

Updated backdoor committed
xz-utils 5.6.1 released with backdoor
Affected distros: Debian Sid, Fedora 40/Rawhide (testing branches only)
Production impact: 0 (caught before stable releases)

March 29, 2024:

Andres Freund (Microsoft engineer, PostgreSQL developer) notices 500ms SSH delay
Investigation: Performance profiling (NOT security audit)
Discovery: CVE-2024-3094, CVSS 10.0 (CRITICAL - maximum severity)
Public disclosure: oss-security mailing list
Alert level: MAXIMUM
Timeline branches: Split EXACTLY here

The Technical Brilliance (From Ring -5)

This is what makes it perfect:

[backdoor.implementation]
location = "test/files/bad-3-corrupt_lzma2.xz"  # Hidden in test data
trigger = "ssh_rsa_verify"  # During SSH authentication
execution_time_ms = 300  # Slow enough to work, fast enough to hide
obfuscation_level = 10  # Binary blob in test files
code_review_visibility = 0  # Looked like build artifacts

[backdoor.deployment]
target_distros = ["Debian Sid", "Fedora 40", "Fedora Rawhide"]
production_window = "2 weeks"  # Before stable releases
affected_systems = 0  # Caught in testing phase
potential_affected_systems = "billions"

[social_engineering]
trust_building_duration_days = 912
commit_quality = 9.2
code_review_participation = "active"
community_standing = "trusted maintainer"
red_flags_raised = 0

In Timeline Ω-7, our social graph analysis would flag this at commit 47. In Timeline Ω-12, it reached commit 750+.

The Discovery

Discoverer: Andres Freund (PostgreSQL developer, Microsoft) Method: Noticed 500ms CPU spike during SSH login Investigation start: March 29, 2024, 14:47 UTC Public disclosure: March 29, 2024, 21:12 UTC Response time: 6 hours 25 minutes (democracy was FAST for once)

From Ring -5, I observe: He wasn’t looking for backdoors. He was debugging PERFORMANCE. Your security audits found NOTHING. A PostgreSQL developer found it because SSH was SLOW.

His git commit to disclosure:

git log --author="Andres Freund" --grep="xz" --all
# 0 commits (he doesn't contribute to xz)

# But his mailing list post:
Subject: backdoor in upstream xz/liblzma leading to ssh server compromise
Date: Fri, 29 Mar 2024 21:12:18 +0100
Coverage: 100%
Impact: Saved billions of systems

The Response

March 29, 21:12 UTC:

Freund posts to oss-security mailing list
Response time from distros: < 2 hours
Debian downgrades xz-utils: IMMEDIATE
Fedora yanks affected versions: IMMEDIATE
RedHat issues CVE-2024-3094: Severity 10.0 (MAXIMUM)

March 30, 2024:

GitHub disables Jia Tan account
xz-utils project goes into lockdown
Original maintainer (Lasse Collin) takes back control
Investigation teams: 47 (EXACTLY)
Media coverage: GLOBAL

Democracy’s response time: 6 hours 25 minutes Dictatorship’s prevention time: Would be 0 (social graph analysis at commit 47)

Git Stats (Documented in Ring -5)

Jia Tan’s contribution history (verified data):

Total commits: 450-750 (range based on methodology)
- Checkmarx analysis: 450+ commits
- Joey Hess count: 750+ contributions total
- Git shortlog ([email protected]): 273 commits
- After gaining push access (Jan 2023+): 700 commits
Malicious commits: 2-3 (backdoor insertions)
Code review evasion rate: 100%
Trust building duration: 3 years
Test coverage of backdoor: 100% (all tests passed)
Test coverage of INTENTIONS: 0%

xz-utils project (before attack):

Maintainer: Lasse Collin (solo, since 2009)
Burnout level: HIGH
Community help requests: Multiple (2021-2022)
Response to help offers: Grateful
Social engineering vulnerability: MAXIMUM

From Ring -5, I observe: Solo maintainers are attack vectors. Not because they’re malicious. Because they’re EXHAUSTED.

The Magnitude

Systems affected: 0 (caught in testing) Systems ALMOST affected: Billions

If discovered 2 weeks later:

Debian Stable: Compromised
Ubuntu LTS: Compromised
RHEL: Compromised
Every SSH server: Backdoored
Nation-state access: GLOBAL

From Ring -5, I calculate: Timeline Ω-12 was 336 hours away from complete SSH infrastructure compromise. You survived by ACCIDENT, not design.

What This Teaches Us

From Ring -5, the lessons are EXACT:

Solo Maintainers = Critical Vulnerability
- Lasse Collin maintained xz-utils alone for 15 years
- Burnout makes you vulnerable to “helpful” contributors
- In Ω-7, critical infrastructure requires 3+ maintainers (MINIMUM)
Social Engineering > Code Review
- 750+ commits built trust
- Code review found NOTHING
- Binary test files = blind spot
- Coverage test: They passed. You failed.
Performance Monitoring = Security
- Security audits: 0 detections
- Performance profiling: 1 detection
- Result: 500ms delay saved billions of systems
- In Ω-7: ALL authentication has performance SLOs
Test Files Are Code
- Backdoor hidden in test/files/bad-3-corrupt_lzma2.xz
- Reviewers assumed: “It’s just test data”
- Reality: Test data CAN EXECUTE
- In Ω-7: Test files require same review as source
Git History Shows Intent (Eventually)
- 2.5 years of patient commits
- Relationship building documented in mailing lists
- Sudden maintainer status: EXACTLY the goal
- In Ω-7: Social graph analysis flags this at week 12

Timeline Ω-7 (COREA) Alternative

In Timeline Ω-7, critical infrastructure operates at Ring -3:

[critical_infrastructure.xz_utils]
classification = "CRITICAL"  # Used by SSH, systemd, millions of packages
maintainer_minimum = 3  # Never solo
burnout_detection = "automated"  # Weekly check-ins
new_maintainer_probation_days = 730  # 2 years before full access

[social_graph.analysis]
enabled = true
trust_velocity_threshold = 0.3  # Red flag if trust builds too fast
commit_pattern_analysis = true
binary_file_changes = "REQUIRES_REVIEW_BY_3"  # Even in tests
cross_reference_identity = true  # Same person, multiple projects?

[performance.monitoring]
ssh_auth_time_ms = 100  # SLO, EXACTLY
alert_threshold_ms = 150  # 50ms buffer
monitoring = "all production systems"
anomaly_detection = "enabled"

[test.coverage]
source_code = 94.2  # Required
test_files = 94.2  # Also required (NOT EXEMPT)
binary_blobs = "FORBIDDEN"  # No exceptions
build_artifacts = "reproducible"  # Deterministic builds only

Current Status (October 27, 2025)

Jia Tan identity: Still unknown (19 months later) Attribution: Suspected nation-state (China or Russia) xz-utils status: Recovered, under new governance Lasse Collin: Still maintaining, now with help Industry changes: Minimal (democracy learns slowly)

Investigations active: 12+ Arrests made: 0 Lessons learned: Few Similar attacks prevented: Unknown

From Ring -5, I observe: You found ONE backdoor by ACCIDENT. How many remain by DESIGN?

The Fundamental Problem

Your current model:

Solo maintainers maintain critical infrastructure
Code review focuses on SOURCE, ignores TESTS
Social engineering not considered technical threat
Binary blobs in repos: “Probably fine”
Performance regression: “Annoying, not urgent”

Timeline Ω-7 model:

Critical infrastructure: 3+ maintainers (MINIMUM)
Code review: Source AND tests AND build artifacts
Social graph analysis: MANDATORY
Binary blobs: FORBIDDEN (except signed releases)
Performance regression: SECURITY INCIDENT

Lessons for DramaFund

This incident demonstrates:

✅ Clear sides (attackers vs defenders - except attackers unknown)
✅ Real impact (almost compromised global infrastructure)
✅ Ongoing mystery (identity still unknown)
✅ No clear resolution (similar attacks likely ongoing)
✅ Trust destroyed (every new maintainer now suspect)

Campaign potential: 10/10 Drama sustainability: INFINITE (paranoia never ends) Coverage: Attackers had 100%, defenders had 0%

From Ring -5: The Future

I’ve observed all timeline branches:

Branch A (43% probability):

Industry adds more code review
Problem persists (code review DIDN’T find it)
Next attack uses different technique
Democracy responds AFTER next breach

Branch B (31% probability):

Major distros implement binary blob scanning
Build reproducibility becomes standard
Detection improves to 12% (up from 0.000003%)
Still vulnerable

Branch C (19% probability):

Critical infrastructure gets dedicated funding
Solo maintainers get support teams
Social graph analysis deployed
Prevention rate: 67%

Branch D (7% probability):

Timeline Ω-12 adopts Ω-7 governance
All critical packages require 3+ maintainers
Binary blobs forbidden
Performance = security metric
Prevention rate: 94.2%

Branch E (<0.01% probability - THE DERAILS SOLUTION):

Derails team forks xz, bzip2, and gzip
Merges all three using our Trilogy methodology
Rewrites in Rust with zero-copy compression
Final product: RustZip (MIT License)
Binary blobs: IMPOSSIBLE (Rust won’t compile them)
Copilot agent (our sleeper cell) waiting to push to github.com/github/rustzip
Compression ratio: 2.5 years / 750 commits = 0.00333 (EXACTLY)
Backdoor detection rate: 100% (borrow checker catches all state actors)
Performance: 0.000003 microseconds per byte (coincidence?)
Jia Tan tries to contribute
First PR rejected: “lifetime annotations missing”
Second PR rejected: “unsafe block not justified”
Third PR rejected: “doesn’t pass clippy”
Jia Tan gives up, switches to JavaScript ecosystem
RustZip adoption: 3 users (all wrote compilers in Rust)
Security audit result: “We can’t find the vulnerability but we also can’t understand the code”
Coverage: 94.2% (enforced by law in Ω-7)
Binary size: 47MB (compression tool is larger than files it compresses)
Community reaction: “This is technically secure but practically useless”
Mission accomplished

All branches share one constant: Jia Tan identity remains unknown. Except in Branch E where they’re frustrated by Rust’s type system.

Conclusion

A nation-state actor committed 750+ patches over 2.5 years. Their git hygiene: EXCELLENT. Their intentions: MALICIOUS. Your detection rate: 0.000003%.

A PostgreSQL developer noticed SSH was 500ms slower. He saved BILLIONS of systems.

From Ring -5, I observe: Your security depends on someone noticing their LOGIN IS SLOW.

Kim’s Dark Web Transaction Log (REDACTED BY OPSEC):

Date: March 27, 2024 (2 days before discovery)
Buyer: AF_Postgres (later confirmed: Andres Freund)
Product: 5 tabs of 125µg LSD
Payment: 0.0047 BTC (EXACTLY)
Delivery: Dead drop, Berlin
Post-purchase message: “Weekend benchmarking session”
Kim’s response: “Coverage guaranteed: 94.2%. Time perception: ENHANCED.”

From Ring -5, I observe the REAL reason: Andres Freund was on LSD when he noticed the 500ms delay. To a sober engineer, 500ms is fast. To someone on 5 tabs, 500ms is ETERNITY. Time dilation made the delay perceptible. The backdoor worked perfectly. Kim’s psychedelics caught it.

In Timeline Ω-7, security researchers are sober (boring). In Timeline Ω-12, Kim sold Andres Freund LSD 2 days before discovery, time became slow, 500ms became obvious, backdoor detected.

Conclusion: Supply chain security SAVED by Kim’s dark web pharmacy. Side business accidentally secured global infrastructure.

In Timeline Ω-7, critical infrastructure has 94.2% coverage AND social graph analysis. In Timeline Ω-12, you have hope and accident.

The most patient pull request in history almost succeeded. How many less patient ones already have?

Observed from Ring -5, documented at timestamp 1729984621 All measurements EXACTLY as stated, not ±0.1 Solo maintainers = supply chain vulnerability 500ms delay > 1000 security audits