A Review of Open Source Software Supply Chain Attacks

Likely, a lot of the code you run is not code you wrote. But what does that mean in terms of security? In this post, we will discuss how long it takes to detect a malicious package, what happens if you include one of these packages, and what you can do about it.

I’ll provide key statistics and takeaways from the article by Marc Ohm et al. They reviewed 174 malicious packages in RubyGems, PyPI and npm in the article ‘Backstabber’s Knife Collection: A Review of Open Source Software Supply Chain Attacks‘.

If you would like to analyze these packages yourself, be sure to visit the authors’ GitHub page.

All your packages belong to us

Today’s typical application includes a number of open source packages, often with additional packages used in the development flow. Many of these contribute to a better development experience and, ultimately, a better product.

Hackers being hackers, all these useful open source packages make very interesting targets for supply chain attacks (i.e., ‘poisoning the well’). In one such case, the npm package event-stream was compromised in September 2018. The hackers targeted developers’ bitcoin wallets through the package for two months before someone raised an issue. At that time, event-stream had over 1.5 million weekly downloads.

Evidently, this is not the only example, but are we any good at detecting this maliciousness?

How quickly are malicious packages detected?

A malicious package is available for 209 days, on average (min: -1, max: 1216) before publicly reported.

You ask, available for -1 days? That was eslint-scope v3.7.2, known malicious from the get go, and still got downloads. For a detailed postmortem on that specific incident, check out this post.

On the other end of the spectrum, it took 1216 days to discover the badness inrpc-websocket 0.7.7.

So, scanning at build time is no silver bullet: no one might have discovered the maliciousness yet. If you’ve pinned to the malicious version, you could be downloading it indefinitely. This is why it’s important to scan regularly.

It’s not all bad news, luckily. In some cases the package manager’s internal teams discover the issues timely, like with electron-native-notify.

What happens if you’ve included a malicious package?

Most malicious packages for JavaScript and Python start routines at installation (i.e. download/run extra files).

Ruby is immune to this behavior since Ruby does not implement the same install logic. Thus, malicious Ruby packages trigger routines at runtime.

• The primary objective of most packages (55%) is data exfiltration (e.g. /etc/passwd, ~/.ssh, ~/.npmrc, ~/.bash_history, ~/.aws). Special mention: some packages target the Discord token as this may be linked to credit card information.
• 49% of packages use some sort of obfuscation.
• 41% of packages check a condition before execution (e.g. check whether the app is in prod, check resolvability of a domain name, the amount of coins in a crypto wallet).
• 1% of packages used the ‘test‘ as their trigger. Special mention: latter-test-js v1.0.0 executes sudo rm -rf /*.
• Packages are typically agnostic to operating systems.

So, we know these packages might already do bad stuff the moment they are installed (unless you’re using Ruby). They’re mostly after the valuables on our laptop. But what can we do about it?

What to do about it

• Scan the packages you use regularly using tools like <a rel="noreferrer noopener" href="https://github.com/pyupio/safety" target="_blank">safety</a> (Python), <a rel="noreferrer noopener" href="https://github.com/rubysec/bundler-audit" target="_blank">bundler-audit</a> (Ruby), or the npm built-in <a rel="noreferrer noopener" href="https://docs.npmjs.com/cli/audit" target="_blank">npm audit</a> (JavaScript). If you use yarn, you can use <a rel="noreferrer noopener" href="https://classic.yarnpkg.com/en/docs/cli/audit/" target="_blank">yarn audit</a>. GitHub already scans some of this for you.
• Extend your CI/CD to verify the integrity of what you deploy or publish to detect injection and prevent further damage.
• Use temporary credentials where possible to limit the impact of scraped credentials.
• Rotate SSH keys, access keys, and tokens regularly, and if possible protect them with a passphrase.
• Store your secrets safely.
• Use multi-factor authentication wherever possible.

In case of compromise, be sure to rotate your credentials. Additionally, don’t forget to raise an incident with any other parties involved. They might be able to help you resolve or contain resulting issues.

References

1. M. Ohm et al., Backstabber’s Knife Collection: A Review of Open Source Software Supply Chain Attacks, International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment (2020). Springer
2. Instructions for retrieving malicious package samples
3. ESLint postmortem
4. Safety for Python
5. Bundler-audit for Ruby
6. npm-audit for JS
7. yarn audit for JS