node_modules directory can easily reach hundreds of megabytes in size, much to the chagrin of engineers who remember the days when an entire hard drive might not even hold 100MB. A brand new create-react-app project comes with 237MB of
node_modules at the time of this writing. There are even memes about this phenomenon:
If you were programming for the web in 2016, you probably recall the infamous
left-pad fiasco. TL;DR: an engineer who was unhappy with npm decided to unpublish all of his packages in protest. One of these packages,
rjust method on strings, a python programmer would discover the identically-named python equivalent, and a PHP programmer would find the helpful
The support matrix is even choppier for web applications. The aforementioned
padStart function doesn’t exist in Internet Explorer 11, and neither do most of the other convenience features added in ES6/ES7. Safari 13 lacks support for BigInt and requestIdleCallback. Edge has caught up a lot since its switch to the Blink rendering engine, but pre-Blink Edge didn’t support setting scroll positions on elements or array
flatMap. Most modern features work in most modern browsers, but you’ll still spend a lot of mental cycles making sure nothing slips through the gaps, especially if you need to support IE11.
Fortunately, there’s a pretty robust toolchain for using the latest language features in web applications while maintaining support for older browsers. It goes something like this:
Internet Explorer 11or queries like
>1%(browser versions with more than 1% global usage),
last 3 Chrome versions, etc.
node_modules. We’ve traded an eye-opening amount of disk space for a great developer workflow and a consistent experience for end users.
Did you know that either npm or yarn will happily install multiple versions of the same package? Imagine you’ve got package A and package B in your dependencies list. Both A and B depend on package C but with incompatible version requirements. In ruby, this produces an installation error and you’re left to work out a consistent dependency tree on your own. npm and yarn, on the other hand, will happily install multiple versions of package C. They accomplish this by giving packages A and B each their own nested
node_modules, so packages without conflicts can be deduped to the top level while conflicted packages are kept in nested directories.
There are certainly some benefits to this approach. I have spent many long hours working through version conflicts in ruby, where seemingly unrelated gems demand inconsistent versions of a shared dependency. But this approach inevitably results in a lot of duplicate packages, and there’s also not much you can do about it. To some extent, this behavior is a necessary consequence of an ecosystem with a greater reliance on helper packages. It would be hellacious trying to get dozens of packages to agree on the same set of helper versions; it’s bad enough in ruby where only a few packages are usually in conflict. Regardless, duplicate package versions should be kept in the back of your mind when trying to understand
The last thought I’ll leave you with is this: don’t stress too much about it. Be honest — when was the last time that you spent even a few minutes dealing with a problem caused by 100MB of used hard drive space? I’m fairly certain that I’ve invested more brain cycles writing this article than I’ve ever spent on that particular class of problem. It feels wrong and can be hard to stomach, especially if you were programming in a time when hard drive space was at a premium. But it’s just not that big of an issue in practice, and it’s a problem that’s easily solved if it does arise by spending a fairly negligible amount of money. As with any issue, you’re best served focusing your mental energy where it creates the most leverage, which is usually solving hard business problems to provide value to your end users.
Originally published on the Aha! engineering blog.