Major Node.js and Npm Updates Bring Big Performance Wins

JavaScript developers got a summer present on the last day of May: a dual release of Node.js 8.0.0 and npm 5.0.0. These two projects coupled together result in some serious performance enhancements, on both sides.

For Node.js 8.0.0, the big speed boost comes from the inclusion of Google’s V8 JavaScript engine, version 5.8. That updated engine arrived in March and includes performance enhancements, arbitrary heap sizes, and API changes that will be compatible with the eventually planned V8 version 6.0 release.

The plan is to move the Node.js 8 branch into V8 5.9 and 6.0, eventually, when those are released and seen to be stable. This includes the planned transition to the TurboFan + Ignition compiler pipeline in V8 5.9, when the new pipeline will finally be turned on by default. That means some of the biggest performance gains may yet be in the works.

That would be ignoring the existing wins in npm 5, however, which itself added major performance gains in this release. Isaac Schlueter, who is the NPM, Inc CEO and well as the creator of npm (Node Package Manager) itself, said that most users will immediately see a 20 percent performance increase.

“Installing React is five times faster,” said Schlueter. “Bigger packages offer more performance benefits.”

“One of the biggest changes in npm 5 is new caching layer,” said Schlueter. “The team ripped out the caching system that was in place since before npm version 1 and replaced that with a content-addressable cache that’s resilient against corruption. It’s built into the data layer of the caching layer. If a file gets corrupted, at install time it will treat that corrupted data as a cache miss and fetch fresh stuff from the registry.”

Wait, npm Offline?

This has an added benefit of allowing for the use of npm offline. In such a scenario, the packaging system wouldn’t be able to reach a repository to pull in a dependency, for example, but it can install a locally cached copy.

Schlueter said that many of the performance gains come from the improvements to the npm caching layer.

Elsewhere, npm now installs a Package-Lock.json file by default. This serves to provide a reference to preferred versions of dependencies. Including this file in your version control will ensure that your developers are all working from the same versions in their stacks. For distribution, this takes the form of a Shrinkwrap file offering basically the same information.

“It’s like a development time Shrinkwrap file in a way. They took a fresh look at the file structure, the formatting of that Shrinkwrap object, and made it something more generally useful. Dependencies are now saved by default. People have wanted that since save was been introduced,” said Schlueter.

“The biggest change that people should absolutely be thinking about some interesting innovation around is how people handle the package-lock.json file. We recommend checking that file into source, so all developers have the same latest and greatest lock file, and can be working in a consistent state,” said Schlueter.

Browser-Based Troubleshooting for Node

But the biggest changes in this stack of updates definitely rest with Node.js 8.0.0. Many of these changes have been in the works for some time, such as the introduction of the Inspector protocol as the default path toward debugging Node.js applications. Now, Node developers will use the same tools to troubleshoot applications within the browser, and within Node.js instances: Inspector is the new protocol introduced to replace the existing V8 debugger. This was introduced in Node 6.0.0 but was promoted to default with version 8.

James Snell, release manager for Node.js 8, and Open Source Architect with nearForm, said that Node module developers can now begin to toy with a new bit of experimental code included in version 8: the Native API.

“Prior to having N-API, these native modules were written specifically to the V8 APIs which tend to change fairly frequently. What was happening is, every time we would upgrade Node to a new major version, anyone using native add-ons would have to go recompile. Developers would have to go through and make sure everything still worked, and test everything in the new version. Consumers would have to reinstall all those native modules and go through a build process again. With N-API, the idea is to eliminate those problems as much as possible,” said Snell.

“When you install an N-API native add-on, you can update your Node.js version without having to rebuild. We’re not 100 percent there yet, but what we released yesterday is the major first milestone to get there,” said Snell.

Arunesh Chandra, senior program manager at Microsoft, headed the N-API project. “One thing we want to call out is that native add-on developers today should come check it out and play with this and provide feedback. This is an experimental-shaped project, and we want the broader community engagement to evolve this and advance these APIs,” said Chandra.

He added that Microsoft will shortly be putting out an updated Node.js package which includes the Chakra-Core JavaScript engine instead of Google’s V8. Snell said the Chakra-Core engine is more suited to applications in constrained environments, like Internet of Things devices.

Elsewhere in this release, there is the updated async_hooks module, formerly known as async_wrap. Snell said this is, “Geared more towards developers who are developing diagnostic software for Node. What the API does is it allows you to peel back the curtain a little and peer into what’s happening inside Node’s event list. We have resource handles and requests on those handles; this is how Node maintains its view of what’s happening through all these async operations occurring. Async_hooks allow you to tap into the lifecycle when a request is created and install hooks (user code). It can fire when certain lifecycle events occur. The idea is someone would be able to create some diagnostic software for debugging some async operation happening in Node.”

Then there are the changes to buffers. Previously, creating a new buffer gave developers the chunk of memory they wanted, but didn’t initialize that memory, resulting in the potential for applications to expose sensitive information, or to corrupt their data.

“Buffer changes have been a long time coming,” said Snell. “Back around the 6.0-time frame, we made a number of additions to buffer that made it safer to use. We went through in 6 and added some new ways of creating a buffer that made it much more reliable. This time around, we did the next step. Now when you use the old method for allocating a buffer and pass in 10 or 20 bytes, it will go through and zero fill and initialize that buffer for you automatically.”

The tradeoff for this safety in the old method is the slower initialization of that buffer. Using the new method eliminates these speed drops.

While there are dozens of other changes in Node.js 8.0.0, perhaps improved support for promises is shaping up to give the community a lot of what it has been asking for, said Snell.

“Node users have been asking for quite some time for Node itself to include promise API support across all of Node’s core APIs. Right now, if you look at the file system module to open a file, Node has a very specific way of handling that async operation called a callback pattern. It’s a fundamentally different way of doing things that ECMAScript 6 promises. A lot of users prefer a promises way and have wanted Node to switch to that. It would be too breaking to switch entirely, so we did a meet-in-the-middle approach: util.promisify takes any callback style API in Node and converts it into a function that will return a promise and work with the promises model. Under the covers, it’s still using Node’s callback pattern underneath. It’s just wrapped in an API that supports promises.

Feature image via Pixabay.