Dygraphs Delivers in JavaScript the Promise of Live Data Visualization – InApps 2022

At the Influx Days conference in San Francisco, time series data was the topic for discussion. While the event was sponsored by Influx Data, makers of the InfluxDB time series database, most of the talks at this one-day event focused on digesting the mountains of information generated by log files from modern systems, rather than on any one specific database. The modern lifeblood of administrator and operator work is intense analysis and tracking, and nothing helps to digest data more than better visualization tools.

The Old Promise of Smalltalk

One man has long been building a tool in JavaScript that helps to bring charts and graphs to giant pools of time series data. In doing so, he’s realized the long-distant dream of the Smalltalk community: the idea that one could have dynamic code rendering data analysis live, as the code is edited. Such a vision was brought to Wall Street in the early ’80s as the Smalltalk community but soon petered out likely because it was simply too early for such an innovative approach to programmatically analyzing data.

Dygraphs originated at Google in 2006, when Dan Vanderkam decided to build a tool for data visualization for his team, there. He forked Alastair Tse’s plotkit, and added interactivity to build a dashboard for the team he’d just joined.

By 2009, another team at Google was using it, and the project was released on GitHub and the now defunct Google Code. Dygraphs really gained traction in 2013, and has since become a popular way to dynamically chart time series data. At Influx Days, Vanderkam gave an overview of the project and where it’s headed.

Today, Dygraphs is still used at Google and elsewhere, in projects like Chronograf and on sites like DuckDuckGo. Vanderkam said he continues to explicitly focus on the dynamic processing of time series data as the strength of the project. Oh, and its speed doesn’t hurt. Dygraphs can chart a million points and show them dynamically in a few seconds.

Dygraphs Delivers

One of the reasons Dygraphs is so fast and responsive is that it renders in canvas, rather than SVG. Scalable Vector Graphics, are handled by the DOM, said Vanderkam, which can be surprisingly slow to render.

“SVG is like XML for drawing. When you define an SVG tag you can style them with CSS and use DOM APIs to manipulate them,” said Vanderkam. “Canvas is nothing like that. Canvas is completely opaque to DOM and CSS. It’s entirely imperative. If you want to change something, you redraw the whole canvas. It sounds like it would be slow, but in practice, the DOM is usually slower. I’ve often had to switch from SVG to canvas. That’s a strength of Dygraphs: its using canvas from the start.”

Today, Vanderkam is a senior software engineer for Sidewalk Labs, but even after 10 years, he remains the primary maintainer of Dygraphs. This isn’t necessarily something he is proud of, and he encouraged the audience to get outside help early if they start an open source project.

As a result, Vanderkam has updated the project in fits and starts over the year, rather than at a reliable cadence. He has come up with a few interesting tools to help outside developers and himself over the years, however.

One of those tools is an automated screenshot comparison tool which he’s using to check test demos between releases. If a result is different, the screenshots of the demo running will be different, and thus these can catch regression bugs.

Elsewhere, at Dygraphs.com, Vanderkam has been working on tools for users to see and try Dygraphs interactively. dygraphs.com/jsbin is a browser-based environment where code can be written on the left side of the browser screen, and rendered into graphs on the right side.

What’s Next for Dygraphs

As for project management tools and tips, Vanderkam advises open source projects to move support to StackOverflow, where topics about Dygraphs are automatically sent to him. Thus, when someone has a problem, there’s a wealth of other knowledgeable people who might first answer it, and if not, Vanderkam himself can respond. Solved issues are SEO’d by StackOverflow, and will hopefully come up the next time someone has the same problem, saving everyone time. He also advises users to post their bug reports first at dygraphs.com/fiddle, which embeds GitHub issue tracking.

Vanderkam is also working on a new tool called Just Chart it, which is based on the Visual Studio Code. This tool includes auto-completion of code, while also rendering a spreadsheet interface for the data being entered. This gives the user an all-in-one time series visualization and manipulation tool with dynamically loading changes as the code is manipulated live.

“Dygraphs has TypeScript type annotations. This editor is the same as Visual Studio Code, so it does autocomplete. You can also set per-series options, and it also has a notion of which series is highlighted,” said Vanderkam, demonstrating the dynamic interactive elements that could be integrated into charts generated with Dygraphs.

Dygraphs reached version 2.0 earlier this year, and Vanderkam has a few ideas where he’d like to take the project going forward. Specifically, he’s planning on doubling down on the strengths of Dygraphs, rather than expanding it into new areas, such as non-time-series data.

“It could be easier to create charts that track the course of data and daily data, and as you go in it fetches hourly, or minutely data. There are opportunities for optional support for themes. There could be framework integrations for React or Angular, so I’d have a chance to rewrite some of the APIs I’d like to get rid of,” said Vanderkam.

As for how Dygraphs compares to other JavaScript charting tools and libraries, Vanderkam said that popular project D3 is more generally focused than Dygraphs. He said that D3 is all about binding data to DOM elements, and that doesn’t work as well with the fast canvas rendering used by Dygraphs.

Similarly, Charts.js, another popular open source JavaScript charting library, is more generally focused on charting, rather than specifically charting time series data, said Vanderkam.

Finally, Vanderkam had some generally relevant advice for all developers: When writing comments, your target audience is you in two years, without any memory of the scope of the work. “When I’m writing a comment, my target audience is my self in two years, when I’ve completely lost all context on what it does and why it does it,” said Vanderkam.

Influx Data is a sponsor of InApps.