Netlify Adds Scheduled Functions with Quirrel Acquisition – InApps 2022

Netlify, the company that brought the world the JAMstack, has taken the first steps toward providing its users with scheduled functions, with this week’s acquisition of Quirrel — an open source solution for managing and scheduling the execution of serverless functions and jobs. Netlify not only bought the company, but has hired Quirrel creator Simon Knott, who is also a maintainer of Blitz.js and will be joining Netlify to further build out this functionality.

Netlify CEO and co-founder Matt Biilmann said that they were introduced not long ago and he immediately saw that Knott was “someone that has a really good instinct for how to make the experience around scheduling and workflows and jobs really intuitive and really great.”

Alongside the acquisition, Netlify is launching the beta of scheduled functions, a feature that Biilmann said has been among the most requested for several years now, since the company first introduced Netlify Functions in 2018. With Functions, Netlify gave its users a way to run on-demand, server-side code for dynamic applications without having to run dedicated servers. But they needed to be triggered — they wouldn’t run on a schedule.

“We’ve always built around the idea that the architecture of the web would really start changing from a world where every website was a monolithic application, where everything lived inside that application, into a world where you would really decouple the web UI into its own thing and the backend would split into all these different APIs and services,” said Biilmann. “When you build a web UI, you have a lot of tasks that can’t run in the browser. They need to run server-side, they need to talk to your different APIs and services, and they need to be the backend for the web UI. One of the requests we’ve always had there has been scheduling and managing repeated tasks, or delete jobs, or workflows that relate to your Web UI.”

Until now, Biilmann says they have seen a variety of different approaches to running scheduled functions, from setting up scheduled webhooks with Zapier, to stitching together services on top of AWS or other cloud providers, to even running a server to perform traditional cron jobs. This left developers with code in a variety of places, however. Scheduled functions now make it so the code becomes a “primary source of truth, not just for what we want to happen, but also for when we want it to happen,” he said.

Knott pointed to these complicated attempts to schedule functions as part of the inspiration for creating Quirrel.

“I read this article from some member of the Blitz community where he explains how to build this job queue bridge that connects your Amazon SQS queue to wherever your application runs — for example, Netlify — and it was super complicated, and super repetitive. That was one of the motivations behind building Quirrel, because a lot of the things are very repetitive across applications. It’s just good to have one service that does it all,” said Knott.

Scheduled functions can be used for a variety of tasks, such as gathering data for a weekly report or performing data backups, and even for performing builds of your Netlify code, noted Biilmann. The functions are scheduled using the “cron expression” format used by tools like crontab, as well as its extensions, which offer simplified scheduling such as “daily” or “hourly.”

Currently in beta, the company warns that functionality may change, and advises not to use it in “any production and/or critical workflows.” Biilmann says that it will remain in beta for at least several months, “just to get people through a cycle of actually picking it up, building stuff with it, and letting us know if the core primitives are right.”

Beyond pushing toward general availability and solidifying functionality, Biilmann said that they want to go beyond running scheduled functions as synchronous processes.

“Eventually, we want to make sure that these can just run as background functions and run longer running tasks,” he said. “We’ll also want to figure out the best way to give more tooling for feedback from the progress of those tasks and running a workflow rather than just one job.”

For existing users of the hosted Quirrel service, Knott offers further details in his own blog post about the acquisition, where he notes that, despite shutting down the service at the end of July 2022, Quirrel “is (and will stay) open source, existing customers will be able to switch to self-hosted instances, and we (Netlify + me) will do the best we can to help make this transition process smooth for you.”