New Relic Insights has technically been in beta since around March, even though it has around 6,000 users. The service is now available to anyone.
Insights adds new kinds of valuable analytics for developers but also takes New Relic deeper into customer organizations where it can be used by marketing, customer care and business executives. Building on the data that New Relic already collects for customers, Insights also pits New Relic against a whole new set of competitors – data visualization and cloud BI providers.
Quizlet, a company that offers an online learning tool for students and teachers that averages 20 million unique hits a month during the school year, has been using Insights from the start. All 20 employees, including developers, customer support and ops people use the service.
Many of the use cases that Quizlet described sound a lot like those on offer from the cloud BI providers (think Birst or Good Data) or the new breed of data visualization services (like Chartio and ClearStory). New Relic seemed blissfully unaware of the competition. “We’re kinda new to data analysis and we don’t know all the BI tools out there,” Jim Gochee, senior vice president of product for New Relic said.
New Relic’s Insight offering is likely to be a relatively easy sell to existing New Relic customers, since it’s a simple add-on. That’s not insignificant. But if New Relic wants to go head-to-head with data visualization or cloud BI providers, it will have to do some work to raise awareness of itself as a contender.
Quizlet was already a New Relic customer. “What New Relic did is reused the same data but put it into an analytics context where instead of looking at it for performance, you can look at it in general for how people are using your site, what parts of the apps they’re using and letting you query over all the ways people can use the site,” said Andrew Sutherland, CTO and founder of Quizlet.
Quizlet has set up around 20 different dashboards. Developers can easily check analytics about how people are using new features or services.
“Any time developers add a feature, the second it gets deployed they can instantly start seeing data about how it’s getting used. That’s really powerful for iteration cycles and the ability to change a product quickly,” Sutherland said.
The infrastructure team uses Insights to measure the performance of various vendors. For instance, they track Quizlet’s CDN provider, noting how much time it takes for servers to render a page as well as to finish loading an entire page. Developers can track that information based on individual data centers and let the vendor know if things are slowing down.
Developers also use Insights to track caching rates over time for audio content available on the site.
Other dashboards show that Quizlet users are seeing error messages. Another shows users based on how they logged in, including with Google or Facebook credentials.
The feedback and support team uses Insights to look at each page that a user viewed before submitting a support message. That can help the support team inform their response.
Business-focused dashboards that Quizlet has set up using Insights track engagement, slicing users by age, how often a user visits the site, whether the user is a paying or free customer and how many people are creating study sets to share with other users. Age is particularly important for Quizlet because it helps the company determine if a visitor is a student or a teacher; each of those segments visits the site for different products and services.
For non-technical users like the feedback and support team, Quizlet has set up a dashboard with pre-formed queries. But because users can see the query behind each graph, they have the freedom to tweak it. “If you don’t know how to generate a query from scratch but had an idea of a small change you wanted to make to tease out additional information, it’s easy to edit and manipulate the query,” Sutherland said.
Insights uses a unique, SQL-like language that lets users easily create new queries, he said. That gives Insights an edge over services like Google Analytics, which Quizlet had been using to try to track usage of its site.
“With Google Analytics, you can only pull up data for things they’ve already thought of. With insights, you can create any metric or dashboard,” he said.
In addition to Google Analytics, before Insights Quizlet was doing its own A/B testing and building custom dashboards around that. It also experimented with other options over time. Sutherland said that what’s particularly powerful about Insights is that New Relic is already collecting the data that Insights analyzes. With most analytics tools, users have to figure out in advance what data to collect and then run queries. “With this, everything is already tracked so we can run queries as soon as we think of them. We don’t have to think what we want, write code then wait a couple days for the data to come in,” he said. “It’s like we’re able to ask it so many more questions and get answers way faster.”
One example of a way that Quizlet has been able to make a notable change based on the analytics stemmed from an examination of the age of users as they work through the process of signing up for a subscription. It turned out that people who started out through the process were heavily skewed toward a young age, around 16 years old. But the age of most people who actually signed up was around 25 years old.
“We realized that we had tons of kids showing up on these upgrade pages whose product wasn’t even relevant for them,” he said. The company ended up increasing the amount of marketing it did to teachers and decreasing the amount it was doing to students, who weren’t actually paying for it.
Sutherland said he’s been impressed at how quickly queries get returned. The average response time for the 250,000 queries that have been made overall since March is 92 milliseconds, New Relic’s Gochee said. New Relic hosts the service itself on non-virtualized servers with SSD drives in a Chicago data center. “We didn’t cloud host it because for us disk I/O is so critical,” he said.
Feature image via Flickr Creative Commons
New Relic is a sponsor of InApps
