New Relic founder and CEO Lew Cirne unveiled the future of his company at its FutureStack16 conference earlier this month, and its name is Seymour. This machine-learning technology mines data from the company’s stack monitoring software and turns it into useful information for customers.
New Relic collects a billion pieces of data per minute, said Bharath Gowda, senior director of product marketing in an interview. With data on this scale, it’s not possible for a person to wend through the complexity and make sense of it. So you need an automated and intuitive process, machine learning, to mine the data.
Cirne explained that his engineers learned from Facebook and other social media companies that the vast amount of data we collect can be surfaced, but the challenge is making it easy to use. “Making something very complex easy for the consumer is the hardest job of IT.”
Think of Seymour as a new way to come into New Relic, he said to the keynote audience. Most people just want to see a feed. “Seymour, tell me what’s interesting that I want to know about.” Seymour then pulls cards based on your role and what you’ve said you want to know and drops them in your feed.
A ‘card’ is created when Seymour sees something interesting in the massive influx of data coming across the New Relic infrastructure all into one nice easy-to-read feed.
Seymour already has some ‘predictive’ reporting built in. For example, users might get an alert saying “if nothing changes, then this server will go down in two weeks.”
Seymour also troubleshoots problems it finds, down to a specific piece of code, who wrote the code, and where the code resides.
More Slack
Because Seymour is integrated with Slack, messaging the errant coder is a snap. Users will have the full range of Slack functionality including starring, thumbs up and thumbs down, which Seymour then reads and uses later to determine which cards to show you.
Security monitoring is a top priority for all companies and is made easier by Seymour’s ability to detect anomalies.
Anomaly Detection
Algorithms are at the core of Seymour. There are currently 20, each focuses on a unique background job, and within each algorithm is plenty of room to tweak and refine as engineers see what is returned when it is run with real customer data.
One of the reasons Seymour was able to come so far so fast is that it is very focused, explained Lee Atchison, Principal Cloud Architect at New Relic, and author of “Architecting for Scale: High Availability for Your Growing Applications“ published by O’Reilly. There are machine-learning components in Seymour, but they only have to learn about performance monitoring.
Once Seymour surfaces an anomaly, it drills down, creating a trail of breadcrumbs across the services used, until it finds the piece of code that caused the anomaly, and sends that information to the developer, leading developer directly to the code instead of spending hours looking for which piece of code in which instance, run on which container, etc.
The algorithms will get more complete and smarter over time, and of course, the more New Relic products a customer has, the more useful Seymour becomes, as some of the 20 algorithms are specific to a product and some range across all New Relic products, providing a wider array of insights.
Peeking Behind the Curtain
Jim Gochee, chief product officer, explained that Seymour is built into the New Relic platform, which collects time series metrics, transaction traces, event data, browser sessions and a variety of data types that are already in the platform all available through API calls and can be queried by the customer through web interfaces.
Seymour runs background jobs periodically that comb through data then do processing on that using algorithms, heuristics, and maybe a dash of machine learning that have been coded by the New Relic engineers (or even Cirne himself).
Seymour Is Barely a Year Old
Cirne is a developer’s developer who has a history of grabbing ideas and running with them, holing himself up in resorts and coding for days, then calling in some developers to work with him before releasing the product to the R&D team. The idea for Seymour showed up about a year ago and Cirne ran with it.
It’s too early to really talk about the stack behind Seymour and which pieces are open source because the program is so new, said Gochee. But he did say the algorithms are fairly compact and fit into the New Relic platform, which is heavily into open source. New Relic has a big MySQL footprint, and it is ramping up its use of Cassandra. It baked Kafka into their architecture for moving data around, with a little MongoDB, and Docker thrown in. They use Kubernetes for scheduling to help with deployment but will be switching to Mesos in the near future. In addition, he said, their internal platform has access to APIs for all the different languages.
The Future Near and Far
The near future is about testing. There is still a lot of testing on the algorithms, said Gochee, They want to be absolutely sure they are working correctly before Seymour goes live.
There’s no specific info as to the release date at this time, but the beta will launch soon. You can sign up to join the beta testers now.
In the far future, Seymour is posed to scale comfortably. There will be horizontal growth across the New Relic stack as they add machines to processing tiers as the number of customers grow, Gochee said. The number of algorithms will also grow with time as customers decide what they want Seymour to do.
Initially, New Relic will be coding Seymour’s algorithms in-house, he explained. But there is talk of making them extensible in the future.
Oh, and don’t get attach to the name – the Insights product was originally called something else and Atchison thinks it’s possible the name will be changed “when marketing gets a hold of it. Cirne named Seymour “So you can see more clearly,” but confided that he was thinking about the “feed me” plant that wanted more and more feeds, and wanted Seymour to have a personality that should behave like a New Relic user would behave if they dredged through data all day.
When asked what he would tell my readers about Seymour, Gochee said, “It’s freaking cool.” And if Seymour can perform as described, he may be right.
InApps Technology is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.
New Relic is a sponsor of InApps Technology. TNS owner Insight Partners is an investor in New Relic.
