Creating a Culture of Experimentation – InApps 2022

When it comes to product strategy, no one is more accountable for creating clear differentiation and competitive advantage than product managers. Despite having clear lines of focus, it’s challenging to avoid ascribing the overarching success to the aptitude and abilities of the product manager in the driving seat.

They take the hit for the pricing strategy, the promotion and the placement. And that’s all before we get to discussing the product itself, sitting at the heart of this execution. Working cross-functionally with engineering, design, marketing and revenue, PMs are more on the hook than anyone. It’s easy to understand how this immense pressure can translate through to inaction.

Despite iterative development being one of the most integral components to an Agile team, PMs often find themselves painted into a corner and unable to put some of their greatest gambles into the field. Andrew Mason, founder of Groupon, put it best in his farewell letter to employees when he remarked that his “biggest regrets are the moments that I let a lack of data override my intuition on what’s best for our customers”.

Stress can cause a special kind of paralysis, preventing those under pressure from acting on their innate intuition and instinct.

Taking an incremental approach to product development is one of the most proven and dependable routes to crafting a product that elicits genuine joy from its users. Historically, experimentation leads to wins. It’s been reported that Booking.com runs around 1,000 concurrent tests to iterate its way toward success. After all, trial cannot exist without error, and growth requires conflict.

While introducing room for failure can feel daunting, these guiding thoughts will empower you to foster an environment of healthy risk-taking that will ultimately change your process for the better. The following four fundamental practices are great steps that can help your team instill a culture of experimentation that uses failure to power its future successes.

Chaos Engineering

One of the best ways to change the narrative around failure is to introduce it as a conscious constant at play. You may be familiar with the concept of a “Chaos Monkey,” but for those who aren’t, let’s go over a brief recap.

Back in 2010, when Netflix was undergoing the enormous task of migrating its systems to the cloud, it needed a way to be able to stress-test the myriad vulnerabilities it would be introducing during this process. The result was Chaos Monkey, a tool designed to disable a selection of services and instances across production at random. The regular interference of a known threat can train teams to unite against a problem, rather than letting it divide and defeat them.

The impact has spawned the entire discipline of chaos engineering, a school of thought seeing engineers practice their incident-response skills in three key steps: Plan your experiment, draw your disaster zone and execute your field tests. Giving your organization a chance to earn its stripes in the realm of incidence recovery, of course, has tangible benefits from a technical perspective, most notably its ability to help you build resilient and robust services. But more than producing product-based benefits, this breed of exercise can train your teams to develop their vocabulary around failure. Discuss the outcomes of your chaos-based experiments with direct and explicit language, while ensuring that everyone involved acknowledges the process as a learning experience. With outages already existing as a present KPI for many engineering teams, it makes sense to give team members the opportunity to flex the muscle that you intend them to use.

Blue-Green Deployment

Blue-green deployment has risen in popularity, in part due to the exponential growth in the number of deployments being made by engineers. Its essential aim is to enable continuous delivery with minimal downtime, providing a safety net for testing and experimentation in a production environment. The process involves creating two nearly identical instances of your application behind a load balancer. At any one time, one of these versions is responding to traffic, while the other provides a perfect breeding ground for your tests and experimentations.

With one of the core deterrents to experimentation being the technical restraints to trialing your ideas in a realistic yet risk-free environment, blue-green can eliminate these issues. While it does pose some logistical challenges, namely its tendency to complicate your overarching infrastructure and introduce additional costs because you’re doubling your microservice usage, its benefits can significantly outweigh the overall costs. Rather than worrying about how your new idea would potentially scale, you can put your work into practice and bring your vision to life, all without sacrificing uptime.

Canary Deployment

Now, if blue-green isn’t for you, luckily there are a few different deployment methods that lend themselves particularly well to experimentation. If the idea of replicating your entire production environment for testing and experimentation sounds too costly, both in terms of time and expenditure, then canary deployment might be for you. Canary deployment involves testing your changes and trialing your releases on a relatively small group of users to get a sense of how they perform in the wild.

This tactic has the intangible benefit of rewarding you with real-world data. You’ll see precisely how your experimentations perform in real-time, with actual results data that you can marry to your organization’s current performance metrics. Rather than working on assumptions, you’ll be able to convert your deployment strategy from being a didactic roadmap to an interactive conversation with your users, giving you the ability to react to their feedback and tweak your methods accordingly.

Review your Recovery

In certain instances, things go wrong. Rather than fixating on the problem, cultivating a culture of experimentation requires you to accept the inevitability of mistakes, even in their most costly and public form. Earlier this year, HBO Max surprised its 44 million user base with an integration test email that went to everyone. That’s right, everyone.

Almost immediately after the event occurred, engineers everywhere responded with compassion. Rather than chastising the engineer behind this mistake, the hashtag #HugOps picked up global traction as a signal to the person who made the misstep. The conversation around HBO’s error evolved constructively into a widespread confessional, giving space for the most seasoned technical leaders to speak out about the times they’d accidentally taken down production.

Dear intern:

We’ve all screwed something up,
So welcome to the club.

“That Sinking Feeling (The #HugOps Song)” pic.twitter.com/KmwgSfx33h

— Forrest Brazeal (@forrestbrazeal) July 7, 2021

In engineering, snafus happen to the best of us. Regardless of skill level, years of experience or even approaching a task with caution, in some cases the old adage applies: If something can go wrong, it will. Instead of chastising those at fault, take the initiative. Create a space for people to share their mistakes in a safe, judgment-free zone.

By enabling the organization as a whole to share its missteps, you enable others to learn from what’s gone wrong. This space can take many forms. It can be a Slack channel, a regular talk track or even just a live Google Document. Irrespective of its structure, make sure that the people in your team have a voice and that they’re not punished for using it. Stifling honest communication doesn’t wipe out outages and problems, it just means that people don’t talk about them.

Experimentation is vital. By introducing failure-based feedback into your cross-functional working, you’ll go from treating criticism as a taboo to working with it as a conversation starter. If you’ve got stories of how experimentation has improved your business and helped your team to thrive, then we’d love to hear from you. Our third annual  Trajectory conference takes place Nov. 9-10, and the call for papers is open. Check out the key themes for this year’s event and to submit your abstract.

InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: LaunchDarkly.

Lead image via Pixabay.