Surveys are great! Surveys are the worst! I have a love-hate relationship with surveys. InApps has parsed many reports about the use of containers and Docker. They provide tremendous insight but for every question they answer, more questions arise. What does it mean to “use” containers? Is Docker actually the dominant container type in use? How are people actually orchestrating the use of containers?
These are questions that InApps will continue to answer in its eBook series on Docker and the Container Ecosystem. In the meantime, this column will help you interpret the numbers you have been reading in the news.
Use of Containers
A year ago Docker was all the rage with a StackEngine (Oracle) survey reporting that 70 percent of IT pros were using or considering using containers. Yet, beyond the hype two-thirds of that number (49 percent) were in the evaluation phase and actual users had a limited deployment. After a year of information gathering and kicking tires, we know more about where containers are in their technology adoption lifecycle.
A recent 451 Research survey (view this SlideShare to see the actual chart) indicates that from the first half to the second half of 2015 cloud computing users in the “discovery and evaluation” phase dropped from 56 percent to 32 percent.
The good news is that almost half of the evaluators have moved on to pilot projects, test/dev or production uses. The bad news is that slightly more than half of the evaluators (13 percent of the sample) have dropped containers from their roadmaps so that only 64 percent are now using or planning to use containers. This drop-off in tangential interest is to be expected and commonly occurs due to the gap between marketing hype and reality.
I used to work with 451 and trust its data team, but at first glance, the figures don’t match up with other published surveys. For example, a recent survey from Robin Systems showed 95 percent of respondents were using, planning or evaluating the use containers and a six-month-old study from Ruxit (Dynatrace) and O’Reilly also reports a figure over 90 percent.
That’s a thirty-point swing between 451 and the other studies. Just looking at non-test/development production use of containers, 451 reports a 14 percent level, Ruxit 26 percent and Robin Systems 36 percent. Again, a twenty-plus percentage difference between the high and low figure. Which study is correct and how should the findings be interpreted? To answer these questions, we compared the findings of all the aforementioned surveys. The reason for the differences in the studies’ findings relates to semantics and samples.
Comparing Roadmap Findings of Recent Container Surveys | |||
|---|---|---|---|
| Sponsor | Details | Data | % Using in Production |
| Robin Systems | December 2015. 200 IT pros were asked, “What is your container adoption status?” | Using in Production (36%), Playing With It (26%), Plan to Adopt (18%), Investigating (15%) | 36% |
| 451 Research | 451 VotE Cloud, Q3 2015; n=960 cloud users | Initial or Broad Production Use (14%) Test and Development Environment (8%), Running Trials/Pilot Projects (10%), Discovery and Evaluation (32%) | 14% |
| Ruxit and O’Reilly Media | May 2015. 130 IT pros were asked, “At this time, in which of the following environments do you use containers?” | Using in Production (26%), Using, but Not in Production Environment (38%), Plans to Use in the Near Future (28%) | 26% |
| StackEngine | January 2015. 765 respondents were asked, “What is your familiarity with Docker and container technology?” | Using (21%), Evaluating (49%) | N/A |
Each questionnaire defined use and consideration of containers in remarkably different ways. After initially asking about their general use or plans for containers, most of the studies tried to elucidate more information. Asking if containers were in “production” use was a popular approach. In other words, are real business applications being run on top of them, and if so how much of the IT infrastructure is using them?
Another way to find this information is to ask about what type of environment is it being used in (e.g., test, development, production). For example, StackEngine and Ruxit allowed users to provide multiple responses when asking if containers were being used for test, development or production.
The problem with this approach is that many survey takers say they’re using containers in multiple environments. While this is likely true in many cases, it is also likely the distinction between QA, test and development are not always clear. For instance, the 451 survey combines test and development environments as one category and it possible that some respondents were testing container solutions as opposed to using them in one step of the software development lifecycle.
Another point of distinction is that while 451 asks about trials and pilots, Robin Systems asks about plans and if someone is “playing” with containers. Overall, we recommendation for future surveys is to better distinguish between people that are testing the use of containers in order to put them in production as opposed to those that are regularly using containers as part of their continuous integration process.
Besides questionnaire wording, who gets asked the questions can impact the results. The 451 study is based on a panel of IT decision-makers while the Robin Systems survey responses were solicited from a network of online publications. The Ruxit and StackEngine surveys were also based on self-selected respondents. This type of survey methodology may result in a bias towards early adopters of containers and is susceptible to a third-party campaign to fill the survey’s “ballots” so to speak. However, by looking just at the respondents that are currently using containers in production, we can still learn a lot.
Feature image via Pixabay
InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.
