3 challenges for using Docker
Discover how containers are being used and which technologies and infrastructures are being adopted.
Port harbour cranes (source: Pixabay)
Containers are one of the hottest topics in IT right now, but how are they actually being used? A survey conducted by O’Reilly Media in collaboration with Ruxit throughout May 2015 invited individuals from the O’Reilly community to share how their organizations currently use or plan to use containers. The results reveal that container technologies—and Docker in particular—are rapidly being adopted to make application deployment faster, easier, and more flexible.
The survey, which ran in the lead up to DockerCon 2015, aimed to discover how containers are being used and which container technologies and infrastructures are being adopted, as well as motivations and challenges associated with opting to use containers.
Learn faster. Dig deeper. See farther.
Containers, sure, but…
Forty percent of survey respondents are currently using containers in production compared to 86% for development. Given that Docker containers are designed to “build once, run anywhere,” why aren’t containers being adopted more widely in production? Challenges to adopting containers identified by respondents include technology maturity, orchestration, monitoring, automation, and environment size, as well as other reasons, including the difficulty of convincing clients, management, or the development team of the benefits of adopting containers.
Technology maturity a barrier to adoption
The biggest barrier to moving to a containerized infrastructure, identified by over 56% of respondents, is the maturity of the technology. It’s difficult to keep track of all of the container-related tools and projects released over the past two years, and with some of them changing names and many key tools still in beta, this rapid rate of development has not yet painted a picture of stability or maturity.
However, the survey was closed before the announcement of the formation of the OCI in June, and before the launch of the Cloud Native Computing Foundation (CNCF) in July, which aims to harmonize existing technologies to foster development of container-packaged, dynamically managed, microservices based applications. It’s too early to say what impact these initiatives will have on container technology in the short term: there may be yet another period of flux as the existing technologies are harmonized.
However, being backed by industry leaders with existing tools in this space, these initiatives offer an auspice of stability, and will provide an entry point for those seeking a set of stable, integrated solutions to get them started with containers.
Orchestration: A key challenge in large deployments
Half of the respondents indicated that orchestration is a challenge when adopting containers. Respondents who are already using Docker were more likely to provide this response. Orchestration involves coordinating and connecting containers within multi-container, multi-host applications, and is particularly important for applications with a microservice architecture. For example, a web application might consist of a cluster of containers running web servers to host multiple instances of the frontend (for failover and load balancing) as well as a number backend services each running in separate containers.
Orchestration tools that explicitly support containers include Kubernetes from Google and Mesosphere’s Marathon, built on Apache Mesos. Docker’s own orchestration tools—Swarm, Compose, and Machine—were released in February, with Compose replacing the earlier Fig tool. When respondents were asked which container-related technologies they were using, 38% indicated that they use or intend to use Docker swarm, while 22% are using or plan to use Kubernetes and 22% have opted for Mesos. Respondents who are using Kubernetes tended to be those from larger organizations. Docker Swarm and Machine are both still in beta, while Version 1.0 of Kubernetes was released on July 21st at the same time that the CNCF was launched, so it will be interesting to compare the rate of adoption for these tools after they have had a chance to mature.
Monitoring scalability a limiting factor
As containers are being used more widely in production, the demand for tools for monitoring them is also increasing. Forty-six percent of respondents identified monitoring as a key challenge to moving toward containerized applications. Traditional Linux-based monitoring and reporting tools that are designed to run on a single host and that rely on analyzing log files on disk (e.g., under /var/log) don’t scale well to multi-container clustered applications and aren’t well suited even to monitoring single-container apps because unless they are written to a data volume, disk contents are not persisted when containers are shut down. A centralized approach to logging and monitoring clusters of applications using container-aware monitoring tools or services is a better fit for managing dynamic multi-container applications.
Self-hosted solutions for monitoring container health include the open source tools cAdvisor, Sensu, and Prometheus.
Docker has supported a Stats API since version 1.5 for streaming resource usage stats from running containers, and a number of vendors provide hosted monitoring solutions built on top of the Docker Stats API, including New Relic, Scout, DataDog, SignalFX, and AppDynamics. These tools monitor the state of the containers themselves, but they rely on data reported through the Stats API and hence don’t provide much application-specific detail about the applications or services running inside of the containers. Ruxit’s Docker Monitoring supports auto-discovery of new containers, and discovery of the services and applications running within them, allowing app-centric monitoring of dynamic distributed applications and microservices running within Docker containers.
Automation
Automation is another challenge identified by 40% of respondents, and is particularly useful for implementing distributed applications based on immutable microservices, i.e., where a service will be replaced with a new container rather than being upgraded in place. Traditional continuous deployment tools like Jenkins (in combination with a CM tool like Ansible for the deployment side) and container-enabled services like CircleCI support pulling Docker images directly from DockerHub and building and deploying containers directly as part of a CI/CD process. Service discovery tools, including etcd and Consul, are an important part of this story, to enable services to dynamically discover other services that they need to communicate with, rather than having to link them advance.
Summary
The high rate of adoption of container technologies by the survey respondents reflects the current industry buzz around containers and Docker. With more than half of the respondents planning to deploy containers in production within the next 6–12 months, there is a clear need for production-ready, stable, integrated solutions that address the key challenge areas identified by the survey: orchestration, monitoring, and automation for distributed containerized applications.
Want to learn how and why people are using Docker? Read the rest of the survey.