artemegorov -

The evolution of containers: Docker, Kubernetes and the future

Container technology is almost as old as VMs, although IT wasn't talking about the topic until Docker, Kubernetes and other tech made waves that caused a frenzy of activity.

The rapid evolution of containers over the past two decades has changed the dynamic of modern IT infrastructure -- and it began before Docker's debut in 2013.

VM partitioning -- which dates to the 1960s -- enables multiple users to access a computer concurrently with full resources via a singular application each. The following decades were marked by widespread VM use and development. The modern VM serves a variety of purposes, such as installing multiple OSes on one machine to enable it to host multiple applications with specific, unique OS requirements that differ from each other.

Then there was chroot

The evolution of containers leaped forward with the development of chroot in 1979, in version 7 of Unix. Chroot marked the beginning of container-style process isolation by restricting an application's file access to a specific directory -- the root -- and its children. A key benefit of chroot separation was improved system security, such that an isolated environment could not compromise external systems if an internal vulnerability was exploited.

The 2000s were alight with container technology development and refinement. Google introduced Borg, the organization's container cluster management system, in 2003. It relied on the isolation mechanisms that Linux already had in place. In those early days in the evolution of containers, security wasn't much of a concern. Anyone could see what was going on inside the machine, which enabled a system of accounting for who was using the most memory and how to make the system perform better.

Nevertheless, this kind of container technology could only go so far. This led to the development of process containers, which became control groups (cgroups) as early as 2004. Cgroups noted the relationships between processes and reined in user access to specific activities and memory volumes. The cgroup concept was absorbed into the Linux kernel in January 2008, after which the Linux container technology LXC emerged. Namespaces developed shortly thereafter to provide the basis for container network security -- to hide a user's or group's activity from others.

evolution of containers

Docker and Kubernetes

Docker floated onto the scene in 2013 with an easy-to-use GUI, and the ability to package, provision and run container technology. Because Docker enabled multiple applications with different OS requirements to run on the same OS kernel in containers, IT admins and organizations saw opportunity for simplification and resource savings. Within a month of its first test release, Docker was the playground of 10,000 developers. By the time Docker 1.0 was released in 2014, the software had been downloaded 2.75 million times. And within a year after that, more than 100 million times.

Unlike VMs, containers have a significantly smaller resource footprint, are faster to spin up and down, and require less overhead to manage. VMs must also each encapsulate a fully independent OS and other resources, while containers share the same OS kernel and use a proxy system to connect to the resources they need, depending upon where those resources are located.

Concern and hesitation arose in the IT community regarding the security of a shared OS kernel. A vulnerable container could result in a vulnerable ecosystem without the right precautions baked into the container technology. Additional complaints early in the modern evolution of containers bemoaned the lack of data persistence, which is important to the vast majority of enterprise applications. Efficient networking also posed problems, as well as the logistics of regulatory compliance and distributed application management.

Container technology ramped up in 2017. Companies such as Pivotal, Rancher, AWS and even Docker changed gears to support the open source Kubernetes container scheduler and orchestration tool, cementing its position as the default container orchestration technology. In April 2017, Microsoft enabled organizations to run Linux containers on Windows Server. This was a major development for Microsoft shops that wanted to containerize applications and stay compatible with their existing systems.

Container vendors have, over time, addressed security and management issues with tool updates, additions, acquisitions and partnerships, although that doesn't mean containers are perfect in 2020.

Cloud container management, accompanied by the necessary monitoring, logging and alert technology, is an active space for container-adopting organizations. Containers offer more benefits for distributed applications, particularly microservices, than for larger, monolithic ones. Each independent service can be fully contained and scaled independently from others with the help of an orchestrator tool, such as Kubernetes, which reduces resource overhead on applications with features that aren't as heavily used as others.

To that end, various public and private cloud providers offer managed container services -- usually via Kubernetes -- to make container deployment in the cloud more streamlined, scalable and accessible by administrators. AI and machine learning technologies are attracting similar interest and participation among enterprises both on and off the cloud for improved metrics and data analysis, as well as error prediction, automated alerts and incident resolution.

The future of containers

And the evolution of container technology has not come to an end. Mirantis, a cloud computing services company, purchased Docker Enterprise in 2019; Kubernetes has cemented its position as the de facto container orchestration standard; and other major vendors regularly acquire smaller startups to bolster their toolchain offerings and fill in gaps with specialized tools.

A Diamanti survey of more than 500 IT organizations revealed that security is still users' top challenge with the technology, followed by infrastructure integration. And Sysdig noted a 100% increase in container density from 2018 to 2019 in its container usage survey.

No matter how container technology evolves, we'll see more of it. In 2018, analyst firm Gartner predicted enterprise container use will increase to as high as 50% adoption in 2020 -- more than double the 20% recorded in 2017. And Gartner predicts that more than 75% of organizations will run containers in production by 2022.

Dig Deeper on Containers and virtualization

Software Quality
App Architecture
Cloud Computing
Data Center