What is DevOps?

Are you trying to get a better handle on what is DevOps? Here are the eight foundational concepts you have to know!

DevOps has become an overloaded buzzword that means a lot of different things to a lot of people. That's a challenge when you are trying to understand what DevOps is or define DevOps. Instead of trying to define DevOps, we are going to describe the foundational concepts that different people associate with DevOps and the history of how the DevOps movement evolved to help you get a holistic view:

1. Where Did DevOps Come From?
DevOps is the offspring of agile software development – born from the need to keep up with the increased software velocity and throughput agile methods have achieved. Advancements in agile culture and methods over the last decade exposed the need for a more holistic approach to the end-to-end software delivery lifecycle.

What is Agile Software Development?
Agile Development is an umbrella term for several iterative and incremental software development methodologies. The most popular agile methodologies include Scrum, Kanban, Scaled Agile Framework® (SAFe®), Lean Development and Extreme Programming (XP).

While each of the agile methodologies is unique in its specific approach, they all share a common vision and core values (see the Agile Manifesto). They all fundamentally incorporate iteration and the continuous feedback that it provides to successively refine and deliver a software system. They all involve continuous planning, continuous testing, continuous integration, and other forms of continuous evolution of both the project and the software. They are all lightweight, especially compared to traditional waterfall-style processes, and inherently adaptable. And what is most important about agile methods is that they all focus on empowering people to collaborate and make decisions together quickly and effectively.

Get the best knowledge on Agile Development through DevOps Online Training

In the beginning, agile teams were primarily made up of developers. As these agile teams became more effective and efficient at producing software, it became clear that having Quality Assurance (QA) and Dev as separate teams was inefficient. Agile grew to encompass QA in order to increase the velocity of delivering software and now agile is once again growing to encompass the delivery and support members to extend agility from ideation to delivery.

The DevOps ideals extend agile development practices by further streamlining the movement of software change thru the build, validate, and deploy and delivery stages, while empowering cross-functional teams with full ownership of software applications – from design thru production support.

DevOps is an IT mindset that encourages communication, collaboration, integration and automation among software developers and IT operations in order to improve the speed and quality of delivering software.

DevOps teams focus on standardizing development environments and automating delivery processes to improve delivery predictability, efficiency, security and maintainability. The DevOps ideals provide developers more control of the production environment and a better understanding of the production infrastructure. DevOps encourages empowering teams with the autonomy to build, validate, deliver and support their own applications. With DevOps, nothing gets “thrown over the wall.” 

2. What Are the Challenges DevOps Solves?
Prior to DevOps application development, teams were in charge of gathering business requirements for a software program and writing code. Then a separate QA team tests the program in an isolated development environment, if requirements were met, and releases the code for operations to deploy. The deployment teams are further fragmented into siloed groups like networking and database. Each time a software program is “thrown over the wall” to an independent team it adds bottlenecks. The problem with this paradigm is that when the teams work separately:
Dev is often unaware of QA and Ops roadblocks that prevent the program from working as anticipated.
QA and Ops are typically working across many features and have little context of the business purpose and value of the software.
Each group has opposing goals that can lead to inefficiency and finger pointing when something goes wrong.

DevOps addresses these challenges by establishing collaborative cross-functional teams that share responsibility for maintaining the system that runs the software and preparing the software to run on that system with increased quality feedback and automation issues.

A Common Pre-DevOps Scenario
The Dev team that has a goal to ship as many features as possible, kicks a new release “over the wall” to QA. Then the tester’s goal is to find as many bugs as possible. When the testers bring their findings to Dev, the developers become defensive and blame the testers that are testing the environment for the bugs. The testers respond that it isn’t their testing environment, but the developer’s code that is the problem.

Eventually the issues get worked out and QA kicks the debugged new release “over the wall” to Ops. The Ops team’s goal is to limit changes to their system, but they apprehensively release the code and the system crashes. The finger pointing resumes.

Ops says that Dev provided them faulty artifacts. Dev says everything worked fine in the test environment. The fire drills begin to debug the system and get production stable. The production environment isn’t Dev’s and QA’s responsibility, so they keep hands off while Ops spends all night fixing the production issues. 

3. What Is the Goal of DevOps?
Improve collaboration between all stakeholders from planning through delivery and automation of the delivery process in order to:
Improve deployment frequency
Achieve faster time to market
Lower failure rate of new releases
Shorten lead time between fixes
Improve mean time to recovery

Get a practical explanation of these goals at DevOps training

According to the 2015 State of DevOps Report, “high-performing IT organizations deploy 30x more frequently with 200x shorter lead times; they have 60x fewer failures and recover 168x faster.”

A Common Pre-DevOps Scenario The software team meets prior to starting a new software project. The team includes developers, testers, operations and support professionals. This team plans how to create working software that is ready for deployment.

Each day new code is deployed as the developers complete it. Automated testing ensures the code is ready to be deployed. After the code passes all the automated testing it is deployed to a small number of users. The new code is monitored for a short period to ensure there are no unforeseen problems and it is stable. The new code is then proliferated to the remaining users once the monitoring shows that it is stable. Many, if not all, of the steps after planning and development are done with no human intervention. 

4.Where Are You on the DevOps Continuum?
The DevOps continuum is a helpful way to look at the different aspects of DevOps. The bottom horizontal axis represents what people perceive DevOps to fundamentally be focused on. Some people adamantly feel that DevOps should focus on culture more than tools, while on the other people tend to value tools over culture.



The vertical axis depicts the three levels of the DevOps delivery chain: continuous integration, continuous delivery and continuous deployment. The DevOps community refers to organizations in the top right of the DevOps continuum as pink unicorns because there are currently so few of them that you don’t see them in the wild very often. Popular examples of these unicorns are companies like Netflix, Etsy, Amazon, Pinterest, Flicker, IMVU and Google. In a recent poll participants indicated where their organizations fit on the DevOps continuum:
55% Bottom Left
26% Bottom Right
14% Top Left
5% Top Right

Thought leaders, coaches and bloggers often portray a vision of DevOps in the upper-right corner and they will often have a strong bias towards either DevOps culture or automation tools. While it is ok to have esoteric debates about whether DevOps culture or tools are more important, the reality is that you can’t have DevOps without tools and all the tools in the world won’t help if you don’t have a strong supporting culture.

Another important point is that moving up and to the right takes time and many organizations’ first step will be a blend of culture, tools and continuous integration, so don’t be discouraged when you read an article about how if you're “not doing DevOps” unless you're deploying all the way into production without any human intervention.

DevOps can be a blend of culture, tools and maturity that make sense for your organization and what makes sense will most likely evolve over time. The important thing is to continually strive to break down the walls and bottlenecks between the phases of software delivery by improving collaboration and automation. In the following sections we dive deeper into each aspect of the DevOps continuum to help you better understand where you fit.

Get more practical knowledge on DevOps at DevOps Online Course 

5. What Are the Phases of DevOps Maturity?
There are several phases to DevOps maturity; here are a few of the key phases you need to know.

Waterfall Development
Before continuous integration, development teams would write a bunch of code for three to four months. Then those teams would merge their code in order to release it. The different versions of code would be so different and have so many changes that the actual integration step could take months. This process was very unproductive.

Continuous Integration
Continuous integration is the practice of quickly integrating newly developed code with the main body of code that is to be released. Continuous integration saves a lot of time when the team is ready to release the code.

DevOps didn’t come up with this term. Continuous integration is an agile engineering practice originating from the Extreme Programming methodology. The terms been around for a while, but DevOps has adopted this term because automation is required to successfully execute continuous integration. Continuous integration is often the first step down the path toward DevOps maturity.

The continuous integration process from a DevOps perspective involves checking your code in, compiling it into usable (often binary executable) code and running some basic validation testing.

Continuous Delivery
Continuous delivery is an extension of continuous integration [DevOps stage 2]. It sits on top of continuous integration. When executing continuous delivery, you add additional automation and testing so that you don’t just merge the code with the main code line frequently, but you get the code nearly ready to deploy with almost no human intervention. It’s the practice of having the code base continuously in a ready-to-deploy state.

Continuous Deployment
Continuous deployment, not to be confused with continuous delivery [DevOps nirvana], is the most advanced evolution of continuous delivery. It’s the practice of deploying all the way into production without any human intervention.

Teams that utilize continuous delivery don’t deploy untested code; instead, newly created code runs through automated testing before it gets pushed out to production. The code release typically only goes to a small percentage of users and there’s an automated feedback loop that monitors quality and usage before the code is propagated further.

There are a very small number of companies that are truly practicing continuous deployment. Netflix, Etsy, Amazon, Pinterest, Flicker, IMVU and Google are popular examples of companies doing continuous deployment.

While DevOps nirvana is often not the end goal for most enterprises, they often focus on moving towards continuous delivery. 

6. What Are the Values of DevOps?
DevOps focuses heavily on establishing a collaborative culture and improving efficiency through automation with DevOps tools. While some organizations and people tend to value one more than the other, the reality is it takes a combination of both culture and tools to be successful. Here’s what you need to know about these two DevOps values.

DevOps Culture
DevOps culture is characterized by increased collaboration, decreasing silos, shared responsibility, autonomous teams, improving quality, valuing feedback and increasing automation. Many of the DevOps values are agile values as DevOps is an extension of agile.

Agile methods are a more holistic way of delivering software. Agile development teams measure progress in terms of working software. Product owners, developers, testers and UX people work closely together with the same goals.

DevOps is just adding the operations’ mindset and maybe a team member with some of those responsibilities into the agile team. Whereas before DevOps progress is measured in terms of working software, with DevOps progress is measured in terms of working software in the customer’s hands.

To achieve this, Dev and Ops must break down the silos and collaborate with one another, share responsibility for maintaining the system that runs the software, and prepare the software to run on the system with increased quality feedback and delivery automation.

DevOps Tools
DevOps tools consist of configuration management, test and build systems, application deployment, version control and monitoring tools. Continuous integration, continuous delivery and continuous deployment require different tools. While all three practices can use the same tools,

Earlier we briefly discussed some of the tools used in DevOps; here are some of the key tools and practices you need to know.

Source Code Repository
A source code repository is a place where developers check in and change code. The source code repository manages the various versions of code that are checked in, so developers don’t write over each other’s work.

Source control has probably been around for forty years, but it’s a major component of continuous integration. Popular source code repository tools are Git, Subversion, Cloudforce, Bitbucket and TFS.

Build Server
The build server is an automation tool that compiles the code in the source code repository into executable code base. Popular tools are Jenkins, SonarQube and Artifactory.

Configuration Management

Configuration management defines the configuration of a server or an environment. Popular configuration management tools are Puppet and Chef.

Virtual Infrastructure
Amazon Web Services and Microsoft Azure are examples of virtual infrastructures. Virtual infrastructures are provided by cloud vendors that sell infrastructure or platform as a service (PaaS). These infrastructures have APIs to allow you to programmatically create new machines with configuration management tools such as Puppet and Chef.

There are also private clouds. For example, VMware has vCloud. Private virtual infrastructures enable you to run a cloud on top of the hardware in your data center.

Virtual infrastructures combined with automation tools to empower organizations practicing DevOps with the ability to configure a server without any fingers on the keyboard. If you want to test your brand-new code, you can automatically send it to your cloud infrastructure, build the environment and then run all of the tests without human intervention.

Test Automation
Test automation has been around for a long time. DevOps testing focuses on automated testing within your build pipeline to ensure that by the time that you have a deployable build, you are confident it is ready to be deployed. You can’t get to the point of continuous delivery where you’re fairly confident without any human intervention that your code is deployable without an extensive automated testing strategy. Popular tools are Selenium and Water.

Pipeline Orchestration
A pipeline is like a manufacturing assembly line that happens from the time a developer says, “I think I’m done,” all the way to the time that the code gets deployed in the production or a late-stage pre-production environment.

Unifying Enterprise Software Development and Delivery
VersionOne VS unifies agile application lifecycle management and DevOps, providing a full picture of your entire software delivery pipeline in a single platform. VersionOne® Continuum™ for DevOps is an enterprise continuous delivery solution for automating, orchestrating, and visualizing the flow of change throughout the software delivery cycle. 

8. What Is the History of DevOps?
2007

Patrick Debois, a software development consultant, had a goal of learning all aspects of IT. Over fifteen years, Patrick had taken on many different roles in IT in order to work in every role of an IT organization and gain a holistic understanding of IT. He worked as a developer, network specialist, system administrator, tester and project manager.

Patrick had taken a consulting job for a large datacenter migration. He was in charge of the testing, which meant he was spending a lot of time with Dev and Ops. Patrick had always been bothered by the differences between how Dev and Ops worked, but he became particularly frustrated with the challenges of managing work across the two groups on this datacenter migration.

Continuous integration was gaining popularity in the agile community and was moving Dev closer to deployment, but there was still nothing out there that fully crossed the divide of Dev and Ops. Patrick was sure there had to be a better way for these two teams to work together.

2008

Andrew Shafer posted an idea for an agile infrastructure “birds of a feather” session at the Agile 2008 Conference. Patrick Debois saw the post and went to the session. Unfortunately, he was the only one who showed up. The idea was so poorly received that Andrew didn’t even show up to his own discussion.

Fortunately, Patrick was so excited to see that someone else was interested in solving the challenges of Dev and Ops working together that he tracked down Andrew and they decided to start a Google group named Agile System Administration.

2009

John Allspaw, senior vice president of technical operations at Flickr, and Paul Hammond, director of engineering at Flickr, gave a presentation at the O’Reilly Velocity Conference in San Jose, “10+ Deploys per Day: Dev and Ops Cooperation at Flickr.” The presentation laid the groundwork for how Dev and Ops can effectively work together to improve software deployment.

Patrick Debois watched the presentation in Belgium via a live stream and was inspired to start his own conference, DevOpsDays, in Ghent, Belgium. The conference brought together an energetic group of forward-thinking minds trying to improve software deployment. What may be even more important is that this group of people kept the conversation going on Twitter with the hashtag #DevOpsDays. In an effort to save Twitter character space, people soon dropped days and the hashtag became #DevOps.

2010

The following year, DevOpsDays were held in Australia and the U.S. Over time, there were more and more DevOpsDays that were hosted in different countries and cities around the world. The face-to-face meetings ignited more and more people to get energized about DevOps until it had become a full-on grassroots movement.

2011

Up until 2011, the DevOps movement has been fueled by individuals and open source tools with little attention from analysts or vendors. But in 2011, the movement began to go mainstream, catching the attention of analysts like Cameron Haight from Gartner and Jay Lyman of 451 Research. Big vendors started to market DevOps.

2012

By 2012 DevOps was quickly turning into a buzzword and DevOpsDays continued to grow.

2013

The public thirst for DevOps information inspired several authors to write books on the topic. Notable examples are The Phoenix Project by Gene Kim, Kevin Behr and George Spafford and Implementing Lean Software Development by Mary and Tom Poppendiek.

2014 Large companies such as Target, Nordstrom, and LEGO became some of the first companies to bring DevOps into the enterprise.