Introducing DevOps Ideas

2022-04-21

What is DevOps?

Wikipedia provides the following definition ^[1]:

DevOps is a set of practices that combines software development Dev and IT operations Ops.

It aims to shorten the system development life-cycle and provide continuous delivery with high software quality. DevOps is complementary with agile software development. Several DevOps aspects came from the agile methodology.

— Wikipedia

DevOps is intended to be a cross-functional mode of working [1, 2, 3]. Practitioners use different toolchains rather than a single one. These toolchains are expected to fit into one or more of the following categories, reflective of key aspects of the development and delivery process.

The major software product development platforms GitLab, GitHub or Bitbucket are classical examples of toolchains integration.

DevOps teams practice daily:

Coding	Code development and review, source code management tools, code merging [4, 5, 6]. Emphasize continuous development and testing. Continuous testing can only be fully automated.
Building	Continuous integration tools, build status. Emphasize continuous integration, continuous delivery, and continuous deployment [7, 8].
Testing	Continuous testing tools that provide quick and timely feedback on business risks. Supports continuous development activities.
Packaging	Artifact repository, application pre-deployment staging. Supports the continuous delivery activities in a corporate environment.
Releasing	Change management, release approvals, release automation. Supports the continuous delivery and deployment to productive servers.
Configuring	Infrastructure configuration and management, infrastructure as code tools. Supports the infrastructure as a code approach.
Monitoring	Applications performance monitoring, end-user experience. Emphasize continuous monitoring and alarming, optimize time to recover metric.

Development

Development in an agile and DevOps environment means:

Continuous integration, where the coding, building, integration, and testing processes are carried out.
Continuous delivery, which includes continuous integration but mainly focuses on product delivery.
Continuous deployment, which aims at automating project deliveries.

Your development teams shall have automatic continuous integration. This pipeline builds the software, runs static code analyzer, and executes a reasonable set of unit and integration tests. If not, they are pretending to do DevOps and lying to you.

The next step is to configure an automatic continuous delivery. This pipeline packages the complete solution and delivers it as an installation package. This package can be deployed in the staging area without manual configuration.

Only professional DevOps teams shall dare to implement continuous delivery.

The areas that a DevOps engineer must be competent are:

Version Control Systems: Any DevOps engineer has expertise in Git, and platforms such as GitHub, GitLab, or BitBucket. Infrastructure as Code IaC is used to build and deploy digital products. Documentation is always synchronized with the source artifacts. Approaches such as static websites are preferred.
Continuous Integration, Continuous Deployment, Continuous Delivery (CI/CD/CD): As for this one, you should look for Jenkins, TeamCity, GitHub, GitLab, or Bitbucket. All products are built and packaged using continuous integration and continuous delivery approaches. No manual steps are required to package a release for production. It is important to note, however, that there are so many DevOps tools out there that it is impossible to cover them.
Testing and Automation Tools: One of the duties of a DevOps engineer is to speed up the delivery of software to the clients. Most companies care about the quality of their software and testing has become an essential part of DevOps engineers' job. Functional and non-functional testing techniques and tools are used to guaranty the quality of the installed solutions. It provides the obvious benefits of automation and helps DevOps engineers save a lot of time.
Containers and Cloud Services: Containers such as Docker and Kubernetes are used to deploy digital solutions. Microsoft Azure, Google Cloud, Amazon Web Services are some examples.
Communication: DevOps is not isolated to a specific role or to a specific team in your organization. A DevOps specialist must have well-developed empathy because their job implies communication with other people. Conflicts are not going to help them. A deep understanding of empiricism and transparency of all available information and measurements is a must for successful communication in an agile and DevOps environment.

Google has published exhaustive insights how they implement DevOps mindset [9].

DevSecOps

Security and corporate governance awareness tremendously grew during the beginning of this decade. Systematic cyberattacks and more stringent citizen protection laws displayed lack of competence in a lot of IT departments.

DevOps engineers shall always also be DevSecOps specialists and master:

Collective Responsibility: Security is not something ephemeral whose progress and contribution cannot be measured. Each person in the organization has their own security responsibility and must be aware of their own contribution to the organization’s security stance.
Collaboration and Integration: Security can only be achieved through collaboration, not confrontation.
Pragmatic Implementation: They focus on application development to ensure safety, privacy, and trust in the digital society. Organizations shall be able to pragmatically approach security in DevOps.
Bridge the Divide between Compliance and Development: The key to addressing the gap between compliance and development is to identify applicable controls, translating them to appropriate software measures, and identifying inflection points within the software lifecycle where these controls can be automated and measured.
Automation: Software quality can be better by improving the thoroughness, timeliness, and frequency of testing. Processes that can be automated should be automated, and the other ones should be considered for elimination.
Measurement, Monitoring, Report, and Action: The qualified people must continuously control the results during software development as well as post-delivery at the right time for DevSecOps to succeed.

Bootstrap Your DevOps Adoption

You are already using agile approaches to develop your digital products. You want to start your journey with DevOps. Success requires actions in three areas. First, your development teams must adopt automated pipelines to create a new version of your application and deploy it. Second, you shall monitor your productive servers to infer service level agreement metrics. Indicators are used to improve the application by changing usage patterns. Third, a continuous improvement initiative shall continuously improve quality and effectiveness.

You shall provide at least the following capabilities for all your teams and products:

Development

Focus on automation and repeatability. Measure deployment frequency and deployment duration.

Each product can be checked out locally from a git repository and built with less than five commands.
Each product is built in a continuous delivery pipeline and packaged for deployment in a staging area or in production. Static code analysis and automated tests are executed in the pipeline. They provide an automated, tracked, and documented quality gate.
Each production version is tagged in the repository, including associated documentation and technical instructions.

Running

Focus is on time to repair rather than time between issues. Measure mean-time to recovery, production failure rate and change failure rate.

Each instance has a monitoring feature, which enabled us to provide realtime information about resource usage, heartbeat, health status, and warnings. You can rely on an automated alert and response system to troubleshoot and monitor your enterprise servers. The developing team can quickly rectify errors or hiccups in the workflow without disturbing the operations teams.
Each instance monitor view is accessible to all involved collaborators.
Each deployed product has a set of measurable targets and the associated metrics to track them.
You have a documented process to handle security issues in a timely manner.
You have a documented process to mitigate proactively potential security, performance, and governance issues. A governance goal could be to always update libraries once a security defect is corrected. Empiricism and data are the building block to quality and sustainable improvements.

Continuous Improvement

Take small steps. Measure the number of improvements, average lead time, and improvement impact.

Each product development team has a continuous improvement initiative. The measures for the next releases are documented and progress is tracked.
Each measure is regularly evaluated for adequacy and success. Does the measure improve the overall quality of the product? Do the actions measurably improve the associated metrics?

Checklist

You give access to a product repository. I can check out the whole project and build it locally with one command.+ I shall install the necessary development tools based on written instruction available in the repository.
You give access to a product repository. I can initiate a complete build and delivery action in a continuous integration and delivery pipeline.+ I shall use the CI/CD tool chosen for the product.
I can check out, build, and deploy the two last releases in a staging area with less than ten commands.
You give me access to the list of all issues and changes ongoing in the repository. The list is specific to one product. I see the information such as priority, cycle-time of these issues and in which release the issue was fixed.
You give me read-only access to the health dashboard of any test and productive server on which the product runs. I see:
- Uptime, availability, last down events with time of occurrence, duration, and explanation.
- Database, memory, CPU, etc. usage, and trends,
- List of alarms with the involved components and resolution status.
You give a list of improvements to continuously improve the quality of the product. For each improvement, a list of actions and their status are available.
You automate everything. All automation solutions are documented. Manual configuration or installation are banned.

Microsoft has published a checklist for Azure DevOps.

A vendor-agnostic checklist is the DevOps checklist.

Here are some of the most critical DevOps metrics ^[2]:

Elite teams have tremendous overall performance when improving in the areas tracked through these metrics.

Deployment Frequency	It analyzes how frequently you are deploying the current release of software into production. Deployment automation is covered through continuous deployment and continuous delivery. Higher frequencies correlate with high-performance teams.
Average Lead Time	It identifies how long it takes to develop, examine, deliver, and deploy a brand-new requirement through lead time tracking. Value stream approaches emphasize optimizing lead time.
Meantime To Recovery	Measures the time between an interruption due to deployment or system failure and full recovery through mean time to recovery MTTR tracking. Focus is on efficient recovery and away from mean time between failure. If your organization can recover in minutes, the failure rate is seldom critical.
Change Failure Rate	Indicates how often a team’s changes or hotfixes lead to failures after the code has been deployed.

Additional metrics often are:

Deployment Speed	It suggests how quickly you can deploy a new software version into the production environment.
Production Failure Rate	It analyzes how frequently your software crashes in production for a particular period through failure rate.

Cultural change

DevOps initiatives create cultural changes in companies by transforming the way operations, developers, and testers collaborate during the development and delivery processes. Getting these groups to work cohesively is a critical challenge in enterprise adoption. DevOps is as much about culture, the toolchains are only instruments to implement the approach.

Organizational culture is a strong predictor of IT and organizational performance. Cultural practices such as information flow, collaboration, shared responsibilities, learning from failures, and encouraging new ideas are central to DevOps.

Team-building and other employee engagement activities are often used to create an environment that fosters this communication and cultural change within an organization. DevOps as a service approach allows developers and operations teams to take greater control of their applications and infrastructure without hindering speed. It also transfers the onus of owning a problem on to the development team, making them much more careful in their stride.

The 2015 State of DevOps Report discovered that the top seven measures with the strongest correlation to organizational culture and successful DevOps are:

Organizational investment,
Team leaders' experience and effectiveness,
Continuous delivery,
The ability of different disciplines development, operations, and security to achieve win-win outcomes,
Organizational performance,
Deployment pain,
Lean management practices.

Lessons Learnt

DevOps will only be successful if your organization adopts agile values.

Support transparency, inspection, and adaption.
These pillars are central to empiricism and lean thinking.
Nurture commitment, focus, openness, respect and courage.

These are the official values of Scrum.

Be honest with yourself.

If you do not have concrete and visible initiatives in the areas of development Dev, running Ops, and continuous improvement, you are just pretending to be a professional. But in reality, you are just an amateur.

Glossary

Continuous Integration	Fowler, 2006 - a software development practice where members of a team usually integrate their work frequently, at least daily. Multiple integrations per day shall be possible. Each integration is verified by an automated build, including tests to quickly detect errors.
Pair Programming	Williams & Kessler, 2003 – style of programming in which two programmers work side by side at one computer, continually collaborating on the same design, algorithm, code, or test. The driver types and writes the design. The navigator observes the driver and looks for defects.
Refactoring	Fowler, 2014 - the process of changing a software system in such a way that it does not change the external structure of the code, only the internal. When you refactor, you are simply improving the design of the code after it has been written.
Single Command Build	Biedenharn, personal communication, October 2018 - describes the goal of automating the process of turning the code into a fully functional program. It functions with a single command, thus not requiring a multistep procedure.
Single Source Repository	Fowler, 2006 - allows teams to keep the main project codebase in one easy-to-access location. This allows for easy version control and for any new team members to get up and running easily.
Technical Excellence	managing constraints on software development via techniques, individual skills, and practices as a team. Generally, this includes test-driven development, collective code ownership, continuous integration, and ego-less programming (Gerald Weinberg). Personal commitment to self-improvement in pursuit of software development as a craft is the driver for technical excellence.
Test-Driven Development	development driven by automated tests published by Beck, 2014: Red: Write a test that does not work. Green: Make the test work quickly. Refactor: Clear up all duplications made in the process to make the test work.
Unit Testing	testing the smallest units possible to prove what was written is true. Unit testing frameworks are Xunit or NUnit for C# and .NET, Junit for Java, PyUnit for Python, Cppunit for C++.

References

[1] N. Forsgren, J. Humble, and G. Kim, Accelerate: The Science of Lean Software and DevOps. IT Revolution Press [Online]. Available: https://www.amazon.com/dp/B07B9F83WM

[2] G. Kim and N. Forsgren, DevOps Handbook, Second. IT Revolution Press, 2021 [Online]. Available: https://www.amazon.com//dp/B09G2GS39R

[3] G. Kim, The Phoenix Project a novel about IT, DevOps, and helping your business win. IT Revolution Press [Online]. Available: https://www.amazon.com/dp/0988262592

[4] R. C. Martin, Clean Code. Prentice Hall, 2009 [Online]. Available: https://www.amazon.com/dp/0132350882

[5] R. C. Martin, Clean Architecture. Pearson, 2017 [Online]. Available: https://www.amazon.com/dp/0134494164

[6] M. Fowler, Refactoring, First. Addision-Wesley, 1999 [Online]. Available: https://www.amazon.com/dp/B004PQQRK2

[7] J. Humble and D. Farley, Continuous Delivery. Addison-Wesley Professional, 2010 [Online]. Available: https://www.amazon.com/dp/0321601912

[8] D. Farley, Continuous Delivery Pipelines. 2021 [Online]. Available: https://www.amazon.com/dp/B096YGZVZ9

[9] B. Beyer, C. Jones, J. Petoff, and N. R. Murphy, Site Reliability Engineering. O’Reilly Media, Inc., 2016 [Online]. Available: https://www.amazon.com/dp/1492029505

1. The first conference dedicated to DevOps themes was held in 2009 in Ghent, Belgium.

2. The DevOps Research and assessment DORA has defined these key metrics and a strong correlation with high-performance organizations.