Is DevOps Dead?
No, but please read this post anyways. DevOps is alive and well, but your job might not be.
It was my first day at a new job. I was remote, but my boss flew me in to meet the team. My role was to be among the company’s “AWS Experts”. My boss walked me over to meet the DevOps team. They had their own space that was a bit isolated from the rest of the building. The room was dark, but was dimly lit up by large TV screens displaying dashboards and charts that covered the walls. My boss introduced me by saying something to the effect of “This is Sheep Code. He’s our AWS guy, so if you have any AWS questions, you know who to go to.” Some of the engineers in the room immediately had questions. I helped one of them craft an IAM policy for a cross-account SQS queue that he was having trouble with. I watched in horror as he simply copied and pasted the policy we wrote into the console and pressed save by hand. I asked him if he would use CloudFormation to deploy the policy in production. He laughed and told me this was production.
As we left the room, I turned to my boss and said “that’s an operations team, not a DevOps team.” He agreed and that was that. I don’t blame him for calling it a DevOps team.
Everyone wants to be DevOps to the point where I think many of us have forgotten what DevOps means.
DevOps is simply when development teams perform their own operations... On-call is handled by the development team that wrote the software. The infrastructure supporting the software is automated and deployed by the development team. The development team might have staff that mainly handle operational work, but the key is that they are on the same team as the developers. This is DevOps in its purest form.
…But not really. Over the years, companies have formed DevOps teams that do not write the services that they operate. Instead, they use the principles and tools that DevOps teams developed over the past decade to run services written by other teams. In my mind, “Site Reliability Engineering” better describes these teams, but these days the two terms are often used interchangeably. (Technically, Site Reliability Engineering is approaching operations from a software engineering perspective).
What are DevOps/SRE teams? They do all the stuff operations teams do, they just do it more effectively using automation and fewer “one-off” commands. I believe there are three types of DevOps teams.
The “Actually Just Operations” DevOps team (discussed in the excerpt above)
The Highly Qualified DevOps team
The Savvy DevOps team.
Let’s discuss 2 and 3.
Highly qualified DevOps/SRE teams consist of what are referred to as T shaped individuals. They have a broad set of skills (programming, networking, cloud, Linux, Windows, IaC, Prolog, database, containers, data center ops, configuration management, storage, etc.), and each senior member has deep expertise in at least one of those areas. These teams often highly value IT certifications (especially cloud certifications) and often are incentivized to collect as many as possible via nice bonuses (one company I worked for offered $10,000 per professional cert). They can take pretty much any software, deploy it to any environment, and keep it healthy. As their maturity grows, the number of services they run grows quadratically instead of linearly. These teams are actually badass and also very expensive.
Savvy DevOps teams have learned they can make their jobs much easier by using modern platforms and tooling. Platforms like AWS Serverless (API Gateway, Lambda, DynamoDB) and Google Cloud Run make it incredibly easy to operate services. And much less technical expertise is needed by the members of team! In many modern platforms, Linux/Windows knowledge is less essential. Configuration management tools like Puppet don’t need to be used either. The networking is cloud based, so knowledge of layer 3/4 networking is less important (what the hell is BGP anyways?). In other words, there are fewer areas T shaped engineers on these teams need to dive deep into. But like the highly qualified DevOps team, the number of services they run grows quadratically instead of linearly as the team matures. Everyone thinks these teams are badass because they accomplish so much with so little and can be much less expensive than the Highly qualified DevOps team.
But if deploying to these platforms is so easy, then why do you need a separate DevOps team at all? The answer: you don’t.
It is getting increasingly easy for development teams to practice DevOps without utilizing a dedicated DevOps team because of these modern platforms where underlying operating system and networking concerns are largely abstracted away from the developer. Tooling like the AWS CDK has made writing automation much more similar to writing application code than ever before. Many companies are going back to DevOps’ roots and putting the operational burden back onto the developers.
There are benefits to this model:
The software engineers will be more incentivized to write operationally efficient code so that they don’t get paged in the middle of the night
Fewer headcount
I could go on, but I’m sure a million other posts cover this
So should you be panicking if you are on a dedicated DevOps team? In most cases, probably not. Companies that require dedicated DevOps teams may not have development teams capable of running their own services whether on a modern platform or much less in a data center. Not all software can run on modern platforms like AWS Serverless. If your company uses Kubernetes, then you’ll likely always need a Platform team to manage your k8s clusters (a Platform team in this context is very similar to DevOps in skillset, but they enable software engineering teams to do their own operations instead of doing operations for them).
So your job is probably safe, but you might find it increasingly difficult to find work at high performing companies because most of them do not have dedicated DevOps teams. Top performing software engineering companies heavily favor development teams that perform their own operations.
If you’re a DevOps engineer looking to grow your career and opportunities, my recommendation would be to pursue employment with a development team that practices DevOps. As part of that team, you should have the opportunity to contribute to the application codebase and improve your software engineering skills. Meanwhile, your team will be very grateful for the knowledge you bring to the team with your DevOps background.
Unfortunately, many software engineering oriented companies with a standardized interview process tend not to favor many of the strengths that DevOps engineers develop in their careers. You’ll get coding exercises that test your knowledge of data structures and algorithms which are skills that don’t always surface in a DevOps role. But you probably won’t be asked about your knowledge of Terraform or Ansible.
So if you want to join a high performing team like that but feel you might have difficulty with the standardized interview process designed for software engineers, then what should you do? In my mind, you have two options.
Practice your problem solving and data structure skills on a site like leetcode.com
Find a company or team that is willing to hire DevOps engineers right onto a software engineering team.
Option 2 is better because learning by doing is how DevOps engineers tend to do the vast majority of their learning. But you’ll have to find a company that hires this way! I really hope more do because many software engineering teams could benefit from having DevOps engineers on them!
If you are an executive reading this, please do not fire your dedicated DevOps teams! I think there is lots of value in development teams owning their own operations, but your company more than likely isn’t ready for such a swift transition. Your software and infrastructure might be too complex for a development team to handle. A better transition would be to put members of your DevOps team members directly onto development teams.