The Insights of Being DevOps

Looking back from the distance of my exciting journey over the last few years in develeap, I have to summarize the major insights I gained. As I was asked a few times about my success points, I want to present a few matters I believe each DevOps professional should know. 

I learned each of these insights from a sometimes bitter experience, and I wish someone had told me all this when I was a green junior.

I will start with the most important one:

The main DevOps strength is in understanding processes

Many professional abilities indicate DevOps expertise, like excelling in IaC and CI/CD tools, problem-solving, and continuous learning. However, the most essential quality is understanding processes. This is related to all the scopes we are in contact with. 

Although this quality is valuable in other tech specialties, it’s the key to success for DevOps. It’s not only important — it is your direct responsibility and main power. 

Historically, DevOps was a missing link in connecting the different teams into a conjoint product delivery process, so a deep comprehension was crucial. Today, as tools and technologies surround us, it’s even more so. 

For each component you face, ask yourself:

• What does it do, and why is it here? 
• How does it work? Why does it work this way? How is it configured?
• How are things connected? Why? 
• What are the surrounding components? What is the data flow? 
• Could something else be used? What are the alternatives?

Continue digging. If you cannot answer by yourself, ask others. 

Don’t start the task if the architecture is unclear. Always draw the architecture – this adds clarity and makes sharing easier.

Taking time to understand the process better – always pays off.

The above leads me to the second point:

Don’t be shy to ask again

“Don’t be shy to ask” is also challenging. There is always this nagging little doubt: If I ask, won’t it uncover the fact that I don’t know? However, most of us are okay here because the opposite point is that we all don’t know something. We learned to admit and use this fact, and we ask when we don’t know.  

Re-asking is more complicated. 

It often happens that what was clear yesterday is fuzzy today. The doubt of asking a second time is even harder: If I had already asked and received explanations, wouldn’t it show that I am not sharp enough, that I’m hard to understand?

Yeah, maybe. And maybe the one who answered wasn’t good enough?

The reasons may differ: I thought I understood but was wrong; maybe I wasn’t concentrated enough, or the scope revealed additional complexity.

The main point is that it happens. When you ask again, you show your responsibility for understanding the process.  

When I started my first position, I asked a 15-year-experienced engineer about the idea of SQL schema. He was glad to explain. The day after, I felt it was not clear enough, so I asked him again. He didn’t look too happy. 

When I asked him a third time, he was distraught. 

“I’m just trying to understand why you use a public schema,” I said. 

“Oh…,” – he looked confused, – “This is just how it works here…”

Unleash the power of “I don’t know”

The more you learn – the more you don’t know. 

You encounter new tools, ideas, and technologies, expand your horizons, and realize how many discoveries you are still far from. 

However, there is a big difference between “I don’t know” and non-acquaintance, nescience. 

My neighbor is a very clever man, a lawyer with extensive experience. Imagine that one day, we both should implement something new. For example, we have to optimize ElasticSearch indexes. Do we have the same level of  “I don’t know”? I don’t think so. 

“I don’t know” – so as “I don’t understand” – is a significant progress from “I don’t even understand the question” or “I have no clue of what this technique serves for.” 

When you learn, you draw away from “I don’t know what I don’t know.”

The difference is sometimes not felt because of the Dunning–Kruger effect. According to it, a low-competent person is often unaware of their mistakes due to incomplete knowledge and skills. They inflate the idea of ​​one’s abilities, even in unfamiliar areas and actions performed for the first time. On the contrary, highly qualified people tend to underestimate their abilities and suffer from insufficient self-confidence.

In one sentence, you should follow a path to reach the “I don’t know” level. 

Value it.

Don’t do things in a “silent mode”

This industry doesn’t like surprises. Don’t apply your excellent features and improvements without telling anyone; don’t perform maintenance activities without sharing them with a team.

I will describe a few scenarios:

• You realize that encryption in transit is not enabled for S3 uploads. You enforce the encryption using the bucket policy and are happy with the great security improvement. Tomorrow, the team receives complaints that some of the uploads stopped working. No one knows the reason, and they spend hours on debugging…

• You apply a new version of CI/CD with enhanced validation and verification. Developers from Team A face more findings and must devote some time to fixing them. Developers from Team B spend some time dealing with new features and reading documentation. As a side effect, two product critical tasks weren’t delivered on the due date…

• You found that developers take tedious manual actions each time they need to deploy a hotfix.  You decide to amaze them and work for two days for automation. When you present it, the developers are astonished. However, your manager is also astonished (in another direction) because he needed you to complete the critical monitoring integration…

• As before, you found that developers take tedious manual actions each time they need to deploy a hotfix. You decide to amaze them by working for two days on an automation. When you present it, the developers are astonished. However, your team partner is also astonished because he had worked on the same feature for three days…

The bottom line:

• Share your plans with the team before implementing something. We aim to match expectations, align priorities, and avoid double work. 
• Update the team before you apply changes.
• Update the team when you complete your work.
• Where possible, leverage self-documented workflows. Yeah, I mean GitOps. A clear history provides effortless evidence of what has changed and facilitates the post-incident investigation. 

The undocumented feature doesn’t exist

No, it’s not a hyperbole. 

What’s the point of having a super cool feature no one knows how to use?

In fact, it’s even worse. If you don’t provide the documentation, it’s not only a non-existent feature. You had wasted time when you could be doing something worthwhile. 

Moreover, the documentation is not always enough. Most people read documentation only when they get into trouble. If you want to show off your work, make a short presentation and demonstrate to the team what you did.

A non-obvious conclusion is that you should add 10-15% of work time to each task and include the time needed for writing the documentation in your work plan. 

Never say: “I’m trying to fix this”

This sounds pathetic.

You are not TRYING to fix, you ARE FIXING!

Imagine a spouse turning to their partner: 

• Dear, I’ve asked for your help to prepare dinner; what are you doing now?
• Uhm, I’m trying to peel potatoes…

It’s a lousy answer, believe me. Your partner doesn’t need you trying; they need dinner. Unexpected trouble can always happen – the knife may break, or the potatoes may turn out rotten. It doesn’t matter; right now, I’m cooking, not attempting to cook. 

The same goes for your employment. Things can break unpredictably, but you are not attempting to work. You are just working. 

Finally, here are a few short clauses that are often under-implemented:

Test everything

Never rely on your change to work without testing it, even if you don’t see any reason it could fail. 

This is widespread advice, but the impact of DevOps teams is more extensive than others. We touch critical infra layers and serve the code delivery to the end clients, so the mistakes are potentially much more destructive. 

A non-obvious conclusion is that you should overhead the time needed for your tasks and plan the testing. Sometimes, this can double your working time; sometimes, most of the task consists of testing. It’s totally okay.

Learn from your mistakes

Nothing new here, isn’t it? 

The trick is that we frequently interpret this as “I got my mistake; I will not repeat it anymore.” But have you really learned?

By learning, I mean taking the practical lessons — post-incident analysis and action items. Understanding the reason is crucial, but it is not enough. The real change happens when you take the steps to prevent the issue from happening again.

Learn to cope with context-switching

I sincerely wish you to concentrate on a single task from the beginning to the end, although the chances of this are small. Context switching can kill your productivity one day, when you find yourself drowning among different teams and tasks, meetings, consultations, and support requests. 

You can continue suffering and reconcile with it. Conversely, if you can’t get rid of it, learning how to deal with it is the alternative. The Internet is full of time management techniques and practices. 

You are responsible for recognizing when context-switching becomes challenging and making a first step. 

I hope you found these experiences interesting, and I’d be glad to hear if you found some useful.