Running Towards Fire: The Key Trait That Separates Senior from Junior Engineers

Production was down for an hour yesterday.

Clickhouse, Parqet’s chosen database for stock market quotes, had too many open connections and too little memory. It took a Google Meet, three developers, and about thirty minutes of looking at various graphs and logs to locate the issue. After another thirty minutes of scaling and redeploying our API, everything returned to normal.

During my first years of coding, I was quiet in these meetings.

I couldn’t make anything of the charts, graphs, metrics. Even if a spike in a chart seemed like a clue, I didn’t know whether it was an actual indicator or the usual pattern.

After a couple of outages and extinguished fires, I felt more confident.

But it took a mindset shift to arrive at that point.

Junior Engineers Run Away From Fires

When production breaks, are you the one who fixes it?

Most junior engineers aren’t. Instead, they expect more experienced colleagues to act. Understandably so: Seniors quickly find the origin of a bug and fix it in a durable way, whereas juniors waste hours running in the wrong direction.

But waiting for seniors quickly turns into a habit.

If an outage happens and your first action is to see whether your coworkers are online, or if a critical bug appears in your monitoring but you ignore it and wait for others to resolve it, you are hiding behind the experience of others.

You are running away from fires.

But fires are where you learn most.

Fires Are Your Best Bet For Growth

Humans learn through feedback.

With implementing features, the feedback is clear and available. If it works, you did a good job; if it doesn’t work, you didn’t. Everybody can achieve results with feedback like that. That’s why a programmer’s skill isn’t visible in what kind of features they can build but in how they build them.

But where do you get feedback for the how?

The quality of your code will be revealed with time. Your code might be difficult to work with when adding new features. Your coworkers might not understand it. Or it might break production.

Whatever it is, you only get feedback when something bad happens.

But if you ignore the bad thing and run away from the fire, you’re skipping the lesson. So run towards it!

You can also learn from others’ fires. An outage might not be caused by your code but by a coworker’s, by an external dependency, or by a setting of your database. To uncover the root of the problem, you must learn about these things — more valuable lessons, for free.

So, appreciate things going south — and run to the fire.

It’s your best bet to learn what matters.

Parting Words

Fires are annoying. They distract from shipping features.

But fires won’t cease to exist, and the experience necessary to fight and prevent them is what makes a great engineer. They are the best opportunity for growth.

The next time you have an outage or a critical bug, be the first to try to solve it.

Soon, your coworkers will look to you to fix it, and you’ll be more than capable of doing so.

That’s it!

All the best,

Julian