The Rational Software Engineer: There is No Perfect Project

I am a software engineer who geeks out on learning, using a scientific approach to solve problems, and optimizing my performance. Throughout my career as a software engineer, I have noticed that the same project can bring different levels of satisfaction to different people. For example, there have been multiple times when I felt happy working on an objectively average project, while my colleagues were often frustrated with it.

Why is that so? Are my standards incredibly low? Not really. Some time ago, I realized that the mindset and ability to adapt to a project could matter even more than the project itself when it comes to satisfaction. This article will show how I changed my behavior to enjoy some projects more, how I gained valuable knowledge from “bad” projects, and what can make me leave a project anyway.

Attitude matters

Let me give you an example of a software engineer that I’m sure you have met yourself. Let’s call him Robert. Every day you can hear him whining about his teammates’ incompetence, boring tasks, and saying, “this is not my responsibility!”, “the managers are all arrogant pricks. They don’t even know what they are doing!”.

Sounds familiar? Overall, Robert feels angry and frustrated with the project - i.e., his level of satisfaction is low. If you ask Robert whether he likes the project, he will reply, “Hell No!”. If asked, “Hey, there should be SOMETHING you like in the project?” - he will struggle to name anything. One may conclude that Robert should change his job as he is apparently at a bad workplace - and so he does. He quits cursing at everybody he worked with and begins a new “dream” project. And guess what? After a few weeks, he starts complaining again. The same old song. Well, this happens because the problem is not in the project - but Robert himself.

Let me introduce you to another character - Linda. Once in a while, she can tell you a story about how her colleague has made a blunder or about some manager who acted like a jerk. However, you will notice that she doesn’t complain. She is not mad - rather, she's got a cool funny story to gossip about during lunch. Overall, Linda is happy with the project even though she'd like to change some things about it.

She says she learns a lot, and she can meet her personal goals on the project if she puts in the effort correctly. Linda considers changing the project when she understands that her development pace has started slowing down and she cannot gain much more from the project - e.g., she keeps doing the same tasks repeatedly.

Linda changes her workplace for a new “dream” project that turns out to have a lot of its own weaknesses. It’s fine - Linda knows that this is natural, she can still get a lot from the project. Linda enjoys what she can about the project. She acknowledges the weaknesses of the project but doesn’t let them spoil the fun.

The funniest thing is that these two characters can work on the same project working on pretty much the same tasks. While Linda will be positive and satisfied with the career, Robert will be constantly pursue an unattainable “perfect” project.

There is no perfect project

This simple realization made my life much easier. Yes, I will meet people I don’t like to interact with; requirements will change when I least expect it. Also, I will have to take over some work instead of a person who is supposed to do it.

You will work on different projects across your career, and you won’t like all of them the same. They will be of different quality and will have different imperfections, but you can gain a lot from a “terrible” project if you adapt to them rationally.

Don’t get me wrong. I am not saying it’s OK to tolerate a bad working environment.

If your eyelid tremors after each workday and your project led you to drink a mix of sedatives and antidepressants to feel okayish, it is pretty likely you should change something. I will explain later what I see as red flags. What I’m saying now is that some level of imperfection is fine.

Strategy is key

Since a “perfect” project is more of a dream than a reality, how can one cope with an “ordinary” project with its imperfections? I need to adapt my behavior for that and to do it correctly - a strategy is a must. For a good “adaptation strategy” I need to find out what I can get out of the project and how to mitigate the weaknesses of the project. I divide my strategy formation into 4 blocks: inception, goals, actions, monitoring.

Inception is the time when I get to know the project and the working environment. I try to understand what people really are that I will work with, what others like and dislike in the project, the technological stack of the project, and what unique and valuable experience I can attain here. It can take me a month or even more to feel more or less natural at a project and get to know most people I will interact with. Until then, I try not to judge fast and draw negative conclusions. Many project weaknesses turn out to make much more sense after I get enough context and feel less frustrating with time, as new opportunities to mitigate them pop up later on.

Goals are the generic things that I see I can achieve on the project and the negative outcomes of the project weaknesses that I want to avoid. For example: “I want to get more knowledge about distributed architecture, and I don’t want to involve in requirements gathering”. My goals must be aligned with the overall career path and development plan. To illustrate, If you want to become a Tech Lead, make sure that the goals you set for the project help you achieve the new role. Here are some examples: Learn new technology, architecture, or developmental approach that is used on the project:

Get knowledge and experience in management, mentoring, or software architecture.
Work without overtime.
Increase other employees’ satisfaction level.

I find that the most common goals I have for a project are about 2 main sections: I want to get valuable knowledge and I want to feel well on a project.

My personal goals will typically result in the former section of goals and the mitigation of the project’s weaknesses in the latter section.

Actions are the concrete steps to fulfill the goals. One goal will typically require a set of actions to achieve. For example, “work without overtime” can include various things like explaining my values to the management or optimizing the testing approach so that I don’t work at night fixing critical bugs as often.

Monitoring is the continuous process of changing my goals and actions on the project that lasts until I end working on the project. My interests can change, some new opportunities will appear as the project goes. I can never calculate everything ahead, and my goals and actions should adapt to the new information and new opportunities. Most of the time, half of the goals I have on my list appear only after a couple of months of working on a project.

Personally, I keep all of the strategy steps I outlined in my head and don’t write them down. However, noting it down or using some tools like TODO-lists can be really useful to keep track of your goals, actions, valuable information you can act upon.

I will now present to you two examples of how I dealt with “imperfect” projects and what my strategy looked like. I will also split the information into the 4 aforementioned sections.

Strategy in a chaotic start-up environment

The first project was a start-up that promised to deliver AI-based insights from news articles, tweets, and so on. There were different teams working on different aspects of the system.

Inception:

As the project was chaotic and its requirements were often changing, it was a challenge to get up to speed and identify my goals.
Some of the opportunities appeared only when the core developers of the system quit, which gave me the ability to take over parts of the system I was interested in.

Weaknesses:

High churn could lead to situations where nobody had enough knowledge about some part of the system to change or fix it.
Chaotic requirements to the extent that my daily work can be pivoted a couple of times (and I’m not exaggerating).
Demanding but poor management pushed me to work overtime.

Strong sides:

CI/CD infrastructure was powerful and new to me.
Architectural patterns I had little experience with.
With a little initiative from my side, I could take over complicated parts of the systems and learn from them.

My goals for the project:

Get as much DevOps experience as possible as it was the most useful part of the project in terms of knowledge.
Get more knowledge about the architectural patterns that were utilized, understand their pros and cons in practice.
Avoid all of the overtime that was caused by the poor management.
Take over parts of the projects that are the most complicated or use technologies I haven’t tried yet.
Leave the project when I get most of the knowledge out of it.

My actions:

Deploy my solutions myself.
Help ou Data Science to deploy their solutions.
Go through the existing CI/CD solutions and nag the DevOps to explain how it works and why it’s made like that.
Implement new solutions in the Event-Driven architecture paradigm, compare the result to the functionally similar solutions I would implement in the Monolithic architecture.
Decline meetings outside my working hours.
Constantly remind the client that we will charge additionally for my overtime.
Provide estimates for the management’s “immediate” tasks with a slight buffer. Okay, maybe not so slight =) As a result, they see how it affects the scope of the sprint and can’t point the blame on me - all their “tiny immediate” tasks are recorded in Jira.
Be proactive about new technologies and approaches that can be a good fit for the new requirements.
Leave the project when I understand most of the project and have had a lot of hands-on experience with new technologies.

Results:

I’ve built a couple of CI/CD for my team’s solutions and helped with the CI/CD for Machine Learning models by our Data Science team.
I’ve gone through most of the CI/CD infrastructures and got a good understanding of how most of them worked.
During the 6 months I was working on this project, there’ve been only a couple of weeks where I worked overtime - but again, just a couple of hours more, not through the weekend.
I built a new API and pipeline in the Event-driven paradigm.
I built a Data Lake + ETL using AWS hosted solutions with no prior knowledge of both of the things.
I got some experience with Kubeflow.
I learned a decent amount of Terraform and Kubernetes.
I got rotated from the project nearly the moment my progress started halting.

The project had a lot of imperfections: poor management, a “work-till-you-collapse” attitude, lack of processes for code reviews, retrospectives, project planning, requirements gathering, product discovery, and many more. It was an example of one of the “start-up environments” most people try to avoid. However, I got very much knowledge out of it, my mood and motivation were high, and I didn’t work overtime much.

Also, this project can back my words that attitude and strategy are important for your satisfaction. I worked with 5 other consultants from my company on the project. However, all of the other software engineers seemed unhappy about the project and whined often. At the same time, I was pretty fine about the project, and what is more important, I got a lot of knowledge that helped me to get a new job.

Strategy on a project with a big timezone difference and little trust

The idea of the project was to provide software for seamless clusterization. It involved a lot of custom close-to-hardware solutions, a large software engineers team, and complex architecture.

Inception:

I understood that I had little control over the team I work at and the tasks I work with.
I was assigned to writing automation testing full-time instead of development as I expected, and I was not happy with that at all.
Quickly I got an opportunity to change a sub-project to a smaller team where the management was more competent, and the meetings were useful and rare.

Weaknesses:

I had 10 hours difference with the main office, so I initially had 3-4 meetings per week at 7-8 pm, making it hard to have other personal plans on these days.
I quickly realized that nobody trusted me with complicated parts of the system because I was a consultant and had less experience than the core team. Some of the things I didn’t manage to change till the very end. For example, for a reason I still don’t understand, I was not given VPN access for the whole 9 months I worked on the project (even though all of my other consultant colleagues got access). I had to use Remote Desktop Access to access Jira instead. If you have ever used Remote Desktop Access to a computer located halfway around the globe, you know how painful it is. It would take me 10 minutes to change a status of a task in Jira. My manager would simply ignore my messages about VPN access.
The processes were slow, sometimes, tremendously slow. SSH keys can be given in terms of weeks. The management would react to my strong desire to stop writing automation testing only after 4 months of nagging them.

Strong sides:

The project was very complex technically and used various technologies new to me, so there was a lot to learn.
There were a couple of people that were great engineers that I could learn from.

Goals:

Minimize useless late meetings.
Gain trust so that more challenging tasks are assigned to me.
Get hands-on experience on TypeScript and Go.
Stop working as an AQA.
Learn from good engineers on the project.

Actions:

Get rid of useless meetings by changing the sub-project.
Show my competence through initiative and good code.
Try to work more with sub-projects that used TypeScript and Go rather than those that were Python-based.
Try to make my way through and work on the sub-projects with the people I can learn from.
Raise my dissatisfaction about working as an AQA with my company’s management and with the project’s management.
Concentrate on self-education.
Learn the codebase and architecture of the project at the parts I didn’t need but was interested in.
Be more involved in the company activities that I liked more than AQA: mentoring, writing articles, knowledge sharing.
Change my schedule and shift some plans from evening to morning.

Results:

1-2 useful late meetings per week instead of 3-4 useless ones.
After 4 months, I managed to stop doing AQA and started working on the development of new and challenging functionality.
Eventually, I was assigned to the team of talented engineers I wanted to learn from.
Hands-on experience with both Go and Typescript.
A much deeper understanding of how distributed systems work and what technologies can be used to build such systems.
I managed not to burn out :)

That project was the most mentally challenging experience for me. Working as AQA for a couple of months, changing requirements that made me rewriting my implementations multiple times, seeing that my work had little impact, late meetings, being ignored by the management. All of these things didn’t promote my happiness, but I was not miserable either. I still learned a lot, managed to do plenty of self-education and enjoyable side-activities, and work with people I honored in the end.

Takeaways from the example

As you can see, even though the projects I listed were not sweet dreams, but it was still possible for me to grow with their helm, get useful experience, and keep my mind stable.

When I see something on a project, and I cannot change it myself, there are 2 strategies that I turn to, which are hilariously similar to the fight-or-flight response. Let’s start with the “fight” response and move to “flight” in the section after.

Talk to the management first

When I say management - it’s not just the direct manager. I mean anybody who can help me to solve my problem: Team Lead, CTO, HR. I think you know who has the power to change things in your company as well. These people should be your friends. They should understand your goals, values, and what you are going through.

It’s good if a company has a process where I talk to the management and share my concerns reasonably often, but If such a process doesn’t exist in the company, I will approach the management myself.

If I struggle on a project because I cannot grow in the manner I want to, I have a colleague you don’t want to work with, or I want to change something, I make my manager aware of that. Usually, there is a very wide range of things that a company is willing to undertake to keep a good software engineer. The company can help you rotate to another sub-project, change your technology stack, increase your salary, and even fire someone you find impossible to work with. There is a good solution or a reasonable compromise for all the problems. Or at least for most of them :)

Sometimes it will happen that management won’t help you to solve the issue. As bad as it can feel, there are a few things to mention here.

Are you sure that you made yourself clear? Be demanding with critical issues that make you frustrated that they can solve. Sometimes, a manager will think you are fine. However, you are broken to pieces, but you didn’t communicate it well or were too humble to say directly. So make sure that if something critical happens, you communicate it clearly.

Your management has more context than you. I try not to forget about it when something doesn’t go as I want. There may be things going on in the company that you don’t know and cannot share that influence their decision. The manager I asked for VPN could receive an order to give a few accesses as possible. The manager who asked me to work overtime could have very tough budget problems in the company (and actually he did). I try to put myself in their shoes to understand such things better.

That being said, it’s still up to me to decide whether I want to tolerate something. I can tolerate doing CSS because the project doesn’t have the budget to hire a dedicated professional for a month, but I won’t do it 70% of the time for a year.

Diversify the project with side-activities

Typically, there are many different activities apart from the project itself I can do in a company. For example, knowledge sharing, interviewing, mentoring, attending courses, company libraries, self-education, competitions, R&D office, pet-projects initiatives, and many more.

I consider them important because they help me organically develop my soft skills, balance some lacking aspects of knowledge on a project, and simply because they are entertaining.

As I find it optimal to only write code for 5 hours per day, I have some time left after the coding and meetings. In that time, I tend to do some activities that I find enjoyable and useful for others, like knowledge-sharing.

For example, If I see that I don’t get enough knowledge on some topic from the project, I turn to self-education, the company’s library, or a pet project. Most of the managers I’ve worked with find it reasonable to spend some hours per week on such activities.

Some of such activities may be rather unofficial or even absent in a company. I typically treat such situations as a good action point where I can help my company introduce a cool or useful activity for all the employees and develop a good process for it.

At times the best option is to leave

Put a Buddhist monk study and graduate with a Ph.D. in Machine Learning. After that, make sure they get 5 years of working on state-of-the-art projects with challenging tasks and good working environments. Now put them to a project where they will need to do CSS fixes to meet the requirements of a pixel-perfect design with a toxic boss, low salary, and 12 hours working day. Will the monk be able to tolerate it being calm and happy? Some will (sorry, but to be honest, even a Buddhist monk can burst here). The question is “What for?” if he can simply quit and find a good project?

I definitely don’t have as much peace of mind and the ability to tolerate things as most Buddhist monks do, and I bet you don’t either. So the question comes down to “Is the project worth dealing with all of the hassles it has?”. Sometimes the answer is “No, I get from it less than I give”. Then it’s reasonable to leave. It’s simple and banal, but I met many people that still wouldn’t leave a project even they think they should.

There 2 main categories of reasons why I can leave a company: I cannot meet my goals as a software engineer at the project, or I find the project working environment unbearable. If I have tried to change them and it didn’t work out, it’s time to say a firewall.

If I cannot develop in the way I want at the project and I cannot change it, it is a good sign I should find something else. For example, If I want to grow as a backend developer but I am assigned to a frontend part of the project and I know have no ability to change it for the next year, I am not progressing with my personal goals.

An unbearable working environment can consist of a lot of things: overtime attitude, toxic management, values of the company that don’t meet mine, big timezone difference, low salary. If I see that I cannot embrace or mitigate one of such things and that the pros of the project are not worth it, I consider leaving the company as well.

I've given an example earlier - Robert, who has a negative attitude to the project without a valid reason as irrational behavior. However, there is also a whole opposite camp of software engineers - who are unhappy with their project for a valid reason, but they continue working on it with zeal.

My goals as a software engineer and overall well-being matter. However, know when you should leave, don’t tolerate what you shouldn’t tolerate

Summary

Here is a quick recap of what I do to get the most out of any project:

Embrace that a “perfect” project doesn’t exist. All projects have their imperfections, and it’s important to mitigate them to get the most out of a project.
I try to turn an “imperfect” project into a “good” one by adapting with a strategy.
I make a strategy that consists of inception, goals, actions, monitoring
Project goals should be in line with my goals as a software engineer.
Turn to side activities to diversify my work and get what I lack on the project.
Talk to my manager about things I don’t like, give them a chance to change them.
Leave when I should leave.

I hope that this article helped you find some aspects you can improve on your current project or in your attitude. If you want to discuss some ideas around the article or help me and give me constructive feedback, reach out.

This article is a part of the "The Rational Software Engineer" series. It covers topics related to optimizing personal processes, education, career development, and being passionate in a software engineering context:

Personal Acknowledgements

Thanks to all my friends who reviewed this article, especially Maksym Bekuzarov and Nastia, for a thorough check and many useful insights.