Here's Why Software Engineers And Data Scientists Get Fed Up And Quit

In our upcoming book, How to be a Master Expert, we discuss the broad dissatisfaction experienced by people working in technical and individual contributor roles. From software engineers to scientists, manufacturing and medical technicians to economists, the contribution made by experts is often not well understood by the broader organisation.

That creates frustration. But more importantly, it prevents the enormous potential of experts from being achieved. This is an idealistic group that wants to do great things - but they are held back by a lack of understanding and development by the broader organisation.

Below we describe a typical performance review and career path of Edward, a business/finance analyst (and yes, a real person, though his name has been changed).

As we say below, Edward’s situation is alarmingly common. There are millions of experts around the world just like Edward. They’re career stuck and feel they could add much more value if provided the opportunity.

Are you willing to help?

The Edward Predicament

Edward is a business analyst in a finance department and has been with his organization for seven years. Edward’s job has constantly evolved. He started out putting together accurate reports for divisional heads, but recently, he has been undertaking much more complex tactical and strategic financial analyses for these same stakeholders.

Edward loves the analysis aspect of his work and wants to be in a position to be part of the conversation the wider organization has about what to do with the insights raised by the data he has collected and analyzed. He wants to be seen as a business person who is also a financial analyst, rather than being seen only as a financial analyst.

Over the past two years, because of the quality of the work he’s clearly capable of, these senior divisional heads have asked Edward to do more and more analysis, almost doubling his workload. Because he loves the work and believes it will lead to a more senior position, Edward has fulfilled all requests.

Why developers and data scientists are fed up

When we first met Edward, he had recently had his annual performance review with his manager, Alex. He told us that this annual review had been a mirror image of the six that had gone before. Alex had scored Edward a mid-level performance ranking (a “3”, which is described as “meets expectations”).

Given the quality of his work and the doubling of his workload, Edward is fuming about the rating. He argues that he delivers work far beyond his position description almost every week. He manages a far higher workload than his peers and is completed trusted by Alex to manage complex tasks. Edward believes, at a minimum, he should be scored a “4” (“exceeds expectations”) or even a “5” (“significantly exceeds expectations”).

Edward tells us that the annual performance review is virtually the only performance conversation he has with his manager each year. He tells us that

“Because I’m performing, because I work independently, because I cause no problems, because my stakeholders are happy with my work, because I am in fact a high performer and low maintenance, my manager focuses on putting fires out elsewhere, knowing he doesn’t need to spend time on me.”

Edward reports that the review discussion centers around key performance indicators that were set at the beginning of the year, which are now out of date. Many more tasks and undertakings have been added, none of which are reflected in the structured process of the annual review. We hear this quite a lot from high-performing experts.

Management "doesn't get what I do"

When we ask Edward why he thinks Alex, his manager, only scores him a “3”, Edward provides a number of theories. Firstly, he thinks that his manager doesn’t really understand the complexity of the work he now completes on a day-to-day basis. Secondly, he believes his manager has virtually no visibility of the more complex and challenging work Edward does for senior business divisional leaders.

Thirdly, given the first and second theories, Alex does not really understand how Edward’s skills and the value he adds have increased. Finally, Edward admits that he believes his manager is a little “old school” and that Alex scores almost all of his reports a “3”, so Edward would need to do something extraordinary and very visible to get a higher rating. This is another sentiment we hear quite often from high-performing experts.

Most of the meetings Edward has with his manager are short, sharp, and task-related. The items Edward wants to discuss aren’t on the agenda.

Why does he get allocated more and more work without either additional reward or any recognition? Why has he scored a mid-ranking rating when all of his senior stakeholders clearly have high confidence in Edward’s abilities and potential by entrusting him with increasingly complex and important analysis? Why isn’t Edward’s career trajectory a topic of regular conversation? Why, in fact, does his manager show no interest in how Edward is feeling about his role or career path?

When we ask him if he has raised these issues directly with his manager, Alex, he tells us that he does so repeatedly and that Alex promises to explore these issues soon, but he never does. Edward explains that Alex awkwardly ends these conversations as quickly as he can.

Edward has come to believe he will never be promoted.

He lists many reasons for this contention. Firstly, there is no obvious successor in place. He’s the only person in the organization who does precisely what he does, which is a typical scenario for experts. Secondly, he feels he’s taken for granted by his manager and the wider organization. He believes that everyone just assumes he has no ambition.

“They think that because I enjoy the parts of the work that involve detail, I’m going to be happy to be a technical analyst for the rest of my career”.

Thirdly, there is no defined career path for him unless he decides he wants to lead a team of people in the finance department. Edward isn’t sure this is a path he wants to take, and since he’s got no experience with leading people so far, he doubts he would ever be offered or entrusted with such a role.

He is career stuck.

Edward is highly ambitious. He’s keen to progress to a greater level of responsibility, a more fulfilling role, and he believes he’s capable of adding significantly more strategic value to his stakeholders and the organization.

He wants to grow his influence and his income. He knows he can add so much more value to his organization. But he can’t see a way forward. It would be fair to say that when we met Edward, he was extremely frustrated and considering his options. He believed that leaving the organization was possibly his only way out of his present predicament.

The Great Paradox of Experts

Edward’s situation is alarmingly common. There are millions of experts around the world just like Edward. They’re career stuck and feel they could add much more value if provided the opportunity.

And there are thousands of organizations that aren’t getting the full organizational value from their experts - their data scientists, developers, architects, and other highly technical roles - then they could and should.

Edward is an example of what we call the Great Paradox of Experts. The more expert he becomes, the less likely it will be that he will be promoted to wider responsibilities. He’s becoming increasingly difficult to replace and the organization grows more and more dependent on him in his current role.

The last thing the organization wants is for anything to change. Edward is a technical star, and this is what the organization wants him to remain, regardless of what Edward wants.

This Great Paradox of Experts is something we have continuously encountered while working with experts. The more brilliant some experts are at their chosen specialty, the more career stuck they become. They have reached an apparently impervious technical ceiling.

Typically, how experts tend to resolve this issue is in one of two ways. The first solution is they leave the organization, believing the only way to get promoted is at another enterprise. This is the grass is greener on the other side of the fence philosophy, which, of course, may or may not be true. The second solution is they stay at their current organization but become withdrawn, cynical, and increasingly unhappy.

Both of these resolutions are poor for the organization, as the experts become high maintenance and difficult to manage or have to be replaced.

Neither one is necessarily the optimal resolution for the expert either.

A third - and better! - way forward

In order to break through these technical ceilings, experts need to change the way in which they think, operate, and connect. Experts like Edward need to progress from performing at the Expert level to what we describe in detail in our book as Master Expert level.

Key transitions include:

• A shift from tactical support to strategic support.
• A shift from departmental focus to an organization-wide focus.
• Replacing jargon-filled communication with cut-through, plain-language messaging that everyone in the organization can understand.
• A shift from designing solutions for them now to designing solutions for the future.
• Replacing transactional stakeholder relationships with transformative stakeholder relationships.
• A shift from a reactive workload strategy to being proactive about which work is prioritized.
• Replacing a mostly internal focus with a much broader external focus.
• A shift from delivering ordinary value to their organization to delivering extraordinary value.

In short, they need to build their mastery of Expertship—the practice of being the very best expert they can be.

This excerpt is taken from chapter one of How to be a Master Expert, coming July 2021. This excerpt was first published on LinkedIn by Alistair Gordon, one of the book's authors.

If you want to know why your organisation's developers and data scientists keep leaving, download the full chapter at expertship.com

Previously published at https://www.linkedin.com/pulse/whose-fault-heres-why-software-engineers-researchers-other-gordon