Excellence in Software Engineering has never been a stationary destination where one can arrive sooner or later. It has always been a lifelong journey and learning process which demands consistency and commitment in order for someone to progress rapidly and to stay relevant over the next few years because of the ever-changing tech scenario. This element of uncertainty and demand for consistency has intrigued me since forever and hence compelled me to choose this a full-time career and what I’d like to do, at least for the foreseeable near future. Being a self-taught developer, I have always been on the edge to keep myself relevant by establishing a disciplined learning schedule and sticking to it religiously. This entire scenario has been fueled by my imposter syndrome multiple times after having the opportunity to work with some extremely smart and competent people. I have experienced my professional survival instincts in the middle of a meeting only after being blown away by the sheer competency and knowledge of the people I was surrounded with. Naturally, we as human beings tend to idolize certain characteristics we found interesting in other individuals and try to shape and model ourselves in a way that resembles those ideals very well. The drawback to this approach is a lot of people tend to lose their unique identity in the process, the 1% differential in the DNA that makes you stand out and end up being submerged in a pool of identity crisis eventually. With all my honesty, I have never seen an individual and said to myself, “I want to be that person!”. My response is slightly different, which is nothing complicated but speaks to me on a very personal level: How can I take the attributes of this individual and amplify the results while maintaining and building a unique identity in the process? A lot of people tend to jump on the answers This is a very important step in critical thinking because it dictates the outcome of our brainstorming and directs our course along the way before we arrive at a conclusion. without refining the question in the first place. To come up with the right answers, you need to ask the right questions. A similar question spawned in my mind 2 years ago which made me think and eventually led me to put myself on a path where I can be as great as my fellow engineers while maintaining my unique identity along the journey. It’s a healthy combination of both technical and soft skills, however, I intend to cover the technical aspect in great detail before jumping on to the “soft” skills. I’m nowhere near to as great as I aspire to be, as I am still undergoing my journey of transformation. However, I believe by trying to understand the following areas in-depth, one can truly transition to be an enterprise-class engineer from being a conventional one. 1. System Architecture Design For a lot of engineers early in their careers, this is one of the most neglected areas of knowledge. On the other hand, this happens to be one of the key factors of determining any platform’s success because the decisions made at this point impact the entire system in the long run. System Architecture Design also referred to as System Design or Architecture is a representation of the behavior of different components & modules that interact with each other in order to solve a problem. This is a high-level representation of which components to keep, which to trade for better/other options and furthermore how to effectively bring them together to address the core challenge in a highly flexible and efficient manner which allows the end solutions to scale, to be reused, and is secure and feasible. The people who perform this duty are known as and . One key thing to notice here is, that these individuals often tend to be present in a lot of client-facing business meetings as they have to understand the problem firsthand in the words of the clients in order to understand the crux of it, and then propose an appropriate solution which checks all the required boxes in that particular situation. This is one position where people who are more experienced tend to do exponentially better than those who are just starting off because there is no definite solution to every problem. The Solutions/System Architect must be very well aware of the bottlenecks of certain components and tradeoffs, along with the decision making ability depending on the context and current situation. System Architects Solutions Architect The reason I put it on the top of my list is that it is a highly valuable skill to acquire. This one key skill sets you apart in another league of developers because when you are thinking with the mindset of building on scale, every decision you will make, every choice you will make will be centered around that notion. We live in a world where except for private or internal applications, the general demand for developers is for the kind of individuals who can build things at scale and further maintain them. Developing applications that accommodate 1,000 users as compared to architecting applications that can conveniently accommodate and serve 100,000+ users at any given time is drastically different in terms of design and architecture. Resources If you are serious about learning System Designs and Architecture, you can check out the resources below. Books Includes but not limited to: Design Patterns: Elements of Reusable Object-Oriented Software Clean Architecture: A Craftsman’s Guide to Software Structure and Design Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems Blogs High Scalability One of my favorites. It has a lot of interesting use cases and general approaches to hows some massive real-world challenges in the realm of system architecture were addressed efficiently. Apart from that, every well-known tech giant has its architecture/engineering blog such as: Netflix TechBlog Uber Engineering Blog Google AI Blog Facebook Engineering 2. Adequate DevOps Skills I’d like to emphasize the word “adequate” in this sentence because I do not want to propagate the notion of mastering all the arts at once. Let’s be honest, competent DevOps Engineers who are willing to take on challenges with an in-depth understanding of their infrastructure are nothing less than a priced possession for their organization. I remember from a few months ago when we were looking for a DevOps at . We ended up making an offer to an individual for the role, which the candidate refused with the following words: TickSmith Corp I want to take on an easy job. I can understand his circumstances. At TickSmith we are dealing with 3 out of the 5 largest stock exchanges groups of the world, producing terabytes of data daily and supporting infrastructure which is capable of ingesting nearly a billion trading messages per day from various stock exchanges (that’s my team!). Just to be clear this is not a paid promotion or whatsoever but rather a statement of facts to build context for the point I’m trying to convey. DevOps skills are highly valuable and highly sought in the market. As a developer, you definitely do not need to be a Jedi master of infrastructure or end up being a Swiss army knife, but knowing your infrastructure in an appropriate depth is priceless. I cannot emphasize this enough. Knowing the and of the infrastructure which your code will eventually run on can be extremely helpful. whats hows Knowing these details beforehand can allow you to make educated assumptions and optimize your code accordingly to squeeze every last bit of performance out of your infrastructure without sending it haywire. Knowing your infrastructure enables you to troubleshoot the hideous of issues that you or your team may incur at any given point in space & time without spending a lot of hours simply trying to figure out what is happening. Knowing the drawbacks, tradeoffs, and caveats of the platform of your choice and the various provided services will allow you to make better system architecture and design decisions upfront. You do not need to be a Master DevOps Engineer, but just have the right amount of information which will allow you to produce work that runs more efficiently and is not a pain to manage in the long run. Resources Following are a few recommended resources to enable you to build adequate knowledge in the above-mentioned domain: Books Includes but not limited to: Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation Hardcover by Jez Humble and David Farley The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations by Gene Kim and others. Site Reliability Engineering: How Google Runs Production Systems by Niall Richard Murphy and others Blogs System Administration Screencasts Cloud Academy Blog Google Cloud - Community FAUN Gruntwork 3. Strong Technical Knowledge A while ago, I was working with AWS Lambdas and trying my best to troubleshoot a network connectivity issue within lambdas which was bothering me for a week at this point. Basically, it was a lambda function written in Python, using the dependencies to connect to an cluster, access a few tables, and run a query on them to invalidate any cache impala had in its metadata store. impyla Apache Impala INVALIDATE METADATA <table>; The problem I was facing was quite simple, AWS Lambdas when running in a VPC, lose access to the internet. In order for me to connect to the Impala Cluster which was a Network Load Balancer in EC2 console, I had to supply the credentials. The password for the cluster was fetched from SSM parameter store on runtime using AWS SDK boto3, which was relying on the internet access my lambda no longer had. The solution I found on the AWS Docs was pretty straight forward: Create 2 subnets at a minimum. Create a NAT gateway, add it on one of the VPC subnets. Create a Route table and add a new entry with 0.0.0.0/0 going to your NAT gateway. Click on Subnet Associations and add your newly created subnets. Assign your lambda to these subnets and you have internet access now. Terraform everything to streamline deployments in other environments. Sounds pretty straightforward, right? Actually, no because there was not only a lot of infrastructure code (terraform) to maintain now for something as simple as my intended task but also a lot of infrastructure associated with it which had to be duplicated with every deployment in a new environment. One of my senior engineers happen to review my solution randomly and he instantly suggested to me, “Why don’t you use a VPC endpoint for SSM?” Why? Because I had no clue that here’s a VPC endpoint for the SSM parameter store. This is one of my early days when I was striving to gain enough knowledge about the AWS platform. I look back at this particular incident and feel low-key stupid, but not bad actually because I learned a lot in the process. His ability to propose a simple solution to an over-engineered problem which was derived from none other than the official AWS docs was something that struck me. I had no answer to that. All this time I was trying to grant internet access to the lambda while completely ignoring the fact that I can achieve the same effect without the internet access by just allowing access to a specific resource. This is the kind of information that you are able to retain and recall after years and years of trial and error. Moreover, having a strong technical foundation in the first place and being able to connect different dots in your head will eventually allow you to come up with innovative solutions that others may ignore or things hidden in plain sight. The most important factor is that you cannot highlight to pinpoint these hidden details without having in-depth knowledge of what you do and how you do it. Again, I’d like to reiterate that this does not mean you should be able to recall a huge amount of information almost perfectly at the perfect moment. In order to decide what information you’d like to retain and which one you’d like to discard, you can take a strategic approach: What is the one tool/programming language which I use every day? Which is the one platform that I use every day? What are the about the tools and platforms I use? not so mainstream and shiny details How do these details correlate with the work that I produce? If you have a plausible answer to (4), then you should write down that specific piece of information somewhere where it is handy and easily accessible in the future. Remember, you do not need to be the absolute master generalist because it will only slow you down. Because one day, you probably will. You just need the right amount of information to retain which you don’t think you need right now but is related to your work. Resources Given the nature of this point, I cannot suggest direct resources but a few tips and tricks that I apply in my daily routine: Take 30 minutes during your work hours to study one in-depth article about a technology you used at work. It has to be very technically detailed and intellectually stimulating. Pick a book about specialization in your choice of field and gradually absorb the information. Read it, teach it, write about it, explain it in your words, and take notes (critical). Ask stupid questions to people who are smarter than you. When I joined my current company, it has heavily based on financial concepts that I had no knowledge of. It was heavily using a tech stack which I was not very experienced with. I approached my Product Owner once and told him: “Look, regardless whatever we might think, I am always going to be the one with the least amount of exposure to and experience with this, so I will definitely make some stupid statements, ask similar questions and suggest things that do not make a lot of sense in design sessions, meetings. When that happens, please free to correct me immediately and explain why it is not a good idea, you will only be doing me a favor.” All I can share is that it was very well received and led me to develop an understanding of some key financial concepts which we daily use. All this information, sourced from some really smart people during meetings. 4. Communication / Presentation Skills This is one of the decisive factors which will determine the weight of your responsibilities within an organization and how your progress from that point on. Communication skills, as we know are critical in any area of life and are not limited to the realm of developers but there’s a very specific reason I decided to group Communication / Presentation skills in order to convey this point of mine. The reason why communication skills are of great importance for someone at this level is that they will find themselves exchanging a lot of complex ideas, explaining certain concepts which are not easy to comprehend by a large number of people, including certain developers themselves but your, yourself being the proven senior developer have the responsibility to communicate them in a manner that is not only easily understandable but stimulates the thought process of your audience in a way that they can use that information to derive different ideas and hence carry on the brainstorming process with you. This is difficult to achieve and depends on a lot of factors such as the experience level of the targetted audience, their familiarity with the subject, and most importantly, their interest in the said domain. This one skill is often categorized as a “soft skill” which I believe is the reason a lot of people tend to overlook it. If you closely look at the field of Software Engineering, it is filled with discussions around various technologies, complex ideas, inventions, and new ways of solving existing problems. But on closer inspection of said phenomena, it is quite evident that the change being discussed, the progress being anticipated is not pushed and carried out by inanimate objects such as machines and algorithms but by the very human beings themselves. If you harness the ability to break down these complicated scenarios and problems into simpler terms for everyone, you have automatically unlocked a superpower that will not only benefit you professionally but on a personal level as well because the challenge of communication goes above and beyond what we are discussing here. While maintaining this emphasis on communication, I’d like to go over some facts why Presentation skills come next in the line and how they can help you propel your career much, much further in a very short span of time. At my current workspace, we hired a DevOps Engineer about two years ago from France who flew all the way to join us at our Montreal HQ. Needless to say, the individual is extremely competent and good at what he does, but he also had a secret superpower which we discovered later on: Impeccable Presentation & Communication Skills. This one factor allowed him to gain tremendous importance within the company and he was a part of all the major technical meetings happening. At one point, we were negotiating with a huge stock exchange group based in the UK and had to send over a technical lead in order to explain certain processes and workflows as a part of preliminary planning and our proposal to that entity in order to address the challenge. The individual sent to the UK for this particular job was the same person working in the DevOps capacity, who wasn’t a technical lead but moving forward from there, he was promoted to the position of one of the DevOps leads within the company and has maintained a very important say in company’s architectural meetings. One important thing I’d like to highlight is that it was not only the person’s technical or verbal skills that helped him grow tremendously in a very short span of time, but The ability to bring together your different skillsets and use in parallel is an extremely handy and useful approach, which is known by the masses but mastered by a few. it was his ability to merge them both in a manner where both of his domains of expertise were working in collaboration to produce excellent results. Resources I can share links to several books & online resources here which will allow you to develop communication/presentation skills but in reality, the theory simply doesn't cut it here. This is one of the key areas where you have to practice more than anything else, make a lot of errors, finding comfort in your vulnerability, and then use that as building blocks to establish yourself as an excellent communicator and presenter. A few tips in this respect are: Try to learn new concepts and then explain them to someone who is either not familiar with them, or it is not related to their domain of familiarity. Try to explain it in the simplest possible terms and then get feedback on what you could have done better to convey your original message. Actively partake in brainstorming/design sessions at work and closely study how the best communicators of your company are carrying out the process. Take notes, notice the highlights, and incorporate that in your style of daily communication. Consume a lot of healthy content available online around this topic and learn the basic tips and tricks which allow you to read your targeted audience, empathize with them. understanding their point of view and constructing arguments around that information to convey your message The journey of becoming a truly outclass engineer goes above and beyond the key areas mentioned in this writing, but my objective was to provide an overview of how you can begin your journey, orchestrate your daily workflows and start modeling yourself around the core skills that will help you take your career as a developer to the next level. Remember, you do not need to make leaps of progress in a short span of time. This is not a race, it is a marathon that you will run at your own pace. Most people are heading forward very fast but in the wrong direction, which is why you need to take your time to observe things, your surroundings, make necessary course corrections and then be on your way. Last but not the least, realize and acknowledge this is a life long process of improvement, to which there are no shortcuts. I hope you enjoyed reading my thoughts on this subject and wish you the best of luck on your journey to becoming a truly distinguished developer.
