Problem Solving in Software Development: Specialization Vs. Generalization

Whether it’s appreciation or discomfort, one of the best way to put your feelings out there on the internet is through memes, which has a great attention span on the user viewing them.

Whenever a meme appears on the internet with respect to software development, most of them have something to do with stack overflow as if the complete software development lifecycle depends on it, and the development of software is not possible if stack overflow was never invented.

But, the endless man-hours that are invested in the design of the architecture of a software, its coding implementation, testing, bug fixing, deploying, and scaling any software product cannot be achieved with the knowledge that stack overflow alone offers. Sure, whenever we get a blockage, whether it’s related to some syntax error in a programming language, networking tweaks, memory corruption, or some here and there common mishaps, then it’s the fastest way for a solution.

What is the actual problem?

First, we need to learn about the exact nature of the problem and then develop an approach for solving the same. Assuming you have a developer profile, there comes a certain requirement for building a feature X. Needless to say, that you are involved in a legacy project whose majority of code is written in the C programming language. There are a few ways through which you can approach this particular problem -

CASE - 1 → You start coding in the same language C (of the project) for this feature from Day 1, and you will modify the architecture or fix the code on the go as and when the issues will arise, and you are done implementing the feature.

Result - Unless you are a very experienced developer or you have the hand of God with you, there are alarming chances that you might have written poor code, or introduced a bad architecture, or might have introduced certain bugs for other parts of fine working code, or the code you have written cannot be scaled, or simply better code could have been written which would have sustained for the future also.

CASE - 2 → You take one or two days, and you devise a proper architecture and strategy of how the execution should suppose to work, divide the feature that has to be implemented into small tasks, and see if those tasks are already been done. If those tasks were already implemented, then we could just reuse that code since it was working in the production also. Upon devising the strategy, the same would have been consulted with the team leader or some experienced developer who might have suggested a better approach, thereby changing your approach to solving the problem.

Result - The implementation would still be in C in this case, so no new programming language experience could have been gained. True, this approach would have taken a little bit longer than the previous approach, but in the end, you would have come out with a better design-solving approach. This will result in a properly documented code with fewer bugs along with multiple peer reviews which will definetly sustain the future requirement.

CASE -3 → There is a lot of urge in you for multiple programming languages, and you feel that the required feature will be pretty easy to implement if done in another programming language, say Python. So, you build the feature in Python and write a C wrapper on the top of your feature, and you communicate with that feature through the APIs from your C Legacy code.

Result - The advantage of this type of implementation is your knowledge gained from learning new programming languages, creating a wrapper around a library, and API communication which is a pretty cool thing.

There may come certain times when a load of delivering the particular feature is tremendous, and most of your colleagues are busy with some other things, then proceeding with CASE-1 seems like a wise choice. However, if the feature is not so crucial for production, then CASE-3 seems like a wise choice. If your feature is going to be deployed in an environment with huge traffic, then CASE-2 seems to be the best choice, keeping scaling in mind.

So, it all comes down to,

where do we study and exactly what it is that we need to study?

This answer can be divided into two parts. You will face two types of problems in your work environment. One will be of the general sense, which includes memory corruption, basic commands and syntax error, firewall issue, routing issues, installation, and environment-specific problems, library dependencies, and things like that. Rote learning of these types of things is not required as they will be automatically absorbed by your mind as and when you will proceed and solve them with the help of resources provided on websites like stack overflow.

The second type of problem is of the specific sense, which includes that there is a certain problem with your architecture, and the code breaks down in production due to that. It may be during scaling of TLS connections or processing asynchronous MySQL requests, complex IP tables rules, composite Gstreamer pipelines or binding multiple ports for media processing, or things like that.

The solutions to these types of problems are not one-liners or come in a paragraph, at least most of the time. They are almost difficult to track on the web, and the solution to the problem comes after reading a ton of resources. The solution for any tricky architecture-based problems/scaling core software comes after multiple hits and trials of connecting the dots between the multiple resources, which is mainly possible through experience.

The main problem is that the number of resources and information available decreases as soon as the technicality and specificity of the problem increases. For, e.g., A ton of resources are available for the IP tables command, but no specific mention of command where IP tables can balance the VoIP protocol (SIP) or the DIAMETER protocol. There are tons of examples of Dynamic IPSec Library Strongswan, but there is little to no information on the load balancing of static XFRM IPSec-based security associations. Nokia, Huawei, and Ericsson are deploying the 5G in the world as we speak, whereas there are little to no concrete resources to easily explain the 5G on the internet.

However, there is a catch. Whatever work you are doing, there is a very high chance that others are doing it too. Very often, some open source project is dedicatedly working for the same cause as your company, except they are providing the service for free. If your work involves deployment, then you have OpenStack, Docker, and Kubernetes for you. If your work involves dealing in CI / CD, then Jenkins and Travis are your buddies. If you are into a media server or telecom domain, then Freeswitch and Asterisk do it for you. If you are into complex media manipulations, then OpenCV, Gstreamer, FFMpeg do it for you. If you are into Databases, then MongoDB and MySQL do it for you. You get a general idea.

It’s inevitable to make mistakes. But, to remain in this competitive world, we need to be very selective as to what we are feeding on. Solving complex problems, studying different architectures, learning about working on multiple open source projects, understanding coding guidelines, and writing clean code along with its tests will help you a lot in the future. As to how to declare an Integer in C or export JAVA_HOME, don’t bother your mind and let stack overflow take care of these small issues.

In other words, you need to make efforts towards the specific issues that you believe are very challenging and demanding while letting the stack overflow take care of small issues like compilation problems due to dependency issues and stuff like that.