As the hype of Artificial Intelligence, Big Data, and Internet of Things appeared into the surface, so many people are interested to enter the data field, either as a Data Engineer, Data Analyst, or Data Scientist. One of the approaches is by learning from online courses, practicing through several tutorials that were freely available on the internet, or maybe doing a conventional approach — back to school.
If you are a developer, you could harness your passion to learn this field by doing this method at your work: Data-driven Development (DDD).
What is it anyway? And what is the correlation between them?
TL;DR.
Data-driven Development (DDD) is a method to develop and maintain our services based on data.
As a developer in a technology company where they grasp the idea of Lifetime Development, we need to conduct the idea of Systems Development Life Cycle.
credit: Husson University Online
In order to iterate this cycle and keep improving our service, we need to maintain our system from any kind of silly or crucial error; plan with our teammate to improve our services, and analyze every business challenge that has appeared in front of us.
As a developer, we could just accept any task that was given by our manager. We could also make any decision based on what kind of system we should build, what kind of features we should make, or based on our intuition. We could also tackle any error when the error rate has been high enough to grab our attention.
However… be careful with it. That could cost you A LOT.
Any error in the production environment is unacceptable. Remember when you were in the restaurant and you were served with awful food just because the chef was trying to improvise the taste of it. What did you feel like?
As a developer, we should care about our stakeholders. Not only about our customers but also our team where their system is integrated with ours. We need to make sure that our stakeholders are happy.
By doing Data-driven Development, you try to minimize the inclination to make any decision based on an invalidated prediction. Instead, we need to reinforce our decision based on data.
It reminds me of the time when one of the APIs that I handled was performing really bad and I need to improve the performance — from an average response time of 3 seconds into 1 second. It would be hard to tackle this issue if I tackle it by using trial-and-error approach or any kind of unsystematic approach.
Instead, you could try to conduct it in a holistic way — assess your data, and use your kick-ass analytical skill!
The existence of analytical platform, such as New Relic, Grafana, or a system that was built by yourself, could be useful to assess our system in real-time. Moreover, we could understand at which section that slowed down the overall performance, and what kind of request that received a slow response. After we started to clearly understand the current behavior of that API, we could easily notice the root problem — faster and more precisely.
What you need after that, is your handful of skills to refactor that system. As a result? My team, including myself, was staggered by the result. We only expected to reduce the response time into an average of 1 second. Instead, we got an average of 3 milliseconds! That’s totally beyond our expectation!
Hence, either you are a front-end developer, back-end developer, system developer, or any kind of developer, if you want to reflect your passion for data field, actually — you could immediately practice your analytical skill by doing Data-driven Development. A sense of sharp eyes to assess the problem by using available data is one of the fundamental skills that you need to have as a data guy.
Surya is a Back-end Engineer in his professional life and a Technology Evangelist in his personal life. This year, he builds a resolution, called “Road to Becoming an Applied AI Scientist”. If you are interested, or want to follow his progress, you can visit him at his personal website, soensurya.com, or on LinkedIn.