Unveiling the Code: Meet the Writer and Lead Data Engineer - Madiha Khalid

In this article, I let Madiha tell her story. With years of experience as a senior data engineer in various companies, her expertise goes beyond lines of code. She helped small and mid-sized companies build their reliable, scalable, cost-effective data platform.

As we talk, Madiha tells her story, shares one of her most challenging projects, and gives advice on climbing the career ladder.

Madiha's Background Story

Madiha works as a Lead Data Engineer and certified Data Platform mentor for startups and aspiring engineers. She inspires startups to recognize the potential of data platforms in the age of AI in their early stage — not just from a technical standpoint, but also from the perspective of building cost and performance-optimized and trustworthy solutions for their data warehouse or data lake.

She generously donates her time to mentor aspiring data engineers in addition to her regular duties. Her dedication to helping others succeed sets her apart as a true leader in the field.

My current goal is two-fold: first, to raise awareness in the business about the need for a Data Platform at the Early Stage and second, to mentor the new generation of Data Engineers." — Madiha said.

Madiha Khalid's foray into technology began in high school in Lahore, Pakistan. Faced with the choice between biology and computer science, she decided to dive into the less-trodden path of computer science.

About 95% of our students chose to go to biology classes and the rest — to computer sciences. It was difficult for me back then, but I made my choice", — Madiha said.

Despite initial setbacks and the challenge of being denied admission to chemical engineering, she shifted to computer science in early college; Madiha's passion for programming ignited during her high school years. Then, her journey continued into college, following in her sister's footsteps and pursuing a bachelor's in computer science.

"I remember my first achievement in 2011 when I developed Image Compression Software for my Image Processing course Assignment. This application reduced the size of .jpg without compromising the real pixel quality, and it became quite popular among my University Friends. This is how I began my career journey towards becoming a Software Engineer" — Madiha said

Her early exposure to website development and freelancing projects marked the beginning of her hands-on experience. Madiha's dedication and late-night coding sessions secured her academic success with a 3rd position in BS and the Erasmus Mundus Scholarship for a competitive Master's in IT4BI, laying the foundation for her future career in the Data Engineering industry.

Advice to Boost Personal Growth for the Aspiring Engineers

For those aspiring to enter the tech field, Madiha advocates building skills with real-world business use cases.

"Employers mostly look for seniors profiles who can solve their current challenges. They want to hear your stories about your impact and the contribution you made to the latest experiences, even though you are applying for Junior Position.

For a young professional, it's a good way to gain experience on your self-learning and apply these skills on some freelance or open-source projects, — Madiha said, — You can easily connect with the real business over Upwork or Fiverr. Contribution to open-source projects in your early career boosts your profile 10x to get your first dream job".

Building a portfolio with real-world projects enhances credibility and storytelling during job interviews.

There is no shortcut to learning skills in one night. It involves hands-on work and applying your learning skills to the real-world business use case.— Madiha said

Maintaining an active LinkedIn profile and networking with industry professionals are crucial for visibility. Madiha's journey emphasizes the significance of perseverance, especially when faced with language barriers and cultural differences during international transitions. As the tech industry evolves rapidly, staying updated on new tools and integrations is essential. Finding one's niche within the extensive tech landscape is critical to a fulfilling career.

Motivation Through Challenges

Madiha's career trajectory took an exciting turn when she saw an internship at a major telecom company in Pakistan through a Facebook group.

I think I had a lot of fortunate incidents during my career path. And this was one of them — fully accidental and without much belief in myself back then. — Madiha said.

Her role involved automating processes for telecom engineers, setting the stage for her future in data engineering. From developing dashboards to optimizing data integration performance, Madiha's expertise grew.

Her journey continued through a Master's in Business Intelligence in Europe, where she developed Big Data projects and deepened her knowledge as a Solution Architect. Studying at universities as an Erasmus student in Belgium and France and a pivotal internship at the Barcelona Supercomputing Center in Spain broadened her skills. Madiha's move to Germany marked the start of her role as a Data Engineer at Jimdo, where she eventually earned the position of Senior Data Engineer.

Madiha's challenges in adapting to different cultures and managing complex data pipelines became stepping stones to her success. Her passion lies in building and optimizing data integration pipelines, a role she found immensely satisfying. Notable projects are the Data Migration of ~8 Billion Event Stream records using novel partitioning techniques from legacy systems to the Data Lake. Implement customer engagement metrics in near real-time with Apache Spark and make them directly available on customer dashboards.

Creating the DWH GDPR Compliant with the data governance access layer. The list could continue, showcasing Madiha's impactful contributions to the field.

Data Engineering Book Recommendations

Here's a list of books and resources Madiha advises for anyone interested in big data:

• Beginners: Fundamentals of Data Engineering

Building the foundation and concepts of Data engineering is very important, and this book's approach is the high level to build your base.

• Intermediate: The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling

This book is a part of my Master's program and teaches data modeling, a highly demanded skill in job interviews.

• Intermediate:  Spark: The Definitive Guide [Book]

This book provides the internal workings of Spark and hands-on sessions. It is written by the original creator of Apache Spark; hence, it is worth reading.

• Advance: Designing Data-Intensive Applications

This was the first book Madiha read in 2017 while building her first real-time streaming application. It is highly recommended to understand how large-scale data applications develop.

Insights for Aspiring Professionals

In concluding her journey, Madiha Khalid shares valuable advice for young professionals entering the tech industry. Emphasizing the need for senior-level skills in the market, she recommends taking on short projects on freelance platforms and open-source projects to build a robust portfolio. Establishing a solid online presence through platforms like LinkedIn and GitHub aids networking and showcasing one's capabilities.

Madiha's story is a testament to the power of determination, adaptability, and continuous learning in navigating the dynamic tech industry landscape. Aspiring professionals can draw inspiration from her journey and understand that challenges are not roadblocks but growth opportunities.