About The Meteoric Rise of the Low Code Data Scientist

Low-code software development, only recently trumpeted as the next big megatrend, is here and it’s already become mainstream. By 2025, 70 percent of new apps developed by organizations will be created using low and no code platform, up from less than 25 percent in 2020 according to Gartner. If you’re not already using low-code platforms, you will be very soon. 

Low and no-code development makes it possible for people without a technical background to create digital products, which is how the term “citizen developer” came about. However, the idea of the citizen developer is a little misleading. While it is true that you really don’t have to be techy by training to use low-code tools…..some transferable skills definitely help. As data scientist and low-code evangelist, Rosaria Silipo, put it, “Codeless does not mean math-less.” 

Furthermore, while organizations and so-called citizen developers are of course benefiting from easier access to digital development, so are technologists. Industry studies show that a significant proportion of professional developers – easily a third and potentially nearly two thirds – are using low-code solutions to speed up their work or create workarounds. 

Low-Code AI: A Story of Supply and Demand?

The only skill that enterprises need even more than IT is data science. The same market forces that are driving up low-code adoption in general are especially acutely felt when it comes to low-code AI solutions. 

Lack of people is not the only challenge for organizations wanting to get their AI projects up and running. AI development often takes a long time and is resource-intensive when large or complex data sets are involved. In other words, it’s costly. Added to that, competition for data scientists is fierce and remuneration generous. Big data and AI projects involve a serious commitment in terms of money and time.  

Until now, that is. Low-code is helping to significantly speed up timelines, while bringing down costs not only on a per-project basis but in terms of headcount – there’s mounting evidence that low-code solutions are helping to bridge the skills gap by reducing the overall number of hires businesses need to make. In the recently updated Emerging Architectures for Modern Data Infrastructure, Andreesen Horowitz noted the ML industry is consolidating around a data-centric approach with low code ML solutions (like Continual and MindsDB) automating the ML modelling process.

Low-code Tools To Suit Every Job

While this might sound like bad news for data scientists, it is worth remembering that data scientists themselves are benefiting from the increased speed and ease of using low-code AI tools. 

The low-code movement is often described as a “democratization” of all things digital, breaking down the barriers to development and to transformational outcomes. This democratization has been gradually taking place at every level of data science from data input to data transformation to machine learning. 

There’s a low-code, simpler way of doing many of the jobs that were previously highly complex or took a long time. Using low-code solutions, project timelines speed up and data scientists no longer find themselves locked into tools and processes based on the particular set of programming skills they happen to have.

Take SQL as an example, a classic skill and the core language of databases. However, to then work with the data, a data scientist used to need to be proficient in a number of other languages, BI tools and ML solutions. No more: rather than extracting the data from the database, uploading it to a BI tool, later an ML tool, running the models, then loading the findings back into the BI tool, there’s a rapidly growing movement seeking to make the magic happen within the data layer, using the established and universal language of data, SQL. 

This example, from the field of in-database machine learning, shows how data scientists using standard SQL commands can now query predictions and build ML models from inside the database. Data pipelines are another example of a process that’s being radically simplified and speeded up thanks to companies like dbt Labs with its data transformation product based on SQL commands.  

Data scientists, who have long debated the relative merits of different programming languages and tools, are increasingly finding humble SQL can get them to their destination quickly and easily. 

Accelerating Transformation with In-Database Tools

It’s an uncomfortable truth for data scientists, but analytics can easily become the bottleneck that slows down digital transformation. There are good reasons behind this – shortage of skills, cost, complexity. Low-code data science tools radically speed up the process of extracting actionable intelligence from data. In the hands of an experienced data pro, the speed, simplicity and cost effectiveness of in-database low-code solutions bring extraordinary possibilities. By making vast amounts of data more accessible, data scientists need only to be constrained by their imagination, rather than the usual reasons things don’t get done (cost, integration issues, time concerns, skills gap). Low-code solutions are helping innovative data scientists shine as they deliver ROI faster and showcase the power of ML for a growing number of scenarios. 

They also help to increase an organization’s overall digital output. After all, the pandemic caused the world to pivot to digital solutions and there has been no going back. Three quarters of CIOs reported an increased demand for new digital products and services, which has continued to grow since. For data and IT teams that were already overworked, overstretch and super stressed, the availability of low-code solutions couldn’t have come at a better time. 

The Challenges of a Low-code World

Although the adoption of low-code data science tools is rising rapidly, it’s still early days for this type of development. Organizations are understandably worried about the security and compliance implications of non-experts handling data (and other) projects. As development and data science become increasingly decentralized, one of the biggest challenges will be for organizations to foster a culture where innovation can occur freely, but remains strategic, relevant, and secure. Most experts agree that company policies will need to be updated to reflect low-code practices.   

Low-code AI solutions, when used by data scientists, are a way of working more efficiently, but they could, indirectly, end up adding to their workload. While “citizen development” sounds like it operates independently of the IT team, the reality for data scientists will mean increased calls to help integrate, rescue or otherwise support digital tools that have been created elsewhere. 

Life as a Low-code Data Scientist

Low-code tools represent a major disruption to the way digital has been done so far. Like every disruptive force, it creates opportunities while threatening the status quo. 

For data scientists, low-code is now becoming a fact of life, a hybrid approach to AI development that should be embraced because of its undeniable speed and simplicity. Furthermore, democratization and decentralization of IT are unstoppable. For data scientists, the question now is not whether to adopt low-code, it is how best to do so. I have seen low code fast-track many data science careers. Understanding their field as they do, they already have an advantage over their non-data colleagues. In the hands of data scientists, low-code solutions help data scientists work faster and smarter while at the same time being more agile and creative.