How SQL Can Eat Enterprise Software

Most B2B applications include a database to store data and an API that interacts with the database to serve and provide an interface to modify the data. Most customers sync their data from their databases to your APIs, which act as an intermediary layer between your database and the customer database.

They may connect data from Excel sheets, data warehouses, CRM, and marketing applications, all of which store data in their database.

While this additional ETL (Extract, Transform, Load) process usually helps isolate different environments and allows you to transform the data in the way you want, it adds an extra layer of complexity in dealing with data stale issues, which becomes a headache for real-time and data-intensive applications.

Cloud data warehouses such as Snowflake and BigQuery address this issue through their data marketplace offerings. Here's how it works:

1. The customer provides data to the same cloud provider (and region for Snowflake Marketplace implementation) as a database linked to your cloud account.

   
   

2. You gain read-only access to customer data, transform it for your use case, and display it in a user interface.

   
   

   1. One of the most common use cases is analytics tools that pre-aggregate your raw data to show some insights, or marketing tools that unify marketing data and gather some statistics.

      
      

3. You combine customer data with your own or third-party data.

   
   

   1. Clean Room Providers allows you to enrich your user data for use cases such as measurement and activation.

      
      

4. You return your data to your customers so that they can use it in their other internal systems as database relations such as tables.

   
   

   1. Analytics providers like Mixpanel and Amplitude have a RESTFul API and also integrate with Snowflake to provide you with access to your event data.

      
      

Reverse ETL tools like Hightouch and Census help cover internal company and provide a variety of out-of-the-box integrations, but BigQuery and Snowflake are also developing a way for you to sync your data with your [Salesforce]() and AWS is developing tools for you to sync your data from your application database to Redshift with Zero ETL.

The data is centralized, and databases have evolved into new APIs that allow you to bundle your application as a collection of SQL scripts. While SQL is an excellent language, it is difficult to write for specific cases such as machine learning, so you can mount your Python and Java code as a user-defined function that can be accessed in SQL.

As a result, SQL becomes the primary interface for your application, with database relations serving as the primary primitive.

Transformation From Analytics to Full Stack Engineers

In today's rapidly changing technological landscape, I believe analytics engineers will be the new full-stack engineers developing B2B applications, moving beyond BI and internal applications to develop customer-facing applications. This is why I'm developing Jinjat.

Jinjat is a pioneering platform that can convert dbt projects into web apps, representing a paradigm shift in how engineers interact with data and develop web applications. Jinjat enables engineers to concentrate on SQL as the primary data interface.

This elevates SQL from a data querying language to a full-fledged tool in application development, allowing engineers to fully master it.

Furthermore, Jinjat supports the use of MDX (Multidimensional Expressions) for web page development, signaling a shift in web development practices. Engineers can now use MDX to build sophisticated web interfaces by referencing components in their markdown files, seamlessly blending data analytics and front-end development.

Jinjat's hybrid nature is especially advantageous for analytics engineers who may lack extensive front-end development skills. Initially, they can build applications using a common set of components. They can work with front-end developers as their needs evolve and become more complex.

This collaboration is critical because it fills knowledge gaps and ensures that the applications developed are not only data-rich but also user-friendly and visually appealing.

Using Marketplace as a Marketing Platform

Using cloud provider marketplaces in a marketing strategy can be a successful approach, particularly for developer-focused teams that excel at product development over traditional marketing tactics. If you bring compute resources to cloud providers, they will reward you with commission and increased visibility in their marketplace.

It's a win-win situation: cloud providers gain more diverse offerings in their ecosystem, and you benefit financially as well as gain increased exposure.

Snowflake is a prime example of this model in action. It has set up a commission structure to entice developers to host their applications on its platform. Developers benefit not only from Snowflake's robust cloud capabilities but also from the financial incentives provided.

Cloud provider marketplaces are essentially digital storefronts where you can showcase your applications or services. Being present in a cloud provider's marketplace increases your visibility among potential customers who use cloud infrastructure.

This targeted exposure is especially beneficial for developer-focused teams that may lack extensive marketing resources or expertise. It enables them to reach a targeted audience with little marketing effort.

In addition, being featured in a well-known cloud provider's marketplace lends credibility to your product. It shows potential customers that the cloud provider trusts and validates your application or service, which can be a powerful endorsement in the tech industry.

Build on Top of SQL and OpenAPI

Building on SQL and OpenAPI is a winning strategy, especially given their popularity and the current technological ecosystem. SQL, one of the most widely taught programming languages in college, provides a solid foundation for data manipulation and management.

OpenAPI, on the other hand, is well-known for its widespread use in designing and describing REST APIs. It complements SQL by facilitating simple and standardized API creation.

Large Language Models (LLMs) such as ChatGPT work well with SQL and OpenAPI as well. The reason for this is that LLMs are trained on diverse datasets that include extensive examples of SQL queries and OpenAPI specifications, allowing them to understand and generate complex queries and API descriptions, making them valuable tools for developers.

ChatGPT functions or plugins are already OpenAPI-based, and "LLM to SQL" is likely more of a "feature" at this point, implemented in a variety of BI tools.

While I experiment with this new concept, I will be dogfooding Jinjat to create a variety of applications ranging from ChatGPT Plugins to Admin tools or Notebooks to demonstrate various use cases.

Through these experiments, I was able to adapt Jinjat to new scenarios and real-world applications. This can also demonstrate Jinjat's potential and the advantages of building on SQL and OpenAPI.

Conclusion

In this article, we looked at new ways to use SQL to easily build applications for engineers with little experience. Platforms like Jinjat can be extremely beneficial in this regard, transforming BI analysts into full-stack engineers.

Furthermore, cloud provider marketplaces such as Snowflake can be an effective marketing tool, allowing businesses to thrive without incurring large marketing costs.

Last but not least, the power of LLM combined with SQL and OpenAPI can be used by developers to create Plugins for Admin tools or Notebooks for a variety of use-case scenarios.