Why Adopting NoSQL is Difficult for Beginners

With the increasing technological breakthroughs, traditional ways of storing data are being challenged. With the adoption of NoSQL by industry giants and the ever-increasing NoSQL database options, it is becoming a trend in considering NoSQL as alternatives to Relational Databases for mainstream software development. However, when it comes to Relational Databases, it has been there for several decades. It is not difficult in finding developers who have the knowledge and experience in writing SQL queries, designing entity relationships, normalizing schemas and ensuring . This is not the case with NoSQL where knowledge, expertise, and experience are scarce. ACID properties This could lead to wrong design decisions in using NoSQL where it doesn’t fit in. Another common mistake in adopting NoSQL happens when developers coming from a Relational Database background designs the NoSQL database adhering to fundamentals of Relational Databases. Although I‘m talking more about NoSQL in this article, I do believe Relational Databases have its strengths where it belongs. So if you are using Relational Databases as of now or going to use for a new software project, don’t worry, stick with your decision and it won’t die in the near foreseeable future. Also, bear in mind that NoSQL doesn’t mean ‘No SQL’, it means ‘Not only SQL’ . In this article, I’m going to highlight several points, which will be useful for beginners in adopting NoSQL. Why NoSQL is Difficult? In short, using NoSQL databases is not difficult. The difficulty comes in using it for the right places in the right way. First of all, it is important to understand that NoSQL doesn’t follow the same principles as Relational Databases such as fixed schemas, normalization, support for expressive queries like SQL. One common mistake developers do is trying to normalize the NoSQL database where they screw up the entire database design making it difficult to retrieve items efficiently. NoSQL databases drastically differ from its other counterparts depending on the type of NoSQL database (Document Database, Key-Value Database, Graph Database & etc.) and the vendor specific implementation ( , , & etc.). Because of this, there are no universal best practices or principals for designing database schema or for queries that fit all types of NoSQL databases. This creates the challenge in designing the unique to the respective NoSQL database type adhering to its own best practices. MongoDB DynamoDB Cassandra For example, if we consider a document database, we need to consider collection size limits,how are they partitioned, available query types, and indexing support so that it is possible to efficiently retrieve items (or nested attributes) required by the application. This is not an easy task since there are several places where things could go wrong. Some of the best practices with Relational Databases such as not storing everything in the database still applies for NoSQL. Although its possible to store files such as images, , etc. in certain NoSQL databases, it makes more sense in storing them in a distributed file system and then uses the database to store the file metadata. transcoded videos Duplicating Data Could Become a Best Practice If you are a developer having a mindset in saying no for data duplication at the database level, its high time to reconsider with NoSQL. With certain types of NoSQL databases, having data attributes duplicated across different tables or collections could become even a best practice to avoid complex queries. For instance, today storage is not that expensive, whereas processing power is more expensive in contrast. Therefore storing data in different forms utilizing more storage for direct retrieval might make sense rather performing computationally heavy queries. So you might wonder, why all these expensive queries in Relational Databases, rather storing the data and its relations with duplication in NoSQL where we could perform direct lookup and retrieval? The answer is that it is not that simple since creating or updating data in NoSQL could become more complex when data is scattered across many tables or collections. Therefore, it is important to evaluate how the data is being consumed and updated by the application and design the database accordingly. Querying NoSQL is Unique for Each NoSQL Database This is another area which is really confusing with NoSQL. Each of these NoSQL databases has their own query language. This is really challenging for developers who used to work with relational databases since SQL could be applicable across many relational database implementations with minor differences. With NoSQL, you have to refer the documentation to understand the specific data types that are supported, query and feature limitations. With certain types of the NoSQL databases (E.g; Certain Document Databases and Key-Value Databases), don’t be surprised by seeing that it’s only possible to query with only key attributes or using predefined indexes in contrast to Relational Databases where you can query almost from any attribute. Another important point to consider with NoSQL databases is the consistency models they support. Although some of these databases support strong consistency, others could be limited with eventual consistency. The consistency support also could vary depending on the type of query, whether its a read, write or an update. Transactions are Not Handled in the Same Way For most of the NoSQL databases, its difficult to create transactions similar to relational databases where the database provide higher level support for commits and rollbacks. Therefore, it is also important to identify the level of support from the NoSQL database to implement transactions. For most of the NoSQL databases, you will need to handle the transactions and rollback at the application level. However, many of these NoSQL databases provide some foundational features which will make your lives easier. For instance, certain NoSQL databases support features such as conditional writes and atomic updates that could be used to implement transactions efficiently. For example, you can create an update operation saying that, if the amount is ‘x’ amount only, do the item update (or throw an exception if its changed which needs to handled in the application accordingly. The advantage is that the collections or tables are not being locked while a transaction is happening which impacts positively for the database performance. Using Both NoSQL and Relational In my experience, many web and mobile application features require different query capabilities where its wise to start with a relational database. If your application requires special support for High Throughput, Availability, Scaling of Storage, you can consider using NoSQL databases for the relevant area of use in addition to the relational database. Over the past few years, adoption of Microservices has also made it logical in using multiple database types depending on the required application capabilities. For example, you can consider using an In-Memory Key-Value NoSQL database like Redis for Session Storage, using Relational Database for storing business entities & etc. So keeping the answer short, it is becoming a common practice in using different database types together to build applications rather sticking with one database for all. Conclusion There are many differences between NoSQL and Relational databases. Although we have discussed some challenges in using NoSQL there are many benefits in using NoSQL for some use cases. For example, few benefits include the capability in handling large amounts of data for storage, scalability, flexible data collections & etc. Therefore, it is important to do a sufficient background research, reach for expertise and having an open mindset before using NoSQL since some of the practices you have already learned on databases needs to be unlearned, regardless of how experienced you are.