Processing Massive Amounts of On Demand Data Without Crashing NodeJS Main Thread
Too Long; Didn't Read
Node.js can have some problems processing big files because the main thread(responsible for the event loop) can stop. For this reason, I decided to write about how to use streams in node.js showing a funny example. Application downloads a huge CSV file from Google Cloud Storage, processes the data, transforms it into a JSON file, extracts some data insights, and finally compresses the file and uploads it again to the Cloud Storage.