Processing Massive Amounts of On Demand Data Without Crashing NodeJS Main Threadby@wesleymr7
334 reads

Processing Massive Amounts of On Demand Data Without Crashing NodeJS Main Thread

tldt arrow
Read on Terminal Reader
Read this story w/o Javascript

Too Long; Didn't Read

Node.js can have some problems processing big files because the main thread(responsible for the event loop) can stop. For this reason, I decided to write about how to use streams in node.js showing a funny example. Application downloads a huge CSV file from Google Cloud Storage, processes the data, transforms it into a JSON file, extracts some data insights, and finally compresses the file and uploads it again to the Cloud Storage.
featured image - Processing Massive Amounts of On Demand Data  Without Crashing NodeJS Main Thread
Wesley Miranda HackerNoon profile picture

@wesleymr7

Wesley Miranda


Receive Stories from @wesleymr7


Credibility

react to story with heart

RELATED STORIES

L O A D I N G
. . . comments & more!
Hackernoon hq - po box 2206, edwards, colorado 81632, usa