Too Long; Didn't Read
The trick is to not load the whole file into memory and keep processing each word as we keep on moving the pointer of the file. With this, we can easily process the whole. file with a minimal amount of memory resources. The solution is we keep multiple pointers at different parts of the. file and each thread. reads chunks of. the file concurrently. We are using channels to unify all the words read by various threads into a single dictionary. The time required to process the 1GB file was halved by doing it in a serial manner.