ExpressJS And Web Sockets: Streaming Tweets in Real-Time Without Any Problems

We are living in an age where data is considered to be more valuable than oil itself. Given that statement, any tool or mechanism that could help us mine and analyze data becomes a powerful resource to get your hands on. Data by itself has the potential to understand the universal behavior of people and assist businesses to design better use-cases. One such rich vein of data-source is Twitter and the data we can get from it concerning any particular topic is fascinating. Not to forget, one more thing which we would be exploring in this article is that we would be fetching data in real-time. Yes, you heard me right. We would be doing so in real-time.

Web Sockets — Real-Time Goodness at Your Fingertips

We have all used chat applications at one time or the other. The magic of sending real-time messages is achieved through something called web sockets, especially on the web (as the name suggests). This is the magic sauce we will be using in our use-case. If I have to describe what web sockets is in simple layman terms, it would go something like this.

Web sockets is a network technology that enables two-way communication between a user’s browser and a server in real-time. Which is to say that you can send and receive messages/information without constantly having to hit the concerned API.

If you wish to know more about it. You can learn about it here. Along with it, we would be using React as our client and Node with Express for our server.

Enough talk already. Let’s dive in.

Less Talk, More Code

There are four sections to be addressed for the code part before we can see the application in-action. These are:

1. Procuring Twitter Developer Application Credentials
2. Setting up the server
3. Configuring the client-side 
4. Witnessing the application come to life

Procuring Twitter Developer Application Credentials

Before we can start writing any actionable code, you should go ahead and sign in/sign up at the Twitter Developers site. Once you have done so, you can create a new personal application here. You will obtain your credentials from there. Copy it somewhere, maybe on a notepad, to keep it handy.

Note: Twitter employs a review process when signing up for a developer’s account. There might be a series of emails you will need to exchange with the review team before you are set up.

You have your credentials. Time to spin up the server.

Setting up the server

In order to stream the twitter data on the server-side, we would require two libraries in particular — Twit (handles streams and connections to Twitter) and Socket.io (helps create socket connection on the server).

Go ahead and install them in your express application.

npm install --save twit socket.io

Having installed the required dependencies, it is time to call the

socket.io 

./bin/www

tweetsRoute

In

tweets

We have reached the fun part of the application. The part where we start reading from twitter and emit the tweets obtained to the client designed. The code is pretty straightforward. Using the Twit object initialized we create a

stream

We primarily use two functions from the socket object. The

on

emit

stream

on

stream.start()

That takes care of the server-side essential. Let’s get our client in order.

Configuring the client-side

Compared to the server, the client is relatively easier to get in shape. We will be using

socket.io-client

1. We use

   socket.io-client

   to connect to the server-side socket. The client will keep attempting to form a connection if it fails to connect at any point in time.
2. The listen event is declared within

   componentDidMount()

   lifecycle function since it is triggered immediately after the DOM is mounted and is the ideal place to declare any generic listeners. It is done since we want the client to listen for any incoming tweet at all times.
3. The

   startStreaming()

   function triggers the emit event which basically indicates to the server to start streaming twitter data to the application.

With that, you have your client set up for receiving the streaming data. I have avoided drilling into the styling manner and the components to display the incoming data. It is not relevant to the topic in question. But you can check out the code. I have kept it super minimal.

After all that work, it is time to see the magic.

Witnessing the application come to life

Server-side in place? Check. React all configured to spin up the sockets? Check. Popcorn in the oven? Um. It is time to run the application and see the result.

We have a twitter streaming application to the ready. Go ahead and play around with it. There are a lot of opportunities in which this feature can be leveraged, both from a business as well as a personal perspective.

Pssst. I am disclosing the source code here. Okay?

https://github.com/tarique93102/article-snippets/tree/master/sockets-twitter-app

Wrap Up

I came across this particular usage in my quest to create a data-analysis application that could track any tweet pertaining to certain social, environmental and political situations around the world in real-time. Emphasis on the word ‘real-time’. You can, of course, go ahead and incorporate this solution for any use-case you deem fit. There are still so many things we can do to massively improve the current implementation.

Web Sockets is a powerful tool and if you know how to use it properly you should actually end up enjoying yourself. Next up, I would be focusing on creating identifiers for each connection and tailor the events according to the need of every individual user.

If you have any queries or feedback, feel free to drop them in the comments section or you can reach out to me on LinkedIn. 

Remember to use your code for good. Till next time. Peace.

Previously published at https://medium.com/@tarique_ejaz/web-sockets-streaming-tweets-in-real-time-with-no-fuss-fcb4749c2078