Laravel Zero/Dusk- Tracking twitter accounts

I was looking for a small application idea to finally test Laravel Zero.I decided to create something to track cryptocurrency Twitter accounts and to detect trends (and offer alerts on this data on Cryptalert).

So we will create a Laravel Zero application and integrate /laravel-console-dusk, a custom version of Laravel Dusk for Laravel Zero, to easily track the evolution of twitter accounts.

What is Laravel Zero ?

Laravel Zero was created by, and is maintained by Nuno Maduro, and is a micro-framework that provides an elegant starting point for your console application. It is an unofficial and customized version of Laravel optimized for building command-line applications.

What is Laravel Dusk ?

Laravel Dusk is a powerful browser automation tool for Laravel. With Dusk you can programmatically test your own applications or visit any website on the internet using a Chrome browser. Dusk can help you to automate repetitive tasks or scrape information from other websites for example.

NB : I discovered the work of Nuno during a meetup at Algolia in November 2018, he did an incredible job for the Laravel community, follow him here: https://twitter.com/enunomaduro

Installation of Laravel Zero and Dusk

Let’s start by installing Laravel Zero by following the documentation: https://laravel-zero.com/docs/installation/

composer create-project — prefer-dist laravel-zero/laravel-zero laravel-zero-dusk

Then install Laravel Dusk:

cd laravel-zero-dusk

php laravel-zero-dusk app:install console-dusk

We will store our data in a database, in Laravel Zero the Laravel’s Eloquent component is an add-on. We can install it like that:

php laravel-zero-dusk app:install database

Remember to change the credentials of your database in the config/database.php file.

Preparation of the database

We will create 2 tables to store our data:

• twitter_account: to save the accounts handles we want to track the statistics of
• twitter_account_data: to save the account statistics of the twitter_account table

So we create 2 migrations:

php laravel-zero-dusk make:migration twitter_account_table

With this content:

And:

php laravel-zero-dusk make:migration twitter_account_data_table

With this content:

We then create a Seeder to add the first accounts in the twitter_account table:

php laravel-zero-dusk make:seeder TwitterAccountSeeder

With this content:

We run our seeder:

php laravel-zero-dusk db:seed --class=TwitterAccountSeeder

Here we just add 4 twitter handles in our twitter_account table. You can do this with basic SQL requests or https://github.com/intonate/tinker-zero (a bridge that allows using laravel/tinker in Laravel Zero applications).

Creation of the command that will store twitter metrics in database

Commands in Laravel Zero are explained here: https://laravel-zero.com/docs/commands/

It’s very similar to the commands of Laravel: https://laravel.com/docs/5.7/artisan#writing-commands

Create a new command named GetTwitterData:

php laravel-zero-dusk make:command GetTwitterData

Here is the code of the command (See details below):

So what are we doing here?

• line 35: we are just getting every handles stored in the twitter_account table, if you create and run the same Seeder as me, you should get 4 handles (@taylorotwell, @enunomaduro, @martin_riedweg and @VitalikButerin)
• line 37: for each handle, we go to the corresponding account page, for example https://twitter.com/taylorotwell for taylorotwell
• line 44 and 47: we retrieve the number of followers and following with CSS selectors: https://laravel.com/docs/master/dusk#interacting-with-elements

The exact value is in the data-count attribute

• line 50 to 57: we store the data in our twitter_account_data table

We just have to test our command:

php laravel-zero-dusk insert:twitter_data

If all goes well you should see this in your console:

Console output

You can easily automate this command using Cron: https://laravel-zero.com/docs/task-scheduling/

In the code of my GetTwitterData command I have:

public function schedule(Schedule $schedule){$schedule->command(static::class)->everyThirtyMinutes();}

So my command will run every 30 minutes ✌🏻

The code is available here: https://github.com/MartinRdw/laravel-zero-dusk

Feel free to ask me your questions/bug reports in the comment section 🙂