Background So today, I will be talking about a Celery alternative named Huey, which comes with a much easier setup than Celery and is much smaller in size compared to Celery. The reason why I decided to try out Huey is because I have faced some issues with Celery sometimes for doing some common tasks because the documentation isn't too great. For those who don't know what Celery is or haven't used it before, Huey is an asynchronous task queue that allows you to perform scheduled tasks or long-running tasks in the background. Prerequisites We will be installing the following packages: redis Django Huey requests (optional, needed for the demo) GitHub Repo The following blog comes accompanied by a GitHub repo that you can use to test out the demo project that we will be creating. Click here to view the repo. Project Setup Create the Project Directory Open the terminal, and type the following to create a directory; you can skip this step and do it from File Explorer itself. mkdir huey_demo Virtual Environment Let us create a virtualenv first to install our project dependencies: python -m venv venv Activate the virtualenv (Linux): source venv/bin/activate Installing Dependencies Type the following command in the terminal to install all the dependencies: pip install Django==4.0.4 redis==4.2.2 huey==2.4.3 At the time of writing this article, these were the versions I tested out this setup with, keep an eye on the Github Repo for any updates as per the latest version in the future. Create the Project Create the Django project by typing the following command in the terminal: django-admin startproject django_huey_demo Change directory into Django project directory: cd django_huey_demo Create the app under our project: python manage.py startapp demo Include the created app in the project settings.py, make the following changes: INSTALLED_APPS = [ # Existing Apps "demo.apps.DemoConfig", # <== Add this line ] Set debug mode to False in settings.py: DEBUG=False We are setting Debug to False so that we can see how Huey runs in production, more on this later. Project Overview Now that we are done setting up our project, it is a good time to take you over what we will be building today. We will fetch "Word of the Day" daily from Wordnik API. Then we will store the word, its definition, and an example of the word in a sentence in our database. We will set up a periodic task using Huey which will fetch the Word of the Day and store it. For storing the word we will be creating a Django Model of the same. Getting the Wordnik API Key You can follow this guide to obtain the API key. Coding Our Project Add Huey to Our Project We need to add Huey to the installed apps of our project, so make the following changes in settings.py file: INSTALLED_APPS = [ # Existing apps 'huey.contrib.djhuey', # <== Add this line ] Install Redis We need to install Redis for Huey to store information about queued tasks in it like we used to do with Celery as well. You can refer to the following link to install Redis based on your specific operating system. If you're comfortable with using Docker, you can use the following command: docker run --name redis_huey -p 6379:6379 -d redis By default, Huey will try connecting to the Redis server running on localhost:6379. If it isn't present, it will raise an error. Model Definition Add the following code to your demo/models.py file: from django.db import models class Word(models.Model): word = models.CharField(max_length=200) part_of_speech = models.CharField(max_length=100) definition = models.TextField() example = models.TextField() def __str__(self): return self.word Make migrations: python manage.py makemigrations demo Apply migrations: python manage.py migrate demo Task Definition Create a file named tasks.py in the demo app directory. The reason why we named our file tasks.py is to help Huey auto-discover the tasks present in our registered apps, if we named our file anything other than that, we would have to manually register our task. If you would like to know more you can check out the Huey documentation here. Before we write the task definition, we need to install an additional dependency requests. Install it by typing the following in your terminal: pip install requests==2.27.1 Now, comes the code: import requests from django.conf import settings from huey import crontab from huey.contrib.djhuey import db_periodic_task from demo.models import Word @db_periodic_task(crontab(hour="18", minute="00")) def fetch_daily_word(): r = requests.get( f"https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key={settings.WORDNIK_API_KEY}") data = r.json() Word.objects.get_or_create( word=data["word"], part_of_speech=data["definitions"][0]["partOfSpeech"], definition=data["definitions"][0]["text"], example=data["examples"][0]["text"] ) Add the following line in your project settings: WORDNIK_API_KEY = "api-key-here" This code block might be a lot to take in, so let's go over the things in it one by one: Huey Decorator from huey.contrib.djhuey import db_periodic_task This is a decorator provided by Huey to register periodic tasks that involve working with the database, this decorator automatically closes the database connection upon task completion, for more details, you can refer here. Crontab Schedule @db_periodic_task(crontab(hour="18", minute="00")) We are passing the argument crontab(hour="18", minute="00") to our periodic task decorator, this tells Huey to run our task at 6 pm every day. You can make use of this website to create your crontab schedules, I use it every time. Wordnik API Key from django.conf import settings # Usage ## settings.WORDNIK_API_KEY from django.conf import settings is the standard way to import any data from our project settings, it is useful in cases where we have multiple settings files set up for different environments so it will know which file to pick from without us having to worry about it. It finds out which settings file we are using from the DJANGO_SETTINGS_MODULE environment variable. But you don't have to worry about these details. Then we use the key in our Wordnik API call. Wordnik API Call r = requests.get( f"https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key={settings.WORDNIK_API_KEY}") Here, we are making use of the requests module to make a GET request to the Wordnik API while passing our API Key for authentication. Storing word in the database data = r.json() Word.objects.get_or_create( word=data["word"], part_of_speech=data["definitions"][0]["partOfSpeech"], definition=data["definitions"][0]["text"], example=data["examples"][0]["text"] ) After parsing the API response, we are storing the word definition in our database. We are making use of get_or_create method instead of create method here so that we don't create multiple copies of the same word in our database if that word is ever repeated by the Wordnik API. Wordnik API Response Here is what the Wordnik API response for the Word of the Day endpoint looks like. Some of the irrelevant sections of the response have been truncated for brevity purposes. { "word": "stolon", "definitions": [ { "source": "ahd-5", "text": "A long thin stem that usually grows horizontally along the ground and produces roots and shoots at widely spaced nodes, as in a strawberry plant.", "note": null, "partOfSpeech": "noun" }, // More definitions here... ], "publishDate": "2022-05-08T03:00:00.000Z", "examples": [ { "title": "4.1 Nursery establishment", "text": "A stolon is a stem that grows along the ground, producing at its nodes new plants with roots and upright stems.", // Additional data here... }, // More examples here... ], // Additional fields here... } Running Huey Worker You can start the Huey worker by typing the following command in your terminal: python manage.py run_huey You can pass multiple flags to the above command which will change what gets logged to the console, such as: -v, --verbose - verbose logging (includes DEBUG level) -q, --quiet - minimal logging -S, --simple - simple logging format (“time message”) To look at various other options for logging, check out the docs here. What Else Can You Do With Huey? Task Decorators Huey comes with multiple task decorators depending on what operations you are performing within the task. I'll explain in brief what all of those do below. Here is the import statement for all the decorators: from huey.contrib.djhuey import task, periodic_task, db_task, db_periodic_task task: A regular task. periodic_task : When you want to run a task periodically based on a schedule. db_task : When you want to perform DB operations within your task. db_periodic_task : When you want to perform DB operations in a periodic task. Crontab Examples Let me show you some more examples of how you can use crontab to schedule your tasks. crontab(minute='*/3') would schedule the task to run every three minutes. crontab(hour='*/3', minute='5') would create a task that will run at 5 minutes past every third hour. crontab(minute='00', hour='10', month='*/2', day_of_week='*/5') would create a task that would run on every 5th day of the week, of every 2nd month at 10:00 AM. Scheduling Tasks For example, you have the following task defined inside tasks.py: from huey.contrib.djhuey import task @task() def count(): for i in range(10): print(i) Now, if you want to call this task but want it to run after 5 seconds, you can do the following: count.schedule(delay=5) delay parameter takes values in seconds, so if you want it to execute after 5 minutes, specify 300 seconds. Retrying Tasks That Fail Suppose you add the following logic to our existing task: @db_periodic_task(crontab(hour="18", minute="00"), retries=2) def fetch_daily_word(): r = requests.get( f"https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key={settings.WORDNIK_API_KEY}") if r.status_code != 200: raise Exception("Unable to fetch data from Wordnik API") ## Add this logic else: data = r.json() Word.objects.get_or_create( word=data["word"], part_of_speech=data["definitions"][0]["partOfSpeech"], definition=data["definitions"][0]["text"], example=data["examples"][0]["text"] ) So, we added the logic to check for the status code of the response, and if it is something other than 200, it will retry that task up to 2 times. But these retries would happen without any time gap between the two attempts. Now, what if you want to delay multiple attempts of this task? We can do that by passing the retry_delay argument, it accepts values in seconds. @db_periodic_task(crontab(hour="18", minute="00"), retries=2, retry_delay=10) This will cause a 10-second delay between multiple attempts. Development Mode Huey comes with a default setting which makes working with Huey during development in Django easier. So, whenever you have DEBUG=True present in your settings.py file, tasks will be executed synchronously just like regular function calls. The purpose of this is to avoid running both Redis and an additional consumer process while developing or running tests. You can read more about this here. For this, we need to add the following line in settings.py: HUEY = {} However, if you want to override this behavior, you can add the following Huey config instead: HUEY = { "immediate": False } If you have the above config mentioned in settings.py, while having DEBUG=True, Huey will require you to set up Redis and run Huey Worker using run_huey command. Celery vs Huey Some observations about Huey as compared to Celery are: Smaller footprint of dependencies as compared to Celery. Celery installs kombu and billiards along with it. Meanwhile, Huey doesn't have any dependencies. Lesser services need to be run for periodic tasks, Celery requires running beat service and a worker service to work with periodic tasks meanwhile we only need to run one service using the run_huey command. References Huey Docs Wordnik API Associated Github Repo Background So today, I will be talking about a Celery alternative named Huey , which comes with a much easier setup than Celery and is much smaller in size compared to Celery. Huey Celery The reason why I decided to try out Huey is because I have faced some issues with Celery sometimes for doing some common tasks because the documentation isn't too great. For those who don't know what Celery is or haven't used it before, Huey is an asynchronous task queue that allows you to perform scheduled tasks or long-running tasks in the background. Prerequisites We will be installing the following packages: redis Django Huey requests (optional, needed for the demo) redis Django Huey requests (optional, needed for the demo) GitHub Repo The following blog comes accompanied by a GitHub repo that you can use to test out the demo project that we will be creating. Click here to view the repo. Click here to view the repo. Project Setup Create the Project Directory Open the terminal, and type the following to create a directory; you can skip this step and do it from File Explorer itself. mkdir huey_demo mkdir huey_demo Virtual Environment Let us create a virtualenv first to install our project dependencies: python -m venv venv Activate the virtualenv (Linux): source venv/bin/activate Let us create a virtualenv first to install our project dependencies: python -m venv venv Let us create a virtualenv first to install our project dependencies: python -m venv venv python -m venv venv Activate the virtualenv (Linux): source venv/bin/activate Activate the virtualenv (Linux): source venv/bin/activate source venv/bin/activate Installing Dependencies Type the following command in the terminal to install all the dependencies: pip install Django==4.0.4 redis==4.2.2 huey==2.4.3 pip install Django==4.0.4 redis==4.2.2 huey==2.4.3 At the time of writing this article, these were the versions I tested out this setup with, keep an eye on the Github Repo for any updates as per the latest version in the future. Create the Project Create the Django project by typing the following command in the terminal: django-admin startproject django_huey_demo Change directory into Django project directory: cd django_huey_demo Create the app under our project: python manage.py startapp demo Include the created app in the project settings.py, make the following changes: INSTALLED_APPS = [ # Existing Apps "demo.apps.DemoConfig", # <== Add this line ] Set debug mode to False in settings.py: DEBUG=False We are setting Debug to False so that we can see how Huey runs in production, more on this later. Create the Django project by typing the following command in the terminal: django-admin startproject django_huey_demo Create the Django project by typing the following command in the terminal: django-admin startproject django_huey_demo django-admin startproject django_huey_demo Change directory into Django project directory: cd django_huey_demo Change directory into Django project directory: cd django_huey_demo cd django_huey_demo Create the app under our project: python manage.py startapp demo Create the app under our project: python manage.py startapp demo python manage.py startapp demo Include the created app in the project settings.py, make the following changes: INSTALLED_APPS = [ # Existing Apps "demo.apps.DemoConfig", # <== Add this line ] Include the created app in the project settings.py , make the following changes: settings.py INSTALLED_APPS = [ # Existing Apps "demo.apps.DemoConfig", # <== Add this line ] INSTALLED_APPS = [ # Existing Apps "demo.apps.DemoConfig", # <== Add this line ] Set debug mode to False in settings.py: DEBUG=False We are setting Debug to False so that we can see how Huey runs in production, more on this later. Set debug mode to False in settings.py : False settings.py DEBUG=False DEBUG=False We are setting Debug to False so that we can see how Huey runs in production, more on this later. Project Overview Now that we are done setting up our project, it is a good time to take you over what we will be building today. We will fetch "Word of the Day" daily from Wordnik API . Then we will store the word, its definition, and an example of the word in a sentence in our database. Wordnik API We will set up a periodic task using Huey which will fetch the Word of the Day and store it. For storing the word we will be creating a Django Model of the same. Getting the Wordnik API Key You can follow this guide to obtain the API key. this guide Coding Our Project Add Huey to Our Project We need to add Huey to the installed apps of our project, so make the following changes in settings.py file: settings.py INSTALLED_APPS = [ # Existing apps 'huey.contrib.djhuey', # <== Add this line ] INSTALLED_APPS = [ # Existing apps 'huey.contrib.djhuey', # <== Add this line ] Install Redis We need to install Redis for Huey to store information about queued tasks in it like we used to do with Celery as well. You can refer to the following link to install Redis based on your specific operating system. link If you're comfortable with using Docker, you can use the following command: docker run --name redis_huey -p 6379:6379 -d redis docker run --name redis_huey -p 6379:6379 -d redis By default, Huey will try connecting to the Redis server running on localhost:6379 . If it isn't present, it will raise an error. localhost:6379 Model Definition Add the following code to your demo/models.py file: from django.db import models class Word(models.Model): word = models.CharField(max_length=200) part_of_speech = models.CharField(max_length=100) definition = models.TextField() example = models.TextField() def __str__(self): return self.word Make migrations: python manage.py makemigrations demo Apply migrations: python manage.py migrate demo Add the following code to your demo/models.py file: from django.db import models class Word(models.Model): word = models.CharField(max_length=200) part_of_speech = models.CharField(max_length=100) definition = models.TextField() example = models.TextField() def __str__(self): return self.word Add the following code to your demo/models.py file: demo/models.py from django.db import models class Word(models.Model): word = models.CharField(max_length=200) part_of_speech = models.CharField(max_length=100) definition = models.TextField() example = models.TextField() def __str__(self): return self.word from django.db import models class Word(models.Model): word = models.CharField(max_length=200) part_of_speech = models.CharField(max_length=100) definition = models.TextField() example = models.TextField() def __str__(self): return self.word Make migrations: python manage.py makemigrations demo Make migrations: python manage.py makemigrations demo python manage.py makemigrations demo Apply migrations: python manage.py migrate demo Apply migrations: python manage.py migrate demo python manage.py migrate demo Task Definition Create a file named tasks.py in the demo app directory. The reason why we named our file tasks.py is to help Huey auto-discover the tasks present in our registered apps, if we named our file anything other than that, we would have to manually register our task. If you would like to know more you can check out the Huey documentation here . tasks.py tasks.py here Before we write the task definition, we need to install an additional dependency requests . Install it by typing the following in your terminal: requests pip install requests==2.27.1 pip install requests==2.27.1 Now, comes the code: import requests from django.conf import settings from huey import crontab from huey.contrib.djhuey import db_periodic_task from demo.models import Word @db_periodic_task(crontab(hour="18", minute="00")) def fetch_daily_word(): r = requests.get( f"https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key={settings.WORDNIK_API_KEY}") data = r.json() Word.objects.get_or_create( word=data["word"], part_of_speech=data["definitions"][0]["partOfSpeech"], definition=data["definitions"][0]["text"], example=data["examples"][0]["text"] ) import requests from django.conf import settings from huey import crontab from huey.contrib.djhuey import db_periodic_task from demo.models import Word @db_periodic_task(crontab(hour="18", minute="00")) def fetch_daily_word(): r = requests.get( f"https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key={settings.WORDNIK_API_KEY}") data = r.json() Word.objects.get_or_create( word=data["word"], part_of_speech=data["definitions"][0]["partOfSpeech"], definition=data["definitions"][0]["text"], example=data["examples"][0]["text"] ) Add the following line in your project settings: WORDNIK_API_KEY = "api-key-here" WORDNIK_API_KEY = "api-key-here" This code block might be a lot to take in, so let's go over the things in it one by one: Huey Decorator from huey.contrib.djhuey import db_periodic_task This is a decorator provided by Huey to register periodic tasks that involve working with the database, this decorator automatically closes the database connection upon task completion, for more details, you can refer here. Crontab Schedule @db_periodic_task(crontab(hour="18", minute="00")) We are passing the argument crontab(hour="18", minute="00") to our periodic task decorator, this tells Huey to run our task at 6 pm every day. You can make use of this website to create your crontab schedules, I use it every time. Wordnik API Key from django.conf import settings # Usage ## settings.WORDNIK_API_KEY from django.conf import settings is the standard way to import any data from our project settings, it is useful in cases where we have multiple settings files set up for different environments so it will know which file to pick from without us having to worry about it. It finds out which settings file we are using from the DJANGO_SETTINGS_MODULE environment variable. But you don't have to worry about these details. Then we use the key in our Wordnik API call. Wordnik API Call r = requests.get( f"https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key={settings.WORDNIK_API_KEY}") Here, we are making use of the requests module to make a GET request to the Wordnik API while passing our API Key for authentication. Storing word in the database data = r.json() Word.objects.get_or_create( word=data["word"], part_of_speech=data["definitions"][0]["partOfSpeech"], definition=data["definitions"][0]["text"], example=data["examples"][0]["text"] ) After parsing the API response, we are storing the word definition in our database. We are making use of get_or_create method instead of create method here so that we don't create multiple copies of the same word in our database if that word is ever repeated by the Wordnik API. Wordnik API Response Here is what the Wordnik API response for the Word of the Day endpoint looks like. Some of the irrelevant sections of the response have been truncated for brevity purposes. { "word": "stolon", "definitions": [ { "source": "ahd-5", "text": "A long thin stem that usually grows horizontally along the ground and produces roots and shoots at widely spaced nodes, as in a strawberry plant.", "note": null, "partOfSpeech": "noun" }, // More definitions here... ], "publishDate": "2022-05-08T03:00:00.000Z", "examples": [ { "title": "4.1 Nursery establishment", "text": "A stolon is a stem that grows along the ground, producing at its nodes new plants with roots and upright stems.", // Additional data here... }, // More examples here... ], // Additional fields here... } Huey Decorator from huey.contrib.djhuey import db_periodic_task This is a decorator provided by Huey to register periodic tasks that involve working with the database, this decorator automatically closes the database connection upon task completion, for more details, you can refer here. Huey Decorator Huey Decorator from huey.contrib.djhuey import db_periodic_task from huey.contrib.djhuey import db_periodic_task This is a decorator provided by Huey to register periodic tasks that involve working with the database, this decorator automatically closes the database connection upon task completion, for more details, you can refer here. here. Crontab Schedule @db_periodic_task(crontab(hour="18", minute="00")) We are passing the argument crontab(hour="18", minute="00") to our periodic task decorator, this tells Huey to run our task at 6 pm every day. You can make use of this website to create your crontab schedules, I use it every time. Crontab Schedule Crontab Schedule @db_periodic_task(crontab(hour="18", minute="00")) @db_periodic_task(crontab(hour="18", minute="00")) We are passing the argument crontab(hour="18", minute="00") to our periodic task decorator, this tells Huey to run our task at 6 pm every day. You can make use of this website to create your crontab schedules, I use it every time. crontab(hour="18", minute="00") this website Wordnik API Key from django.conf import settings # Usage ## settings.WORDNIK_API_KEY from django.conf import settings is the standard way to import any data from our project settings, it is useful in cases where we have multiple settings files set up for different environments so it will know which file to pick from without us having to worry about it. It finds out which settings file we are using from the DJANGO_SETTINGS_MODULE environment variable. But you don't have to worry about these details. Then we use the key in our Wordnik API call. Wordnik API Key Wordnik API Key from django.conf import settings # Usage ## settings.WORDNIK_API_KEY from django.conf import settings # Usage ## settings.WORDNIK_API_KEY from django.conf import settings is the standard way to import any data from our project settings, it is useful in cases where we have multiple settings files set up for different environments so it will know which file to pick from without us having to worry about it. It finds out which settings file we are using from the DJANGO_SETTINGS_MODULE environment variable. But you don't have to worry about these details. from django.conf import settings DJANGO_SETTINGS_MODULE Then we use the key in our Wordnik API call. Wordnik API Call r = requests.get( f"https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key={settings.WORDNIK_API_KEY}") Here, we are making use of the requests module to make a GET request to the Wordnik API while passing our API Key for authentication. Wordnik API Call Wordnik API Call r = requests.get( f"https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key={settings.WORDNIK_API_KEY}") r = requests.get( f"https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key={settings.WORDNIK_API_KEY}") Here, we are making use of the requests module to make a GET request to the Wordnik API while passing our API Key for authentication. Storing word in the database data = r.json() Word.objects.get_or_create( word=data["word"], part_of_speech=data["definitions"][0]["partOfSpeech"], definition=data["definitions"][0]["text"], example=data["examples"][0]["text"] ) After parsing the API response, we are storing the word definition in our database. We are making use of get_or_create method instead of create method here so that we don't create multiple copies of the same word in our database if that word is ever repeated by the Wordnik API. Storing word in the database Storing word in the database data = r.json() Word.objects.get_or_create( word=data["word"], part_of_speech=data["definitions"][0]["partOfSpeech"], definition=data["definitions"][0]["text"], example=data["examples"][0]["text"] ) data = r.json() Word.objects.get_or_create( word=data["word"], part_of_speech=data["definitions"][0]["partOfSpeech"], definition=data["definitions"][0]["text"], example=data["examples"][0]["text"] ) After parsing the API response, we are storing the word definition in our database. We are making use of get_or_create method instead of create method here so that we don't create multiple copies of the same word in our database if that word is ever repeated by the Wordnik API. get_or_create create Wordnik API Response Here is what the Wordnik API response for the Word of the Day endpoint looks like. Some of the irrelevant sections of the response have been truncated for brevity purposes. { "word": "stolon", "definitions": [ { "source": "ahd-5", "text": "A long thin stem that usually grows horizontally along the ground and produces roots and shoots at widely spaced nodes, as in a strawberry plant.", "note": null, "partOfSpeech": "noun" }, // More definitions here... ], "publishDate": "2022-05-08T03:00:00.000Z", "examples": [ { "title": "4.1 Nursery establishment", "text": "A stolon is a stem that grows along the ground, producing at its nodes new plants with roots and upright stems.", // Additional data here... }, // More examples here... ], // Additional fields here... } Wordnik API Response Wordnik API Response Here is what the Wordnik API response for the Word of the Day endpoint looks like. Some of the irrelevant sections of the response have been truncated for brevity purposes. { "word": "stolon", "definitions": [ { "source": "ahd-5", "text": "A long thin stem that usually grows horizontally along the ground and produces roots and shoots at widely spaced nodes, as in a strawberry plant.", "note": null, "partOfSpeech": "noun" }, // More definitions here... ], "publishDate": "2022-05-08T03:00:00.000Z", "examples": [ { "title": "4.1 Nursery establishment", "text": "A stolon is a stem that grows along the ground, producing at its nodes new plants with roots and upright stems.", // Additional data here... }, // More examples here... ], // Additional fields here... } { "word": "stolon", "definitions": [ { "source": "ahd-5", "text": "A long thin stem that usually grows horizontally along the ground and produces roots and shoots at widely spaced nodes, as in a strawberry plant.", "note": null, "partOfSpeech": "noun" }, // More definitions here... ], "publishDate": "2022-05-08T03:00:00.000Z", "examples": [ { "title": "4.1 Nursery establishment", "text": "A stolon is a stem that grows along the ground, producing at its nodes new plants with roots and upright stems.", // Additional data here... }, // More examples here... ], // Additional fields here... } Running Huey Worker You can start the Huey worker by typing the following command in your terminal: python manage.py run_huey python manage.py run_huey You can pass multiple flags to the above command which will change what gets logged to the console, such as: -v, --verbose - verbose logging (includes DEBUG level) -q, --quiet - minimal logging -S, --simple - simple logging format (“time message”) -v, --verbose - verbose logging (includes DEBUG level) -v, --verbose -q, --quiet - minimal logging -q, --quiet -S, --simple - simple logging format (“time message”) -S, --simple To look at various other options for logging, check out the docs here . here What Else Can You Do With Huey? Task Decorators Huey comes with multiple task decorators depending on what operations you are performing within the task. I'll explain in brief what all of those do below. Here is the import statement for all the decorators: from huey.contrib.djhuey import task, periodic_task, db_task, db_periodic_task from huey.contrib.djhuey import task, periodic_task, db_task, db_periodic_task task: A regular task. periodic_task : When you want to run a task periodically based on a schedule. db_task : When you want to perform DB operations within your task. db_periodic_task : When you want to perform DB operations in a periodic task. task : A regular task. task periodic_task : When you want to run a task periodically based on a schedule. periodic_task db_task : When you want to perform DB operations within your task. db_task db_periodic_task : When you want to perform DB operations in a periodic task. db_periodic_task Crontab Examples Let me show you some more examples of how you can use crontab to schedule your tasks. crontab(minute='*/3') would schedule the task to run every three minutes. crontab(hour='*/3', minute='5') would create a task that will run at 5 minutes past every third hour. crontab(minute='00', hour='10', month='*/2', day_of_week='*/5') would create a task that would run on every 5th day of the week, of every 2nd month at 10:00 AM. crontab(minute='*/3') would schedule the task to run every three minutes. crontab(minute='*/3') crontab(hour='*/3', minute='5') would create a task that will run at 5 minutes past every third hour. crontab(hour='*/3', minute='5') crontab(minute='00', hour='10', month='*/2', day_of_week='*/5') would create a task that would run on every 5th day of the week, of every 2nd month at 10:00 AM. crontab(minute='00', hour='10', month='*/2', day_of_week='*/5') Scheduling Tasks For example, you have the following task defined inside tasks.py : tasks.py from huey.contrib.djhuey import task @task() def count(): for i in range(10): print(i) from huey.contrib.djhuey import task @task() def count(): for i in range(10): print(i) Now, if you want to call this task but want it to run after 5 seconds, you can do the following: count.schedule(delay=5) count.schedule(delay=5) delay parameter takes values in seconds, so if you want it to execute after 5 minutes, specify 300 seconds. delay Retrying Tasks That Fail Suppose you add the following logic to our existing task: @db_periodic_task(crontab(hour="18", minute="00"), retries=2) def fetch_daily_word(): r = requests.get( f"https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key={settings.WORDNIK_API_KEY}") if r.status_code != 200: raise Exception("Unable to fetch data from Wordnik API") ## Add this logic else: data = r.json() Word.objects.get_or_create( word=data["word"], part_of_speech=data["definitions"][0]["partOfSpeech"], definition=data["definitions"][0]["text"], example=data["examples"][0]["text"] ) @db_periodic_task(crontab(hour="18", minute="00"), retries=2) def fetch_daily_word(): r = requests.get( f"https://api.wordnik.com/v4/words.json/wordOfTheDay?api_key={settings.WORDNIK_API_KEY}") if r.status_code != 200: raise Exception("Unable to fetch data from Wordnik API") ## Add this logic else: data = r.json() Word.objects.get_or_create( word=data["word"], part_of_speech=data["definitions"][0]["partOfSpeech"], definition=data["definitions"][0]["text"], example=data["examples"][0]["text"] ) So, we added the logic to check for the status code of the response, and if it is something other than 200, it will retry that task up to 2 times. But these retries would happen without any time gap between the two attempts. Now, what if you want to delay multiple attempts of this task? We can do that by passing the retry_delay argument, it accepts values in seconds. retry_delay @db_periodic_task(crontab(hour="18", minute="00"), retries=2, retry_delay=10) @db_periodic_task(crontab(hour="18", minute="00"), retries=2, retry_delay=10) This will cause a 10-second delay between multiple attempts. Development Mode Huey comes with a default setting which makes working with Huey during development in Django easier. So, whenever you have DEBUG=True present in your settings.py file, tasks will be executed synchronously just like regular function calls. The purpose of this is to avoid running both Redis and an additional consumer process while developing or running tests. You can read more about this here . DEBUG=True settings.py here For this, we need to add the following line in settings.py : settings.py HUEY = {} HUEY = {} However, if you want to override this behavior, you can add the following Huey config instead: HUEY = { "immediate": False } HUEY = { "immediate": False } If you have the above config mentioned in settings.py , while having DEBUG=True , Huey will require you to set up Redis and run Huey Worker using run_huey command. settings.py DEBUG=True run_huey Celery vs Huey Some observations about Huey as compared to Celery are: Smaller footprint of dependencies as compared to Celery. Celery installs kombu and billiards along with it. Meanwhile, Huey doesn't have any dependencies. Smaller footprint of dependencies as compared to Celery. Celery installs kombu and billiards along with it. Meanwhile, Huey doesn't have any dependencies. Lesser services need to be run for periodic tasks, Celery requires running beat service and a worker service to work with periodic tasks meanwhile we only need to run one service using the run_huey command. Lesser services need to be run for periodic tasks, Celery requires running beat service and a worker service to work with periodic tasks meanwhile we only need to run one service using the run_huey command. run_huey References Huey Docs Wordnik API Associated Github Repo Huey Docs Huey Docs Wordnik API Wordnik API Associated Github Repo Associated Github Repo
