You are reading the third part of a three-part series. It is highly encouraged to that you read the previous two articles before proceeding with this part! Flix-Finder [Part 1/3] Flix-Finder [Part 2/3] By the end of the previous blog, Flix-Finder supported movie searches based on rating parameters from various agencies. The functionality of our app met expectations and achieved its intended function. But after a while, Flix-Finder's customers began to gripe that the app did not provide the most recent information about films. Because Flix-Finder employs static data, this is something we anticipate. Flix-Finder needs some process through which its movie data can be updated to reflect the new movies. And, this is something we would be implementing in this final part! Despite the fact that there are several ways to implement this functionality, we'll be creating a new endpoint that can be used to update the data. Your data would change instantaneously, much as when you press a button. But there are other strategies as well, such utilising a cronjob. Once the specifics of one strategy are described in this blog, any new technique may be easily created using the same concepts. Planning it all out… Before we start to code, let us take a step back and understand all our requirements. We want to create a new endpoint and hitting that endpoint should update the data Flix-Finder uses. Let us use endpoint and any hitting this endpoint should update the data our app uses. /update GET request With the interface planned out, let us now think about ‘ . ‘ How to update the data? The process of updating the data can be broken into three steps: (1) Re-Run the Collectors on Bright Data (2) Fetch the Results obtained from running the collector (3) Overwrite the results to static files For the first two steps, we can use the APIs provided by Bright Data. Once we have the data, we can use vanilla Python to overwrite the existing static files. Great! With everything planned out, let us start the implementation. Adding the new Endpoint to Flix-Finder Start by creating a new method inside the file of recommender app in . We will call this method . This is where all our logic of updating the static data will reside. view.py Django updateData # recommender/views.py def update(request): # TODO: Implement this method return HttpResponse("Movie Data Updated!") Let us set up our views as well. We need to add the mapping for the newly created method in views. # recommender/urls.py from django.urls import path from . import views urlpatterns = [ path("", views.index, name = "index"), path("update", views.updateData, name = "update data") ] With this, we have successfully created a new endpoint in Flix-Finder. We will be using this endpoint to update the static data. For now, this endpoint does not do anything and simply returns a String message. ➜ ~ curl localhost:8000/update Movie Data Updated! Generating API Token on Bright Data Before we can start using Bright Data’s API for data update, we need to generate an API Token. All communications to Bright Data should contain this token and it is used for authentication purposes. Steps to generate API Token : Head over to . https://brightdata.com/cp/setting In the API Token section, click on button. Add Token In the dialog box, select the user, the permission and the expiration date. Click on the save button and enter the 2FA code. Copy and securely keep the generated token. Make sure to not leak your API Token. Anyone with access to your API Token can literally mimic you on Bright Data!!! Once the API Token is generated, we are ready to start using the APIs for running the collectors and fetching data from collector runs. APIs for Initiating Collectors Head over to Initiate by API section of your collectors. Here you will find curl requests that you can use to re-run collectors and fetch the results. Bright Data provides an endpoint to initiate collector run. The endpoint expects a POST request. It also expects a query parameter ‘ ’ that is used to uniquely identify the collector. The header should contain the API token. API to initiate collector run: https://api.brightdata.com/dca/trigger_immediate collector Here is the curl request for the same. curl -H "Authorization: Bearer API_TOKEN" -H "Content-Type: application/json" -d '{}' "https://api.brightdata.com/dca/trigger_immediate?collector=c_rj3i0wspnhr0ataba" The response of this endpoint is a response ID. This can be used to later fetch the results. With the response ID obtained from running a collector, Bright Data provides us with another API to fetch results from the collector run. The endpoint expects a GET request. It also expects a query parameter ‘response_id’ that is used to uniquely identify the collector. The header should contain the API token. API to fetch results from a collector’s run: https://api.brightdata.com/dca/get_result Here is how the curl request for this endpoint would look. curl "https://api.brightdata.com/dca/get_result?response_id=RESPONSE_ID" -H "Authorization: Bearer API_TOKEN" Implementing the update endpoint With the knowledge of endpoints to use and API token handy, we can start implementing the update endpoint. Let us implement this endpoint according to the subtasks we identified initially. (1) Re-Run the Collectors on Bright Data (2) Fetch the Results obtained from running the collector (3) Overwrite the results to static files First we need to use the re-run collector API and get a response ID. We will be using Python’s requests library to perform this task. API_TOKEN = "###########" # Replace by your API Token response = requests.post( "https://api.brightdata.com/dca/trigger_immediate?collector=c_rj3i0wspnhr0ataba", headers={"Authorization": f"Bearer {API_TOKEN}", "Content-Type": "application/json"}, json={} ) if response.status_code < 200 or response.status_code > 299 or 'response_id' not in response.json(): return HttpResponseServerError('Something Wrong Happened') response_id = response.json().get('response_id') Next step is to use the response ID to fetch the results. We would wait for some time before trying to fetch the results. time.sleep(30) # Sleeping for 30 seconds get_results_response = requests.get( f"https://api.brightdata.com/dca/get_result?response_id={response_id}", headers={"Authorization": f"Bearer {API_TOKEN}"} ) With data in hand, we can overwrite the static file with this new data. movie_data = get_results_response.json() data_file = open(ROTTEN_TOMATO_DATA_FILE_PATH, encoding="utf8", mode="w") json.dump(movie_data, data_file, indent=4) data_file.close() With that the implementation of the update method is completed. Using the new endpoint A simple curl request can be made to the endpoint to initiate the data updation process. Once the API returns, the data would have been updated. localhost:8000/update ➜ ~ curl localhost:8000/update Movie Data Updated! Concluding Remarks Finally, we have reached the conclusion of this section. In the Flix-Finder, we were able to integrate data updating functionality. As a result, our Flix-Finder will always be available to its users with fresh data! I hope you liked the three-part series. This contains the whole collection's source code. Github repository You can for more such amazing articles. See you in the next article… Till then, Happy Learning! 🙂 follow me on HackerNoon
![featured image - Flix-Finder: Building a Movie Recommendation App With Django and Bright Data [Part 3/3]](https://hackernoon.imgix.net/images/hxgB3A2KjtPk4sJXB1ezGnusKBx1-k9935ab.jpeg?auto=format&fit=max&w=3840)
