Mohtadi Ben Fraj

@mohtedibf

Metflix: Because You Watched X

Netflix because you watched feature

Where are we at?

This is what we did so far

  • In part 0, we downloaded our data from MovieLens, did some EDA and created our user item matrix. The matrix has 671 unique users, 9066 unique movies and is 98.35% sparse
  • In part 1, we described 3 of the most common recommendation methods: User Based Collaborative Filtering, Item Based Collaborative Filtering and Matrix Factorization
  • In part 2, we implemented Matrix Factorization through ALS and found similar movies
  • In part 3, this part, we recommend movies to users based on what movies they’ve rated. We also make an attempt to clone Netflix’s “because you watched X” feature and make a complete page recommendation with trending movies

Recommending Movies to users

We pick up our code where we trained the ALS model from implicit library. Previous code to load and process the data can be found in the previous posts in this series or on my Github.

model = implicit.als.AlternatingLeastSquares(factors=10,
iterations=20,
regularization=0.1,
num_threads=4)
model.fit(user_item.T)

First let’s write a function that returns the movies that a particular user had rated

def get_rated_movies_ids(user_id, user_item, users, movies):
“””
Input
-----
   user_id: int
User ID
   user_item: scipy.Sparse Matrix
User item interaction matrix
   users: np.array
Mapping array between user ID and index in the user item matrix
   movies: np.array
Mapping array between movie ID and index in the user item matrix
  Output
-----
   movieTableIDs: python list
List of movie IDs that the user had rated
  “””
user_id = users.index(user_id)
# Get matrix ids of rated movies by selected user
ids = user_item[user_id].nonzero()[1]
# Convert matrix ids to movies IDs
movieTableIDs = [movies[item] for item in ids]
  return movieTableIDs
movieTableIDs = get_rated_movies_ids(1, user_item, users, movies)
rated_movies = pd.DataFrame(movieTableIDs, columns=[‘movieId’])
rated_movies
def get_movies(movieTableIDs, movies_table):
“””
Input
-----
   movieTableIDs: python list
List of movie IDs that the user had rated
   movies_table: pd.DataFrame
DataFrame of movies info
   Output
-----
   rated_movies: pd.DataFrame
DataFrame of rated movies
   “””
   rated_movies = pd.DataFrame(movieTableIDs, columns=[‘movieId’])
   rated_movies = pd.merge(rated_movies, movies_table, on=’movieId’, how=’left’)
   return rated_movies

movieTableIDs = get_rated_movies_ids(1, user_item, users, movies)
df = get_movies(movieTableIDs, movies_table)
df

Now, let’s recommend movieIDs for a particular user ID based on the movies that they rated.

def recommend_movie_ids(user_id, model, user_item, users, movies, N=5):
“””
Input
-----
   user_id: int
User ID
   model: ALS model
Trained ALS model
   user_item: sp.Sparse Matrix
User item interaction matrix so that we do not recommend already rated movies
   users: np.array
Mapping array between User ID and user item index
   movies: np.array
Mapping array between Movie ID and user item index
   N: int (default =5)
Number of recommendations
   Output
-----
   movies_ids: python list
List of movie IDs
“””
   user_id = users.index(user_id)
   recommendations = model.recommend(user_id, user_item, N=N)
   recommendations = [item[0] for item in recommendations]
   movies_ids = [movies[ids] for ids in recommendations]
   return movies_ids
movies_ids = recommend_movie_ids(1, model, user_item, users, movies, N=5)
movies_ids
> [1374, 1127, 1214, 1356, 1376]
movies_rec = get_movies(movies_ids, movies_table)
movies_rec
display_posters(movies_rec)
movies_ids = recommend_movie_ids(100, model, user_item, users, movies, N=7)
movies_rec = get_movies(movies_ids, movies_table)
display_posters(movies_rec)

Because You watched

Let’s implement Netflix “Because You Watched” feature. It’s about recommending movies based on what you’ve watched. This is similar to what we already did, but this time, it’s more selective. Here’s how we will do it: We will choose random 5 movies that a user had watched and for each movie recommend similar movies to it. Finally, we display all of them in a one page layout

def similar_items(item_id, movies_table, movies, N=5):
“””
Input
-----
item_id: int
MovieID in the movies table
     movies_table: DataFrame
DataFrame with movie ids, movie title and genre
     movies: np.array
Mapping between movieID in the movies_table and id in the item user matrix
     N: int
Number of similar movies to return
     Output
-----
df: DataFrame
DataFrame with selected movie in first row and similar movies for N next rows
“””
    # Get movie user index from the mapping array
user_item_id = movies.index(item_id)
# Get similar movies from the ALS model
similars = model.similar_items(user_item_id, N=N+1)
# ALS similar_items provides (id, score), we extract a list of ids
l = [item[0] for item in similars[1:]]
# Convert those ids to movieID from the mapping array
ids = [movies[ids] for ids in l]
# Make a dataFrame of the movieIds
ids = pd.DataFrame(ids, columns=[‘movieId’])
# Add movie title and genres by joining with the movies table
recommendation = pd.merge(ids, movies_table, on=’movieId’, how=’left’)
    return recommendation
def similar_and_display(item_id, movies_table, movies, N=5):
     df = similar_items(item_id, movies_table, movies, N=N)
     df.dropna(inplace=True)
     display_posters(df)
def because_you_watched(user, user_item, users, movies, k=5, N=5):
“””
Input
-----
user: int
User ID
     user_item: scipy sparse matrix
User item interaction matrix
     users: np.array
Mapping array between User ID and user item index
     movies: np.array
Mapping array between Movie ID and user item index
     k: int
Number of recommendations per movie
     N: int
Number of movies already watched chosen
     “””
     movieTableIDs = get_rated_movies_ids(user, user_item, users, movies)

df = get_movies(movieTableIDs, movies_table)
     movieIDs = random.sample(df.movieId, N)
     for movieID in movieIDs:
title = df[df.movieId == movieID].iloc[0].title
print(“Because you’ve watched “, title)
similar_and_display(movieID, movies_table, movies, k)
because_you_watched(500, user_item, users, movies, k=5, N=5)

“Because you watched “, ‘Definitely, Maybe (2008)’

“Because you watched “, ‘Pocahontas (1995)

“Because you watched “, ‘Simpsons Movie, The (2007)’

“Because you watched “, ‘Catch Me If You Can (2002)’

“Because you watched “, ‘Risky Business (1983)’

Trending movies

Let’s also implement trending movies. In our context, trending movies are movies that been rated the most by users

def get_trending(user_item, movies, movies_table, N=5):
“””
Input
-----
   user_item: scipy sparse matrix
User item interaction matrix to use to extract popular movies
   movies: np.array
Mapping array between movieId and ID in the user_item matrix
   movies_table: pd.DataFrame
DataFrame for movies information
   N: int
Top N most popular movies to return
   “””
   binary = user_item.copy()
binary[binary !=0] = 1
   populars = np.array(binary.sum(axis=0)).reshape(-1)
   movieIDs = populars.argsort()[::-1][:N]
   movies_rec = get_movies(movieIDs, movies_table)
   movies_rec.dropna(inplace=True)
   print(“Trending Now”)
   display_posters(movies_rec)
get_trending(user_item, movies, movies_table, N=6)

Trending Now

Page recommendation

Let’s put everything in a timeline method. The timeline method will get the user ID and display trending movies and recommendations based on similar movies that that user had watched.

def my_timeline(user, user_item, users, movies, movies_table, k=5, N=5):
     get_trending(user_item, movies, movies_table, N=N)
     because_you_watched(user, user_item, users, movies, k=k, N=N)
my_timeline(500, user_item, users, movies, movies_table, k=5, N=5)

Trending Now

“Because you watched “, ‘Definitely, Maybe (2008)’

“Because you watched “, ‘Pocahontas (1995)’

“Because you watched “, ‘Simpsons Movie, The (2007)’

“Because you watched “, ‘Catch Me If You Can (2002)’

“Because you watched “, ‘Risky Business (1983)’

Export trained models to be used in production

At this point, we want to get our model into production. We want to create a web service where a user will provide a userid to the service and the service will return all of the recommendations including the trending and the “because you’ve watched”.

To do that, We first export the trained model and the used data for use in the web service.

import scipy.sparse
scipy.sparse.save_npz(‘model/user_item.npz’, user_item)
np.save(‘model/movies.npy’, movies)
np.save(‘model/users.npy’, users)
movies_table.to_csv(‘model/movies_table.csv’, index=False)
from sklearn.externals import joblib
joblib.dump(model, ‘model/model.pkl’)

Conclusion

In this post, we recommend movies to users based on their movie rating history. From there, we tried to clone the “because you watched” feature from Netflix and also display Trending movies as movies that were rated the most number of times. In the next post, we will try to put our work in a web service, where a user requests movie recommendations by providing its user ID.

Stay tuned!

More by Mohtadi Ben Fraj

Topics of interest

More Related Stories