Implementing The Perceptron Algorithm From Scratch In Python

In this post, we will see how to implement the perceptron model using breast cancer data set in python. A perceptron is a fundamental unit of the neural network which takes weighted inputs, process it and capable of performing binary classifications. This is a follow up to my previous post on the . Perceptron Model If you want to skip the theory and jump into code directly click . here Disclaimer: The content and the structure of this article is based on the deep learning lectures from One-Fourth Labs — Padhai . Perceptron Recap In the perceptron model inputs can be real numbers unlike the Boolean inputs in MP Neuron Model. The output from the model will still be binary {0, 1}. The perceptron model takes the input if the weighted sum of the inputs is greater than threshold output will be 1 else output will be 0. x b Fig 1— Mathematical RepresentationLearning Algorithm The main goal of the learning algorithm is to find vector capable of absolutely separating Positive (y = 1) and Negative (y = 0) sets of data. Perceptron learning algorithm goes like this, w P N (Fig 2— Perceptron Algorithm) To understand the learning algorithm in detail and the intuition behind why the concept of updating weights works in classifying the Positive and Negative data sets perfectly, kindly refer to my previous post on the . Perceptron Model Lets Code Image Source The data set we will be using is breast cancer data set from . The data set has 569 observations and 30 variables excluding the class variable. The breast cancer data is an imbalanced data set, that means the classes ‘0’ and ‘1’ are not represented equally. In this example, we are not going to perform any sampling techniques to balance the data because this is a simple implementation of the perceptron model. sklearn (Class Imbalance) Before start building the Perceptron Model, first we need to load the required packages and the data set. The data set is present in the sklearn datasets module. Once we load the data, we need to grab the features and response variables using breast_cancer.data and breast_cancer.target commands. # packages sklearn.datasets numpy np pandas pd matplotlib.pyplot plt sklearn.model_selection train_test_split #load the breast cancer data breast_cancer = sklearn.datasets.load_breast_cancer() #convert the data to pandas dataframe. data = pd.DataFrame(breast_cancer.data, columns = breast_cancer.feature_names) data[ ] = breast_cancer.target data.head() data.describe() #plotting a graph to see = ) plt.xlabel( ) plt.ylabel( ) plt.show() sklearn.preprocessing MinMaxScaler #perform scaling on the data. X = data.drop( , axis = ) Y = data[ ] mnscaler = MinMaxScaler() X = mnscaler.fit_transform(X) X = pd.DataFrame(X, columns=data.drop( ,axis = ).columns) #train test split. X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = , stratify = Y, random_state = ) import import import as import as import as from import "class" [' ']. (). ( class imbalance data class value_counts plot kind "barh" "Count" "Classes" from import "class" 1 "class" "class" 1 0.1 1 (Perceptron Preprocessing) After fetching the X and Y variables, we will perform Min-Max scaling to bring all the features in the range 0 — 1. Before building the model, we will split the data so that we can train the model on training data and test the performance of the model on testing data. We will use sklearn’s train_test_split function to split the data in the ratio of 90:10 for training and testing respectively. Now that we are done with preprocessing steps, we can start building the model. We will build our model inside a class called . perceptron In the perceptron class, we will create a constructor function def__init__. The constructor initializes the weights vector and threshold to None. w b = None self.b = None #model def model(self, x): (np.dot(self.w, x) >= self.b) #predictor to predict on the data based on w def predict(self, X): Y = [] x X: result = self.model(x) Y.append(result) np.array(Y) def fit(self, X, Y, epochs = , lr = ): self.w = np.ones(X.shape[ ]) self.b = accuracy = {} max_accuracy = wt_matrix = [] # all epochs i range(epochs): x, y zip(X, Y): y_pred = self.model(x) y == and y_pred == : self.w = self.w + lr * x self.b = self.b - lr * elif y == and y_pred == : self.w = self.w - lr * x self.b = self.b + lr * wt_matrix.append(self.w) accuracy[i] = accuracy_score(self.predict(X), Y) (accuracy[i] > max_accuracy): max_accuracy = accuracy[i] chkptw = self.w chkptb = self.b #checkpoint (Save the weights and b value) self.w = chkptw self.b = chkptb print(max_accuracy) #plot the accuracy values over epochs plt.plot(accuracy.values()) plt.xlabel( ) plt.ylabel( ) plt.ylim([ , ]) plt.show() # the weight matrix, that contains weights over all epochs np.array(wt_matrix) : # ( ): . class Perceptron constructor def __init__ self self w return 1 if else 0 for in return 1 1 1 0 0 for for in for in if 1 0 1 0 1 1 if "Epoch #" "Accuracy" 0 1 return return (Perceptron Model) The function model takes input values as an argument and perform the weighted aggregation of inputs (dot product between ) and returns the value 1 if the aggregation is greater than the threshold else 0. Next, we have the predict function that takes input values as an argument and for every observation present in , the function calculates the predicted outcome and returns a list of predictions. x w.x b x x Finally, we will implement fit function to learn the best possible weight vector and threshold value for the given data. The function takes input data( & ), learning rate and the number of epochs as arguments. w b x y perceptron = Perceptron() #epochs = and lr = wt_matrix = perceptron.fit(X_train, Y_train, , ) #making predictions on test data Y_pred_test = perceptron.predict(X_test) #checking the accuracy the model print(accuracy_score(Y_pred_test, Y_test)) 10000 0.3 10000 0.3 of (Perceptron Model Execution) Once we have our class ready, we initialize a new perceptron class object and using that object we will call fit method on our training data to learn the best possible parameters. We will evaluate the model performance on the test data by calculating the testing accuracy. The entire code discussed in the article is present in this GitHub . Feel free to fork it or download it. repository Continue Learning If you want to learn more about Artificial Neural Networks using Keras & Tensorflow 2.0 (Python or R). Check out the by Abhishek and Pukhraj from . They explain the fundamentals of deep learning in a simplistic manner. Artificial Neural Networks Starttechacademy Further Improvements You can try out a few possible improvements to increase the accuracy of the model, Vary the train-test size split and see if there is any change in accuracy. Choose larger epochs values, learning rates and test on the perceptron model and visualize the change in accuracy. Take random weights in the perceptron model and experiment. Conclusion In this article, we have seen how to implement the perceptron algorithm from scratch using python. Connect with Me GitHub: LinkedIn: https://github.com/Niranjankumar-c https://www.linkedin.com/in/niranjankumar-c/ — There might be some affiliate links in this post to relevant resources. You can purchase the bundle at the lowest price possible. I will receive a small commission if you purchase the course. Disclaimer