How to Classify when linear Regression Fails?

is a simple solution to our classification problems but what happens when it fails. As we will see in below problem Linear regression Suppose we want to classify and are . It is binary classification. Let’s try it with linear Regression Y= {0,1} X data samples Wow, Linear Regression has done the job.It is really a good fit but what happens if i add a new data to given data set. It’s really a bad . Linear Regression didn’t worked. What should we do next? If there is problem, there is solution and the solution is Let’s start with formal definition and have an idea what is linear and logistic Regression are: solution Logistic Regression . In , the outcome (dependent variable) is continuous. It can have any one of an infinite number of possible values. In , the outcome (dependent variable) has only a limited number of possible values. is used when the response variable is categorical in nature. linear regression logistic regression Logistic regression Intuitively, it also doesn’t make sense for h(x) to take values larger than 1 or smaller than 0 when we know that y ∈ {0, 1}.To fix this, lets change the form for our hypotheses h(x). We will choose hypothesis as follows : g(z) is called or L** **. It look like as follows: Sigmoid function ogistic function Let’s assume that if we combine above equations together it can be re written as follows: As in above equation when , h(x) will become 1, p(y|x;theta) = (1-h(x)) and when ,(1- h(x)) will become 1, p(y|x;theta) = h(x) y =0 y =1 Now, are given by likelihood of the parameters and when we will be given by : Maximize log likelihood, it Now we can use , we already know that h(x) is given by sigmoid function. Gradient Descent For ease, the partial derivative of g(z) with respect to z is given by Now if apply gradient descent by taking the partial derivative of log likelihood with respect to theta If we compare the above rule to the least mean square, it looks identical but it’s not. This is a different algorithm because h(x) is now defined as non-linear function of theta transpose * x[i]. learning If you find any inconsistency in my post, please feel free to point out in the comments. Thanks for reading. If you want to connect with me. Please feel free to connect with me on LinkedIn. _View Sameer Negi's profile on LinkedIn, the world's largest professional community. Sameer has 3 jobs listed on their…_www.linkedin.com Sameer Negi - autonomous Vehicle Traniee - Infosys | LinkedIn