is a statistical method used for analyzing a dataset in which there are one or more independent variables that can be used to predict the outcome of a categorical dependent variable. In simpler terms, it's a technique used for classification problems where the target variable is binary (i.e., it can take on two possible values, such as 0 or 1). Logistic regression The main goal of logistic regression is to find the best-fitting model to describe the relationship between the independent variables and the probability of a particular outcome. It uses the logistic function (also known as the sigmoid function) to map predicted values to probabilities. Difference Between Linear Regression and Logistic Regression is used when the target variable is continuous and can take any numerical value. It predicts a value that is unbounded, which means it can be any real number. Linear regression uses a straight line to model the relationship between the independent variables and the dependent variable. Linear regression , on the other hand, is used when the target variable is categorical and represents two classes (binary classification). It predicts the probability that a given instance belongs to a particular class. The output of logistic regression is bounded between 0 and 1, thanks to the logistic (sigmoid) function. Logistic regression Real World Applications of Logistic Regression Logistic regression finds applications in various fields due to its simplicity, interpretability, and effectiveness in binary classification problems. Some real-world applications include: : Medical Diagnostics Predicting whether a patient has a particular disease based on symptoms and test results. : Finance Credit scoring to determine if a borrower is likely to default on a loan. Predicting the likelihood of a customer subscribing to a service based on demographic data. The logistic regression model uses the logistic function (also known as the sigmoid function) to model the relationship between the independent variables and the probability of a particular outcome. Mathematical Formula for Logistic Regression The logistic regression model can be mathematically expressed as: Logistic Function (Sigmoid Function): The logistic function is defined as: Here, ( ) is the output (probability), and is the linear combination of the independent variables and their coefficients: S t t The logistic function transforms any value of to a value between 0 and 1. This is crucial in logistic regression because we want to model probabilities. t Transformation of Linear Combination into Probabilities The linear combination t represents the weighted sum of the independent variables. When we apply the logistic function to t, we obtain a value between 0 and 1. This value represents the estimated probability of the event =1 given the values of the independent variables. Y In logistic regression, the goal is to find the best-fitting set of coefficients that maximizes the likelihood of the observed data. This is typically done through optimization techniques like gradient descent. Binary and Multinomial Logistic Regression Binary Logistic Regression: is used when the dependent variable has two possible categories or classes. These classes are typically coded as 0 and 1, representing the absence or presence of an outcome or event. Binary logistic regression For example, binary logistic regression can be used for tasks like: Predicting whether an email is spam (1) or not spam (0). Determining if a patient has a certain disease (1) or does not have it (0). In binary logistic regression, the logistic function is used to model the probability of belonging to one of the two classes. Multinomial Logistic Regression: , also known as softmax regression, is used when the dependent variable has more than two categories. It's an extension of logistic regression that can handle multiple classes. Multinomial logistic regression For example, multinomial logistic regression can be used for tasks like: Predicting the type of fruit in a basket (e.g., apple, banana, orange). Classifying different species of animals (e.g., cat, dog, bird). In multinomial logistic regression, the logistic function is extended to handle multiple classes. The softmax function is used to compute the probabilities of each class, and the class with the highest probability is predicted. Here's how it works: : Softmax Function The softmax function takes a vector of scores (one for each class) and transforms them into probabilities that sum to 1. It assigns a probability to each class based on the scores. : Prediction The class with the highest probability is predicted as the outcome. Code Step 1: Import necessary libraries import pandas as pd from sklearn.model_selection import train_test_split from sklearn.linear_model import LogisticRegression from sklearn.metrics import accuracy_score Step 2: Load and prepare the Titanic dataset data = pd.read_csv('train.csv') Step 3: Select relevant features and target variable X = data[['Pclass', 'Sex', 'Age', 'SibSp', 'Parch']] y = data['Survived'] Step 4: Data Cleaning and Encoding # Convert categorical variable 'Sex' to numeric X['Sex'] = X['Sex'].map({'male': 0, 'female': 1}) # Handle missing values in 'Age' (replace with mean) X['Age'].fillna(X['Age'].mean(), inplace=True) Step 5: Split Dataset # Split the dataset into training and testing sets X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42) Step 6: Initialize the Model model = LogisticRegression() model.fit(X_train, y_train) Step 7: Evaluate the Model # Step 5: Make predictions y_pred = model.predict(X_test) # Step 6: Evaluate the model accuracy = accuracy_score(y_test, y_pred) print(f'Accuracy: {accuracy * 100:.2f}%') # ------------------ OUTPUT -------------- # Accuracy: 82.68%
