---

Logistic Regression §

Logistic Regression is a supervised machine learning algorithm for binary classification that models the probability that an input belongs to a class .

See also:

• ML1 - Logistic Sigmoid Function -> useful to predict the output in a neural network
• SC - Lecture 17 - Maximization of the Likelihood and Logistic Regression -> Logistic regression is an application of the maximization of the likelihood

Features are considered as linear combination of features passes through a sigmoid function to output probability:

• is the weight vector
• is the bias
• is the sigmoid activation

The decision rule is:

The standard threshold is usually

Usually the parameters and are learned. A maximum likelihood estimation through gradient descent

The most natural choices is to minimize the log-loss (binary cross-entropy):

• the index is the -step iteration of the gradient descent
• is the true label of the -th training example.
• (y hat) is the predicted probability that the -th example belongs to class 1
• is the number of training examples in the dataset

So is the average log-loss per sample. is computed using the sigmoid function: where is the i-th input.