We can’t use the same cost function for logistic regression as for linear regression. To find a better cost function, we’ll exploit the fact that the model output approaches 0 or 1 and the outputs in the training set are always either 0 (indicating the input is not in the class) or 1 (indicating the input is in the class). We want a cost function with the following properties:

  1. If the model output is near the training output, the cost should near 0.

  2. If the model output is near 1 when the training output is 0, the cost should be very large.

  3. If the model output is near 0 when the training output is 1, the cost should be equally large.

The following cost function has all three of these properties. It looks quite long, but you can see that the left addend becomes zero when the training output and the right addend becomes zero when .

Summary

Click the arrows to see different ways of expressing the same thing: