How to Find Binary Cross Entropy Loss Value Using TensorFlow

Binary cross-entropy loss can be calculated two ways: with a custom implementation of the formula in NumPy, and with TensorFlow’s built-in tf.keras.losses.BinaryCrossentropy. This article walks through both and shows that they produce approximately the same loss value.

What binary cross-entropy measures

A loss function measures how far a model’s predictions are from the true labels. Binary cross-entropy — also known as log loss — is the standard loss for binary classification problems, where each true label is either 0 or 1 and the model outputs a probability that the sample belongs to class 1.

Binary cross-entropy is the negative average of the log of corrected predicted probabilities.

The binary cross-entropy over a batch of N samples is:

$$
\text { Log loss }=\frac{1}{N} \sum_{i=1}^{N}-\left(y_{i} * \log \left(p_{i}\right)+\left(1-y_{i}\right) * \log \left(1-p_{i}\right)\right)
$$

Here y_i is the true label (0 or 1) and p_i is the model’s predicted probability for that sample. For each sample only one of the two log terms contributes: when y_i = 1 the loss is -log(p_i), and when y_i = 0 it is -log(1 - p_i). A confident correct prediction contributes almost nothing, while a confident wrong prediction contributes a large value. The sum is divided by N, so the result is the average per-sample loss.

Calculate binary cross-entropy with NumPy

The formula translates directly into a NumPy function. y_true holds the actual labels and y_pred holds the model’s predicted probabilities, both as column vectors:

import numpy as np


def BinaryCrossEntropy(y_true, y_pred):
    y_pred = np.clip(y_pred, 1e-7, 1 - 1e-7)
    term_0 = y_true * np.log(y_pred + 1e-7)
    term_1 = (1 - y_true) * np.log(1 - y_pred + 1e-7)
    return -np.mean(term_0 + term_1, axis=0)


print(
    BinaryCrossEntropy(
        np.array([1, 0, 1]).reshape(-1, 1), np.array([0, 0, 1]).reshape(-1, 1)
    )
)

[5.14164949]

np.clip(y_pred, 1e-7, 1 - 1e-7) bounds every prediction into the interval [1e-7, 1 - 1e-7], so [0, 0, 1] becomes [1e-7, 1e-7, 0.9999999]. This is needed because the formula contains log(p) and log(1 - p): the natural logarithm of 0 is undefined, so an exact prediction of 0 or 1 would make the loss uncomputable. Clipping keeps every logarithm’s argument finite, and the small 1e-7 added inside each log call guards the same way. np.mean(..., axis=0) then averages the per-sample losses, which is the 1/N factor in the formula.

In this run the first sample is confidently wrong (true label 1, predicted 0), so its large loss dominates the mean and the result is about 5.14.

Calculate binary cross-entropy with TensorFlow

TensorFlow ships the same loss as the built-in class tf.keras.losses.BinaryCrossentropy. Using it requires the tensorflow package — install it with pip install tensorflow. Because that dependency is not assumed here, the call is shown as inline code rather than as an executed output block; run it locally to reproduce the result.

After import tensorflow as tf, build the loss object with bce = tf.keras.losses.BinaryCrossentropy(from_logits=False, reduction=tf.keras.losses.Reduction.SUM_OVER_BATCH_SIZE) and call it on the same y_true and y_pred arrays used in the NumPy example. Calling bce(y_true, y_pred) takes two arguments:

1. y_true (true label): either 0 or 1 for each sample.
2. y_pred (predicted value): the model’s output — either a logit, a value in [-inf, inf] used when from_logits=True, or a probability in [0., 1.] used when from_logits=False.

Two settings control how the call interprets these inputs. from_logits=False declares that y_pred already contains probabilities, and reduction=tf.keras.losses.Reduction.SUM_OVER_BATCH_SIZE averages the per-sample losses over the batch — the same 1/N factor as the formula. The call returns a tensor, and loss.numpy() retrieves the scalar value; on the same inputs it works out to about 5.37, close to but not identical with the NumPy result. See the official tf.keras.losses.BinaryCrossentropy API reference for the remaining arguments.

Interpret the loss and choose probabilities or logits

Match the input format to from_logits. With from_logits=False, pass probabilities in [0, 1] — typically the output of a sigmoid layer. With from_logits=True, pass logits: raw, unbounded scores in [-inf, inf] that the loss maps through a sigmoid internally. Feeding logits while from_logits=False, or probabilities while it is True, yields a meaningless loss value rather than a warning.

On identical inputs both implementations compute the same mean per-sample loss, but they agree only approximately: the printed 5.14164949 comes from clipping and then adding 1e-7 inside each log call, while tf.keras.losses.BinaryCrossentropy clips predictions at its own internal epsilon instead. The reduction setting and floating-point precision can also shift the trailing digits.

A lower loss means the predictions sit closer to the true labels on that batch, which is why the loss is useful as a training signal. A low value on its own does not prove the model is good overall — judge quality with metrics on held-out data as well.