The Sigmoid Function in Python

  1. Implement the Sigmoid Function in Python Using the math Module
  2. Implement the Sigmoid Function in Python Using the numpy.exp() Method
  3. Implement the Sigmoid Function in Python Using the SciPy Library

In this tutorial, we will look into various methods to use the sigmoid function in Python. The sigmoid function is a mathematical logistic function. It is commonly used in statistics, audio signal processing, biochemistry, and the activation function in artificial neurons. The formula for the sigmoid function is F(x) = 1/(1 + e^(-x)).

Implement the Sigmoid Function in Python Using the math Module

We can implement our own sigmoid function in Python using the math module. We need the math.exp() method from the math module to implement the sigmoid function.

The below example code demonstrates how to use the sigmoid function in Python.

import math

def sigmoid(x):
    sig = 1 / (1 + math.exp(-x))
    return sig

The problem with this implementation is that it is not numerically stable and the overflow may occur.

The example code of the numerically stable implementation of the sigmoid function in Python is given below.

import math

def stable_sigmoid(x):

    if x >= 0:
        z = math.exp(-x)
        sig = 1 / (1 + z)
        return sig
    else:
        z = math.exp(x)
        sig = z / (1 + z)
        return sig

Implement the Sigmoid Function in Python Using the numpy.exp() Method

We can also implement the sigmoid function using the numpy.exp() method in Python. Like the implementations of the sigmoid function using the math.exp() method, we can also implement the sigmoid function using the numpy.exp() method.

The advantage of the numpy.exp() method over math.exp() is that apart from integer or float, it can also handle the input in an array’s shape.

Below is the regular sigmoid function’s implementation using the numpy.exp() method in Python.

import numpy as np

def sigmoid(x):
  
    z = np.exp(-x)
    sig = 1 / (1 + z)

    return sig

For the numerically stable implementation of the sigmoid function, we first need to check the value of each value of the input array and then pass the sigmoid’s value. For this, we can use the np.where() method, as shown in the example code below.

import numpy as np

def stable_sigmoid(x):

    sig = np.where(x < 0, np.exp(x)/(1 + np.exp(x)), 1/(1 + np.exp(-x)))
    return sig

Implement the Sigmoid Function in Python Using the SciPy Library

We can also use the SciPy version of Python’s sigmoid function by simply importing the sigmoid function called expit in the SciPy library.

The example code below demonstrates how to use the sigmoid function using the SciPy library:

from scipy.special import expit

x = 0.25
sig = expit(x)

The expit() method is slower than the above implementations. The advantage of the expit() method is that it can automatically handle the various types of inputs like list, and array, etc.

from scipy.special import expit

sig = expit(np.array([0.25, 0.5, 0.6, 0.7, 0.4]))
print(sig)

Output:

[0.5621765  0.62245933 0.64565631 0.66818777 0.59868766]
Contribute
DelftStack is a collective effort contributed by software geeks like you. If you like the article and would like to contribute to DelftStack by writing paid articles, you can check the write for us page.