How to Subtract Background in OpenCV

1. OpenCV Background Subtraction Using the cvzone Library
2. OpenCV Background Subtraction Using MOG2 and KNN

This tutorial will discuss removing background from an image using the cvzone library, MOG2, and KNN in OpenCV.

OpenCV Background Subtraction Using the cvzone Library

We can use the cvzone library to remove the background of an image which uses the mediapipe library to remove the background. We can also use it to remove or change the background in real-time.

The mediapipe library provides two models for background subtraction, one is slow but has high accuracy, and the other is fast but has low accuracy. The first model uses the 256x256x3 tensor to remove the background, and the other uses the 144x256x3 tensor.

First, we have to install the cvzone and mediapipe to use them with OpenCV. We can install them using pip.

We have to import the selfie segmentation module from the cvzone library. To use its removeBG() method, we can remove the background from an image.

For example, let’s remove the background from an image. See the code below.

import cv2
import cvzone
from cvzone.SelfiSegmentationModule import SelfiSegmentation

segmentor = SelfiSegmentation()
img = cv2.imread("cat.jpg")
img_Out = segmentor.removeBG(img, (255, 255, 255), threshold=0.99)

cv2.imshow("img", img_Out)
cv2.waitKey(0)
cv2.destroyAllWindows()

Output:

In the output, the background is removed from the given image. In the above code, the first argument of the removeBG() function is the input image, and the second argument is the color that we want to use as the new background color.

The third argument is the threshold we can set according to our given image. We can also remove or change the background of a video.

We have to use a loop and process each frame one by one. We can also pass a background image instead of background color in the removeBG() to change the background of a video.

For example, let’s change the background of a video. See the code below.

import cv2
import cvzone
from cvzone.SelfiSegmentationModule import SelfiSegmentation

segmentor = SelfiSegmentation()
img_b = cv2.imread("background.jpg")
capture = cv2.VideoCapture("Man.mp4")

while True:
    ret, frame = capture.read()
    if frame is None:
        break

    frame = cv2.resize(frame, [512, 512])
    out = segmentor.removeBG(frame, img_b, threshold=0.6)

    cv2.imshow("Frame", out)

    keyboard = cv2.waitKey(30)
    if keyboard == "q" or keyboard == 27:
        break
cv2.destroyAllWindows()

Output:

In the above code, we used the resize() function of OpenCV to resize the frames because the frame and the background image should have the same size.

OpenCV Background Subtraction Using MOG2 and KNN

We can also use the subtraction methods of OpenCV like MOG2 and KNN to highlight the moving objects present in a video.

The algorithm will make a background model from the video, and then it will subtract the image from the background model to get the foreground mask of moving objects.

The algorithm compares two frames to check if the position of a pixel is changed or not. If the position changes, the pixel will be added to the mask.

So this algorithm can only be used if the camera is stationary and the objects are moving like humans or cars. We can use this method to highlight moving objects like humans and cars.

For example, let’s use the MOG2 background subtractor to highlight traffic from a video. See the code below.

import cv2

backSub_mog = cv2.createBackgroundSubtractorMOG2()
cap_v = cv2.VideoCapture("cars.mp4")

while True:
    ret, frame = cap_v.read()
    if frame is None:
        break

    fgMask = backSub_mog.apply(frame)

    cv2.imshow("Input Frame", frame)
    cv2.imshow("Foreground Mask", fgMask)

    keyboard = cv2.waitKey(30)
    if keyboard == "q" or keyboard == 27:
        break
cv2.destroyAllWindows()

Output:

In the above code, we used a loop to get all the frames of the video one by one and apply the background subtractor using the apply() function. We can also use the KNN subtractor by changing the characters from MOG2 to KNN in the above code.

We cannot use this method to remove the background from an image because no moving objects are present in an image.