How to Ignore SSL Security Certificate Check in Python Requests

1. Understanding the Reason Behind SSL Security Checks and Why It Fails
2. Ignore SSL Security Check in Python
3. Conclusion

Accessing a URL without a secure SSL certificate raises exception warnings when HTTP requests are sent to it. A lot of times, the SSL certificate of these URLs expires, thus creating all sorts of security issues.

If the information is not sensitive, these warnings can be subsided when programs use requests in Python. This article will provide multiple ways to disable security certificate checks using requests.

Understanding the Reason Behind SSL Security Checks and Why It Fails

If a program uses Python requests to get requests from a URL whose SSL certificate is expired, it raises two exceptions. The below program displays what those exceptions are.

Scenario 1

This program uses a URL provided by the SSL community with an expired security certificate for testing purposes. It must be noted that the SSL security exceptions only raises with URLs having expired SSL certificates.

The Python requests do not raise any exception with URLs with a valid SSL certificate or a revoked one. So this article will mainly focus on URLs with expired security certificates.

The example below shows a simple program that imports requests in the first line. The second line of the program sends a post request to the URL to modify occurrences of 'bar' as 'baz'.

It is done to send a post request to the URL and holds no other significance inside the program.

import requests

requests.post(url="https://expired-rsa-dv.ssl.com/", data={"bar": "baz"})

Executing this Python script throws SSLError exceptions.

....
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='expired-rsa-dv.ssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:841)'),))

This is the first exception to be considered in learning how to disable security certificate checks using requests.

Scenario 2

This program turns off the SSL certificate verification using verify=False to disable the security certificate check using requests.

import requests

requests.post(url="https://expired-rsa-dv.ssl.com/", data={"bar": "baz"}, verify=False)

The requests library is built in a way that it can turn off verification for SSL certificates, but the program throws another exception with links having expired SSL certificates.

InsecureRequestWarning: Unverified HTTPS request is being made to host 'expired-rsa-dv.ssl.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,

These two exceptions are needed to be dealt with to disable security certificate checks using requests.

Ignore SSL Security Check in Python

This section will explain various methods that either disable security certificate check using requests or provides a turnaround with the problem. Every method has its purpose.

Create a Monkey Patch for the requests Library

If a third-party library requires the security checks disabled, the requests library can be monkey patched. A context manager is used for patching to disable security certificate checks using requests.

After the requests is patched, the verify field is given a False value by default, suppressing the warning. Another warning is raised when verify=false is used, as explained in scenario 2 of the previous section.

The patch will then add an exception handling block to disable security certificate check using requests and suppresses the warnings. It will be easier to understand with the following examples.

The program below patches the requests library. Let’s understand what this code does.

Imports:

1. warnings: This library package is a sub-package of the Exceptions library.
2. contextlib: This Python library is used for patching the requests library.
3. requests: The requests library package of Python.
4. urllib3: It is a module that handles HTTP requests and URLs in Python. This library is used to import a sub-module InsecureRequestWarning, which raises an exception for expired SSL certificates.

Patch:

At first, the program saves the default environment settings of the requests library in a variable - old_merge_environment_settings. This variable will be used to bring requests back to their default state after the opened adapters are closed.

old_merge_environment_settings = requests.Session.merge_environment_settings

The method no_ssl_verification is created and decorated with @contextlib.contextmanager. A new variable opened_adapters is created and assigned a set() to it.

A set is a datatype that stores items in an unordered format.

@contextlib.contextmanager
def no_ssl_verification():
    opened_adapters = set()

Inside the no_ssl_verification method, another nested method merge_environment_settings is created. This method has six parameters, similar to the merge_environment_settings from the requests module, and it will act as the patch for the original module.

def merge_environment_settings(self, url, proxies, stream, verify, cert):

Inside the method, the opened_adapters variable is updated with matching adapter pair from the parameter url. Every connection is made with some matching adapter pair, which gets returned in this step.

Since verification only occurs once per connection, we must shut all opened adapters after we’re done. If not, the effects of verify=False last after this context manager has ended.

def merge_environment_settings(self, url, proxies, stream, verify, cert):

    opened_adapters.add(self.get_adapter(url))

It is important to remember the first section of the article to understand the next line of code. When the requests.post function was used on the URL with an expired SSL certificate, it threw two exceptions.

The first exception was caused by verify, which is set with a True value. Though the verify field was switchable, it could be given a False value.

The second exception was a non-mutable entity that stopped the program from making a connection.

The below code modifies the verify field to have a False value by default to solve this problem. A new variable settings is created to execute this step, which gets assigned with data from the variable old_merge_environment_settings.

Once the data is stored inside the variable settings, the verify field is turned to False.

This changes the default value of verify. Lastly, this variable is returned.

settings = old_merge_environment_settings(self, url, proxies, stream, verify, cert)
settings["verify"] = False

return settings

The data from the updated merge_environment_settings method is assigned to the function requests.Session.merge_environment_settings. This will assure that the field verify has a False value by default.

requests.Session.merge_environment_settings = merge_environment_settings

We have dealt with the first exception. The second exception, which is not mutable, will be solved using an exception handling block.

Let’s understand what the code does here.

At first, inside the try block, a with block is created that catches all the warnings raised. Inside this with block, warnings.simplefilter function is used to give an ignore value to the urllib3 sub-module InsecureRequestWarning.

As we learned in the imports section, the InsecureRequestWarning sub-module raises an exception for expiring SSL certificates; assigning an ignore value to it will bypass the second exception caused.

try:
    with warnings.catch_warnings():
        warnings.simplefilter('ignore', InsecureRequestWarning)
        yield

Inside finally block, the original settings stored inside old_merge_environment_settings is assigned back to the function requests.Session.merge_environment_settings.

As all the opened adapters need to be closed before the program exits, a for loop is created that iterates for the number of times opened adapters are inside the program.

All the adapters are closed using the function adapter.close() using a try-except block; inside the except block, the iteration is passed.

for adapter in opened_adapters:
    try:
        adapter.close()
    except:
        pass

The complete code is given below.

import warnings
import contextlib

import requests
from urllib3.exceptions import InsecureRequestWarning

old_merge_environment_settings = requests.Session.merge_environment_settings


@contextlib.contextmanager
def no_ssl_verification():
    opened_adapters = set()

    def merge_environment_settings(self, url, proxies, stream, verify, cert):

        opened_adapters.add(self.get_adapter(url))

        settings = old_merge_environment_settings(
            self, url, proxies, stream, verify, cert
        )
        settings["verify"] = False

        return settings

    requests.Session.merge_environment_settings = merge_environment_settings

    try:
        with warnings.catch_warnings():
            warnings.simplefilter("ignore", InsecureRequestWarning)
            yield
    finally:
        requests.Session.merge_environment_settings = old_merge_environment_settings

        for adapter in opened_adapters:
            try:
                adapter.close()
            except:
                pass

Use Monkey Patch in Python

The monkey patch can be invoked using the no_ssl_verification() method. Any request sent under this method will follow the directives of the monkey patch and disable security certificate checks using requests.

Let’s look at an example.

The method no_ssl_verification is called under a with block. This will run the method inside the block and then closes itself when the compiler comes out of the block.

Inside the block, a requests.get call is sent to the URL with an expired SSL certificate. With the default settings, this would have caused an exception throw, but this time the requests.get call is sent successfully.

Another request is sent with the requests, where the verify field is set to True. Unlike the default scenario, this time, no error exception is thrown.

with no_ssl_verification():
    requests.get("https://expired-rsa-dv.ssl.com/")
    print("Modifications working Properly")

    requests.get("https://expired-rsa-dv.ssl.com/", verify=True)
    print("`Verify=true` has its meaning changed")

Output:

Modifications working Properly

`Verify=true` has its meaning changed

It was seen that Verify=False had the directives to reset the patch with default settings. When this field was set inside requests.get(), the monkey patch resets to the default settings, allowing to throw error exceptions.

requests.get("https://expired-rsa-dv.ssl.com/", verify=False)
print("It resets back")

Output:

connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 'expired-rsa-dv.ssl.com'.
  InsecureRequestWarning,
It resets back

The requests sub-module session can also be used with the monkey patch.

In a variable named session, the function requests.Session() is loaded. The session.verify value is set to True.

Inside a with block, the session.get() function is used to send the get request to the URL, and the verify field is set to True. A print statement prints a message if the connection is made.

session = requests.Session()
session.verify = True

with no_ssl_verification():
    session.get("https://expired-rsa-dv.ssl.com/", verify=True)
    print("Works in Session")

Output:

Works in Session

When the context manager exits, this code closes any open adapters that handle a patched request. Since requests maintains a connection pool for each session and certificate validation only occurs once per connection, unexpected events like the following will occur.

try:
    requests.get("https://expired-rsa-dv.ssl.com/", verify=False)
except requests.exceptions.SSLError:
    print("It breaks")

try:
    session.get("https://expired-rsa-dv.ssl.com/")
except requests.exceptions.SSLError:
    print("It breaks here again")

Output:

C:\Users\Win 10\venv\lib\site-packages\urllib3\connectionpool.py:1052: InsecureRequestWarning: Unverified HTTPS request is being made to host 'expired-rsa-dv.ssl.com'.
  InsecureRequestWarning,
It breaks here again

Use urllib3 to Disable Warnings in Python

If the program requires disabling security certificate checks without using requests or monkey patching, then using the urllib3 provides a simple solution.

Import requests and urllib3, and from urllib3 import the submodule InsecureRequestWarning. The warning is disabled using the urllib3.disable_warnings function.

The requests.post() statement is placed under the try block, and the verify field is set to False. This will disable the security check for expired security certificates.

Inside the except block, the SSLError is raised with an error message.

import requests
from urllib3.exceptions import InsecureRequestWarning

requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)

try:
    requests.post(
        url="https://expired-rsa-dv.ssl.com/", data={"bar": "baz"}, verify=False
    )
    print("Connection made successfully")

except requests.exceptions.SSLError:
    print("Expired Certificate")

Output:

Connection made successfully

Conclusion

This article explains various methods to disable security certificate checks using requests in Python. The reader, through the article, can disable security checks easily.