How to Fix Python UnicodeDecodeError: ASCII Codec Can't Decode Byte in Position: Ordinal Not in Range

1. Unicode Decode Error in Python
2. How to Solve the Unicode Decode Error in Python

In this article, we will learn how to resolve the UnicodeDecodeError that occurs during the execution of the code. We will look at the different reasons that cause this error.

We will also find ways to resolve this error in Python. Let’s begin with what the UnicodeDecodeError is in Python.

Unicode Decode Error in Python

If you are facing a recurring UnicodeDecodeError and are unsure of why it is happening or how to resolve it, this is the article for you.

In this article, we go in-depth about why this error comes up and a simple approach to resolving it.

Causes of Unicode Decode Error in Python

In Python, the UnicodeDecodeError comes up when we use one kind of codec to try and decode bytes that weren’t even encoded using this codec. To be more specific, let’s understand this problem with the help of a lock and key analogy.

Suppose we created a lock that can only be opened using a unique key made specifically for that lock.

What happens when you would try and open this lock with a key that wasn’t made for this lock? It wouldn’t fit.

Let’s create the file example.txt with the following contents.

𝘈Ḇ𝖢𝕯٤ḞԍНǏ
hello world

Let’s attempt to decode this file using the ascii codec using the following code.

Example 1:

with open("example.txt", "r", encoding="ascii") as f:
    lines = f.readlines()
    print(lines)

The output of the code:

Traceback (most recent call last):
  File "/home/fatina/PycharmProjects/examples/main.py", line 2, in <module>
    lines = f.readlines()
  File "/usr/lib/python3.10/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 0: ordinal not in range(128)

Let’s look at another more straightforward example of what happens when you encode a string using one codec and decode using a different one.

Example 2:

string = "𝘈Ḇ𝖢𝕯٤ḞԍНǏ"
encoded_string = string.encode("utf-8")
decoded_string = encoded_string.decode("ascii")
print(decoded_string)

In this example, we have a string encoded using the utf-8 codec, and in the following line, we try to decode this string using the ascii codec.

The output of the code:

Traceback (most recent call last):
  File "/home/fatina/PycharmProjects/examples/main.py", line 4, in <module>
    decoded_string = encoded_string.decode('ascii')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 0: ordinal not in range(128)

This happens because the contents of the file in example 1 and the string in example 2 were not encoded using the ascii codec, but we tried decoding these scripts using it. This results in the UnicodeDecodeError.

How to Solve the Unicode Decode Error in Python

Resolving this issue is rather straightforward. If we explore Python’s documentation, we will see several standard codecs available to help you decode bytes.

So if we were to replace ascii with the utf-8 codec in the example codes above, it would have successfully decoded the bytes in example.txt.

Example code:

with open("example.txt", "r", encoding="utf-8") as f:
    lines = f.readlines()
    print(lines)

The output of the code:

['𝘈Ḇ𝖢𝕯٤ḞԍНǏ\n', 'hello world']

As for the second example, you need only to do the same thing.

Example code:

string = "𝘈Ḇ𝖢𝕯٤ḞԍНǏ"
encoded_string = string.encode("utf-8")
decoded_string = encoded_string.decode("utf-8")
print(decoded_string)

The output of the code:

𝘈Ḇ𝖢𝕯٤ḞԍНǏ

It is important to mention that sometimes a string may not be completely decoded using one codec.

So if the need arrives, you can develop your program to ignore any characters that it cannot decode by simply adding the ignore argument like this:

with open("example.txt", "r", encoding="utf-8", errors="ignore") as f:
    lines = f.readlines()
    print(lines)

While this will skip any errors the compiler encounters while decoding some characters, it is important to mention that this can result in data loss.

We hope you find this article helpful in understanding how to resolve the UnicodeDecodeError in Python.