How to Fix the SyntaxError: Invalid Token in Python

1. the SyntaxError: invalid token in Python
2. Fix the SyntaxError: invalid token in Python

Every programming language has a set of rules and regulations known as the language syntax that needs to be followed to avoid an error. Similarly, Python has its syntax where every statement component is known as a token; white spaces separate these tokens, so the Python compiler understands and behaves accordingly.

The essential components called tokens can be variable names, keywords, operators, identifiers, delimiters, or any other built-in or user-defined features.

Let’s understand what tokens are and how whitespaces separate them.

token_demo = 39

In the above line of code, we have three different tokens, including token_demo, =, and 39. These tokens are further separated by white space, so the Python compiler understands them and behaves accordingly.

the SyntaxError: invalid token in Python

As discussed, every programming language has some syntax rules you must follow to avoid errors. Similarly, when you use the token invalidly, the Python compiler gives you an error message specifying the error.

In Python, octal, binary, decimal, and hexadecimal are all considered as integer literals. Let’s say you want to find the data type of an octal number; it will show the data type int or similarly for binary, decimal, and hexadecimal.

Suppose you want to store the date as a tuple in the format DD,MM,YYYY.

date = (04, 08, 2022)
print(date)

Output:

SyntaxError: invalid token

This gives you the SyntaxError: invalid token because the syntax rules were violated. The reason is that we are not allowed to use 0 as a prefix in any numbers until and unless you have correctly defined it as hexadecimal or string.

In Python 2x, you could use 04 and 08 because it has a special meaning that the number is in the octal format. But, later on, in Python 3x, this functionality was removed; now, 0 as a prefix has no meaning.

You can either define 04,08 as a single digit number like 4,8 or a string and then type cast the string to manipulate it accordingly.

Fix the SyntaxError: invalid token in Python

To fix the SyntaxError: invalid token in Python 3x, you can define the token properly as a string or hexadecimal to manipulate accordingly. In the above code, (04,08,2022) was throwing the SyntaxError, but there are multiple ways to overcome this error.

Let’s understand it through an example by defining the tokens as strings.

date = ("04", "08", "2022")
print(date)

Output:

('04', '08', '2022')

Defining the date format as a string has resolved the invalid token error. You got the date in your desired format.

And to make it more useful and understandable, we can use dictionaries in this scenario to fix the SyntaxError: invalid token in Python.

date = {"Day": "04", "Month": "08", "Year": "2022"}

for i, j in date.items():
    print(i, "\t", j)

Output:

Day      04
Month    08
Year     2022

Dictionaries are always handy for storing and presenting data with keys and values.