How to Get All Keys in Redis Database With Python

1. Use keys() to Get All Keys in Redis Database
2. Use scan_iter() to Get All Keys in Redis Database
3. Use zip_longest to Get All Keys in Redis Database

The Redis design system will be familiar if you know JSON. It uses a key-value structure and a distributed and in-memory approach to achieve a resilient database.

Hashes, lists, sets, sorted sets, strings, JSON, and streams are one of the many data structures that Redis supports. This open-source database supports different languages, including Python, and if you are developing a backend system with it, some modules and packages help with that.

One of the many operations you would often do with databases is retrieving data, and in a database like Redis, the keys are important to achieve such an operation.

This article will discuss getting all the keys in a Redis database.

Use keys() to Get All Keys in Redis Database

To make use of redis, we need to have it installed; you can check through the Redis download page to find out how. For Linux and macOS users, it’s quite easier; however, for Windows users, you might have to make use of the Windows Subsystem for Linux (WSL2), and you can follow their instructions video guide.

With the assumption that you have your Redis database set up, we will install the redis package, which provides client access to the Redis database. To install it, we will use the pip command.

pip install redis

Output:

Collecting redis
  Downloading redis-4.3.4-py3-none-any.whl (246 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 246.2/246.2 kB 794.4 kB/s eta 0:00:00
Collecting deprecated>=1.2.3
  Downloading Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)
Collecting async-timeout>=4.0.2
  Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting packaging>=20.4
  Downloading packaging-21.3-py3-none-any.whl (40 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.8/40.8 kB 1.9 MB/s eta 0:00:00
Collecting wrapt<2,>=1.10
  Downloading wrapt-1.14.1-cp310-cp310-win_amd64.whl (35 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
  Downloading pyparsing-3.0.9-py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.3/98.3 kB 624.8 kB/s eta 0:00:00
Installing collected packages: wrapt, pyparsing, async-timeout, packaging, deprecated, redis
Successfully installed async-timeout-4.0.2 deprecated-1.2.13 packaging-21.3 pyparsing-3.0.9 redis-4.3.4 wrapt-1.14.1

So, the redis package uses wrapt, pyparsing, async-timeout, packaging, and deprecated modules to power its module.

To make use of the redis package, we need to import it.

import redis

Moving on to the main issue, we can use the keys() method provided by the redis module to access and get all the keys in.

The key() method returns a list of keys matching the pattern passed within its arguments from a given Redis database. There is a default pattern if no argument is not passed, which is *, which means all the keys.

To show the method at work, we have manually prepopulated the redis database with the alias Temp with some keys using the +Key button or the following code.

import redis

redisHost = "localhost"
redisPort = 6379
redisDecodeRes = True

r = redis.StrictRedis(host=redisHost, port=redisPort, decode_responses=redisDecodeRes)

r.set("ConnectionPool", "Ox1212af34w3141")

The above code results in the key ConnectionPool being added to the Temp database with the corresponding value.

The set() method applies the key-value pair to the database, and the StrictRedis() method creates a Redis connection object that gives us access to the database.

To show the database (with the alias Temp) and its keys via a GUI, we can use the RedisInsight application, as seen in the image.

A total of 11 keys have been added to the database manually to test the key() method.

Now, to the key() method, we will have to use the StrictRedis() method to create a Redis connection object to access the keys. The host, port, and decode_responses arguments are passed to define the parameters of the connection.

The host and port define the hostname and port number, and the decode_responses defines that the data passed is decoded to Python strings that we can easily work with. The keys() method then accesses all available keys since no argument isn’t passed.

import redis

redisHost = "localhost"
redisPort = 6379
redisDecodeRes = True

r = redis.StrictRedis(
    host=redisHost, port=redisPort, decode_responses=redisDecodeRes, db=0
)

print(r.keys())

Output:

['bar-two', 'information', 'bar-one', 'details', 'foo', 'jinku', 'bar', 'User-One', 'delft', 'bar-three', 'ConnectionPool']

We have a list of all the keys in the Temp database, which we can work with.

If we have a key pattern we need, we can pass it as an argument. Let’s list all the keys that start with bar.

print(r.keys(pattern="bar*"))

Output:

['bar-two', 'bar-one', 'bar', 'bar-three']

Use scan_iter() to Get All Keys in Redis Database

With a large database, scan_iter() allows us to manage the data better within our Python application. Also, the key() method blocks the server and prevents other usage operations, and with scan_iter(), its batch-based operation allows other usage operations.

Though keys() might be faster, it is not great for multiple request-based systems.

Now, let’s see it in action.

import redis

redisHost = "localhost"
redisPort = 6379
redisDecodeRes = True

try:
    r = redis.StrictRedis(
        host=redisHost, port=redisPort, decode_responses=redisDecodeRes, db=0
    )
    for key in r.scan_iter(match="bar*"):
        print(key)
except Exception as e:
    print(e)

Output:

bar-three
bar-one
bar-two
bar

Using the try/except helps deal with connection issues when we try to work with the database. After the connection using the StrictRedis(), we use the match argument to define the key pattern we are looking for and loop through the result to give the key.

Use zip_longest to Get All Keys in Redis Database

As we stated, for larger databases with lots of keys, the scan_iter() method is better, but we can improve it further by retrieving the keys in batches of a specified number to better manage the result.

To create the batch, we need the itertools module which provides different iterators (or methods) that can be used for different cases. Within the itertools module, we have the zip_longest method which returns a zip_longest object whose .next() method returns a tuple and aggregates elements from the iterable passed to it.

We can use the zip_longest() method to make a function that creates a batch of a specified number of keys depending on the arguments passed. For example, we create a batch of 2, which can be used for many cases.

import redis
from itertools import zip_longest

redisHost = "localhost"
redisPort = 6379
redisDecodeRes = True

try:
    r = redis.StrictRedis(
        host=redisHost, port=redisPort, decode_responses=redisDecodeRes, db=0
    )

    def batch(iterable, num):
        initIter = [iter(iterable)] * num
        return zip_longest(*initIter)

    for keyBatch in batch(r.scan_iter("bar*"), 2):
        print(keyBatch)
except Exception as e:
    print(e)

Output:

('bar-three', 'bar-one')
('bar-two', 'bar')