How to Query DynamoDB in Python

1. Introduction to DynamoDB
2. Introduction to Boto3
3. Create Table in DynamoDB Using Boto3
4. Delete Tables in DynamoDB via Boto3
5. List Tables in DynamoDB via Boto3
6. Pagination in DynamoDB via Boto3
7. Sorting in DynamoDB via Boto3
8. Get Items in DynamoDB via Boto3
9. Scan Items in DynamoDB via Boto3
10. Global Secondary Index in DynamoDB
11. Backup a DynamoDB Table Using Boto3
12. Conclusion

This article will discuss how we can query Amazon DynamoDB using python. We shall also discuss what Boto3 is about and why it needs to query DynamoDB.

Introduction to DynamoDB

DynamoDB is a NoSQL database service that provides built-in security mechanisms, continuous backup, memory caching, and useful data import/export tools.

DynamoDB automatically replicates data across different Availability Zones (AZs) in an AWS Region, enhancing security against outages and data loss. Numerous security features are available with DynamoDB, including fine-grained access control, encryption at rest, and user activity recording.

Introduction to Boto3

Boto3 is a built-in python library developed for Amazon Web Services (AWS). This library is useful for interacting, creating, configuring, managing and using different services of Amazon, including DynamoDB.

Install and Import Boto3

To install the Boto3 library in our Python workbench, use the following command.

pip install boto3

Output:

Code:

import boto3

This shall import the boto3 library to our notebook.

Connecting Boto3 to DynamoDB

We will use the following code to connect to our DynamoDB using Boto3.

Code:

import boto3

client = boto3.client(
    "dynamodb",
    aws_access_key_id="yyyy",
    aws_secret_access_key="xxxx",
    region_name="us-east-1",
)

Keep in mind that the Database must already be created on AWS DynamoDB to connect to it via the Python Boto3 library.

Create Table in DynamoDB Using Boto3

Tables on DynamoDB can be created in multiple ways. This includes making use of Amazon CLI, AWS Console or by using Boto3.

Here we will be using Boto3 with the Database that we already are connected to by following the steps mentioned previously. Knowing how the DynamoDB client and table resource vary can let you use either one depending on your needs since this table resource can significantly simplify some operations.

Developers can create, update, and delete DynamoDB tables and all contents using the boto3.resource('dynamodb') resource. This resource supports item-level security using Conditional Expressions and table-level security.

For serverless access to DynamoDB data, utilize the boto3.resource('dynamodb') resource in conjunction with AWS Lambda functions.

import boto3

dynamodb = boto3.resource("dynamodb", region_name="us-west-2")

table = dynamodb.create_table(
    TableName="Movies",
    KeySchema=[
        {"AttributeName": "year", "KeyType": "HASH"},  # Partition key
        {"AttributeName": "title", "KeyType": "RANGE"},  # Sort key
    ],
    AttributeDefinitions=[
        {"AttributeName": "id", "AttributeType": "N"},
        {"AttributeName": "createdAt", "AttributeType": "S"},
    ],
    ProvisionedThroughput={"ReadCapacityUnits": 10, "WriteCapacityUnits": 10},
)
print("Table status:", table.table_status)

Table creation via Boto3 takes some time before it is active and shown. A slight wait is recommended before retrying, or we can use a waiter function to let us know when the table is active.

import botocore.session

session = botocore.session.get_session()
dynamodb = session.create_client(
    "dynamodb", region_name="us-east-1"
)  # low-level client

waiter = dynamodb.get_waiter("table_exists")
waiter.wait(TableName="my-table-name")

Delete Tables in DynamoDB via Boto3

Deleting tables in DynamoDB is quite simple. We need to type in the following code snippet if we change our mind and want to delete a table from our Database.

import boto3

dynamodb = boto3.resource("dynamodb", region_name="us-east-1")

dynamodb.delete_table(TableName="Movies")

Keep in mind that the table we’re trying to delete already exists in the Database with the same name or will give us an error message(Table not found).

List Tables in DynamoDB via Boto3

If we are interested in finding the list of all tables available in our region, we need to use the list() function to list all the available tables in the region.

import boto3

dynamodb = boto3.resource("dynamodb", region_name=region)

tables = list(dynamodb.tables.all())
print(tables)

It must be made sure that the total number of tables is less than 100, or we will have to paginate through the list.

Pagination in DynamoDB via Boto3

A single list call returns results up to 1MB of items. For a further listing, we need to issue a second call. Another call with ExclusiveStartKey should be made to get more items from this table if LastEvaluatedKey was present in the response object.

dynamodb = boto3.resource("dynamodb", region_name=region)

table = dynamodb.Table("my-table")

response = table.query()
data = response["Items"]

# LastEvaluatedKey indicates that there are more results
while "LastEvaluatedKey" in response:
    response = table.query(ExclusiveStartKey=response["LastEvaluatedKey"])
    data.update(response["Items"])

Sorting in DynamoDB via Boto3

On the database side, DynamoDB only provides one method of sorting the results. Your ability to sort items is restricted to doing so in the application code after receiving the results if your table lacks one.

However, you may use the following syntax to sort DynamoDB results using the sort key either in descending or ascending.

import boto3

dynamodb = boto3.resource("dynamodb", region_name=region)

table = dynamodb.Table("my-table")

# true = ascending, false = descending
response = table.query(ScanIndexForward=False)
data = response["Items"]

Get Items in DynamoDB via Boto3

The GetItem() function is used if we’re interested in getting a specific item from our DynamoDB Database.

import boto3

dynamodb = boto3.resource("dynamodb", region_name=region)

table = dynamodb.Table("my-table")

response = table.get_item(Key={primaryKeyName: "ID-1", sortKeyName: "SORT_2"})

Scan Items in DynamoDB via Boto3

All the information in tables can be accessed through scanning. The data will be returned by the scan() method after reading each item in the table.

The table scan operation can return fewer and the desired results when you add other choices, like the FilterExpression.

from boto3.dynamodb.conditions import Key, Attr
import boto3

dynamodb = boto3.resource("dynamodb")

table = dynamodb.Table("Employees")

response = table.scan()
response["Items"]

print(response)

# or


dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table("Employees")

response = table.scan(FilterExpression=Attr("Department").eq("IT"))

print("The query returned the following items:")
for item in response["Items"]:
    print(item)

Global Secondary Index in DynamoDB

You can query properties that aren’t included in the primary key of the main table by using a global secondary index. Doing this can avoid the delays and inefficiencies brought on by a full table scan procedure.

The global secondary index will also include properties from the primary table but will be structured using a different primary key, making queries faster.

Backup a DynamoDB Table Using Boto3

To create on-demand backups for the DynamoDB table using Boto3, utilize the create_backup() method and pass the destination backup table name together with the table name.

import boto3

client = boto3.client("dynamodb")

response = client.create_backup(TableName="Employees", BackupName="Employees-Backup-01")
print(response)

Conclusion

This article explained how to query DynamoDB tables, create, list, and do other CRUD activities on Amazon DynamoDB using Python Boto3 and perform other maintenance tasks.