Creating a Lambda Triggered by an SQS Queue with AWS CDK

In this article, we will explore how to create an AWS-based architecture where a Lambda is triggered by an SQS queue. We will use AWS CDK (Cloud Development Kit) in TypeScript to define our infrastructure. This type of setup is very common when creating an asynchronous processing system that handles large-scale events.

Context

To illustrate this solution, imagine a system that works as follows: a Next.js application exposes a route that allows uploading PDF files to an S3 bucket. Once the files are uploaded, a status document is created in a DynamoDB database. Then, a message is sent to a predefined SQS queue. This SQS queue triggers a Lambda that processes the content and saves the results to a file in S3.

The goal is to show how to create this architecture using AWS CDK.

Preparation: Installing and Creating the CDK Project

To get started, make sure you have AWS CDK installed globally on your machine:

npm install -g aws-cdk

Next, create the structure of our CDK project in TypeScript:

cdk init app --language typescript

AWS Environment Configuration

Before starting, we need to verify that our AWS account is properly configured. To do this, run:

aws sts get-caller-identity --profile default --query "Account" --output text

Then, create an .env file to define the default account and region settings:

1CDK_DEFAULT_ACCOUNT=default
2CDK_DEFAULT_REGION="eu-west-3"

Bootstrapping the Environment

To initialize our CDK environment:

cdk bootstrap

This command creates a CloudFormation stack that will help manage CDK resources.

Building and Synthesizing the CDK Application

Let's build our TypeScript project:

npm run build

To generate the associated CloudFormation template:

cdk synth

Defining the SQS Queue and Lambda in CDK

Now, let's add the SQS queue and the Lambda that will be triggered by this queue. Here is the code to add in the file lib/poc-sqs-lambda-cdk-stack.ts:

1import * as cdk from 'aws-cdk-lib';
2import { Construct } from 'constructs';
3import * as lambda from 'aws-cdk-lib/aws-lambda';
4import * as sqs from 'aws-cdk-lib/aws-sqs';
5import * as lambdaEventSources from 'aws-cdk-lib/aws-lambda-event-sources';
6
7export class AwsCdkStackStack extends cdk.Stack {
8  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
9    super(scope, id, props);
10
11    // Create the SQS queue
12    const queue = new sqs.Queue(this, 'AwsCdkStackQueue', {
13      visibilityTimeout: cdk.Duration.seconds(300),
14      queueName: 'AwsCdkStackQueue',
15    });
16
17    // Create the Lambda function
18    const fn = new lambda.Function(this, 'AwsCdkStackLambda', {
19      runtime: lambda.Runtime.NODEJS_18_X,
20      code: lambda.Code.fromAsset('lambda'),
21      handler: 'index.handler',
22      functionName: 'AwsCdkStackLambda',
23    });
24
25    // Add the SQS queue as an event source for the Lambda
26    fn.addEventSource(new lambdaEventSources.SqsEventSource(queue, {
27      batchSize: 1,
28      maxBatchingWindow: cdk.Duration.seconds(20),
29    }));
30  }
31}
32
33// Main entry point to create the application
34const app = new cdk.App();
35new AwsCdkStackStack(app, 'AwsCdkStackStack');

Creating the Lambda

The Lambda code is simple and is located in the /lambda/index.js directory:

1exports.handler = async (event) => {
2  console.log("Received: ", event);
3  return {};
4};

Deploying the CDK Stack

To deploy the stack:

cdk deploy

Testing and Modifications

To test the SQS queue and the Lambda, simply send a message to the queue. The Lambda should be automatically triggered and display the message in CloudWatch logs.

If you need to modify your application, you can run:

cdk diff

This will allow you to see the differences before redeploying with:

cdk deploy

To delete the stack:

cdk destroy

Conclusion

In this article, we have seen how to create an asynchronous architecture based on an SQS queue and an automatically triggered Lambda, all defined with AWS CDK. This allows for elegant and scalable asynchronous processing. Feel free to explore more of the possibilities offered by AWS CDK to automate and organize your cloud deployments.