Microservices Tutorial: Node.js + DynamoDB + React

Serverless Framework + Node.js + DynamoDB + React on AWS

In the movie Diehard with a Vengeance, Detective John McClane must solve a riddle in order to disarm a bomb. Measure out exactly 4 gallons of water, and place the resulting weight on a scale, using only a 3 gallon, a 5 gallon jug, and a fountain of unlimited water to complete this task. 

In this tutorial we will build an application that consists of a frontend interface built in React, and a microservice API endpoint deployed on AWS Lambda, storing data to DynamoDB. 

The frontend application consists of a form to input the size of each jug and the target amount. And a graphical representation of the two jugs, animating the steps to reach the solution. When you submit the form, the app makes a request to the API endpoint and uses the solution returned to animate the jugs.

The API endpoint Lambda function will check DynamoDB to see if the solution is stored for the jug sizes and target amount from the request, and return the solution if found. If not found, it will calculate the solution, store it to DynamoDB, and return the solution.

Demo: https://react-diehard.s3.amazonaws.com/index.html

GitHub repository: https://github.com/jcummins54/diehard

Project Requirements:

• AWS account (this project can run on the free tier)

• Node.js 16

• Serverless framework

To get started, set up an AWS account if you don’t already have one. AWS provides a free tier for the first year of your account. Install Node 16 and the Serverless framework on your system. I recommend using NVM to install Node as it will allow you to easily switch Node version if needed. Then follow the instructions in the Serverless framework’s readme to install it.

Microservice

The Serverless framework uses AWS programmatic access credentials to build your infrastructure (Lamda and DynamoDB table) from the command line. The next section describes how to set up an AWS user with these credentials step-by-step. Skip ahead to Configuring the Serverless Microservice if you already have AWS credentials set up on your system.

Setting up AWS credentials

Once you have your AWS account created, you will create a user, and for that user you will create Security Credentials, that will be used for the Serverless framework. 

Log into the AWS console in your browser and go to the IAM dashboard user panel to create a user.

Click on the Add users button. You can name this user “serverless”, and check on the box labeled “Access key - Programmatic access”.

Click on Next: Permissions, then click on Create group. We are going to create a group named “Administrators” to assign to the “serverless” user. In the Create group dialog, enter “Administrators” in the Group name box, then check the policy “Administrator Access” in the policy list below. Then click on the Create group button at the bottom right of the dialog. There should now be an “Administrators” group in the Add user to group list. Check the box next to the “Administrators” group, then click Next: Tags. Note that in a production environment, you would want to create a group with only the permissions necessary for your serverless application, but we are using full admin privileges for expediency in this tutorial.

Tags are useful for organizing objects, and services in AWS, but we can skip this for now. Click on Next: Review. Verify the User name is “serverless” and the Permissions summary has the Group “Administrators” and click on Create user.

Here you will see your newly created user with an Access key ID and a hidden Secret access key. These are your AWS credentials. Copy them to a safe place, or click the Download .csv button to download the credentials to your system. The secret access key will never be available again, so make sure you copy it now, or you will have to create a new set of credentials later.

These credentials will allow you to run AWS CLI commands on your computer, and will be used by the Serverless framework to set up the infrastructure on your AWS account. Follow these instructions to set up the credentials on your system.

Configuring the Serverless Microservice

Clone the Github repo. Follow the steps in the readme file.

The code for the microservice is in the ‘serverless’ folder.

Serverless applications are configured using a YAML file. Open ‘./serverless/serverless.yml’ and let’s go through the config for an explanation of what each section does.

line 1: service: solutions
This is the name of the service in AWS, “solutions”. This will be used to name the Lambda functions that are created.

line 3: frameworkVersion: "3"
The version of the Serverless framework this serverless.yml is written for.

line 5: plugins:
This section defines which Serverless framework plugins the project uses. We are using the serverless-dynamodb-local and serverless-offline plugins to run the Lambda functions and an instance of DynamoDB locally. This will allow us to run and test code changes in the serverless functions without waiting for a code deploy after each change.

line 9: custom:
This section defines the configuration for the local DynamoDB instance when running locally.

line 11: stages:
This defines a whitelist of stage names where the local DynamoDB will be allowed to run. This is due to an issue documented on the serverless-dynamodb-local plugin’s Git repo.

line 12: ${sls:stage}
In Serverless YAML files, variables are declared with the syntax “${variable_name}”. Here we are defining a stage name for the stages whitelist dynamically to whatever stage name is set in the Serverless framework. The “sls” is an abbreviation for “serverless”. The default stage name is “dev”, so ${sls:stage} = “dev” by default. 

line 20: provider:
Declaration for the cloud service to deploy on; in this case, AWS.

line 22: runtime: nodejs16.x
The language and version the Lambda functions will run. The “x” in “nodejs16.x” indicates it will use the latest subversion available.

line 23: environment:
This section declares any supporting services the Lambda functions will use. In this case a DynamoDB table.

line 24: DYNAMODB_TABLE: ${self:service}-${sls:stage}
The name of the table. “self:service” refers to the service name defined on line 1, “solutions”. “sls:stage” is the stage name the Serverless framework is set to, default being “dev”. By default the DynamoDB table will be named “solutions-dev”. Once the service is deployed, you can login to your AWS console and you will see this table under DynamoDB in the default region you set in your AWS credentials (typically us-east-1, N. Virginia).

line 25: iam:
This section creates a user role with the permissions needed by the Lambda functions. In this case the Lambda functions will need the DynamoDB permissions listed under line 29 “Action”.

line 38: Resource: "arn:aws:dynamodb:${aws:region}:*:table/${self:provider.environment.DYNAMODB_TABLE}"
This is the resource these permissions will be allowed for. In this case we are allowing the Lambda functions to perform the DynamoDB actions listed on the table we defined above. Assuming your default AWS region is “us-east-1”, and Serverless stage is “dev”, the resource variable translates to "arn:aws:dynamodb:us-east-1:*:table/solutions-dev".

line 40: functions:
This section declares the Lambda functions we are creating. In this case a GET handler to query solutions, and a DELETE handler to drop the “solutions-dev” table and create a new empty “solutions-dev” table. The DELETE function serves as an example of how to perform a DynamoDB table drop and create.

line 41: get:
The name of the Lambda function we are defining, which will be “[service name]-[stage]-get”. Once you deploy, you will find in the AWS console, the Lambda function called “solutions-dev-get”.

line 42: handler: solutions/get.get
Declares the handler for this endpoint, which is the location of the js file and function in that file to run. Here the file is ./solutions/get.js and it will execute the function “get” in that file. 

line 45: path: solutions/{id}
The path with a variable “id” in braces which will get passed to the get handler. Example, a GET request is made to the Lambda endpoint https://[AWS_Lambda_URL]/solutions/3-5-4 it will run get(“3-5-4”) in the solutions/get.js file.

line 46: method: get
Defines the REST method (GET) that will execute this handler.

line 47: cors: true
This will allow CORS requests to this endpoint from the React app. If you would like to restrict the origin for CORS requests, update the file ./serverless/solutions/config.js responseHeaders object and edit the Access-Control-Allow-Origin header. You can also add other response headers here like Access-Control-Allow-Methods, Access-Control-Allow-Headers, etc…

line 49: delete:
Declares a DELETE handler which will drop and create the DynamoDB table. Runs the “resetSolutionsTable” function in the ./solutions/delete.js file.

line 56: resources:
Declares additional services to support the serverless application we are creating. Here we are creating a DynamoDB table that will be accessible by the Lambda functions.

line 62: TableName: ${self:provider.environment.DYNAMODB_TABLE}
The name of the table declared on line 24. In this case, “solutions-dev”.

line 63: AttributeDefinitions:
The schema for this table. In the following lines we are declaring a primary key “id”. This table will store the results for each id passed to the GET handler.

Deploying the Microservice

In the serverless folder, you can run the command “serverless deploy”. This will create the the Lambda functions and DynamoDB table in your AWS account. The output from this command should look like:

Deploying solutions to stage dev (us-east-1)

✔ Service deployed to stack solutions-dev (104s)

endpoints:
  GET - https://5nnnwdo6nc.execute-api.us-east-1.amazonaws.com/dev/solutions/{id}
  DELETE - https://5nnnwdo6nc.execute-api.us-east-1.amazonaws.com/dev/solutions
functions:
  get: solutions-dev-get (75 MB)
  delete: solutions-dev-delete (75 MB)

You will have a unique GET endpoint when you deploy. Copy the URL, and in the React app, ./front/src/App.js, replace the URL on line 9 with the URL you get, removing the “{id}” at the end of the URL.

If you wish to remove your serverless project, you can use the command “serverless remove”. This will remove your Lambda functions and all associated infrastructure that was created on deploy. 

Infrastructure

The basic infrastructure of an AWS serverless project includes an AWS Lambda function and an associated API Gateway. The API Gateway is an endpoint that will accept HTTP requests, execute the Lambda function, and return the response from the function. 

Note that AWS has more recently introduced Lambda Function URLs, which also serve as an endpoint to execute a Lambda function. Function URLs are more limited than API Gateways, but also can be far cheaper, as you pay no more than the Lambda execution fee when a Lambda Function URL is accessed, whereas an API Gateway under heavy load can get quite expensive just for the API Gateway fees alone. This blog post has a good overview of the differences between the two.

Lambda Functions

Let’s review the GET handler and take a look at the way a Lambda function handles the request-response cycle.

Open the file ./serverless/solutions/get.js. When the GET endpoint is called, the get function will be executed. This is the Lambda handler function defined in the serverless.yml file. When a Lambda function is called by a request to the API Gateway endpoint, the handler will be called with the parameters event and context.

export async function get(event, context) {

The event object contains the information provided by the request event. The context object provides information about the Lambda execution, including invocation, function, and execution environment.

We defined the GET handler path in serverless.yml as “solutions/{id}”. On deploy the endpoint I was given was “https://5nnnwdo6nc.execute-api.us-east-1.amazonaws.com/dev/solutions/{id}”. So if we make the GET request https://5nnnwdo6nc.execute-api.us-east-1.amazonaws.com/dev/solutions/3-5-4, the string “3-5-4” is accessible as the path parameter “id” in the event object received by the get handler. See line 61 in ./serverless/solutions/get.js:

event.pathParameters.id

An async Lambda function returns the response to the request. For the microservice, we are returning a JSON body. A successful response might look like this:

{
  statusCode: 200,
  headers: {
    "Content-Type": "application/json",
    "Access-Control-Allow-Origin": "*",
  },
  body: JSON.stringify({ foo: "bar" }),
}

You define what your response will look like. See the documentation for an overview of the Lambda response object. An example error response from ./serverless/solutions/get.js, line 33:

{
  statusCode: error.statusCode || 501,
  headers: responseHeaders,
  body: JSON.stringify({ error: "Couldn't create the solution item." }),
}

We have defined responseHeaders in ./serverless/solutions/config.js as they are the same for all responses.

Frontend

The frontend is a React app that consists of a form that allows the user to input the size of jug 1 and jug 2, and the target amount to find. It defaults to the amounts in the classic puzzle, a 3 gallon jug, a 5 gallon jug and the target amount of 4 gallons.

The file ./front/src/App.js contains the functionality that makes requests to the backend service and updates the view. On line 8 is defined the BASE_URL which sets the URL of the API service that provides a result.

The handleSubmit function on line 55 does input validation, and if valid, submits the form data to the BASE_URL.

The handleResponse function checks the result to see if there’s a possible solution, then starts an animation of the solution with the least number of steps (filling bucket 1 first, or filling bucket 2 first).

Because the React app is made up of static files which require no server-side rendering, it can be hosted on an S3 bucket, which can potentially be far cheaper than hosting on servers.

Following are instructions to set up the React app to an S3 bucket:

1. Build the React app: In your project folder, in terminal run: cd front && yarn build

2. In the AWS console, navigate to S3 buckets.

3. Click Create bucket

4. Give the bucket a name, like “diehard-puzzle”

5. Under Object Ownership, select ACLs enabled

6. Under Block Public Access settings for this bucket, uncheck Block all public access and check the “I acknowledge…” box that appears below

7. You can leave all the settings to the default and on the bottom of the page, click Create bucket

8. In the Buckets list, click on the new bucket you just created

9. Click on the Properties tab

10. At the bottom of the page, under Static website hosting, click Edit

11. For Static website hosting select Enable

12. For Index document enter “index.html”

13. At the bottom click on Save changes

14. Click on the Objects tab

15. Click on Upload

16. Add all the files from the ./front/build folder (drag and drop from your system file browser is easier here)

17. Scroll down and click on Permissions to open

18. Under Predefined ACLs, click on Grant public-read access, and then check “I understand the risk…” checkbox that appears

19. Then at the bottom click Upload, and your files should upload to the S3 bucket

The app should now be available at

http://diehard-puzzle.s3-website-us-east-1.amazonaws.com

assuming you named the bucket “diehard-puzzle”. Note that bucket names have to be unique in the AWS region across all accounts, so if that name is taken, just pick another. 

Conclusion

You should now have a working application, a React app that loads data from an API endpoint deployed on AWS Lambda. 

Thanks to 99x - this project is based on their serverless-react-boilerplate example. There are many more example projects to get you started with the Serverless framework.