How to decouple microservices using AWS Cloud Development Kit (CDK)

Leveraging a microservices architecture can offer numerous benefits to a solution, including scalability, maintainability, and flexibility. However, without careful implementation, the promise of microservices can fall short, leading to tightly coupled services that are difficult to manage and evolve.

This article explores how to effectively decouple services in AWS using the Cloud Development Kit (CDK) and Parameter Store. By leveraging Parameter Store for configuration management, developers can ensure that services facilitate seamless updates while reducing the risk of interdependencies.

What are microservices and why choose microservices over monolithic architecture?

Microservices are an architectural style that structures an application as a collection of significantly smaller, loosely coupled services, each of which are fine-grained and adopt lightweight protocols. 

Advantages of microservices

An alternative to more traditional monolithic architecture, microservices offer a number of advantages both during and post-development, including:

• Scalability: Microservices can be scaled independently, meaning resources can be allocated more efficiently based on the needs of each service
• Fault isolation and recovery: A failure in one service is won’t necessarily affect neighbouring services, while the segmented nature of microservices makes faults easier to detect and isolate
• Flexibility: Services can utilise different frameworks, programming languages, and databases, as per their individual needs
• Independent development: Development teams can work on services autonomously without relying on other teams to complete their tasks
• Independent deployment: Likewise, services can be deployed independently, enabling the implementation of continuous integration and continuous deployment (CI/CD) pipelines

This focus on single-responsibility principles makes microservices a popular option for organisations looking to expedite and enhance their delivery processes and system performance.

How does AWS facilitate microservices frameworks?

Microservices architecture relies heavily on the principle of loosely coupled services, which underpins most benefits offered by this framework. Loosely coupled services themselves typically communicate through well-defined interfaces that use lightweight protocols. 

This communication can occur both synchronously, usually via HTTP/REST, and asynchronously, using messaging queues. Amazon Web Services (AWS) helps facilitate the former via API Gateway and the latter via its Simple Notification Service (SNS) and/or Simple Queue Service (SQS) – often a combination of the two.

But while AWS offers the necessary components to facilitate microservices, the key advantage of its AWS Cloud Development Kit (CDK) is how it enhances the development phase by enabling the creation and deployment of services as stacks. In AWS, a stack is a collection of AWS resources that are managed and ultimately deployed as a single unit via AWS CloudFormation.

AWS resources that are managed and ultimately deployed as a single unit via AWS CloudFormation.

Utilising Infrastructure as Code (IaC), the entire infrastructure for a microservice can be defined in a template file, ensuring reusability and consistency. Each microservice can be deployed as its own stack, allowing for independent updates, scaling, and management. AWS CloudFormation also supports nested stacks, meaning you can create smaller, reusable templates for common components such as VPCs and databases, and include them in higher-level templates.

Entire stacks are deployed and destroyed in a single action, improving deployment performance and efficiency while negating much of the time and effort involved with managing individual resources.

This article may be of interest to you: Building Infrastructure as Code: A guide to AWS CloudFormation and CDK

Pitfalls: avoiding tightly coupled services and circular dependencies

Of course, loosely coupled services are still coupled to an extent, namely because services rely on the exchange of configuration data necessary to enable communication with one another. 

AWS provides a simple means of supporting the importing and exporting of component details such as API URLs and keys between stacks, or services. However, this approach throws a potential spanner in the works when building microservices by risking the establishment of circular dependencies, which can prove prohibitive during development.

Circular dependency issues arise where multiple components rely on each other directly or indirectly, creating a cycle. This inadvertent tight coupling of components typically results in added complexity and maintenance challenges, whereby changes in one component often necessitate changes in the dependent component.

This is particularly problematic in the context of AWS and IaC, where circular dependencies can prevent developers from destroying or deploying updates to stacks. Consequently, the development of entire services can be stunted by the configuration of a single component, turning an efficiency gaining feature into a substantial blocker.

Though circular dependencies can be resolved in AWS by reviewing and amending the applicable CloudFormation templates via the AWS console, this is a somewhat onerous hack that is best avoided.

Example scenario: Illustrating circular dependencies in microservices

The diagram below of a very simple framework provides an illustration of a scenario where a microservices design could result in circular dependencies. Here, our Request Handler Service is accepting requests from an external client and surfacing a response before distributing the request payload to the Logging Service. In turn, the Logging Service writes the request to a DynamoDB table before sending a message back to the Request Handler Service to confirm the status of the operation.

In this example, the Request Handler and Logging Services use a combination of SNS topics and SQS queues to communicate with one another asynchronously. However, this implementation involves either service importing the other’s SNS topic, resulting in a two-way circular dependency.

How to decouple services with AWS Systems Manager (SSM) and Parameter Store

Fortunately, AWS offers a solution by way of Systems Manager (SSM), a comprehensive service that enables you to manage AWS resources and applications at scale. Amongst its range of capabilities is configuration and secrets management via Parameter Store.

Rather than exporting and importing components directly between services, Parameter Store provides a centralised, secure location for the storage of configuration data, which is maintained in a hierarchical structure.

Crucially, Parameter Store enables microservices to retrieve the required configuration data at runtime, meaning the success of efforts to build, deploy, and destroy stacks are not contingent on the imported data being readily available at the time of the command in question.

Harnessing AWS SSM and Parameter Store for effective decoupling of services can be boiled down to a five-step process, which we illustrate below by resolving our hypothetical microservices framework using stacks developed using TypeScript.

1. Create an SSM parameter

The first step is to navigate to your stack file for the service in question and create an SSM parameter for the component that you need to export. The code snippet below demonstrates how we would implement this within the stack for our Request Handler Service, where we have created an SSM parameter for the service’s SNS topic ARN, which the Logging Service needs to access in order to consume messages that are published to the topic.

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import {
 aws_sns as sns,
 aws_ssm as ssm,
} from 'aws-cdk-lib';


export class RequestHandlerStack extends cdk.Stack {
 constructor(
   scope: Construct,
   id: string,
   props: cdk.StackProps,
   env: NodeJS.ProcessEnv,
 ) {
   super(scope, id, props);
  
   // Create Request Handler Service SNS topic
   const requestHandlerServiceSNSTopic = new sns.Topic(
     this,
     'requestHandlerServiceSNSTopicId',
     {
       displayName: 'Request Handler Service SNS Topic',
     },
   );


   // Store SNS topic ARN in SSM Parameter Store for cross referencing
   new ssm.StringParameter(this, 'requestHandlerServiceSNSTopicArnId', {
     description: 'Request Handler Service SNS topic',
     parameterName: env.REQUEST_HANDLER_SSM_PARAMETER_NAME_SNS_TOPIC_ARN,
     stringValue: requestHandlerServiceSNSTopic.topicArn,
   });
 }
}

The topic ARN is the specific component that is required by the consuming application in this instance, so we have used dot notation to access it before passing it through as a string value. We have also assigned a parameter name that will be used consistently throughout this process.

2. Define an environment variable for source and consumer services

Next, navigate to the .env file at the service level of your directory, or create one if there is none preexisting. Define your environment variable specifying the parameter name referenced during step one as the key, and a value of your choosing.

 

Note that the value that you use to populate your environment variable will specify the path within AWS Parameter Store that your value will be stored. As such, in the example below, we have specified a sensible directory structure for our SNS topic ARN which specifies the service from which it is sourced.

REQUEST_HANDLER_SSM_PARAMETER_NAME_SNS_TOPIC_ARN='/cdk/microservices/requestHandler/snsTopicArn'

Having defined your environment variable in the exporting stack, open the stack for the service that you intend to use to consume the parameter value and repeat this process in the adjacent ..env file, using the same key/value pair.

3. Inject SSM parameter into the consuming stack

Next, still within the consuming service, locate your entry point script. In a standard CDK repository, this will be named after your service by default and can be found within the /bin directory. This file is responsible for setting up the CDK application and defining which stacks are to be deployed.

Install the @aws-sdk/client-ssm package locally so that you can make use of the SSM client. If you are using NPM, the required terminal command is: npm install @aws-sdk/client-ssm

Your entry point script will need to be expanded, with an asynchronous function created to instantiate an SSM client before fetching the SSM parameter from the Parameter Store. It then needs to inject the retrieved value into the existing script which instantiates and deploys your CDK stack, as illustrated in the LoggingStack example below.

import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { LoggingStack } from '../lib/logging-stack';
import * as dotenv from 'dotenv';
import { SSMClient, GetParametersCommand } from '@aws-sdk/client-ssm';


dotenv.config();


const app = new cdk.App();


(async () => {
 try {
   // Create SSM client and command
   const client = new SSMClient({
     region: 'eu-west-2',
   });
   const command = new GetParametersCommand({
     Names: [
       process.env.REQUEST_HANDLER_SSM_PARAMETER_NAME_SNS_TOPIC_ARN as string,
     ],
   });
   // Execute command
   const output = await client.send(command);
   console.log(output);
   new LoggingStack(
     app,
     'LoggingStack',
     {
       env: {
         account: '01234567890',
         region: 'eu-west-2',
       },
       // Pass SSM parameter value through to stack
       ssmParameters: {
         requestHandlerSnsTopicArn: output.Parameters?.find(
           (value) =>
             value.Name ===
             process.env.REQUEST_HANDLER_SSM_PARAMETER_NAME_SNS_TOPIC_ARN,
         )?.Value,
       },
     },
   );
 } catch (error) {
   console.error(error);
 }

This function underpins the dynamic configuration as it is executed at runtime, ensuring that the latest and correct configuration is used for deploying the stack. It also helps maintain clean code by externalising configuration data.

4. Generate component instance in consuming stack file

Finally, you will need to update the stack file for your consuming service to generate a replica component from the value retrieved from the Parameter Store via your entry point script.

As illustrated in the script below, having imported the necessary modules, this initially means creating an interface which extends cdk.StackProps and adds the custom property ssmParameters that you injected your SSM parameter value into within the entry point script.

Populate this object with a string/undefined parameter with a name matching the key specified in step three. Note that the properties (props) of the stack in the example below have been updated to include this interface.

Then, introduce some conditional logic to generate an instance of the imported component, should the expected parameter be present within your properties. In our example, in addition to building the SNS topic, the conditional logic involves subscribing our SQS queue to the topic in question.

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import {
 aws_sqs as sqs,
 aws_sns as sns,
 aws_sns_subscriptions as sns_subscriptions,
} from 'aws-cdk-lib';


// Extend stack props interface to incorporate SSM params
interface LoggingStackProps extends cdk.StackProps {
 ssmParameters: {
   requestHandlerSnsTopicArn: string | undefined;
 };
}


export class LoggingStack extends cdk.Stack {
 constructor(
   scope: Construct,
   id: string,
   props: LoggingStackProps,
 ) {
   super(scope, id, props);


   // Create SQS queue to subscribe to SNS topic
   const requestHandlerServiceTopicSubscriber = new sqs.Queue(
     this,
     'requestHandlerServiceTopicSubscriberId',
     {
       retentionPeriod: cdk.Duration.days(7),
       visibilityTimeout: cdk.Duration.seconds(30),
     },
   );


   // Conditional actions based on presence of SSM parameter
   if (props.ssmParameters.requestHandlerSnsTopicArn) {
     // Create SNS topic from topic ARN retrieved
     const requestHandlerServiceSNSTopic = sns.Topic.fromTopicArn(
       this,
       'requestHandlerServiceSNSTopicId',
       props.ssmParameters.requestHandlerSnsTopicArn,
     );
     // Subscribe SQS queue to SNS topic
     requestHandlerServiceSNSTopic.addSubscription(
       new sns_subscriptions.SqsSubscription(
         requestHandlerServiceTopicSubscriber,
       )
     );
   }
 }
}

5. Repeat and deploy

By this stage, you will now have the configuration required to successfully share a component between services while ensuring they remain decoupled. Remember to repeat the process for any other configuration data that needs to be shared, before deploying each of your stacks.

Conclusion

Decoupling microservices effectively is crucial to realise their benefits of scalability, maintainability, and flexibility. This article has demonstrated how to use the AWS Cloud Development Kit (CDK) and Parameter Store to manage configuration data and avoid common pitfalls like circular dependencies.

Key insights include:

1. Avoiding Circular Dependencies: Ensuring services do not rely on each other directly to prevent development and deployment issues.
2. Leveraging AWS SSM and Parameter Store: Storing configuration data centrally and securely to enable runtime retrieval and decoupled service operation.
3. Utilising Infrastructure as Code (IaC): Streamlining infrastructure management with reusable, consistent templates.

By implementing these practices, organisations can build efficient and independent microservices.

As AWS Partners, we are here to help you design and implement effective microservices architectures. Contact us for expert assistance with your AWS projects and to ensure your microservices framework is optimised for success.