Parallel CDK stack deployments with GitHub Actions
How
Here on the Billing team our primary application is a single monorepo with 12 CDK stacks deployed using GitHub Actions. When we dive into the pipeline, we realize that we have a number of redundant steps that are increasing deployment times. These duplicate steps are a result of the way CDK deploys dependent stacks.
For instance, let’s take four of our stacks: Secrets, API, AsyncJobs, and Dashboard. The API stack relies on Secrets, while Dashboard relies on API and AsyncJobs. If we only need to update the Dashboard stack, CDK will still force a no-op deployment of Secrets, API, and AsyncJobs. This pipeline will always start from square one and run the full deployment graph for each stack.
package.json) were changed. We alter the standard cdk deploy –all because we have some stacks listed that we don’t want to deploy. Instead, we use cdk deploy –exclusively Secrets-Stack API-Stack AsyncJobs-Stack Dashboard-Stack, which decreases our median deployment time by a full minute and decreases our slowest deployment time by 10 minutes.need: [...] option.cdk synth) for each stage we'd like to deploy and then parse the resulting manifest.json in the cdk.out directory.const execSync = require("child_process").execSync;
const fs = require('fs');
const path = require('path');
const { parseManifest } = require('./stackDeps');
const stages = ['demo'];
const stackGraphs = {};
stages.map((stage) => {
execSync(`STAGE=${stage} npx cdk synth`, {
stdio: ['ignore', 'ignore', 'ignore'],
});
stackGraphs[stage] = parseManifest();
});
const data = JSON.stringify(stackGraphs, undefined, 2);
fs.writeFileSync(path.join(__dirname, '..', 'generated', 'graph.json'), data);
Our stack graph:
{
"demo": {
"stacks": [
{
"id": "Secrets-demo",
"name": "Secrets-demo",
"region": "us-east-1",
"dependencies": []
},
{
"id": "Datastore-demo",
"name": "Datastore-demo",
"region": "us-east-1",
"dependencies": []
},
{
"id": "AsyncJobs-demo",
"name": "AsyncJobs-demo",
"region": "us-east-1",
"dependencies": [
"Datastore-demo",
"Secrets-demo"
]
},
{
"id": "Api-demo",
"name": "Api-demo",
"region": "us-east-1",
"dependencies": [
"Datastore-demo",
"Secrets-demo"
]
},
{
"id": "Dashboards-demo",
"name": "Dashboards-demo",
"region": "us-east-1",
"dependencies": [
"AsyncJobs-demo",
"Api-demo"
]
}
]
}
}
The workflow below is easy to define programmatically from the stack graph, which allows GitHub Actions to do all the heavy lifting of orchestrating the jobs for us.
Due to network latency and variance in the GitHub runner setup, this change sometimes causes our fastest deployments to slow down. However, the median deployment performs 1 minute faster. Most importantly, our p99 deployment times always perform 12 minutes faster than before: 18 minutes!
How to get started
The sample repo provides an un-opinionated example with just one stage to deploy. You could build on this in a few ways, such as specifying stage orders, implementing integration tests or whatever else is needed in your stack.
Get blog posts delivered to your inbox.