This will be a 3 part blog posting that will address a transitionary architectural plan, the second part of this series focuses on leveraging Lambda to facilitate your build process to a completely server-less implementation.
Part 2: Making your Build Environment Server less
If you have grown to a point where you can isolate your production resources to one particular account and every subsequent developer or technical resource gets their own AWS account with federated access to the production account after they have proven their capacities you have created an extensible implementation for on boarding technical resources. The next step is to now create a server less implementation of your build process such that you leverage CodeCommit, CodeDeploy, CodePipeline, and all native container services within AWS.
This transition is conceptually the hardest for most startups and smaller scale enterprise technical companies to conceptualize and accept. You have an existing pipeline that builds your product effectively, this procedure is stable and your core engineering group have built an entire technical business on it. Throwing that away to adopt an AWS centric development and deployment process so that future growth is capable is the first hurdle. Let’s take the previous example of an architecture below as a starting point.
This implementation is fantastic if you are 4 developers, and need to produce an application suite quickly. The problem with this is that it is not scalable at all and unnecessarily maintaining a process that doesn’t need to utilize servers at all. You can simply write a lambda script, modify your jenkins server to accept API calls, and deploy to the elastic container service, then within your lambda function call the infrastructure management tool via an API [rundeck] to deploy new docker image to the machine. Now you don’t have to manage anything but a lambda function to build your environment. It’s easier if I break it into steps visually with the cloudcraft images.
1) Modify Your Jenkins machine to accept API calls
Your Jenkins server must have a way to accept API calls, your goal now is to spin up a machine that has some form of persistence with the jenkins job definition such that you can call the “build” execution via the API.
2) Move your application github repo to CodeCommit
Do not check in code into github anymore, put it into a private github repository within your production AWS account, the purpose for this is so that you can replicate the production environment to a developers environment with the github repository already created.
3) Launch a Infrastructure Management Tool [Rundeck]
This will be your deployment machine API this will be utilized to deploy your docker images to machines within your system.
The interim diagram should look very similar to this.