Pulumi – IaC on steroids

When it comes to Infrastructure-as-a-code, Terraform has emerged as the clear winner among other platforms, especially when comparing it to its prime competitor – good ‘ol AWS CloudFormation (Sorry Jeff… :)). This open-source project has swept the DevOps community, peaking with over 17k stars on GitHub. Its main advantage is being cloud agnostic supporting hundreds of providers, often less verbose than CloudFormation & has a great module system. But what about efficiency?

Contrary to Terrafom, Pulumi enables you to describe the same infrastructure resources as real code, providing huge productivity gains, and has deep support for cloud native technologies such as Kubernetes and serverless programming. Philosophically, this technology obliterates the last traces of the dev/ops divide, for better and for worse. It brings cloud infrastructure into your code the way system libraries bring operating system details into your code. And on top of that, with language facilities for abstraction, some of the low-level details can be hidden, in the name of portability, in the same way that Java and its successors tried hiding details specific to operating systems.

Deploying Containers

Following example demonstrates how easy it is to create, deploy and provision infrastructure hosting Docker container (Simple web-server written in Crystal) on AWS Fargate, using Pulumi:

Prerequisites

  • Download and install Pulumi
  • Configure AWS credentials. Make sure the corresponding IAM Role has full VPC/ECS/IAM/ECR permissions.
  • Download & install and run Docker
  • Download & instal Crystal Lang (Optional, as we’re encapsulating the code into Docker image)

Bootstrap Pulumi project

$ mkdir crystal-web-server
$ cd crystal-web-server
$ pulumi new javascript

Following file describes the desired infra using JavaScript. It provisions high level object service, which encapsulate all dependencies needed to spin-up AWC ECS cluster with 2 running containers. Replace the contents of index.js with the following:

const cloud = require("@pulumi/cloud");

let service = new cloud.Service("pulumi-crystal-server", {
    containers: {
        nginx: {
            build: ".",
            memory: 128,
            ports: [{ port: 80 }],
        },
    },
    replicas: 2,
});

exports.url = service.defaultEndpoint.apply(e => `http://${e.hostname}`);

Add package.json which contains the required dependencies for JavaScript Pulumi:

{
    "name": "javascript",
    "main": "index.js",
    "dependencies": {
        "@pulumi/cloud": "^0.18.0",
        "@pulumi/cloud-aws": "^0.18.0",
        "@pulumi/pulumi": "latest"
    }
}

Create Pulumi.yaml project configuration file:

name: crystal-web-server
runtime: nodejs
description: A Javascript Pulumi do deploy Crystal Lang server on AWS Fargate container

Create Pulumi.prod.yaml stack manifest file. Set the desired region you want the stack to be hosted at:

config:
  aws:region: us-east-1
  cloud-aws:useFargate: "true"
  cloud:provider: aws

Install Pulumi dependencies:

$ npm install

Create Crystal Lang Dockerfile

In root directory, create Dockerfile:

FROM crystallang/crystal:latest

WORKDIR /app
ADD . /app
RUN crystal build src/crystal-web-server.cr --release

EXPOSE 80

CMD "./crystal-web-server"

Create subfolder src and add crystal-web-server.cr containing the Crystal web-server code:

require "http/server"
require "json"

def generate_response(request)
 json = JSON.build do |json|
 json.object do
   json.field "remote_address", "#{request.remote_address}"
   json.field "method", "#{request.method}"
   json.field "host", "#{request.host}"
   json.field "path", "#{request.path}"
   json.field "headers" do
     json.array do
       request.headers.each do |key, value|
         json.object do
           json.field "#{key}", "#{value}"
         end
       end
     end
   end
   json.field "body", "#{request.body}"
   json.field "query_params", "#{request.query_params}"
   json.field "resource", "#{request.resource}"
   json.field "version", "#{request.query_params}"
 end
end
 return json
end

server = HTTP::Server.new do |context|
 response_json = generate_response(context.request)
 context.response.content_type = "application/json"
 context.response.print "#{response_json}"
end

address = server.bind_tcp "0.0.0.0", 80
puts "Listening on http://#{address}"
server.listen

Run Pulumi

Preview and deploy changes via pulumi up. This will take a few minutes. Pulumi automatically builds and provisions a container registry (ECR or ACR), builds the Docker container, and pushed the image into the repository. This all happens automatically and does not require manual configuration on your part.

$ pulumi up
Previewing update of stack 'crystal-web-server-prod'
Previewing changes:
...
Diagnostics:
    ...
    global: global
    info: Building container image 'pulum-134fa290-container': context=.
...
Do you want to perform this update? yes
Updating stack 'crystal-web-server-prod'
...

---outputs:---
url: "http://83dec887-42a040f-3858d3cec4b44f45.elb.us-east-1.amazonaws.com"

info: 20 changes performed:
    + 20 resources created
Update duration: 14m53.44141303s

curl the url given in the output:

$ curl http://83dec887-42a040f-3858d3cec4b44f45.elb.us-east-1.amazonaws.com
{
    "remote_address": "172.31.47.183:27248",
    "method": "GET",
    "host": "83dec887-42a040f-3858d3cec4b44f45.elb.us-east-1.amazonaws.com",
    "path": "/",
    "headers": [
        {
            "Host": "[\"83dec887-42a040f-3858d3cec4b44f45.elb.us-east-1.amazonaws.com\"]"
        },
        {
            "User-Agent": "[\"curl/7.54.0\"]"
        },
        {
            "Accept": "[\"*/*\"]"
        }
    ],
    "body": "",
    "query_params": "",
    "resource": "/",
    "version": ""
}

Crystal server, which runs on AWS Fargate, has responded with the request body and headers.

Conclusions

Plumi is a great tool – Instead of writing repeatable and cumbersome code, it enables engineers to focus on efficiency. Although the project itself is far from being mature, I truly believe that in the near future it’ll be a must-have tool in the DevOps tool-belt.

GitHub repo: https://github.com/pasha1986/crystal-web-server

Leave a Reply