This article is the first in many – as mentioned in the last article, I will be writing more articles over the course of the next several months on AWS, touching on as much of the service as I can get my hands on.

For my first article, I am starting with the basics – CloudFormation, Amazon VPC (Virtual Private Cloud), Elastic Load Balancing, and finally, EC2. The services that are covered here serve as some of the basic building blocks of an Amazon infrastructure, and some of the oldest components of AWS. This will serve as a entry point not only into further articles, but for myself, and you the reader, into learning more about AWS, and being more comfortable with the tools that manage it.

However, this article got so large that I have had to separate it into 3 parts! so, for the first article, I will be mainly covering CloudFormation, the second one will cover VPC, and the third one will cover ELB and EC2.

Viewing the Technical Demo

All of the items covered in this article have been assembled into a CloudFormation template that can be downloaded from the github page:

https://github.com/vancluever/aws-basics-using-cloudformation

There is a README there that provides instructions on how to download and use the template.

Introduction

I selected the first features of AWS to cover from a way that could give someone that is already familiar with the basic concepts of modern cloud computing and devops (which includes virtual infrastructure, automation, and configuration management) an idea of what that means when dealing with AWS and its products. Ultimately, this meant creating an example that would create a full running basic “application” that could be created and destroyed with a single command.

CloudFormation is Amazon’s primary orchestration product, and covers a wide range of services that make up the core of AWS’s infrastructure. It is used in this article to manage every service I touch – besides IAM and access keys, which are not covered here, nothing in this example has been set up through the AWS console. Given that the aforementioned two items have been set up, all that is necessary to create the example is a simple aws cloudformation CLI command.

Amazon VPC is the modern (and current) virtual datacenter platform that makes up the base of AWS. From a VPC, networks, gateways, access lists, and peer connections (such as VPN endpoints and more) are made to cover both the needs of a public-facing application and the private enterprise. It is pretty much impossible to have a conversation about AWS these days without using VPC.

Amazon EC2 is one of Amazon’s oldest and most important products. It is the solution that gave “the cloud” its name, and while Amazon has created a large number of platform services that have removed the need for EC2 in quite a few applications (indeed, one can run an entire application these days in AWS without a single EC2 instance), it is still highly relevant, and will continue to be so long as there is ever a need to run a server and not a service. Products such as VPC NAT instances (covered in part 2) and Amazon EC2 Container Service (not covered here) also use EC2 directly with no transparency, so its importance in the service are still directly visible to the user.

I put these three products together in this article – with CloudFormation, a VPC is created. This VPC has two subnets, a public subnet and a private subnet, along with a NAT instance, so that one can see some of the gotchas that can be encountered when setting up such infrastructure (and hopefully avoid some of the frustration that I experienced, mentioned in the appropriate section). An ELB is also created for two EC2 instances that will, upon creation, do some basic configuration to make themselves available over HTTP and serve up a simple static page that allows one to see both the ELB and EC2 instances in action.

CloudFormation

CloudFormation is Amazon’s #1 infrastructure management service. With features that cover both deployment and configuration management, the service supports over two dozen AWS products, and can be extended to support external resources (and AWS processes not directly supported by CloudFormation) via custom resources.

One does not necessarily need to start off with CloudFormation completely from scratch. There are templates available at the AWS CloudFormation Templates page that have both examples of full stacks and individual snippets of various AWS services, which can be a great time saver in building custom templates.

The following few sections cover CloudFormation elements in further detail. It is a good idea to consult the general CloudFormation User Guide, which provides a supplemental to the information below, and also a good reference while designing templates, be it starting from scratch or using existing templates.

CloudFormation syntax synopsis

Most CloudFormation items (aside from the root items like template version and description) can be summarized as being an name/type pairing. Basically, given any certain type, be it parameters, resources, mappings, or anything else, items in CloudFormation generally are assigned a unique name, and then a type. Consider the following example parameter:

"Parameters": {
  "KeyPair": {
    "Type": "AWS::EC2::KeyPair::KeyName",
    "Description": "SSH key that will be used for EC2 instances (set up in web console)",
    "ConstraintDescription": "needs to be an existing EC2 keypair (set up in web console)"
  }
}

This parameter is a AWS::EC2::KeyPair::KeyName parameter named KeyPair. The latter name can (and will be) referenced in resources, like the EC2 instance names (see the below section on EC2).

Look in the below sections for CloudFormation’s Ref function, which will be used several times; this function serves as the basis for referencing several kinds of CloudFormation elements, not just parameters.

Parameters and Outputs

Parameters are how data gets in to a CloudFormation template. This can be used to do things like get IDs of SSH keys to assign to instances (as shown above), or IP addresses to assign to security group ACLs. These are the two items parameters are used for in the example.

Outputs are how data gets out of CloudFormation. Data that is a useful candidate for being published through outputs include instance IP addresses, ELB host names, VPC IDs, and anything else that may be useful to a process outside of CloudFormation. This data can be read thru the UI, or through the JSON data produced by the aws cloudformation describe-stacks CLI command (and probably the API as well).

Parameter syntax

Let’s look at the other example in the CloudFormation template, the SSHAllowIPAddress parameter. This example uses more generic data types and gives a bigger picture as to what is possible with parameters. Note that there are several data types that can be used, which include both typical generic data types, such as Strings and Integers, and AWS-speciifc types such as the AWS::EC2::KeyPair::KeyName parameter used above.

"SSHAllowIPAddress": {
  "Type": "String",
  "AllowedPattern": "\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\\.\\d{1,3}\/32",
  "Description": "IP address to allow SSH from (only /32s allowed)",
  "ConstraintDescription": "needs to be in A.B.C.D/32 form"
}

This parameter is of type String, which also means that the AllowedPattern constraint can be used on it, which is used here to create a dotted quad regular expression, with the /32 netmask being explicitly enforced. JSON/Javascript syntax applies here, which explains the somewhat excessive nature of the backslashes.

Parameters are referenced using the Ref function. The snippet below gives an example of the SSHAllowIPAddress‘s reference:

"VCTSNatSecurityGroup": {
  "Type": "AWS::EC2::SecurityGroup",
  "Properties": {
    ...
    "SecurityGroupIngress": [
      ...
      { "IpProtocol": "tcp", "CidrIp": { "Ref": "SSHAllowIPAddress" }, "FromPort": "22", "ToPort": "22" }
    ],
    ...
  }
}

Ref is a very simple function and usually just used to refer back to a CloudFormation element. It is not just restricted to parameters is used with both parameters, mappings, and resources. Further examples will be given below, so there should be a good idea on how to use it by the end of this article.

Output syntax

Below is the NatIPAddr output, pulled from the example.

"Outputs": {
  "NatIPAddr": {
    "Description": "IP address of the NAT instance (shell to this address)",
    "Value": { "Fn::GetAtt": [ "VCTSLabNatGw", "PublicIp" ] }
  },
  ...
}

The nature of outputs are pretty simple. The data can be pulled any way that allows one to get the needed data. Most commonly, this will be from the Fn::GetAtt function, which can be used to get various attributes from resources, or possibly Ref, which in the event of resources, usually references a specific primary attribute.

Mappings

Mappings allow a CloudFormation template some flexibility. The best example of this is allowing the CloudFormation template to be used in multiple regions, by mapping AMIs (instance images) to their respective regions.

Mapping syntax

This is the one in the reference’s template, and maps to Amazon Linux AMIs. These are chosen because they support cfn-init out of the box, which was going to be used in the CloudFormation template to run some commands via the AWS::CloudFormation::Init resource type in the EC2 section, but I opted to use user data instead (I cover this in further detail in part 3).

"Mappings": {
  "RegionMap": {
    "us-east-1": { "AMI": "ami-1ecae776" },
    "us-west-1": { "AMI": "ami-e7527ed7" },
    "us-west-2": { "AMI": "ami-d114f295" }
  },
  "NatRegionMap": {
    "us-east-1": { "AMI": "ami-303b1458" },
    "us-west-1": { "AMI": "ami-7da94839" },
    "us-west-2": { "AMI": "ami-69ae8259" }
  }
}

The above RegionMap is then referenced in EC2 instances like so:

"VCTSLabNatGw": {
  "Type": "AWS::EC2::Instance",
  "Properties": {
    "ImageId": { "Fn::FindInMap": [ "NatRegionMap", { "Ref": "AWS::Region" }, "AMI" ] },
    "InstanceType": "t2.micro",
    ...
  }
}

This is one of many ways to use mappings, and more complex structures are possible. Check the documentation for further examples (such as how to expand the above map to make use of processor architecture the the region map).

Resources

Resources do the real work of CloudFormation. They create the specific elements of the stack and interface with the parameters, mappings, and outputs to do the work necessary to bring up the stack.

Since resources vary so greatly in what they need in real world examples, I explain each service that the template makes use of in their respective sections (ie: the VPC, ELB, and EC2 sections). However, some common elements are explained here in brief, as to give a primer on how they can be used to further control orchestration of the stack. Again, further detail on how to use these are shown as examples with the various AWS services explained below.

Creation Policies, Dependencies, and Metadata

A CreationPolicy can be used as a constraint to determine when a resource is counted as created. For example, this can be used with cfn-signal on an EC2 instance to ensure that the resource is not marked as CREATE_COMPLETE until all reasonable post-installation work has been done on an instance (for example, after all updates have been applied or certain software has been installed).

A dependency (defined with DependsOn) is a simple association to another resource that ties its creation with said parent. For example, the web server instances in the example do not start creation until the NAT instance is complete, as they are created in a private network and will not install properly unless they have internet access available to them.

Metadata can be used for a number of things. The example commonly explained is the use of the AWS::CloudFormation::Init metadata type to provide data to cfn-init, which is a simple configuration management tool. This is not covered in the example, as the work that is being done is simple enough to be done through UserData.

All of these 3 concepts are touched up on in further detail in part 3, when EC2 and the setup of an instance in CloudFormation is discussed.

Next Article – Amazon VPC

That about covers it for the CloudFormation part of this article. Stay tuned for the next part, in which I cover Amazon VPC basics, in addition to how it is set up in CloudFormation!