How to Build a Highly Available and Secure Infrastructure on AWS using Amazon Autoscaling, Application Load Balancer , VPC and Ansible.
We were given a task for our holiday challenge in AltSchool to :
Set up 2 EC2 instances on AWS(use free tier).
Deploy an Nginx web server on these instances(you are free to use Ansible)uh
Set up an ALB(Application Load balancer) to route requests to your EC2 instances
Make sure that each server displays its Hostname or IP address. You can use any programming language of your choice to display this.
Important points to note:
Access to your web servers through their respective IP addresses is not allowed. Access must be only via the load balancer
You should define a logical network on the cloud for your servers.
Your EC2 instances must be launched in a private network.
Your Instances should not be assigned public IP addresses.
You may or may not set up autoScaling
You must submit a custom domain name(from a domain provider e.g. Route53) or the ALB’s domain name.
What will this project achieve?
Modularity: This is the principle of system design that deals with how two or more components are related or connected such that each tier of the application can be managed independently. This way different teams can focus on different aspects of the application and changes can be made quickly, leading to a quick recovery in case of an unexpected disaster.
Scalability: A system is scalable when it can accommodate a greater amount of usage. Each tier of the application can be scaled horizontally to support and manage traffic requests. This we can achieve with EC2 Autoscaling groups as traffic demand increases it scales up more ec2 instances and reduces as the traffic demand diminishes.
High availability: A highly available system should be able to quickly recover from any sort of failure state to minimize interruptions for the end user. Our application will be created in different availability zones in case of a disaster like an earthquake or flooding in one zone, the application will still be reachable and hosted by another zone
Fault tolerance: Fault-tolerant systems use backup components that automatically take the place of failed components, ensuring no loss of service. This is usually done by adding a redundant system that will account for a hike in traffic when it does occur. So for example instead of having two EC2 instances working at 50% each, such that when one instance goes bad, the other instance will be working at 100% capacity until a new instance is brought up by our Auto Scaling Group, we have extra instance making it three instances working at approximately 35% each.
Security: we are trying to create a highly y secure infrastructure so for this to happen our application backend and frontend will be communicating through a private IP. The presentation (frontend) tier of the infrastructure will be in a private subnet (the subnet with no public IP assigned to its instances) within the VPC. Users can only reach the front end through the application load balancer. The backend and the database tier will also be in the private subnet because we do not want to expose them over the internet. We will set up the Bastion host for remote SSH and a NAT gateway for our private subnets to access the internet. The AWS security group helps us limit access to our infrastructure setup.
I solved this task using two methods:
First, I practised using Ansible to provision the infrastructure on AWS then also used ansible to deploy Nginx and my app. The playbook is in my GitHub repository while my app repository is My-App
The second method was using the AWS GUI to manually provision the infrastructure, setting up the logical network and EC2 Instances.
AWS services to design and build a three-tier cloud infrastructure: Elastic Compute Cloud (EC2), Auto Scaling Group, Virtual Private Cloud(VPC), Elastic Load Balancer (ELB), Security Groups and the Internet Gateway. Our infrastructure will be designed to be highly available and fault tolerant
I created 3 EC2 instances, one instance(BASTION-HOST) on a public subnet and 2 instances (INSTANCE A and B) on the private subnet using AutoScaling groups. I added the two private instances to a target group in the Load balancer then redirected traffic from the private instance through the bastion host to the ALB. Next, I deployed the nginx server and my app using ansible then I provided a Nat-gateway to make the site highly available.
After connecting to the public internet using Nat-gateway, I deleted the Natgateway and released the Elastic IP attached to it.
My-App is live and running with ALB URL http://project-alb-362817232.us-east-1.elb.amazonaws.com/
Steps to do this;
Step 1: Logical Network(Virtual Private Cloud)
Defining a logical network on the cloud simply means creating your VPC and its components to make your app highly available on the internet.
A highly available infrastructure is designed to handle failures and continue operation without interruption dues to its high availability in different availability zones and regions.
For this project, we are going to be designing the infrastructure building the following for our logical network.
A VPC, Subnets, Route tables, Internet gateway and a Nat gateway.
To do this, search for the keyword VPC, navigate to the VPC dashboard then do the following;
Virtual Private Cloud (VPC)
Select VPC and click on create VPC.
Name your VPC and include a CIDR block 10.0.0.0/16.
Subnets
Create your subnets, two subnets each for public and private subnets in two availability zones. I used us-east-1a and b zones.
I used the following CIDR blocks for my subnets
private-subnet a 10.0.16.0/20
private subnet b 10.0.32.0/20
public subnet a 10.0.0.0/24
public subnet b 10.0.1.0/24
On the "Edit subnet" settings of your public subnet, click on Enable Auto-assign ipv4 address so your instances created on the public subnet can be automatically assigned a public IP address.
Internet gateway
Internet Gateway is used to allow resources in your VPC to access the internet.
Create an Internet gateway to allow internet access through our instances and attach it to your VPC. To do this, click on the internet gateway, go to actions then click on attach to VPC. Make sure to attach it to the VPC created for your project.
Route Tables
Create route tables and make sure to attach them to the right VPC, the one created.
Click on edit route tables under actions. Add a route with the CIDR 0.0.0.0/0 as your destination, then use your created Internet gateway as your target and save changes.
under actions click on edit subnet associations and add your public subnet to your public route table.
Nat Gateway
A NAT device forwards traffic from the instances in the private subnet to the internet or other AWS services, and then sends the response back to the instances. This helps us allow our private instances to access the Internet through the load balancer.
It is important to note that this service is not under the free tier
Create a NAT gateway and allocate an elastic IP to it. Go back to your route tables and edit route tables under actions. Add a route with the CIDR 0.0.0.0/0 as your destination, set the target as Nat-gateway then save changes.
under actions click on edit subnet associations and add your public subnet to your public route table.
Hurrayy, we have completed the process of defining a logical network on the cloud.
Next thing is to provide the environment to host our highly available application. For this, we'll need three EC2 instances, 1 public instance which will serve as our Bastion host and 2 private instances which we will be deploying our apps.
Step2: CREATING OUR INSTANCES
Provisioning the Bastion Host instance.
To do this, create an instance in the public subnet of your VPC and name it Bastion-Host.
Assign a keypair because you will need it to ssh into the private instances we will create shortly
Create a security group and give it the following rules
Inbound rule: allow ssh anywhere CIDR 0.0.0.0/0
Outbound rule: allow All traffic CIDR 0.0.0.0/0
Next, we create our Private Instances using Auto Scaling groups and a Load balancer to direct the traffic to the Internet. This helps our app to remain highly available as it improves redundancy and failovers.
By redundancy and failovers, I mean it can automatically switch to a backup system and redirect traffic there. This is why we built our infrastructure in two availability zones (creating our subnet into two zones)
Creating Private Instances using AutoScaling groups
Navigate to Autoscaling groups at the bottom of your EC2 dashboard and click on create.
Choose a name for your ASG and create a launch template.
Choose the name of your ASG
Choose your Linux distribution and select t2micro
Select a Key Pair. Make sure to use the same key with your Bastion host or copy the newly created key pair to your Bastion host. We need this to be able to SSH from the bastion host to the private instances we'll be provisioning with ASG.
Do not include a subnet in the launch template. we will include the subnet in the autoscaling group itself.
Create security groups with the rules as seen below in the image;
(Please make sure to include the security group of the ALB after creating the Application Load Balancer as I took screenshots after I had completed my process which is why we can see the rule with the ALB security group at this point. At this point it should only contain the rule to allow ssh traffic from the bastion host)
Create a launch template and move on to continue creating the ASG with the launch template we just created
Select your VPC and select a private subnet
Skip all other settings until you get to the Group size and scaling policy then select your desired, max and min capacity. I used 2 because I want 2 instances at all times
Click create autoscaling group.
This should automatically create your EC2 instances for you after a few minutes.
Application Load-Balancer
Create an Application load balancer, give it a name and make sure it's internet-facing.
Select your VPC and select the network mappings across your subnets
Create a security group with the following rules:
Inbound rule: Allow HTTP on port 80 from anywhere CIDR 0.0.0.0/0
Outbound rule: Allow HTTP on port 80 to destination private instance security
Under Listeners and Routing, create a target group comprising of the instances that your autoscaling group scaled up for you. Make sure to include all the instances you want to deploy your app to, that is, the two private instances.
Step 3: DEPLOY APP WITH ANSIBLE PLAYBOOK
Connect to your bastion host using ssh or aws connect and make sure to copy the .pem key you created for your instances to your bastion host if it isn't there already. SSH into your instances from your bastion host to confirm your connection is working perfectly then deploy your playbook to the app.
On your Bastion Host, create your playbook using this sample playbook to test if you do not have one already.
then deploy to your instances.
Do not forget to edit the variables, ansible config and IP addresses under the host file to yours.
Also, note there is a UI template attached to our setup. you can edit it out if you do not want it.
Target group
Confirm that your target groups are showing healthy after the successful deployment.
Viola, your app is highly available. Use the ALB link to view it and it should look like this with our two host ips displaying on each refresh.
Delete your Nat gateway as it is not a free service and release the Elastic Ip.
Your App will be on and running as long as you do not do any further updates on it. So unless you are completely done with the setup, do not delete your Nat gateway.
We have come to the end of this project, I hope you enjoyed it as much as I did.
Thank you for reading my process.
I will be documenting the process of Configuring the entire process (Infrastructure as code) with an updated Ansible playbook or Terraform shortly.
Watch this space!