KP

CI/CD with AWS

Github Repo

Overview

This project is about building a CI-CD pipeline with Jenkins to AWS ECS (Elastic Container Service) for a simple microservices e-commerce application.

🔥 The tech

  • E-commerce application:
    • Front-end service: NextJS
    • Back-end services: NodeJS
    • Database: PostgreSQL (AWS RDS)

       

  • DevOps Tools:
    • GitHub
    • Jenkins
    • Docker

 

  • Aws Services:
    • ECR: Stores docker images
    • ECS: Orchestrates all containers
    • RDS: Hosts PostgreSQL database server
    • SES: Sends simple email notifications

💻 Application structure

  • client/ : NextJS application
    • Dockerfile: Building instructions for Docker
  • product-service/ : Express application for product APIs
    • Dockerfile: Building instructions for Docker
  • checkout-service/ : Express application for sending email notification
    • Dockerfile: Building instructions for Docker
  • Jenkinsfile: CI-CD pipeline

👀 Review application code

Client (port 3000)

  • app/api/checkout/

    export async function POST(request: Request) {
      const { email, items, total } = await request.json();
    
      try {
        await fetch(`${process.env.CHECKOUT_SERVICE_URL}/checkout`, {
          headers: {
            "Content-Type": "application/json",
          },
          method: "POST",
          body: JSON.stringify({ email, items, total }),
        });
    
        return NextResponse.json(
          { message: "Checkout successful" },
          { status: 200 }
        );
      } catch (error) {
        console.log(error);
        return NextResponse.json({ error: "Failed to checkout" }, { status: 500 });
      }
    }
  • app/api/products?q=seach-keyword

    export async function GET(request: NextRequest) {
      const { searchParams } = new URL(request.url);
      const search = searchParams.get("search") || "";
    
      try {
        const response = await fetch(
          `${process.env.PRODUCT_SERVICE_URL}/products?search=${search}`
        );
    
        if (!response.ok) {
          throw new Error(`Backend responded with status: ${response.status}`);
        }
    
        const data = await response.json();
        return NextResponse.json(data);
      } catch (error) {
        console.error("Error fetching products:", error);
        return NextResponse.json(
          { error: "Failed to fetch products" },
          { status: 500 }
        );
      }
    }

     

Product Service (Port 3001)

/products?seach=search-keyword

app.get("/products", async (req, res) => {
  const search = req.query.search || "";

  let query = `SELECT * FROM products WHERE name ILIKE $1`;

  if (!search) {
    query += ` LIMIT 5;`;
  }

  const { rows } = await pgPool.query(query, [`%${search}%`]);

  res.json(rows);
});

 

Checkout Service (Port 3002)

/checkout (Sends email with SES - AWS SDK)

app.post("/checkout", async (req, res) => {
  const { email, items, total } = req.body;
  // Logic: Clear cart, create order
  const result = await pgPool.query(
    "INSERT INTO orders (email, total) VALUES ($1, $2) RETURNING id",
    [email, total]
  );
  const orderId = result.rows[0].id;

  for (const item of items) {
    await pgPool.query(
      "INSERT INTO order_items (order_id, product_id, quantity) VALUES ($1, $2, $3)",
      [orderId, item.id, item.quantity]
    );
  }
  // Send email
  const orderDetails = items
    .map(
      (item, idx) =>
        `${idx + 1}. Product ID: ${item.id}, Product Name: ${
          item.name
        }, Quantity: ${item.quantity}`
    )
    .join("\n");

  const emailBody = `Order confirmed!\n\nOrder ID: ${orderId}\nTotal: $${total}\n\nItems:\n${orderDetails}`;

  const params = {
    Destination: { ToAddresses: [email] },
    Message: {
      Body: { Text: { Data: emailBody } },
      Subject: { Data: "E-commerce Order Confirmation" },
    },
    Source: process.env.SES_SOURCE,
  };

  await ses.sendEmail(params).promise();

  res.send("Checkout complete");
});

 

☁️ AWS Infrastructure Setup

AWS Infrastructure

 

VPC

  • Name: ecommerce-vpc
  • 2 AZ: ap-south-east-2a, ap-southeast-2b
  • 2 public subnets:
    • 10.0.1.0/24
    • 10.0.2.0/24
  • 4 private subnets:
    • 10.0.10.0/24
    • 10.0.11.0/24
    • 10.0.20.0/24
    • 10.0.21.0/24
  • NAT Gateway

 

Load Balancer & Target Groups

  • Target Groups
    • Port: 3000 (Client Service)
    • Private subnets:
      • 10.0.10.0/24
      • 10.0.11.0/24
  • Application Load Balancer
    • Name: ecommerce-alb
    • Listening port: 80
    • Public subnets:
      • 10.0.1.0/24
      • 10.0.2.0/24
    • Security group:
      • Name: ecommerce-alb-sg
      • HTTP: 80 - 0.0.0.0/0 (From anywhere)

 

EC2 Setup (Single Jenkins)

  • OS: Ubuntu 24.04
  • Instance type: m3.large
  • Storage: 20GB
  • VPC: ecommerce-vpc
    • Subnet: 10.0.1.0/24
  • Security Group:
    • Port 8080: Jenkins default UI application
    • Port 50000: Optional for communication to Jenkins agents
  • Port 22: SSH
  • IAM Role:
    • AmazonEC2ContainerRegistryPowerUser (for ECR)
    • AmazonECS_FullAccess (for ECS)
  • User data script (initialise instance with Jenkins and Docker):
#!/bin/bash
sudo apt update -y && sudo apt upgrade -y


# Install Java
sudo apt-get install fontconfig openjdk-17-jre -y


# Download Jenkins
sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
 https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
 https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
 /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update -y


# Install Jenkins
sudo apt-get install jenkins -y
sudo systemctl enable jenkins
sudo systemctl start jenkins


# Download & Install Docker (Optinal if need Docker as a builder)
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo \
 "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
 $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
 sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update -y
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo systemctl enable docker
sudo systemctl start docker


# Add the Jenkins User to the Docker Group
# sudo chmod 666 /var/run/docker.sock (Less Recommended)
sudo usermod -aG docker jenkins
sudo systemctl restart docker
sudo systemctl restart jenkins


# Installing AWS CLI
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
sudo apt install unzip -y
unzip awscliv2.zip
sudo ./aws/install

 

Services Security Group

  • Name: ecommerce-services-sg
  • Ports:
    • 3000 - From ecommerce-alb-sg
    • 3001 & 3002 - From itself (ecommerce-services-sg)

 

RDS PostgreSQL Setup

  • Engine: PostgreSQL
  • Private subnet group:
    • 10.0.20.0/24 - ap-south-east-2a
    • 10.0.21.0/24 - ap-southeast-2b
  • Parameter groups:
    • rds.force_ssl: 0 (For demonstrating purpose)
  • Security group:
    • PostgreSQL: 5432 - From ecommerce-services-sg

       

ECR Setup (Elastic Container Registry)

3 repositories for 3 microservices:

  • ecommerce-cicd-project/product-serivce
  • ecommerce-cicd-project/checkout-serivce
  • ecommerce-cicd-project/client

 

ECS Setup (Elastic Container Service)

Task definitions:

  • ecommerce-client

    • Launch type: AWS Fargate
    • Container: ecommerce-cicd-project/client:latest
    • Port: 3000
    • Environment variables:
      • PRODUCT_SERVICE_URL: http://product-service:3001
      • CHECKOUT_SERVICE_URL: http://checkout-service:3002

     

  • ecommerce-product-service

    • Launch type: AWS Fargate
    • Container: ecommerce-cicd-project/product-serivce:latest
    • Port: 3001
    • Environment variables:
      • PG_URL: postgresql://postgres:(RDS DB password)@(RDS DB hostname):5432/ecommerce

     

  • ecommerce-checkout-service
    • Launch type: AWS Fargate
    • Container: ecommerce-cicd-project/checkout-serivce:latest
    • Port: 3002
    • IAM Role:
    • AmazonSESFullAccess
    • Environment variables:
      • PG_URL: postgresql://postgres:(RDS DB password)@(RDS DB hostname):5432/ecommerce
      • SES_SOURCE: keithphan2909@gmail.com (Source Email)

 

Cluster:

  • Infrastructure: Fargate
  • Namespace setup for service-to-service communications
  • Services:
    • ecommerce-client-service:
      • Launch type: Fargate
      • Task definition: ecommerce-client
      • VPC: ecommerce-vpc
      • Security group: ecommerce-services-sg
      • Private subnets:
        • 10.0.10.0/24
        • 10.0.11.0/24
      • Namespace: Cluster’s namespace
      • Loadbalancer: Application Balancer (above)
        • Target group (above)

           

    • ecommerce-product-service:
      • Launch type: Fargate
      • Task definition: ecommerce-product-service
      • VPC: ecommerce-vpc
      • Security group: ecommerce-services-sg
      • Private subnets:
        • 10.0.10.0/24
        • 10.0.11.0/24
      • Namespace: Cluster’s namespace
        • DNS: product-service
        • Port: 3001
           
    • ecommerce-checkout-service:
      • Launch type: Fargate
      • Task definition: ecommerce-checkout-service
      • VPC: ecommerce-vpc
      • Security group: ecommerce-services-sg
      • Private subnets:
        • 10.0.10.0/24
        • 10.0.11.0/24
      • Namespace: Cluster’s namespace
        • DNS: checkout-service
        • Port: 3002

 

 

Jenkinsfile breakdown

Environment Variables:

environment {
       AWS_REGION = "ap-southeast-2"
       AWS_ACCOUNT_ID = "412381745022"
       AWS_ECR_REGISTRY = "${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com"
      
       CLIENT_REPO_NAME = "ecommerce-cicd-project/client"
       CLIENT_REPO_URL = "${AWS_ECR_REGISTRY}/${CLIENT_REPO_NAME}"
      
       PRODUCT_SERVICE_REPO_NAME = "ecommerce-cicd-project/product-service"
       PRODUCT_SERVICE_REPO_URL = "${AWS_ECR_REGISTRY}/${PRODUCT_SERVICE_REPO_NAME}"


       CHECKOUT_SERVICE_REPO_NAME = "ecommerce-cicd-project/checkout-service"
       CHECKOUT_SERVICE_REPO_URL = "${AWS_ECR_REGISTRY}/${CHECKOUT_SERVICE_REPO_NAME}"


       CLUSTER_NAME = "ecommerce-cicd-project"
       CLIENT_SERVICE = "ecommerce-client-service"
       PRODUCT_SERVICE = "ecommerce-product-service"
       CHECKOUT_SERVICE = "ecommerce-checkout-service"
 }

 

Stage 1: Docker Login

stage ("Docker Login") {
	steps {
		sh "aws ecr get-login-password --region ${AWS_REGION} | docker login --username AWS --password-stdin ${AWS_ECR_REGISTRY}"
	}
}

 

Stage 2: Build Docker Images & Push To ECR

Use changeset to detect file based changes (Only build images on services that changed)

stage ("Build Docker Images & Push To ECR") {
	parallel {
		stage ("Client") {
			when {
				changeset "client/**"
			}
				steps {
					echo "************************ Client ************************"
                    sh "docker build -t ${CLIENT_REPO_URL}:latest ./client"
                    sh "docker push ${CLIENT_REPO_URL}:latest"
               	}
    		}
             
		stage ("Product Service") {
			when {
				changeset "product-service/**"
			}
				steps {
 					echo "************************ Product Service ************************"
                  	sh "docker build -t ${PRODUCT_SERVICE_REPO_URL}:latest ./product-service"
                   	sh "docker push ${PRODUCT_SERVICE_REPO_URL}:latest"
                    }
                }

     	stage ("Checkout Service") {
          	when {
               	changeset "checkout-service/**"
           	}
         		steps {
            		echo "************************ Checkout Service ************************"
                	sh "docker build -t ${CHECKOUT_SERVICE_REPO_URL}:latest ./checkout-service"
                   	sh "docker push ${CHECKOUT_SERVICE_REPO_URL}:latest"
        	}
    	}
	}
}

 

Stage 3: Deploy

Use AWS CLI to force new deployment and changeset to detect services changed

stage ("Deploy") {
 	parallel {
     	stage ("Client") {
     		when {
             	changeset "client/**"
           	}
        	steps {
           		echo "************************ Client ************************"
               	sh "aws ecs update-service --cluster ${CLUSTER_NAME} --service ${CLIENT_SERVICE} --force-new-deployment"
         	}
   		}

     	stage ("Product Service") {
           	when {
              	changeset "product-service/**"
          	}
        	steps {
             	echo "************************ Product Service ************************"
              	sh "aws ecs update-service --cluster ${CLUSTER_NAME} --service ${PRODUCT_SERVICE} --force-new-deployment"
        	}
     	}

   		stage ("Checkout Service") {
           	when {
               	changeset "checkout-service/**"
          	}
         	steps {
              	echo "************************ Checkout Service ************************"
            	sh "aws ecs update-service --cluster ${CLUSTER_NAME} --service ${CHECKOUT_SERVICE} --force-new-deployment"
          	}
     	}
  	}
}

 

Post Stage: Docker Logout

post {
   	always {
  		sh "docker logout ${AWS_ECR_REGISTRY}"
	}
}

 

🚀 Final Application

Application is available at application load balancer DNS

 

Search for products by keywords

 

Get email confirmation after checking out

 

Thank You For Reading!