How to update Python Flask code to AWS EC2 with an Automated Process: Set Up a Continuous Deployment Process by GitHub Action

Setting up a continuous deployment (CD) process for your Flask application hosted on AWS EC2 can greatly streamline the process of updating your app. CD automates the deployment of your application every time you make changes to your code, ensuring that your app is always up-to-date without manual intervention. Below are brief overviews of three popular tools for setting up CD: GitHub Actions, Jenkins, and AWS CodeDeploy.

This article is about using GitHub Actions. Before we start, ensure that your Flask application code is hosted on GitHub.

Step 1: Creating a Workflow File for GitHub Actions

In your GitHub repository, navigate to the Actions tab. Choose to set up a workflow yourself or start from an existing Python template provided by GitHub.

This will create a .github/workflows directory in your repository, where you'll define your CD pipeline in a YAML file.

Step 2: Defining the CD Workflow

Edit the workflow file (e.g., deploy.yml) to define the steps for your CD process.

Before using GitHub Action, I need to manually operate every time:

0. Abstract the requirements.txt from the local venv:
pip3 freeze > requirements.txt 

1. Update the app from local to AWS EC2 instance(from the local end of the terminal)
rsync -av --exclude='.git/' --exclude='.DS_Store' --exclude='venv_winjob' --exclude='__pycache__' --exclude='.gitattributes' --exclude='.gitignore' --exclude='history templates' --exclude='history version' --exclude='.env' -e "ssh -i /Users/henrywu/MyDrive/98_Products/config/AWS_EC2_key_pair_web_winjob.pem" /Users/henrywu/MyDrive/99_Coding/01-Github/winjob/ ec2-user@ec2-35-164-114-135.us-west-2.compute.amazonaws.com:/home/ec2-user/winjob/

2. Log in AWS EC2 instance, in the terminal, the next steps are the same:: 
ssh ec2winjob
(“nano ~/.ssh/config” to check the config file)

3. After log in, delete the former venv: 
rm -rf venv_winjob

4. Create a new venv:
python3 -m venv venv_winjob

5. Activate the venv:
source /home/ec2-user/winjob/venv_winjob/bin/activate

6. Install the packages:
pip3 install -r requirements.txt

7. Restart the app:
sudo systemctl restart gunicorn

Here’s the deploy.yml code based on the above commands. Basically, it automates those codes:

name: Deploy to AWS EC2

on:
  push:
    branches:
      - main  # Set the branch to trigger the deployment

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout repository
      uses: actions/checkout@v2

    - name: Deploy to AWS EC2
      run: |
        # Setting up SSH Key
        mkdir -p ~/.ssh
        echo "${{ secrets.SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa
        chmod 600 ~/.ssh/id_rsa

        # SSH options to bypass host key checking
        SSH_OPTIONS="-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i ~/.ssh/id_rsa"

        # Rsync to transfer the project to EC2
        rsync -avz --exclude='.git/' --exclude='.DS_Store' --exclude='venv_winjob' --exclude='__pycache__' --exclude='.gitattributes' --exclude='.gitignore' --exclude='history templates' --exclude='history version' --exclude='.env' -e "ssh $SSH_OPTIONS" $GITHUB_WORKSPACE/ ec2-user@${{ secrets.EC2_HOST }}:/home/ec2-user/winjob/

        # SSH into EC2 and set up the virtual environment and install dependencies
        ssh $SSH_OPTIONS ec2-user@${{ secrets.EC2_HOST }} "
          # Remove the existing virtual environment (if necessary)
          # rm -rf /home/ec2-user/winjob/venv_winjob
          # Create a new virtual environment
          # python3 -m venv /home/ec2-user/winjob/venv_winjob
          # Activate the virtual environment
          source /home/ec2-user/winjob/venv_winjob/bin/activate
          # Install the required packages
          pip3 install -r /home/ec2-user/winjob/requirements.txt
          # Additional commands to restart your application (if necessary)
          sudo systemctl restart gunicorn
        "
        
      env:
        SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
        EC2_HOST: ${{ secrets.EC2_HOST }}

Step 3: Managing Secrets and Credentials

In your repository, go to Settings -> Secrets.

Add necessary secrets like SSH_PRIVATE_KEY (your EC2 SSH private key) and EC2_HOST (your EC2 instance IP or domain). GitHub Actions will use these to securely SSH into your EC2 instance for deployment.

SSH_PRIVATE_KEY

The value of SSH_PRIVATE_KEY is the content of your AWS EC2 key pair file:

Open the file in a text mode or VS Code, it should be looks like this:

Copy the whole content (including “ — — -BEGIN RSA PRIVATE KEY — — -” and “ — — -END RSA PRIVATE KEY — — -”) to your GitHub.

EC2_HOST

Go to your EC2 dashboard, click Connect :

Copy the Public DNS to your GitHub.

If everything goes alright, every time you update the code to your repo, your website (the EC2 instance) will update automatically.

One more thing, to enhance the security of your GitHub Actions workflow, you can add the EC2 instance’s SSH host key to the known_hosts file:

How to prevent Man-in-the-Middle (MITM) attacks by adding EC2 instance’s SSH host key to GitHub…

This article is helped by ChatGPT 4.0