A Complete Guide to Amazon EC2 and Amazon EFS Integration

Overview

Managing scalable and highly available storage in modern cloud environments is crucial for applications running across multiple instances. One common challenge developers face is setting up a shared file system that multiple Amazon EC2 instances can access, whether in a public or private subnet.

AWS Elastic File System (EFS) provides a fully managed, scalable, and serverless NFS-based storage solution, allowing multiple Amazon EC2 instances to read and write data simultaneously. However, configuring Amazon EFS properly, especially for public and private subnet instances, requires careful setup, including Amazon VPC networking, security group adjustments, AWS IAM permissions, and NFS mounting.

This guide walks you through setting up Amazon EFS with Ubuntu 22.04 EC2 instances, ensuring smooth integration in public and private environments. Whether you are working with a web application, a data processing workload, or a shared storage need, this blog will help you configure Amazon EFS efficiently to enhance scalability, durability, and ease of management.

Key Challenges Addressed

• Shared Storage Across Instances – Setting up a central file system accessible by multiple Amazon EC2 instances.
• Public & Private Subnet Access – Configuring Amazon EFS to work seamlessly in public and private environments.
• Network & Security Setup – Ensuring proper Amazon VPC, security groups, and IAM roles for secure access.
• NFS Mounting & Automation – Mounting Amazon EFS correctly on Ubuntu 22.04 and automating the process.
• High Availability & Scalability – Leveraging Amazon EFS for a fault-tolerant, scalable storage solution.

Prerequisites

1. AWS Account: You will need an active AWS account. If you don’t have one, you can sign up here.
2. AWS IAM User/Role with Permissions: Access to create and manage Amazon EC2, Amazon EFS, Amazon VPC, and Security Groups.
3. Basic Knowledge of AWS Services: A basic understanding of AWS services such as Amazon EC2, Amazon VPC, Security Groups, and AWS IAM is highly recommended for a smoother setup.

Steps to Set Up Amazon EFS

Step 1: Launching an Amazon EC2 Instance (Ubuntu 22.04)

1. Navigate to the Amazon EC2 Dashboard:
   • Go to the AWS Management Console and open the Amazon EC2 Dashboard.
2. Launch an Instance:
   • Click Launch Instance.
   • Choose the Ubuntu Server 22.04 LTS AMI.

• • Select the Instance Type (e.g., t2.micro for free-tier usage).

3. Configure Network Settings:

• • Public Amazon EC2:
    • Enable auto-assign public IP.
    • Attach a security group allowing:
      • SSH (Port 22) for remote access.
      • NFS (Port 2049) for EFS communication.
  • Private Amazon EC2:
    • Ensure the instance resides in a private subnet (without an internet gateway).
    • Attach a security group with the same rules as above.
    • Use a bastion host or AWS Systems Manager Session Manager for access.

4. Add Storage and Launch:

• • Add a root volume if needed (default is sufficient).
  • Launch the instance.

5. Connect to the Instance:

• • Public Amazon EC2:
    • SSH directly:
    • ssh -i <your-key>.pem ubuntu@<public-ip>
  • Private Amazon EC2:
    • Use a bastion host or Systems Manager to connect.

Step 2: Creating and Configuring Amazon EFS

1. Navigate to the Amazon EFS Console:
   • Open the Elastic File System Dashboard.
2. Create a File System:
   • Click Create File System.
   • Provide a name for your file system.
   • Select the Amazon VPC where your Amazon EC2 instance resides.

3. Configure Access Points (Optional):

• • Create access points for specific permissions if needed.

4. Set Up Mount Targets:

• • AWS automatically creates mount targets in each subnet of your Amazon VPC.
  • Ensure:
    • Private EC2: Mount target is in the same private subnet.
    • Public EC2: Mount target can be in any subnet.

5. Security Group for Amazon EFS:

• • Ensure the security group allows inbound NFS traffic (Port 2049) from the Amazon EC2 instance’s security group.

Step 3: Mounting Amazon EFS on Amazon EC2 Instances

1. Install NFS Utilities:
   • Connect to your Amazon EC2 instance and install NFS utilities:

2. Create a Mount Directory:

• • Create a directory to mount Amazon EFS:

sudo mkdir -p /mnt/efs

3. Mount the EFS:

• • Use the mount command provided in the Amazon EFS console:

sudo mount -t nfs4 -o nfsvers=4.1 <efs-id>.efs.<region>.amazonaws.com:/ /mnt/efs

4. Persist Mount Across Reboots:

• • Add the following line to /etc/fstab:

<efs-id>.efs.<region>.amazonaws.com:/ /mnt/efs nfs4 defaults,_netdev 0 0

1. • Test the /etc/fstab entry:

Step 4: Verifying Amazon EFS Attachment

1. Check Mounted File Systems:
   • Verify using:

df -h

1. • Output should display the Amazon EFS mounted at /mnt/efs.
2. Test File Creation:
   • Navigate to the mount directory:

cd /mnt/efs

sudo touch testfile.txt

echo “Testing EFS” | sudo tee testfile.txt

cat testfile.txt

3. Differentiate Between Attached Storage:

• • Amazon EFS: Appears as a network-mounted file system.
  • Amazon EBS: Appears as a block device (e.g., /dev/xvda).

Step 5: Public vs. Private Amazon EC2 with Amazon EFS

Tips for Private EC2:

• Use a NAT Gateway for internet access.
• Ensure the private Amazon EC2 can access the Amazon EFS mount target in the same subnet or through Amazon VPC routing.

Real-Life Example – Using Amazon EFS with Jenkins

Scenario: Use Amazon EFS to store Jenkins build artifacts, logs, and configuration for high availability and scalability.

1. Install Jenkins:
   • Follow the steps to install Jenkins (as described earlier).
2. Configure Jenkins Home Directory:
   • Update the Jenkins home directory to use Amazon EFS:
     • Navigate to Jenkins > Manage Jenkins > Configure System.
     • Set the JENKINS_HOME directory to /mnt/efs/jenkins.
3. Shared Workspaces:
   • Multiple Jenkins nodes can use the same workspace in Amazon EFS for distributed builds.
4. Benefits:
   • Scalability: Amazon EFS automatically scales as data grows.
   • Durability: Highly available across multiple AZs.
   • Ease of Use: Simple integration with Jenkins and other applications.

Conclusion

This guide demonstrates creating and configuring an Amazon EC2 instance with Amazon EFS, including detailed steps for public and private instances.

Drop a query if you have any questions regarding Amazon EC2 and we will get back to you quickly.

About CloudThat