Update 2019-09-01: Integrating EFS with Lightsail instances does not seem to work anymore. The information described in this article might be helpful nevertheless. Please get into touch with me if you find a working solution.
Scaling web applications can be tricky. One obstacle is the storage of data, e.g. images uploaded by users, in the local file system. If such a web application should be run on multiple instances for meeting requirements regarding scalability and availability this creates a challenge. A common solution is the usage of a shared filesystem, e.g. one mounted via NFS.
In AWS the Elastic File System (EFS) service provides exactly this kind of shared storage. It’s completely managed, the data is stored durably and the service is highly available. Originally, the shared file systems managed by EFS are intended to be attached to EC2 instances.
In a previous blog post I have shown how AWS Lightsail, a service providing standard infrastructure components like load balancer, virtual server, and database, can be automatically set up through scripts using the AWS CLI. AWS Lightsail does not offer a shared file system. Fortunately, with some simple steps EFS volumes can also be attached to AWS Lightsail Instances. This blog post shows how this is done.
1. Peer the VPCs
Peer the Lightsail VPC with the Default VPC in the account and region:
aws lightsail peer-vpc
You can check whether peering has been successful
aws lightsail is-vpc-peered
2. Create the EFS File System
Next, we create a filesystem with the EFS service. This can be done manually via the AWS Console. A much better way is to use a CloudFormation template for this purpose. The important part for this step is to configure the Security Group associated with the EFS volume correctly.
The CloudFormation template looks like this:
AWSTemplateFormatVersion: 2010-09-09
Parameters:
Subnet:
Type: AWS::EC2::Subnet::Id
Description: The subnet to which the EFS volume should be exposed
SourceCidrIp:
Type: String
Description: CIDR block of allowed connections
Resources:
SharedFileSystem:
Type: AWS::EFS::FileSystem
Properties:
PerformanceMode: generalPurpose
MountTargetSecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: Security group for EFS mount target
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: '2049'
ToPort: '2049'
CidrIp:
Ref: SourceCidrIp
MountTarget:
Type: AWS::EFS::MountTarget
Properties:
FileSystemId:
Ref: SharedFileSystem
SubnetId:
Ref: Subnet
SecurityGroups:
- Fn::GetAtt: MountTargetSecurityGroup.GroupId
The template has two parameters: the subnet into which the EFS volume should expose a mount target and the CIDR block from which connections to the EFS volume should be allowed. The latter configuration helps in securing access to the EFS volume since only connections from the Lightsail VPC will be permitted.
The CIDR block of the Lightsail VPC can either be determined by looking it up in the AWS Console in the “Peering Connections” page for the Default VPC. Alternatively, it can be retrieved by using the AWS CLI:
vpcid=$(aws ec2 describe-vpcs --filters "Name=isDefault,Values=true" \
--query Vpcs[0].VpcId \
--output text)
aws ec2 describe-vpc-peering-connections --filters "Name=accepter-vpc-info.vpc-id,Values=$vpcid" \
--query "VpcPeeringConnections[0].RequesterVpcInfo.CidrBlock" \
--output text
Running these two commands prints out the CIDR block for the AWS Lightsail VPC. This is the appropriate parameter when creating the CloudFormation stack from the template shown above.
Based on this template a CloudFormation stack can either be created through the AWS Console or via the AWS CLI.
The EFS volume provides a set of mount targets. Each exposes an IP address which needs to be used in the next step to actually mount the volume to the Lightsail Instance. The IP address can be looked up in the AWS Console or via the following AWS CLI command:
aws efs describe-mount-targets --file-system-id fs-c4f6859d \
--query "MountTargets[0].IpAddress" \
--output text
Note: The EFS volume also has a DNS name assigned to it. In the examples mentioned in the AWS documentation about EFS this DNS name is used. Unfortunately, it can not be resolved in the Lightsail VPC. Therefore, the IP address needs to be used instead.
3. Mount the File System
On the Lightsail Instance the EFS tools need to be installed and the volume must be mounted:
sudo yum -y install amazon-efs-utils
sudo mkdir /mnt/efs
sudo mount -t nfs4 \
-o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 \
172.31.0.27:/ /mnt/efs
The IP address used in this command, e.g. 172.31.0.27 is the one associated with the EFS mount target set up before.
It’s a best practice to run these commands when an instance is launched. To achieve this they can be put in the user data script provided to a Lightsail Instance when it is launched.
That’s it! Lightsail Instances have access to shared storage managed by EFS now.
Automatically mount the File System
Probably, it has to be ensured that the shared file system is mounted on system boot. To achieve this it needs to be added to /etc/fstab:
echo "172.31.0.27:/ /mnt/efs nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2" >> /etc/fstab
Summary
AWS Lightsail is intended for more simple use cases which only require a smaller feature set than the one offered by classic AWS services. However, with a little bit of effort it is possible to integrate Lightsail with other AWS services and leverage their features for building more complex solutions. By using EFS as a shared file system it becomes easier to build more scalable and more resilient solutions for hosting web applications.