Amazon Elastic File System (EFS)
Amazon EFS or Elastic File System is a new shared file storage service Amazon announced on April 9th 2015 at the San Francisco AWS Summit.
While I don’t often blog about new services from Amazon Web Services, this particular one has captured my interest. Currently, it’s not widely available and at the time of writing you can only sign up for the preview that’s coming soon to the US West (Oregon) region. So it could be some time until we’re actually able to access the service in the EU regions.
What is it?
Amazon EFS is a shared file storage service for EC2 instances, which support the Network File System version 4 (NFSv4) protocol. The storage capacity is elastic so it goes up and down as you add and remove files so you don’t need to worry about how much space to provision.
Multiple EC2 instances can access an EFS file system simultaneously and thanks to being SSD-based it is designed to deliver the performance and low latency required for a broad range of uses.
Why is it useful?
One of the headaches when running web servers at AWS is how to enable a website or web application to be load balanced. While it’s simple to run a website on a single server, there is a lot more to consider when you switch that to a load balanced configuration of two or more servers.
The two major considerations that spring to mind are:
- Cache invalidation across all instances
- Session storage across all instances
I don’t want to get into to the various different approaches for solving these problems as many already exist and are perfectly workable on AWS currently. But what I will say is that EFS has huge potential to solve these and many other problems that come with running infrastructure in AWS with a far simpler (and hopefully reliable) solution than is probably in place currently.
“Have you heard of S3?!”
Amazon’s Simple Storage Service (S3) does also provide shared file storage but it has a number of features/problems that prevent it from being useful in some situations:
S3 is an object store not a file system
Although S3 has the concept of folders and files it doesn’t include the permissions and locks that a normal file system does. Therefore S3 can’t be mounted as a drive in the same way a file system can. I’m aware software solutions exist to allow an S3 bucket to be mounted as a drive but the performance in my experience has been terrible.
S3 provides eventual consistency
If you write a file to S3 its possible that if you immediately tried to read it or another instance did, that file might not yet exist. It will do eventually but there’s no guarantee it will immediately. This obviously presents problems for use cases when an application needs a 100% consistent storage service, which is actually probably most!
"Can’t you do this using EBS volumes?"
No, EBS volumes can only be attached to one instance at a time so there’s no way to share them between instances.
Granted there are services running on AWS that provide some of the features of EFS by using EBS volumes with EC2 instances but not without considerable cost. We’re hoping EFS will bring this much need feature to AWS at a cost that is affordable for all.
