Gateway Endpoints: The secure way to access S3 from your VPC

Introduction

In this blog post, we will explore the differences between using gateway endpoints and internet routing for S3 access.
We will discuss why you should use gateway endpoints and how to set them up.

We'll also have a look at some common pitfalls and how to avoid them.

As with most of our articles, we'll also provide you with a complete project that you can run and deploy to your own AWS account.

awsfundamentals-hq / gateway-endpoints-s3

🔒 A project demonstrating gateway endpoints s3

⭐ Stars

🍴 Forks

🐛 Issues

s3

View on GitHub →

Feel free to clone the repository and play around with the code!

Make sure to deploy it to your AWS account with SST, not just running the npx sst dev command.
This is due to the fact that you won't be able to invoke your function locally or via the console.

What are Gateway Endpoints?

Gateway endpoints are a type of VPC endpoint that allows you to access S3 (and other services like DynamoDB) from within your VPC.
You may ask: this is already possible from within the VPC using the public internet, so why do we need gateway endpoints?

The answer is: mostly due to better security.
You'll also get a small performance benefit.

But back to the question: why is it better?

Well, if you use the public internet, you're exposed to all the security threats that come with it.
If you for example want to access S3 from within your private VPC, you'll need to open your VPC to the public internet.
At least for the outgoing traffic.

So this is the public internet routing.
Even though our S3 bucket resides in the same AWS account, we'll route the traffic through the public internet.
For a set up with a private subnet, we'll need to route traffic through a NAT gateway or an NAT instance.

How would this look like with a gateway endpoint?

With a gateway endpoint, we'll route the traffic through the VPC endpoint.
This way, we don't need to go through multiple hops and we don't need to expose anything to the public internet.

The best part of this is: you can use gateway endpoints at no additional cost.
The downside: gateway endpoints do not allow access from on-premises networks.
For this you'll need to use interface endpoints.

Setting up an Gateway Endpoint for Amazon S3

Let's have a look at how to set up a gateway endpoint for Amazon S3.

We'll look at a trivial example: a Lambda function that lists the contents of an S3 bucket.

The Lambda function will reside in a private subnet and will only be able to access the bucket through our gateway endpoint.

We'll make sure that the bucket is not accessible elsewhere by restricting access to the gateway endpoint.

Let's get started!

Prerequisites

Surprise surprise: we'll need to have an AWS account and our CLI needs to be configured.

If you have done that, you can follow the documentation specific to your operating system. If you are using macOS, you can easily install the CLI through Homebrew by executing the command: brew install awscli.

Your credentials can be configured by running the command: aws configure.
You can obtain your credentials from the AWS Console.

Creating our VPC and Subnets

Now to the interesting part: creating our VPC and subnets.
You can either follow along or clone our repository and run the SST commands yourself.

SST will do a lot of heavy lifting for us:

    const { privateSubnets, id: vpcId } = new sst.aws.Vpc('awsf-vpc', { az: 3 });
  

As you see, it will create a VPC with three private subnets in three different availability zones.
We'll also directly get the VPC ID and the private subnets.
This way, we can directly use them in our next steps.

Before we create our gateway endpoint, we'll also need to find the route tables of our private subnets:

    const routeTableIds = privateSubnets.apply((subnets) =>
    subnets.map((subnet) =>
        subnet.apply((subnet) =>
            aws.ec2
                .getRouteTable({
                    subnetId: subnet,
                })
                .then((rt) => rt.id),
        ),
    ),
);
  

Now we're ready to create our gateway endpoint!

Creating a Gateway Endpoint

Let's do exactly that:

    const endpoint = new aws.ec2.VpcEndpoint('awsf-s3-endpoint', {
    vpcId,
    serviceName: 'com.amazonaws.us-east-1.s3',
    vpcEndpointType: 'Gateway',
    routeTableIds,
});
  

As you can see, we're creating a VPC endpoint with the type Gateway and the service name com.amazonaws.us-east-1.s3.
We'll also pass our route table IDs to the endpoint.