Copy a file from AWS S3 to an EC2 instance

My goal was to successfully copy a file I had stored in a Amazon Web Services (AWS) Simple Storage Service (S3) bucket to an Elastic Compute Cloud (EC2) instance I had created.  I ran into some problems. I hope documenting my experience will help someone else. Or perhaps, as is often the case, only I seem to be able hit these kinds of problems.

I used the Amazon Linux machine image when I created my EC2 instance. The nice thing about a Amazon Linux flavored EC2 instance is that the AWS Command Line Interface (CLI) comes pre-installed.

When I issued a “aws s3 ls” command on my EC2 instance I received an error message. The error I encountered was stated as follows.

A client error (SignatureDoesNotMatch) occurred when calling the ListObjects operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.

I had created an IAM user and gave that user full access for S3.

Using “aws config” I stored the IAM user’s secret access key (and key ID) in the AWS Command Line Interface (CLI) configuration file (~/.aws/config).

I initially thought that maybe there is some connection between the IAM user and the EC2 Linux user. I had logged into he EC2 instance using the “root” user (ec2-user).

A thought hit me that maybe the “trick” was to use a named profile (in the AWS CLI configuration) where the name of the profile matches the IAM user name. So, I tried that and it worked! I was ready to give myself the Genius award, but then I changed the profile name so that it didn’t match the IAM user name. Things (e.g., “aws s3 cp”) still worked. Hmmm… maybe I am not a genius… my wife certainly thinks quite the opposite. So, then I went back to what I had tried before (just putting the access key data for the IAM user in the [default] section of the ~/.aws/config file) and that worked!

The only thing I can think that might have made a difference is I had stopped and restarted the EC2 instance prior to getting things to work.

I also added a region=us-west-2 line to the ~/.aws/config file, but previously I had tried specifying the region using the –region command line option. So, I doubt adding the region to the config file made a difference, but I’ve been surprised before.

Anyhow, the bottom line is:

  • I can log in to my EC2 instance as ec2-user and then use the access key of a IAM user to execute “aws s3” commands
  • There is no connection between the IAM user and the EC2 Linux user.

I hope these ramblings help someone in the future. If so, please leave me a comment. Thanks!


4 thoughts on “Copy a file from AWS S3 to an EC2 instance”

  1. It’s actually a great and useful piece of info. I’m happy that you shared this useful info with us. Please keep us up to date like this. Thanks for sharing.

  2. Some really fantastic blog posts on this web site, thanks for contribution. “A conservative is a man who sits and thinks, mostly sits.” by Woodrow Wilson.

Comments are closed.