Category Archives: Amazon Web Services

Tricks, tips, etc. about AWS

Copy a file from AWS S3 to an EC2 instance

My goal was to successfully copy a file I had stored in a Amazon Web Services (AWS) Simple Storage Service (S3) bucket to an Elastic Compute Cloud (EC2) instance I had created.  I ran into some problems. I hope documenting my experience will help someone else. Or perhaps, as is often the case, only I seem to be able hit these kinds of problems.

I used the Amazon Linux machine image when I created my EC2 instance. The nice thing about a Amazon Linux flavored EC2 instance is that the AWS Command Line Interface (CLI) comes pre-installed.

When I issued a “aws s3 ls” command on my EC2 instance I received an error message. The error I encountered was stated as follows.

A client error (SignatureDoesNotMatch) occurred when calling the ListObjects operation: The request signature we calculated does not match the signature you provided. Check your key and signing method.

I had created an IAM user and gave that user full access for S3.

Using “aws config” I stored the IAM user’s secret access key (and key ID) in the AWS Command Line Interface (CLI) configuration file (~/.aws/config).

I initially thought that maybe there is some connection between the IAM user and the EC2 Linux user. I had logged into he EC2 instance using the “root” user (ec2-user).

A thought hit me that maybe the “trick” was to use a named profile (in the AWS CLI configuration) where the name of the profile matches the IAM user name. So, I tried that and it worked! I was ready to give myself the Genius award, but then I changed the profile name so that it didn’t match the IAM user name. Things (e.g., “aws s3 cp”) still worked. Hmmm… maybe I am not a genius… my wife certainly thinks quite the opposite. So, then I went back to what I had tried before (just putting the access key data for the IAM user in the [default] section of the ~/.aws/config file) and that worked!

The only thing I can think that might have made a difference is I had stopped and restarted the EC2 instance prior to getting things to work.

I also added a region=us-west-2 line to the ~/.aws/config file, but previously I had tried specifying the region using the –region command line option. So, I doubt adding the region to the config file made a difference, but I’ve been surprised before.

Anyhow, the bottom line is:

  • I can log in to my EC2 instance as ec2-user and then use the access key of a IAM user to execute “aws s3” commands
  • There is no connection between the IAM user and the EC2 Linux user.

I hope these ramblings help someone in the future. If so, please leave me a comment. Thanks!