How to use AWS Systems Manager Parameters with only cURL and OpenSSL

Sometimes your project may need to communicate with AWS Application Programming Interface (Amazon API) without any access to AWS Command Line Interface (AWS CLI) or Software Development Kits (AWS SDK). For me it was the moment when I had to extract AWS Systems Manager (AWS SSM) Parameters from inside a Docker container running in Amazon Elastic Container Service (Amazon ECS) on Amazon Elastic Compute Cloud (Amazon EC2) Instance. The container was quite tiny and had no AWS CLI on it. Base image used was busybox and it was making the situation even more complicated. Installing AWS CLI would mean bringing in Python with all its dependencies — not an option. The only solution left was to communicate with Amazon API via Representational state transfer (REST) with cURL.

The issue (and the reason why I am writing this post) was, that there are not many tutorials on the Internet on how to do that with cURL. Moreover, there are no posts on how it works with AWS EC2 instance profile credentials (the ones we inherit from the Amazon EC2 instance) and nothing on how to do all of that for AWS SSM API. Below, I will briefly go through all the steps and highlight the details you should know. In the end, I will attach the script that performs the whole process.

Prerequisites:

1. Your AWS Identity and Access Management (IAM) user/IAM instance profile of your Amazon EC2 instance must have IAM policy that allows executing “ssm:getparameters”.
2. Your environment must have cURL and OpenSSL command line tools installed.

In order to send requests to the Amazon API via REST, you have to sign them to make identifiable. The signing of signature version 4 (the one AWS uses on almost all its services in 2019) generally contains the next fours steps:

1. Generate canonical request
2. Use it to create a string to sign
3. Generate signing key, create signature based on the key and the string to sign
4. Put the signature as part of the Hypertext Transfer Protocol (HTTP) request (e.g. as an HTTP header or a query string parameter)

Step 1: Canonical Request

First of all, you need to specify the information from your request such as HTTP method, headers, payload, etc. in the precise order described in the pseudo-code snippet 1. When you are running the script with help of instance profile credentials (as I did) you have to add an additional header x-amz-security-token:< your-token > to it. The security token as well, as access and secret keys can be derived from the instance metadata endpoint from within the Instance. Note: in the < SignedHeaders > you must specify the headers in the same order you do in < CanonicalHeaders >.

Code Copied!

$HTTPMethod
$CanonicalURI

content-type:application/x-amz-json-1.1
host:$AwsService.$MyRegion.amazonaws.com
x-amz-date:$MyDateAndTime
x-amz-security-token:$Token
x-amz-target:$Target

content-type;host;x-amz-date;x-amz-security-token;x-amz-target
$HashedPostData

Snippet 1. Example of canonical request in Bash. Empty lines and newlines must be preserved

In most tutorials about signing process, authors use Amazon Simple Storage Service (Amazon S3) API to list objects as an example. The difference is that AWS SSM GetParameter expects POST method instead of GET (more about it), and therefore you need to provide a proper hashed (JSON) payload to it. Make sure that you have a well-formatted JSON string and do not try to hash a JSON file instead. For hashing payload, you can use OpenSSL.

Step 2: String to Sign

Next step is creating the string to sign, which you will eventually turn into a valid signature. For that you need the hashed canonical request (from step 1), hashing algorithm (AWS4-HMAC-SHA256), timestamp and scope in format < yyyyMMdd >/< Region >/< Service >/aws4_request. In the snippet 2 below you can see the order you should use for it.

Code Copied!

AWS4-HMAC-SHA256
$MyDateAndTime
$MyDate/$MyRegion/$AwsService/aws4_request
$CanonicalRequestHash

Snippet 2. Example Bash code for a string to sign

Step 3: Create Signature

Now you need to calculate the signing key and use it to create the signature. The signing key is derived through four-step HMAC-SHA256 hashing. It starts with hashing of your secret access key with date, then it hashes the result with region, then with AWS service name and possibly with “aws4_request”, which is a constant string. After that, you need to hash the string to sign from the previous step with the signing key you just derived. This will be your signature. The step is displayed on snippet 3.

Code Copied!

dateKey=$(hmac_sha256 key:"AWS4$SecretAccessKey" $MyDate)
dateRegionKey=$(hmac_sha256 hexkey:$dateKey $MyRegion)
dateRegionServiceKey=$(hmac_sha256 hexkey:$dateRegionKey 
$AwsService)
HexKey=$(hmac_sha256 hexkey:$dateRegionServiceKey "aws4_request")

Snippet 3. Template for a signing key and signature creation

Step 4: Build HTTP Request

Finally you have everything you need to make the request. You only need to wrap it in cURL command and execute it. You have to specify all canonical headers from step 1, add authorisation header and plain JSON payload. Authorisation header consists of hashing algorithm, credentials, signed headers (step 1) and the signature from step 4. The snippet 5 is a cURL command with all flags configured to send the request to AWS SSM API.

Code Copied!

curl -vvv https://$AwsService.$MyRegion.amazonaws.com/ \
   -X $HTTPMethod \
   -H "Authorization: AWS4-HMAC-SHA256 \
      Credential=$AccessKeyId/$MyDate/$MyRegion/$AwsService/aws4_request, \
      SignedHeaders=content-type;host;x-amz-date;x-amz-security-token;x-amz-target, \
      Signature=$HashSignature" \
   -H "x-amz-security-token: $Token" \
   -H "x-amz-target: $Target" \
   -H "content-type: application/x-amz-json-1.1" \
   -d $RequestPostData \
   -H "x-amz-date: $MyDateAndTime"

Snippet 4. cURL command for AWS SSM API

If you did everything right, you will finally see HTTP/1.1 200 OK response.

Summary

As you can see, the process can be characterised as “hashing and mixing the same set of inputs in different ways”, and that’s what makes it look so complicated. I published the script on GitHub; it allows, with little to no changes' communication with most AWS APIs. If you have problems with it or any other topic-related thing, please do not hesitate to leave questions in comments and I will do my best to answer it.