How to manage AWS Secrets through IaC

In recent years, with Cloud computing infrastructure being more and more used by providing IT infrastructure, the concept of providing those resources using code (IaC) with tools like AWS CloudFormation or Terraform becoming very popular.

In short words, Infrastructure as Code or IaC is the concept of managing resources using configuration files instead of them manually through a user interface.

Some benefits of IaC:

• Consistency: with resources being managed by code, this code became the source of truth of the environments, which reduces manual tasks, so the same resource can be provisioned a lot of times with the same configuration, without risk of human error.
• Version control: code can be versioned using Git repositories and safely stored, can full traceability of who changed what.
• Speed: new environment or resource with the same configuration can be provisioned more quickly.

These benefits are just a few of them, there many more reasons to use IaC.

However, some resources are still difficult to manage as IaC, and one of them is secrets.

Secrets cannot be stored in standard configuration files, because they have sensitive data, such as API Keys, Password, Connection Strings, etc. Having an inventory of which secrets exist in the environment or stored the plain text value in a secure is a difficult task.

AWS has two services that are possible to store secrets, AWS SSM Parameter Store or AWS Secrets Manager.

AWS Secrets Manager has a CloudFormation resource, which enables the option to create the secret through a CloudFormation stack, and if you use a tool to manage these stacks, like Sceptre, the parameters need to be provided by configuration files, so the plain text value will be exposed, and cannot be stored directly in the source version tool.

AWS SSM Parameter Store at this moment does not have a CloudFormation resource, so any SSM parameter needs to be created manually or using custom scripts.

Solution

AWS Secrets CLI is a tool to manage SSM Parameter Store (SecureString and String) and AWS Secrets Manager through a CLI and uses AWS KMS to encrypt the secrets so you can store the configuration files in the Git repository without exposing the sensitive data for anyone who has access to the repo.

How the CLI works?

AWS Secrets CLI uses AWS KMS to encrypt the secret and store it in an encrypted configuration file, enabling the user to store the secret in the repo in a safe way.

This configuration file also uses AWS KMS to decrypt the secret to manage the resources in the AWS environment.

The AWS Secrets CLI provides an inventory of all the secrets you have in the AWS environment, which makes it easy to control permission and avoid duplications.

Getting Started

Now to use this tooling, follow the next steps to install, configure and manage the secrets.

Requirements

• Python 3.8 or greater
• AWS KMS Key

It is necessary to create a KMS key before starting to create the parameter using the CLI.

You can create this key using AWS CLI, AWS SDK, console, or CloudFormation.

Example using CloudFormation:

Description: "KMS Key for Secrest"
Resources:
  Key:
    Type: AWS::KMS::Key
    Properties:
      KeyPolicy:
        Statement:
          - Action:
              - kms:Create*
              - kms:Describe*
              - kms:Enable*
              - kms:List*
              - kms:Put*
              - kms:Update*
              - kms:Revoke*
              - kms:Disable*
              - kms:Get*
              - kms:Delete*
              - kms:ScheduleKeyDeletion
              - kms:CancelKeyDeletion
              - kms:GenerateDataKey
              - kms:TagResource
              - kms:UntagResource
            Effect: Allow
            Principal:
              AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
            Resource: "*"
          - Action:
              - kms:Decrypt
              - kms:Encrypt
              - kms:ReEncrypt*
              - kms:GenerateDataKey*
            Effect: Allow
            Principal:
              AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root"
            Resource: "*"
        Version: "2012-10-17"
      Description: AWS KMS Key for secrets
    UpdateReplacePolicy: Retain
    DeletionPolicy: Retain
  KeyAlias:
    Type: AWS::KMS::Alias
    Properties:
      AliasName: alias/infra-scripts-secrets
      TargetKeyId: !GetAtt Key.Arn
Outputs:
  KeyArn:
    Description: KMS Key Arn
    Value: !GetAtt Key.Arn

This CloudFormation template has the basic configuration, which allows anyone who has access to the KMS in the account to encrypt/decrypt the secrets.
You can manage the AWS KMS Key the way you want.

Install

To install the CLI open the terminal and execute the following command:

pip install aws-ssm-secrets-cli

Our fist config

For naming convention, you should give the environment name for the file name (e.g., dev.yaml)

kms:
  arn: KMS_KEY_ARN (String) #Required
parameters:
  - name: myparametername
    value: "MySecretValueHere"
    type: SecureString
secrets:
  - name: mysecretname
    value:
      "MySecretValueHere"
  - name: mysecretname
    value:
      user: "myuser"
      password: "mypassword"

Encrypt

To encrypt the secrets, use this command:

aws-secrets encrypt -e dev.yaml --profile <aws-profile> --region <aws-region>

Decrypt

To edit the values, you can decrypt and re-encrypt the secrets, use this command:

aws-secrets decrypt -e dev.yaml --profile <aws-profile> --region <aws-region>

At this moment, a new file has been created dev.yaml.dec. If you want to decrypt in overwrite mode put the --output option with the same file name that you are decrypting.

aws-secrets decrypt -e dev.yaml --output dev.yaml --profile <aws-profile> --region <aws-region>

After your changes you need to re-encrypt, you can do it using this command:

aws-secrets encrypt -e dev.yaml --profile <aws-profile> --region <aws-region>

Managing secrets

You can also add or edit the secrets using the command aws-secrets set-parameter or aws-secrets set-secret

aws-secrets set-parameter -e dev.yaml \
--name "<ssm-parameter name>" \
--profile <aws-profile> \
--region <aws-region>

or

aws-secrets set-secret -e dev.yaml \
--name "<ssm-parameter name>" \
--profile <aws-profile> \
--region <aws-region>

For security reasons, the secret value needs to be provided in the CLI prompt format.

The CLI also supports managing secrets with custom KMS Keys.

Deploying secrets into AWS Account

To deploy the secrets that you created on the last step, use this command:

aws-secrets deploy -e dev.yaml --profile <aws-profile> --region <aws-region>

Now your secrets have been created in AWS Account.

Full CLI documentation: https://github.com/lucasvieirasilva/aws-ssm-secrets-cli#command-line-interface

Summary

With this tooling, it is possible to manage AWS Secrets using configuration files that can be safely stored in the Git repository.
Enables a manageable way to deploy the secrets and also provides visibility about which secrets are managed by the team.