How to Use AWS Secrets Manager for Managing Credentials?

AWS offers the secrets manager service for managing secrets easily such as database credentials, passwords, third-party API keys, & even arbitrary text. With the help of the Secrets Manager console, the Secrets Manager command line interface (CLI), or the Secrets Manager API and SDKs, storing & accessing the secrets are possible. Let us know more about AWS Secrets Manager in this blog post.

What is AWS Secrets Manager?

AWS Secrets Manager refers to a secrets management service that protects access to your applications, services, & IT resources. It lets you rotate, manage, and retrieve database credentials, API keys, & other secrets. With the help of the secrets manager, you will be able to protect & manage secrets used for accessing resources in the AWS Cloud, on third-party services, as well as on-premises.

Why should you use AWS Secrets Manager?

It can protect access to applications, services, & IT resources. Even, you don’t need to pay the upfront investment and ongoing maintenance costs for operating your infrastructure. Mainly, IT administrators who are willing to store & manage secrets securely use this AWS Secrets Manager. Security administrators need to meet regulatory and compliance needs. They use the Secrets Manager for tracking secrets & rotating secrets without impacting applications. Developers who are willing to replace hard coded secrets, will be able to retrieve secrets programmatically from this secrets manager.

Secrets Manager Features:

Secure secrets storage:

It uses your encryption keys to encrypt secrets at rest. While you retrieve a secret, the service will decrypt the secret & transmit this securely over TLS to the local environment.

Automatic secrets rotation without disrupting applications:

It allows you to use the Secrets Manager console, AWS SDK, or AWS CLI for rotating secrets on a schedule.

This service helps to rotate credentials for databases that are hosted on Amazon RDS & Amazon DocumentDB as well as clusters that are hosted on Amazon Redshift.
Extending the service helps to rotate secrets that are used with AWS or 3P services by modifying sample Lambda functions.

Automatic replication of secrets to multiple AWS Regions:

It lets you replicate the secrets to many AWS Regions for fulfilling the requirements of disaster recovery and cross-regional redundancy. Remember that you need to indicate the AWS Regions where a secret will be replicated. The service generates regional read replicas. So, you don’t have to maintain a complex solution for such functionality. Give the multi-region applications access to the replicated secrets in the Regions you require and depend on the Secrets Manager for keeping replicas in sync with the primary secret.

Programmatic retrieval of secrets:

You can produce an app with security secrets.

The service offers code samples to call the Secrets Manager APIs from the common programming languages.
You need to configure endpoints of Amazon VPC to keep traffic between the Secrets Manager & VPC.

For improving availability and decreasing latency during secret retrieval, Secrets Manager client-side caching libraries can be used.

Audit & monitor secrets usage:

It lets you audit & track secrets via integration with AWS logging, monitoring, & notification services. Once you enable AWS CloudTrail for an AWS Region, it is possible to audit after viewing AWS CloudTrail logs and rotate the secret.

Configuring Amazon CloudWatch is possible to receive email messages with the help of the Amazon Simple Notification Service if secrets remain unused for a long period. Otherwise, you can configure Amazon CloudWatch Events to receive push notifications while the secret manager will rotate the secrets.

Compliance:

The service can fulfill compliance needs. You can use the AWS Config Rules to verify that the secrets are configured as per the protection of your company and compliance requirements.

You can manage secrets for workloads that are subject to the following:

Department of Defense Cloud Computing Security Requirements Guide (DoD CC SRG IL2, DoD CC SRG IL4, and DoD CC SRG IL5),
Federal Risk and Authorization Management Program (FedRAMP),
The U.S. Health Insurance Portability and Accountability Act (HIPAA),
Information Security Registered Assessors Program (IRAP),
Outsourced Service Provider’s Audit Report (OSPAR),
ISO/IEC 27001, ISO/IEC 27017, ISO/IEC 27018, ISO 9001
TPayment Card Industry Data Security Standard (PCI-DSS), or System and Organization Control (SOC).

How does AWS Secrets Manager use AWS KMS?

This one is an AWS service that can encrypt and store the secrets, decrypt transparently & return them to you in plaintext. Its main purpose is storing application secrets, like login credentials, which change periodically. You should neither hard code it nor store them in plaintext in the application. Your app will call Secrets Manager in place of table lookups or hard-coded credentials.

It can support features that help to rotate the secrets periodically which are connected to the commonly used databases. This manager can encrypt the newly rotated secrets before they are stored.

It integrates with AWS KMS for encrypting each version of secret value with a unique data key that an AWS KMS key protects. The integration keeps the secrets protected under encryption keys that will never unencrypt AWS KMS. With the help of this, it is possible to set custom permissions on the KMS key. Besides, you can audit operations that are capable of creating, encrypting, and decrypting data keys which help to protect the secrets.

What can you do with AWS Secrets Manager?

It lets you manage, retrieve & rotate database credentials, application credentials, OAuth tokens, API keys, and so on. This Secrets manager allows you to improve the security posture, as you don’t need hard-coded credentials in the application source code. When you store the credentials in this service, it helps to avoid possible compromises by anyone who is able to inspect the app or the components. Replacing hard-coded credentials is possible with a runtime call for retrieving credentials dynamically if you require them.

This service allows you to configure an automatic rotation schedule for the secrets. Besides, you are able to replace long-term secrets with short-term ones to decrease the risk of compromise. As credentials aren’t stored with the app, there is no need to update the app to rotate credentials and even no need to deploy changes to application clients.

Other tasks which you can do with secrets are:

Creating and managing secrets
Controlling access to your secrets
Retrieving secrets
Rotating secrets
Monitoring secrets
Auditing secrets for compliance
Creating secrets in AWS CloudFormation

Secrets to manage in AWS Secrets Manager:

The Secrets, including

database credentials,
on-premises resource credentials,
SaaS application credentials,
third-party API keys,
and Secure Shell (SSH) keys — can be managed. This manager lets you store a JSON document using which you can manage any text blurb of 64 KB size or smaller.

The Secrets to rotate with AWS Secrets Manager:

Rotating credentials for Amazon Relational Database Service (RDS), Amazon DocumentDB, and Amazon Redshift is possible. Extending this service is possible to rotate other secrets like credentials for Oracle databases hosted on EC2 or OAuth refresh tokens.

How to Retrieve Secrets from AWS Secrets Manager:

You need to use the console (https://console.aws.amazon.com/secretsmanager/) or the AWS CLI to retrieve the secrets. In applications, retrieving the secrets is possible by calling GetSecretValue in any of the AWS SDKs. Suppose you get to see Get a secret value in the AWS SDK Code Examples Library. If you want, you can call the HTTPS Query API directly. However, it is recommended that you should use client-side caching to cache the secret values. Caching secrets can help you to improve speed & decrease costs.

Java applications:

When you store database credentials in secret, you need to use Secrets Manager SQL connection drivers so that you can connect these to a database with the credentials in secret.

However, you need to use the Secrets Manager Java-based caching component for other secrets.

Python applications: Hence, you have to use the Secrets Manager Python-based caching component.

.NET applications: You should use its .NET-based caching component.

Go applications: Use the Secrets Manager Go-based caching component.

JavaScript applications: Call the SDK directly with getSecretValue.

PHP applications: Call the SDK directly with GetSecretValue.

Ruby applications: Call the SDK directly with get_secret_value.

GitHub Actions: See Use AWS Secrets Manager secrets in GitHub jobs. The secret manager is capable of producing a CloudTrail log entry when you want to retrieve a secret.

Within other systems and AWS services:

Retrieving secrets is possible through the following:

When it comes to AWS Batch, your task is to reference secrets in a job definition.

For AWS CloudFormation, you can create secrets and reference secrets in a CloudFormation stack. Regarding Amazon ECS, referencing secrets is possible in a container definition. Additionally, there are several cases:

Amazon EKS: Use ASCP or AWS Secrets and Configuration Provider, for mounting secrets as files in this.

GitHub: Use the Secrets Manager GitHub action so that you are capable of adding secrets as environment variables in your GitHub jobs.

AWS IoT Greengrass: It can reference secrets in a Greengrass group.

AWS Lambda: Reference secrets in a Lambda function.

Parameter Store: Users will be able to reference secrets in a parameter.

AWS CLI:

Let’s see the following get-secret-value example, which has got the current secret value.

aws secretsmanager get-secret-value \

–secret-id MyTestSecret

This following get-secret-value example has got the previous secret value.

aws secretsmanager get-secret-value \

–secret-id MyTestSecret

–version-stage AWSPREVIOUS

How can your application use these secrets?

Your first job is to write an AWS IAM or AWS Identity and Access Management policy which will permit the app to access specific secrets.
After that, your task is to replace secrets in plain text with code in the application source code to retrieve secrets programmatically with the help of the Secrets Manager APIs.

How do you get started with AWS Secrets Manager?

If you want to start using the service, you need to follow these below-mentioned steps:

First, You need to detect the secrets and find the location in the applications where these are used.
Next, your job is to use AWS credentials to sign into the AWS Management Console. Then, you have to move to the Secrets Manager console.
Now, you should upload the identified secret using the Secrets Manager console. Otherwise, you can use the AWS SDK or AWS CLI for uploading a secret. Users are able to write a script to upload several secrets.
You may find your secret unused yet. In that case, your task is to follow the instructions for configuring automatic rotation on the console. Whether you find the applications using your secret, you need to complete the 5th and 6th steps before configuring automatic rotation. When other applications retrieve the secret, you should write an IAM policy granting permissions to the secret.
Finally, you need to update the applications that retrieve secrets from the Secret Manager.

AWS Secrets Manager Pricing:

If you use this service, you need to pay only for what you use. It doesn’t require any setup fees. Besides, you don’t have to pay for secrets marked for deletion. The AWS-managed key aws/secretsmanager ( made by Secrets Manager) allows you to encrypt your secrets for free. When you make your KMS keys to encrypt the secrets, you need to pay them at the current AWS KMS rate. You need to check AWS Key Management Service Pricing if you want more details.

Once you activate the automatic rotation (except managed rotation), you will see the service using an AWS Lambda function for rotating the secret. You need to pay for the rotation function at the current Lambda rate. People charged the service as of December 2018 on a per-user basis. You need to pay $0.40 per secret monthly, whereas $0.05 for each 10,000 API calls.

The service includes the default AWS KMS key which is free but you may need to pay extra charges when an administrator generates a custom master key through AWS KMS. However, you need to pay extra to use many management events in CloudTrail, Amazon S3 and Amazon Simple Notification Service fees.

Once you activate AWS CloudTrail on your account, it is possible to obtain logs of the API calls sent out by the service. In addition, the service can log all events as management events. AWS CloudTrail is capable of storing the first copy of each management event for free. Remember, for Amazon S3 for log storage, you will incur charges. Once you set up extra trails, extra copies of every management event incur charges.

Available Now:

The service is now available in US East (N. Virginia), US East (Ohio), US West (N. California), US West (Oregon), Canada (Central), Asia Pacific (Seoul), EU (Frankfurt), Asia Pacific (Mumbai), Asia Pacific (Sydney), Asia Pacific (Tokyo), EU (Ireland), EU (London), Asia Pacific (Singapore), and South America (São Paulo).

Secrets Manager Integration:

The main purpose of integrating AWS services with Secrets Manager is to manage the credentials securely. With the help of these integrations, users are able to exchange credentials with different AWS services. AWS-managed KMS keys or customer-managed keys are used to encrypt the credentials. It rotates secrets periodically so that the level of the security bar remains high. As soon as the service stores the secrets, instead of a plain text, you can give the ARN of a secret to an AWS service.

These are the services that provides Secret Manager support for customers to exchange credentials securely:

Alexa for Business
AWS App2Container
Amazon AppFlow
AWS AppSync
Amazon Athena
AWS CodeBuild
AWS Direct Connect
AWS Directory Service
Amazon DocumentDB (with MongoDB compatibility)
AWS Elemental MediaLive
AWS Elemental MediaConnect
AWS Elemental MediaConvert
Amazon CodeGuru Reviewer
AWS Elemental MediaPackage
AWS Elemental MediaTailor
Amazon EMR
Amazon EventBridge
Amazon FSx
AWS Glue DataBrew
AWS Glue Studio
AWS IoT SiteWise
Amazon Kendra
AWS Launch Wizard
Amazon Lookout for Metrics
Amazon Managed Streaming for Apache Kafka known as mazon MSK
Amazon Managed Workflows for Apache Airflow known as mazon MWAA
AWS Migration Hub
AWS OpsWorks for Chef Automate
Amazon Relational Database Service (Amazon RDS)
Amazon Redshift
Amazon Redshift query editor v2
Amazon SageMaker
AWS Toolkit for JetBrains
AWS Transfer Family

How do you request access to AWS Secrets Manager in your AMS account?

You have to submit an RFC with the Management to request access to the service.

The RFC can provision these following IAM roles to the account: customer_secrets_manager_console_role and customer-rotate-secrets-lambda-role. The customer_secrets_manager_console_role will be used as an Admin role for the purpose of provisioning & managing the secrets. Whereas, the customer-rotate-secrets-lambda-role acts as the Lambda execution role for the Lambda function that is used to rotate the secrets. Once this will be provisioned in your account, you need to onboard the customer_secrets_manager_console_role role in your federation solution.

What are the restrictions to using AWS Secrets Manager in my AMS account?

The AMS account will have the full functionality of AWS Secrets Manager along with the secret’s automatic rotation functionality. When you use ‘Create a new Lambda function to perform rotation,’ you will find this unsupported. The reason is that it needs elevated permissions for generating the AWS CloudFormation stack (IAM Role and Lambda function creation) that helps to bypass the Change Management process. AMS Advanced Ssuits ‘Use an existing Lambda function to perform rotation and allow people to manage their Lambda functions to rotate secrets with the help of the AWS Lambda SSPS Admin role. Remember that the AMS Advanced never creates or manages Lambda to rotate the secrets.

What are the prerequisites or dependencies to using AWS Secrets Manager in your AMS account?

AMS reserves the following namespaces for use. But these can not be accessed directly by AWS Secrets Manager:

arn:aws:secretsmanager:*:*:secret:ams-shared/*

arn:aws:secretsmanager:*:*:secret:customer-shared/*

arn:aws:secretsmanager:*:*:secret:ams/*

How can you share the keys with AMS using Secrets Manager?

You need to use your federated access & the correct role for logging into the AWS Management console:

For SALZ, the Customer_ReadOnly_Role

For MALZ, AWSManagedServicesChangeManagementRole.

Now, you have to move to the AWS Secrets Manager console. Then, your job is to tap on the option Store a new secret.
Then, you have to choose Other type of secret.
Next, your task is to put the secret value as plain text. Then, your job is to use the default KMS encryption. Now, tap on Next.
Now, you should put the secret name and description. In this case, ensure that your name begins with customer-shared/. Suppose you can use customer-shared/mykey2022. Then, tap on Next.
Now, you need to leave automatic rotation disabled and after that click on Next.
Then, you have to review and tap on Store to save the secret.
You can reply using your secret name via the Service request, RFC, or incident report.

What permissions are required for sharing the keys using Secrets Manager?

SALZ: You need to find the customer_secrets_manager_shared_policy managed IAM policy. Then, your task is to verify that the policy document is the same as the one which is attached in the creation steps underneath. Ensure that the policy is linked to these IAM Roles: Customer_ReadOnly_Role.

MALZ: You should validate that the AMSSecretsManagerSharedPolicy, is attached to the AWSManagedServicesChangeManagementRole role, which lets you the GetSecretValue action in the ams-sharednamespace.

Example:

{

“Action”: “secretsmanager:*”,

“Resource”: [

“arn:aws:secretsmanager:*:*:secret:ams-shared/*”,

“arn:aws:secretsmanager:*:*:secret:customer-shared/*”

“Effect”: “Allow”,

“Sid”: “AllowAccessToSharedNameSpaces”

}

AWS Secret Manager encryption and decryption:

This service encrypts the secret value in secret by following the procedure.

The service can invoke the AWS KMS GenerateDataKey operation using a secret ID & request for a 256-bit AES symmetric key. The KMS is able to deliver a plaintext data key & the data key’s encrypted duplicate under the KMS key.

The secret value is encrypted outside of AWS KMS with the help of the plaintext data key and the AES or Advanced Encryption Standard algorithm. It can delete the plaintext key from memory once you use this.

It keeps the encrypted data key in the secret’s metadata to decrypt this. However, the encrypted secret or the encrypted data key will not be returned by Secrets Manager APIs.

If you want to decrypt a secret value which has been encrypted, you should follow the procedure.

The AWS KMS Decrypt method is invoked by Secrets Manager, supplying the encrypted data key.
For decrypting the data key, the KMS key is employed by AWS KMS for the secret. In return, it will give the data key in plaintext.
The service applies the plaintext data key for decoding secret values. After that, the key is removed from the memory.

The bottom line:

Secret management plays an important role in any type of business regarding data protection. Besides, when it comes to talking about secret management solutions, you can notice that this service can provide a wide range of feature sets. It is compatible with multiple secrets like database credentials, on-premises resource credentials, SaaS application credentials, API keys, SSH keys, etc. Multiple secret management systems are available in the security industry. However, AWS Secrets Manager works efficiently in the AWS environment & offers outstanding compatibility with other settings.

Frequently Asked Questions

What type of secrets must be shared using Secrets Manager?

Some examples are pre-shared keys for VPN creation, confidential keys like Authentication keys (IAM, SSH), License keys, and Passwords.

What is the difference between AWS KMS & Secrets Manager?

When it comes to AWS KMS, you can get a plaintext data key as a return, along with a copy of the data key, which is encrypted under the KMS key. On the other hand, the secret manager uses AES & the plaintext data key to encrypt the secret value outside of AWS KMS and removes the plaintext key from the memory after using it.