Configure OpenID Connect in AWS to retrieve temporary credentials

DETAILS: Tier: Free, Premium, Ultimate Offering: GitLab.com, Self-managed, GitLab Dedicated

WARNING: CI_JOB_JWT_V2 was deprecated in GitLab 15.9 and is scheduled to be removed in GitLab 17.0. Use ID tokens instead.

In this tutorial, we'll show you how to use a GitLab CI/CD job with a JSON web token (JWT) to retrieve temporary credentials from AWS without needing to store secrets. To do this, you must configure OpenID Connect (OIDC) for ID federation between GitLab and AWS. For background and requirements for integrating GitLab using OIDC, see Connect to cloud services.

To complete this tutorial:

Add the identity provider
Configure the role and trust
Retrieve a temporary credential

Add the identity provider

Create GitLab as a IAM OIDC provider in AWS following these instructions.

Include the following information:

Provider URL: The address of your GitLab instance, such as https://gitlab.com or http://gitlab.example.com. This address must be publicly accessible.
Audience: The address of your GitLab instance, such as https://gitlab.com or http://gitlab.example.com.
- The address must include https://.
- Do not include a trailing slash.

Configure a role and trust

After you create the identity provider, configure a web identity role with conditions for limiting access to GitLab resources. Temporary credentials are obtained using AWS Security Token Service, so set the Action to sts:AssumeRoleWithWebIdentity.

You can create a custom trust policy for the role to limit authorization to a specific group, project, branch, or tag. For the full list of supported filtering types, see Connect to cloud services.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::AWS_ACCOUNT:oidc-provider/gitlab.example.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "gitlab.example.com:sub": "project_path:mygroup/myproject:ref_type:branch:ref:main"
        }
      }
    }
  ]
}

After the role is created, attach a policy defining permissions to an AWS service (S3, EC2, Secrets Manager).

Retrieve temporary credentials

After you configure the OIDC and role, the GitLab CI/CD job can retrieve a temporary credential from AWS Security Token Service (STS).

assume role:
  id_tokens:
    GITLAB_OIDC_TOKEN:
      aud: https://gitlab.example.com
  script:
    - >
      export $(printf "AWS_ACCESS_KEY_ID=%s AWS_SECRET_ACCESS_KEY=%s AWS_SESSION_TOKEN=%s"
      $(aws sts assume-role-with-web-identity
      --role-arn ${ROLE_ARN}
      --role-session-name "GitLabRunner-${CI_PROJECT_ID}-${CI_PIPELINE_ID}"
      --web-identity-token ${GITLAB_OIDC_TOKEN}
      --duration-seconds 3600
      --query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]'
      --output text))
    - aws sts get-caller-identity

ROLE_ARN: The role ARN defined in this step.
GITLAB_OIDC_TOKEN: An OIDC ID token.

Working examples

See this reference project for provisioning OIDC in AWS using Terraform and a sample script to retrieve temporary credentials.
OIDC and Multi-Account Deployment with GitLab and ECS.
AWS Partner (APN) Blog: Setting up OpenID Connect with GitLab CI/CD.
GitLab at AWS re:Inforce 2023: Secure GitLab CD pipelines to AWS w/ OpenID and JWT.

Troubleshooting

Error: `Not authorized to perform sts:AssumeRoleWithWebIdentity`

If you see this error:

An error occurred (AccessDenied) when calling the AssumeRoleWithWebIdentity operation:
Not authorized to perform sts:AssumeRoleWithWebIdentity

It can occur for multiple reasons:

The cloud administrator has not configured the project to use OIDC with GitLab.
The role is restricted from being run on the branch or tag. See configure a conditional role.
StringEquals is used instead of StringLike when using a wildcard condition. See related issue.

`Could not connect to openid configuration of provider` error

After adding the Identity Provider in AWS IAM, you might get the following error:

Your request has a problem. Please see the following details.
  - Could not connect to openid configuration of provider: `https://gitlab.example.com`

This error occurs when the OIDC identity provider's issuer presents a certificate chain that's out of order, or includes duplicate or additional certificates.

Verify your GitLab instance's certificate chain. The chain must start with the domain or issuer URL, then the intermediate certificate, and end with the root certificate. Use this command to review the certificate chain, replacing gitlab.example.com with your GitLab hostname:

echo | /opt/gitlab/embedded/bin/openssl s_client -connect gitlab.example.com:443

`Couldn't retrieve verification key from your identity provider` error

You might receive an error similar to:

An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation: Couldn't retrieve verification key from your identity provider, please reference AssumeRoleWithWebIdentity documentation for requirements

This error might be because:

The .well_known URL and jwks_uri of the identity provider (IdP) are inaccessible from the public internet.
A custom firewall is blocking the requests.
There's latency of more than 5 seconds in API requests from the IdP to reach the AWS STS endpoint.
STS is making too many requests to your .well_known URL or the jwks_uri of the IdP.

As documented in the AWS Knowledge Center article for this error, your GitLab instance needs to be publicly accessible so that the .well_known URL and jwks_uri can be resolved. If this is not possible, for example if your GitLab instance is in an offline environment, you can follow issue #391928 where a workaround and more permanent solution is being investigated.