Connect to Vertex AI without a key

The AI Gateway can reach Google Vertex AI with no stored key. It uses Google Cloud Workload Identity Federation: the gateway proves its identity with a short-lived credential it already has, and Google exchanges that credential for a short-lived access token. No private key exists that could leak, and you have no key to rotate.

The gateway supports two identity sources:

Identity source	Works on	Google grants access to
Kubernetes ServiceAccount token	Any cluster whose OIDC issuer Google can reach	The gateway's ServiceAccount
AWS IAM role	Amazon EKS with EKS Pod Identity or IAM roles for service accounts (IRSA)	The IAM role the gateway runs as

Use the Kubernetes ServiceAccount token unless your security team ties GCP access to AWS IAM roles. It works on any cluster, including Amazon EKS and Azure Kubernetes Service (AKS), and the grant follows exactly one ServiceAccount.

Prerequisites

• The gcloud CLI, signed in as a principal that can create workload identity pools and service accounts and set IAM policy on your project.

• A GCP project with Vertex AI enabled. This guide uses one project, referred to as <PROJECT_ID>, for the pool, the service account, and Vertex AI. They can be separate projects.

• The name of the gateway's Kubernetes ServiceAccount. The chart names it <RELEASE_NAME>-ai-gateway-v2. Confirm it on a running install:

  kubectl get deployment <RELEASE_NAME>-ai-gateway-v2 -n <NAMESPACE> \
    -o jsonpath='{.spec.template.spec.serviceAccountName}'

Enable the APIs the token exchange uses, and look up the project number:

gcloud services enable iam.googleapis.com sts.googleapis.com \
  iamcredentials.googleapis.com aiplatform.googleapis.com \
  --project=<PROJECT_ID>
gcloud projects describe <PROJECT_ID> --format='value(projectNumber)'

Create a workload identity pool for the gateway:

gcloud iam workload-identity-pools create <POOL_ID> \
  --project=<PROJECT_ID> \
  --location=global \
  --display-name="AI Gateway"

Choose how Google grants access

Google can grant Vertex AI access in two ways. The steps for each identity source show both.

• Service account impersonation: the federated identity impersonates a GCP service account that holds roles/aiplatform.user. Vertex AI sees the service account. Set serviceAccountEmail in the gateway's configuration. Impersonation works with every Google Cloud API and fits teams that already grant access to service accounts.
• Direct resource access: the federated identity holds roles/aiplatform.user itself. No service account is involved. Leave serviceAccountEmail unset. Some Google Cloud products limit what a federated identity can do directly.

A service account you create for impersonation has no key. Create it with:

gcloud iam service-accounts create ai-gateway-vertex \
  --project=<PROJECT_ID> \
  --display-name="AI Gateway Vertex AI"
gcloud projects add-iam-policy-binding <PROJECT_ID> \
  --member="serviceAccount:ai-gateway-vertex@<PROJECT_ID>.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

Use the Kubernetes ServiceAccount token

The chart projects a short-lived, audience-bound ServiceAccount token into the gateway pod. The Google Security Token Service (STS) validates it against your cluster's OIDC issuer.

Find your cluster's OIDC issuer

Amazon EKS

aws eks describe-cluster --name <CLUSTER_NAME> \
  --query "cluster.identity.oidc.issuer" --output text

Azure AKS

Enable the OIDC issuer on the cluster if it isn't on yet, then read the URL:

az aks update -g <RESOURCE_GROUP> -n <CLUSTER_NAME> --enable-oidc-issuer
az aks show -g <RESOURCE_GROUP> -n <CLUSTER_NAME> \
  --query "oidcIssuerProfile.issuerUrl" -o tsv

Google STS fetches the issuer's signing keys, so the issuer must be publicly reachable. Amazon EKS and AKS issuers are. For a private or self-managed cluster, upload the cluster's JSON Web Key Set (JWKS) to the provider with --jwk-json-path. Upload it again after every signing key rotation, or every token exchange fails.

Create the OIDC provider

Create the provider with an attribute condition that admits only the gateway's ServiceAccount. Without the condition, any workload in the cluster can use the provider.

gcloud iam workload-identity-pools providers create-oidc <PROVIDER_ID> \
  --project=<PROJECT_ID> \
  --location=global \
  --workload-identity-pool=<POOL_ID> \
  --issuer-uri=<ISSUER_URL> \
  --attribute-mapping="google.subject=assertion.sub" \
  --attribute-condition="assertion.sub == 'system:serviceaccount:<NAMESPACE>:<SERVICE_ACCOUNT>'"

Leave the allowed audiences unset. The chart projects the provider's default audience, https://iam.googleapis.com/ followed by the provider's resource name.

Use one pool per cluster

A federated principal belongs to the pool and is keyed on the token's sub claim alone. Two clusters that run the gateway under the same namespace and ServiceAccount name produce the same principal. Give each cluster its own pool so one cluster's gateway can't use another cluster's grant.

Grant Vertex AI access to the ServiceAccount

The gateway's federated principal is:

principal://iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/subject/system:serviceaccount:<NAMESPACE>:<SERVICE_ACCOUNT>

Impersonation

Let the principal impersonate the service account:

gcloud iam service-accounts add-iam-policy-binding \
  ai-gateway-vertex@<PROJECT_ID>.iam.gserviceaccount.com \
  --project=<PROJECT_ID> \
  --role="roles/iam.workloadIdentityUser" \
  --member="principal://iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/subject/system:serviceaccount:<NAMESPACE>:<SERVICE_ACCOUNT>"

Direct resource access

Grant the principal Vertex AI access on the project:

gcloud projects add-iam-policy-binding <PROJECT_ID> \
  --role="roles/aiplatform.user" \
  --member="principal://iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/subject/system:serviceaccount:<NAMESPACE>:<SERVICE_ACCOUNT>"

Configure the provider

Replace credentialSecretRef with gcpWorkloadIdentity on the vertex provider:

values.yaml

ai-gateway-v2:
  providers:
    - kind: vertex
      baseUrl: https://aiplatform.googleapis.com
      project: <PROJECT_ID>
      region: global
      models: ['gemini-2.5-pro']
      gcpWorkloadIdentity:
        audience: //iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/providers/<PROVIDER_ID>
        # Set for impersonation. Omit for direct resource access.
        serviceAccountEmail: ai-gateway-vertex@<PROJECT_ID>.iam.gserviceaccount.com
        # Token lifetime in seconds. Defaults to 3600, minimum 600.
        expirationSeconds: 3600

The chart mounts the projected token in the pod. The kubelet rotates the token before it expires, and the gateway reads the new token on its next refresh, so rotation needs no restart. The gateway doesn't start if the token file is missing, so a broken volume fails the rollout instead of the first request.

The chart refuses to render when gcpWorkloadIdentity is on a provider other than vertex, when it's combined with credentialSecretRef, when the audience isn't a workload identity pool provider, or when serviceAccountEmail isn't a *.gserviceaccount.com address.

With networkPolicy.allowExternalEgress: false, the gateway needs no extra egress rule. The provider rule already allows port 443, which covers sts.googleapis.com and iamcredentials.googleapis.com.

Use the gateway's AWS IAM role

On Amazon EKS, the gateway can sign in to Google with the IAM role it runs as through EKS Pod Identity or IRSA. The gateway signs an AWS Security Token Service (STS) GetCallerIdentity request with that role's credentials, and Google verifies the request with AWS. The chart projects no token.

The gateway uses the role its pod has. If the gateway already reaches Claude on AWS Bedrock without a credential Secret, that's the same role. Google sees only the role's ARN, so any principal that can assume the role reaches Vertex AI. The role's trust policy is the boundary, and it must admit only the gateway.

Restrict the role's trust policy

Allow only the gateway's ServiceAccount to assume the role. With EKS Pod Identity:

trust-policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": { "Service": "pods.eks.amazonaws.com" },
      "Action": ["sts:AssumeRole", "sts:TagSession"],
      "Condition": {
        "StringEquals": {
          "aws:RequestTag/eks-cluster-arn": "arn:aws:eks:<AWS_REGION>:<AWS_ACCOUNT_ID>:cluster/<CLUSTER_NAME>",
          "aws:RequestTag/kubernetes-namespace": "<NAMESPACE>",
          "aws:RequestTag/kubernetes-service-account": "<SERVICE_ACCOUNT>"
        }
      }
    }
  ]
}

With IRSA, use the cluster's OIDC provider as a federated principal, and set a StringEquals condition on its :sub key to system:serviceaccount:<NAMESPACE>:<SERVICE_ACCOUNT>.

If the gateway's ServiceAccount has no role yet, associate one:

aws eks create-pod-identity-association \
  --cluster-name <CLUSTER_NAME> \
  --namespace <NAMESPACE> \
  --service-account <SERVICE_ACCOUNT> \
  --role-arn arn:aws:iam::<AWS_ACCOUNT_ID>:role/<ROLE_NAME>

The role needs no IAM permissions for Vertex AI. Google calls AWS STS to verify the signed request.

Create the AWS provider

Create an AWS provider for your account, with an attribute condition that pins it to the gateway's role. Without the condition, any role in the account can use the provider.

gcloud iam workload-identity-pools providers create-aws <PROVIDER_ID> \
  --project=<PROJECT_ID> \
  --location=global \
  --workload-identity-pool=<POOL_ID> \
  --account-id=<AWS_ACCOUNT_ID> \
  --attribute-condition="attribute.aws_role == 'arn:aws:sts::<AWS_ACCOUNT_ID>:assumed-role/<ROLE_NAME>'"

Leave the attribute mapping unset. The default mapping already sets attribute.aws_role to the role's assumed-role ARN.

Grant Vertex AI access to the role

The gateway's federated principal is:

principalSet://iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/attribute.aws_role/arn:aws:sts::<AWS_ACCOUNT_ID>:assumed-role/<ROLE_NAME>

Impersonation

gcloud iam service-accounts add-iam-policy-binding \
  ai-gateway-vertex@<PROJECT_ID>.iam.gserviceaccount.com \
  --project=<PROJECT_ID> \
  --role="roles/iam.workloadIdentityUser" \
  --member="principalSet://iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/attribute.aws_role/arn:aws:sts::<AWS_ACCOUNT_ID>:assumed-role/<ROLE_NAME>"

Direct resource access

gcloud projects add-iam-policy-binding <PROJECT_ID> \
  --role="roles/aiplatform.user" \
  --member="principalSet://iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/attribute.aws_role/arn:aws:sts::<AWS_ACCOUNT_ID>:assumed-role/<ROLE_NAME>"

Configure the provider for the AWS role

Set gcpWorkloadIdentity.aws on the vertex provider:

values.yaml

ai-gateway-v2:
  providers:
    - kind: vertex
      baseUrl: https://aiplatform.googleapis.com
      project: <PROJECT_ID>
      region: global
      models: ['gemini-2.5-pro']
      gcpWorkloadIdentity:
        audience: //iam.googleapis.com/projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/providers/<PROVIDER_ID>
        # Set for impersonation. Omit for direct resource access.
        serviceAccountEmail: ai-gateway-vertex@<PROJECT_ID>.iam.gserviceaccount.com
        aws:
          # The AWS region to sign for, such as us-east-1.
          region: <AWS_REGION>

aws.region is the AWS region the gateway signs for. The provider's region is the Vertex AI region. The chart refuses to render when aws.region isn't an AWS region name, and when expirationSeconds is set, because no token is projected.

With networkPolicy.allowExternalEgress: false, the chart adds an egress rule for the EKS Pod Identity Agent so the gateway can fetch its role credentials.

Verify the connection

Upgrade the release and wait for the rollout to finish:

helm upgrade <RELEASE_NAME> <CHART> -n <NAMESPACE> -f values.yaml
kubectl rollout status deployment/<RELEASE_NAME>-ai-gateway-v2 -n <NAMESPACE>

Send a request for a model the vertex provider serves:

curl -s https://<AI_GATEWAY_HOST>/v1beta/models/gemini-2.5-pro:generateContent \
  -H "Authorization: Bearer <ACCESS_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"contents": [{"role": "user", "parts": [{"text": "ping"}]}]}'

A response confirms the token exchange, the grant, and the route. The gateway fetches a Google token on the first Vertex AI request, so the pod starts cleanly even when the GCP side is misconfigured. See Troubleshooting if the request fails.

Remove the old service account key

After the request succeeds, delete the key Secret and the key itself:

kubectl delete secret <VERTEX_SECRET_NAME> -n <NAMESPACE>
gcloud iam service-accounts keys list \
  --iam-account=<KEY_SERVICE_ACCOUNT_EMAIL>
gcloud iam service-accounts keys delete <KEY_ID> \
  --iam-account=<KEY_SERVICE_ACCOUNT_EMAIL>

Next steps

• Control model access to decide which users and groups can call the Vertex AI models.
• Route models to providers to understand how requests reach the vertex provider.

Related information

• Connect model providers for the other provider kinds and credentials.
• Workload Identity Federation with other clouds in the Google Cloud documentation.

Troubleshooting

Vertex AI requests fail with "mint gcp access token"

The gateway couldn't get a Google token, so it sent nothing to Vertex AI. The gateway logs carry the Google STS error after mint gcp access token. Check these causes:

• The attribute condition doesn't match. Compare the condition with the gateway's namespace and ServiceAccount, or with the role's assumed-role ARN.
• The audience in gcpWorkloadIdentity.audience names a different pool or provider.
• Google STS can't fetch your cluster's signing keys. Confirm the issuer is public, or upload the JWKS again.
• With impersonation, the principal lacks roles/iam.workloadIdentityUser on the service account.

Vertex AI returns 403 permission denied

The token exchange worked, but the identity Vertex AI sees has no access. Grant roles/aiplatform.user on the Vertex AI project to the service account you impersonate, or to the federated principal with direct resource access. IAM changes can take a few minutes to apply.