Secure Azure DevOps Pipelines Using OIDC and Self-Hosted Linux Agents

Introduction

Modern DevOps practices demand secure, automated pipelines without storing long-lived credentials. With OpenID Connect (OIDC) federation, Azure DevOps pipelines can dynamically assume roles in AWS, GCP, or other cloud providers, eliminating the need for static access keys.

This guide focuses on:

1. Overview of Azure Pipelines with OIDC
2. Setting up a self-hosted Linux Azure DevOps agent
3. Configuring OIDC federation with cloud identity providers

Prerequisites

Before you begin, ensure you have the following:

• AWS Account: With permissions to create IAM roles and identity providers.
• Azure DevOps Project: With access to configure service connections.
• AWS Toolkit for Azure DevOps: Version 1.15.0 or higher installed in your Azure DevOps project.

Overview of Azure Pipelines with OIDC

Azure Pipelines automates the build, test, and deployment process. Traditionally, pipelines required static credentials to access cloud resources. OIDC changes this by:

• Issuing short-lived credentials at runtime
• Eliminating the need to store secrets in pipelines
• Supporting least-privilege access to cloud services

Workflow Example:

1. Developer pushes code to Azure Repos
2. Pipeline triggers on a commit
3. Self-hosted agent picks up the job
4. Pipeline uses OIDC to assume a cloud role securely
5. Pipeline performs cloud operations (deploy, configure, test)

Setting up a self-hosted Linux agent

Azure DevOps pipelines run on Microsoft-hosted or self-hosted agents (Windows/Linux).

• Microsoft-hosted agents already include common build tools and have outbound internet access to AWS.
• Self-hosted agents must:
  • Have outbound internet access on port 443
  • Run with sufficient privileges to install AWS CLI or Toolkit
  • Be secured to prevent unauthorized pipeline executions

Restrict pipeline edit access to authorized team members and configure service connections with least privilege.

Install Dependencies

On your Linux server/VM (Ubuntu example):

Download and Configure Azure Pipelines Agent

1. Go to Azure DevOps → Project Settings → Agent Pools → New Agent
2. Select Linux and download the agent package
3. Extract and configure:
   1. Create a Directory for the Agent

2. Configure the agent

You will be asked for:

• Azure DevOps URL (e.g., https://dev.azure.com/myorg/)
• PAT (Personal Access Token) with Agent Pools → Read & Manage
• Agent Pool name

To run the agent, run the below command to run the agent interactively

Setting Up OIDC Federation Between Azure DevOps and AWS

Step 1: Create a Service Connection in Azure DevOps

This step establishes a connection between Azure DevOps and AWS using OIDC.

1. Navigate to Project Settings in Azure DevOps.
2. Under Pipelines, select Service Connections.
3. Click on New service connection and choose AWS.
4. In the Role to assume field, enter the ARN of the AWS IAM role you will create in AWS.
   (e.g., arn:aws:iam::123456789012:role/az-serviceconnection).
5. Check the Use OIDC box.
6. Name the service connection (e.g., aws-oidc-federation).
7. Save the connection.

Step 2: Obtain the Organization GUID from Azure DevOps

The Organization GUID is required to configure the identity provider in AWS.

1. Create a new pipeline in Azure DevOps using the YAML editor.
2. Use the following YAML template:

1. Run the pipeline.
2. In the pipeline logs, look for the sts get-caller-identity output.
3. Extract the arn field containing the Organization GUID in the format vstoken.dev.azure.com/{OrganizationGUID}.

Step 3: Configure an OpenID Connect Identity Provider in AWS

This step allows AWS to trust Azure DevOps as an identity provider.

1. In the AWS Management Console, navigate to IAM.
2. Select Identity providers and click on Add provider.
3. Choose OpenID Connect as the provider type.
4. For the Provider URL, enter https://vstoken.dev.azure.com/{OrganizationGUID} (replace {OrganizationGUID} with the actual GUID obtained earlier).
5. Set the Audience to api://AzureADTokenExchange.
6. Click Add provider.

Step 4: Create an AWS IAM Role for Azure DevOps

This role defines the permissions Azure DevOps will have when interacting with AWS.

1. In the AWS Management Console, navigate to IAM.
2. Select Roles and click on Create role.
3. Choose Web Identity as the trusted entity type.
4. Select the identity provider you just created.
5. For the Audience, enter api://AzureADTokenExchange.
6. Attach the necessary policies to the role, such as AmazonS3FullAccess, AmazonEC2ReadOnlyAccess, etc., depending on your requirements.
7. Name the role (e.g., az-serviceconnection).
8. Click Create role.

Step 5: Update the Trust Relationship of the AWS IAM Role

This step allows the Azure DevOps service connection to assume the AWS IAM role.

1. In the AWS Management Console, navigate to AWS IAM.
2. Select Roles and choose the role you created (azdo-federation).
3. Under the Trust relationships tab, click on Edit trust relationship.
4. Replace the existing policy with the following:

5. Click Update Trust Policy.

Step 6: Execute the Existing YAML Pipeline in Azure DevOps
With the federation configured, trigger the pipeline you created in Step 2 to validate the integration. Once completed successfully, review the pipeline logs to verify that the sts get-caller-identity command runs without issues. This confirms that the federation is functioning correctly.

Conclusion

By following these steps, you’ve successfully federated Azure DevOps with AWS using OpenID Connect. This setup allows your Azure DevOps pipelines to authenticate with AWS securely, leveraging temporary credentials and eliminating the need for static IAM credentials.

Drop a query if you have any questions regarding Azure DevOps or AWS and we will get back to you quickly.

About CloudThat