Use an Azure Virtual Machine Scale set as Azure DevOps agents

Author: Akhil M Anil || DevOps Engineer

An agent in  Azure DevOps is a computing infrastructure with installed agent software that runs a job to perform a task/ a set of tasks.

There are different types of agents available for the users such as:

  • Microsoft Hosted Agents
  • Self-hosted Agents
  • Scale Set Agents
Azure Virtual Machine Scale Set Agents

Azure Virtual Machine Scale Set agents is a feature within Azure DevOps. It allows you to use an existing Virtual Machine Scale Set to host your Azure DevOps agents and allows Azure DevOps to manage the scaling out and in of those agents. Therefore, you get the ability to deploy your agents in a private network and control the dependencies that are installed on the agents.

Note:
  • You cannot run Mac agents using scale sets. You can only run Windows or Linux agents this way.
  • Scale set agents currently supports Ubuntu Linux, Windows Server/DataCenter 2016/2019, and Windows 10 client.
In this article, we will configure the Linux Scale Set agent, install some software and run a sample ADO pipeline.

Steps:

  • Provisioning a Linux VM in Azure
    • Open portal.azure.com and click on resources. Select Virtual Machines. Here for this tutorial, I have used Linux 20.04LTS V2.

  • Install the software in VM.
    • To install software in a Linux VM, we need to connect to the VM using SSH. Here I'm using Putty to connect to VM using SSH. Initially copy the public IP of the VM, open Putty and enter the public IP and click open.
    • Connect to VM using username and password which gave at the time of VM creation.
    • Now we install software to VM. For testing purposes, we will install python3 and nodejs. 
  • Generalize the VM, Create a VM image definition
    • Open the Azure portal and open the VM which we created. As a first step stop the VM.
    • After the VM is stopped, click on capture to create an image of the VM.

    • For the target compute gallery, either you can use an existing azure compute gallery or create a new one.
    • Select the operating system state as Generalized.
    • Then create a new image definition.
    • Enter version no. as 0.0.1, the rest all keep as default values.
    • Now click Review and create.
  • Create a VMSS using the VM image definition
    • Open the VM Image definition which we created and click on create VMSS.

    • Make sure you select the scaling policy as manual and uncheck the use load balance checkbox if it is already checked.
    • Rest all keep the default values.
    • click on Review + Create and create VMSS.

  • Add VMSS agent pool in Azure DevOps
    • Open Azure DevOps, click on project settings, and click on Agent pools.
    • Click on Add pool.
    • Select pool type as Azure virtual machine scale set.
    • For Azure Subscriptions, either uses the subscription name and click authorize or create a service connection and use the service connection.
    • Select the VMSS that we created in the previous step.
    • Give a name to the agent pool.
    • For the Maximum number of virtual machines in the scale set, give any number, here I will give 5.
    • For the Number of agents to keep on standby, keep it as 1 or 2
    • Make sure to check the Grant access permission to all pipelines checkbox.

    • Now create the agent pool
    • In order to add the agents to the agent pool. Navigate back to the portal and open the VMSS that we created.
    • Click on Instances, select all the instances and click upgrade.

    • Click on Extensions + applications and we can see  Microsoft.Azure.DevOps.Pipelines.Agent extension gets created automatically.

  • Run the Pipeline 
    • Download the validation.yml from here: Azure-DevOps/VMSS-yml
    • For creating a pipeline, click on pipelines, and click on the new pipeline. select Azure repos git, select the repository, click on existing azure pipelines YAML file and select the validation.yml file
    • Click on Save and Run and Run the pipeline.

The pipeline is successful, and we can see the software are also detected!

Connect me via:

LinkedIn: Akhil M Anil | LinkedIn

References: 

Comments

Post a Comment

Popular posts from this blog

Install Java silently using powershell in Azure Windows VM

Image
Written by Akhil M Anil . To install Java silently using PowerShell in Windows VM, follow the following steps: Downloading Java can be done in two ways:   Download your required Java version ( Java Downloads | Oracle ) to a folder in your Windows VM.     Use PowerShell to Download Java to a folder.  Follow the below steps for downloading Java through PowerShell : Initially copy the Download link for your required Java version, Java file name, and Location where you want to download Java in your VM to a notepad Open PowerShell  Run the following command in PowerShell $url = "<your-copied url>" $filename = "<file_name>" $dest = "<outputfile-location\$filename>" Invoke-WebRequest -Uri $url -OutFile $dest For Example:  $url = "https://download.oracle.com/java/18/latest/jdk-18_windows-x64_bin.exe" $filename = "jdk-18_windows-x64_bin.exe"  $dest = "C:\Users\akhilanil\Desktop\Java\$filename" Invoke-WebRequest -Uri
Read more

Configure an Azure DevOps self-hosted Windows agent in Docker

Image
Author: Akhil M Anil || DevOps Engineer An agent is computing infrastructure with installed agent software that runs job to perform a task/ a set of tasks. There are different types of agents available for the users such as: Microsoft Hosted Agents Self-hosted Agents Scale Set Agents Self-hosted Agents An agent that we set up and manage on our own to run jobs is a  self-hosted agent .  Self-hosted agents give us more control to install dependent software needed for our builds and deployments. Also, machine-level caches and configuration persist from run to run, which can boost speed. We  can install the agent on Linux, macOS, or Windows machines. We can also install an agent on a Docker container. This Article covers configuring an agent on a Docker container and use the agent to run job to perform tasks. Run a self-hosted agent in Docker We can set up a self-hosted agent in Azure Pipelines to run inside a Windows Server Core (for Windows hosts), or Ubuntu container (for Linux hosts)
Read more

List of Repositories - Authorize Rest API calls using Job Access Token

Image
  Author: Akhil M Anil || DevOps Engineer Getting the complete list of Repository names  in Azure DevOps Project using REST Api and Publishing as a CSV file. In the last blog I have talked about Authorizing REST API call using Job Access Tokens. If missed, please find that blog here:  Job Access Tokens . In this blog we will use Job Access Token to Authorize Rest API call and retrieve the list of repositories.  Full code can be found in GitHub:  Repository-List Steps: Push the developed pipeline code  (azure-pipelines-repos.yml) and python code  (repos.py) to Azure Repos. We are using  Repositories - List - REST API  to fetch the repository list using python. Job Authorization Scope for Job Access token. In order to access repositories using system access token, we need to change our default job authorization scope. In my last blog I have already mentioned about Job Authorization Scopes, feel free to take a look.  To set job authorization scope at the organization level for all projec
Read more