Q1) What is an Azure Assessment tool?
A1) Azure Assessment tool is an automated cloud visibility tool developed by Lucidity which streamlines the process of discovering, reporting and analyzing Azure managed disk storage to help improve cost optimization for managed disks.
Q2) What are the uses of the Azure Assessment tool?
A2) Assessment tool provides a summary of Azure-managed disks across your subscriptions and resource groups. It helps identify idle, under-utilized, and over-utilized disks and also provides recommendations for usage optimization. It helps calculate the potential savings that can be achieved across your managed disk storage.
Q3) My Azure cloud subscriptions are spread across multiple Azure regions. Can I still run the Azure Assessment Tool and collect data across regions?
A3) Yes, the good thing about Azure Assessment Tool is its independence on Azure regions and availability zones. This script can traverse across locations to collect the metrics.
Q4) What type of network access and permissions are required to run Azure Assessment Tool?
A4) The Assessment Tool just requires network connectivity from your system and appropriate IAM permissions to execute and run the script from a terminal window. The required IAM permissions are detailed in the Azure Disk Assessment - Permissions and Prerequisites link of this knowledge base.
Q5) Can I run the Azure Assessment Tool against multiple Azure subscriptions?
A5) Yes, Assessment Tool has the option to run the script against multiple Azure subscriptions. The multiple subscription IDs can be specified in the command with comma-separated values.
Q6) Should I run the Assessment tool on the server we plan to run the pilot on?
A6) Assessment tool can be executed on any server that has the capability to access Azure APIs. While we recommend running the tool from a virtual machine, it is also possible to run it from a personal laptop.
Q7) Why are the listed permissions in the Assessment app necessary to run an Assessment?
A7) The permissions required to run an Assessment are essential for collecting metrics that provide valuable insights into your environment. Below is a detailed breakdown of why each permission is necessary (CSP wise):
AWS:
Permission Name | Description |
|---|---|
ce:GetCostAndUsage | To retrieve cost and usage information from AWS Cost Explorer. |
ec2:DescribeIamInstanceProfileAssociations | To query and list the IAM instance profile associations with an EC2 instance. |
ec2:DescribeImages | To retrieve details about AMIs on an AWS account. |
ec2:DescribeInstances | To retrieve EC2 instance details such as state, tags, storage configuration, placement, etc. |
autoscaling:DescribeAutoScalingGroups | To retrieve details about one or more Auto Scaling groups. |
ec2:DescribeRegions | To retrieve a list of all available AWS regions for an account. |
ec2:DescribeVolumes | To retrieve detailed information about one or more (EBS) volumes such as size, volume type, AZ, etc. |
eks:DescribeNodegroup | To retrieve detailed information about a specific node group. |
iam:ListPolicies | To list the policies attached to an EC2 instance |
iam:ListRoles | To list the IAM roles attached to an EC2 instance. |
ssm:DescribeInstanceInformation | To list all EC2s managed by SSM. |
ssm:GetCommandInvocation | To view details about commands executed using ssm such as Command execution status, Command output and error messages, etc. |
ec2:AssociateIamInstanceProfile | To associate an instance profile with an EC2 instance to interact with the SSM agent. |
ec2:DisassociateIamInstanceProfile | To detach an instance profile from the EC2 instance post assessment. |
eks:ListNodegroups | To retrieve a list of all node groups in a specified EKS cluster. |
iam:AddRoleToInstanceProfile | To add the temporarily created Lucidity IAM Role with ssm permissions to the instance profile. |
iam:CreateInstanceProfile | To create an instance profile, which is used to activate each VM’s SSM agent, and fetch its disk details and share it to the SSM server. |
iam:DeleteInstanceProfile | To delete the temporarily created instance profile post assessment. |
iam:RemoveRoleFromInstanceProfile | To detach the Role from the Instance Profile, before deleting it permanently. |
iam:GetInstanceProfile | To retrieve information about an instance profile, such as Profile name, ARN, Associated IAM role, associated tags, etc. |
iam:CreateRole | To create a temporary IAM role, with the above mentioned AWS policy. |
iam:DeleteRole | To delete the temporarily created Role post assessment. |
eks:ListClusters | To retrieve a list of all EKS clusters in the account. |
iam:GetRole | To fetch the existing roles of an EC2 instance. If a role already exists, an Instance Profile will be attached to that role without creating a new role. |
ssm:SendCommand | To run commands that allow collection of disk utilization metrics on Lucidity managed ec2 instances. |
iam:PassRole | To allow an IAM principal to delegate or pass permissions to an AWS service by configuring a resource such as an EC2 instance with an IAM role. |
iam:ListAttachedRolePolicies | To list the policies attached to a particular role |
iam:AttachRolePolicy | To attach the temporary AWS policy to the AWS IAM Role |
iam:DetachRolePolicy | To detach the AWS Policy from the role, before deleting it permanently. |
cloudwatch:GetMetricStatistics | To fetch historical IOPS metrics for EBS volumes |
Azure:
Permission Name | Description |
|---|---|
Microsoft.Authorization/locks/read | The Assessment app requires these permissions to detect and skip locked resource groups. |
Microsoft.Authorization/roleAssignments/read | To identify which roles are assigned to which entities within the current scope. |
Microsoft.Authorization/roleDefinitions/read | To read the role definitions of the above mentioned role assignments. |
Microsoft.Compute/disks/read | To access details about managed disks like size, disk tier etc. |
Microsoft.Compute/virtualMachineScaleSets/read | To access VMSS details and metadata. |
Microsoft.Compute/virtualMachineScaleSets/virtualMachines/read | To access details about the VMs that are part of VMSS. |
Microsoft.Compute/virtualMachineScaleSets/virtualMachines/runCommand/action | To run commands to get disk utilization of VMs within VMSS. |
Microsoft.Compute/virtualMachines/extensions/delete | During the Assessment, the Assessment application may add extensions (Azure Monitor agent and Log Analytics agent). If added by the Assessment app, this permission is required to remove the extensions during the cleanup process, ensuring the system configuration remains consistent |
Microsoft.Compute/virtualMachines/extensions/read | The Assessment app checks if the Azure Monitor agent and Log Analytics agent extensions are already attached to Azure VMs |
Microsoft.Compute/virtualMachines/extensions/write | The Assessment app adds Azure Monitor agent and Log Analytics agent extensions to Azure VMs, if they do not exist. |
Microsoft.Compute/virtualMachines/instanceView/read | To access the instance state (STOPPED, RUNNING etc) |
Microsoft.Compute/virtualMachines/read | To get VM metadata (VM size, type, configuration, IDs etc.) |
Microsoft.Compute/virtualMachines/runCommand/action | To perform Azure Run command to get disk utilization of VMs. This is an alternative incase the Log Analytics couldn't fetch utilization% |
Microsoft.Compute/virtualMachines/write | To deploy VMInsights within the VM, we need write permission on the instance |
Microsoft.CostManagement/query/read | To access managed disks' usage using Azure cost management service. |
Microsoft.Insights/DataCollectionRuleAssociations/Delete | During the Assessment, the Assessment application may create DCR associations. After the Assessment is complete, this permission is required to remove the associations during the cleanup process, ensuring the system configuration remains consistent |
Microsoft.Insights/DataCollectionRuleAssociations/Read | To analyze current DCR associations in the scope and assess if they provide access to utilization %. |
Microsoft.Insights/DataCollectionRuleAssociations/Write | To associate a DCR with specific Azure Monitor extensions, enabling the collection of data from Azure Monitor. |
Microsoft.Insights/DataCollectionRules/Delete | During the Assessment, the Assessment application may create DCRs. After the Assessment is complete, this permission is required to remove the DCRs during the cleanup process, ensuring the system configuration remains consistent |
Microsoft.Insights/DataCollectionRules/Read | To retrieve the current data collection rules within the specified scope and determine if an existing rule collects utilization %. |
Microsoft.Insights/DataCollectionRules/Write | If Azure Monitor is used to collect metrics, a DCR must be defined to gather disk utilization data, as required by Azure. |
Microsoft.Insights/Logs/Read | To read utilization metrics logged by Log Analytics and Azure Monitor extensions from the VM. |
Microsoft.Insights/MetricDefinitions/Read | To read metric definitions from Azure monitor workspace. |
Microsoft.Insights/Metricnamespaces/Read | To read metric namespaces in Azure monitor and access relevant metric definitions. |
Microsoft.Insights/Metrics/Read | To read performance metrics (CPU, memory usage, disk I/O, etc.). |
Microsoft.ManagedIdentity/userAssignedIdentities/assign/action | The Azure monitoring or Log Analytics Workspace (LAW) extensions connect to the logging service using this permission. |
Microsoft.OperationalInsights/workspaces/delete | During the Assessment, the Assessment application creates a Log analytics workspace. After the Assessment is complete, this permission is required to remove the workspace during the cleanup process, ensuring the system configuration remains consistent |
Microsoft.OperationalInsights/workspaces/query/InsightsMetrics/read | To access the metrics collected by VM Insights that are sent to Log Analytics Workspaces. |
Microsoft.OperationalInsights/workspaces/query/read | To query the Log analytics workspace and retrieve the necessary data for performing an Assessment. |
Microsoft.OperationalInsights/workspaces/read | To get details about the Log analytics workspaces and see if an existing LAW can be utilised. |
Microsoft.OperationalInsights/workspaces/sharedKeys/action | For VM extensions to send logs to a Log Analytics Workspace, Azure requires the use of a shared key for validation. |
Microsoft.OperationalInsights/workspaces/write | To create a log analytics workspace to retrieve disk utilization data from VMInsights. |
Microsoft.OperationsManagement/managementAssociations/read | To understand the current management associations and see if an existing association can be utilised. |
Microsoft.OperationsManagement/managementAssociations/write | To ensure logs are sent from all resources, an association between Azure resources and the LAW needs to be created. |
Microsoft.OperationsManagement/managementAssociations/delete | During the Assessment, the Assessment application may establish associations between the VM and the newly created workspace, allowing the VM to send data to the workspace via Log Analytics. After the Assessment is complete, this permission is required to remove the associations during the cleanup process, ensuring the system configuration remains consistent |
Microsoft.OperationsManagement/managementConfigurations/read | To read the details of existing management configurations applied to resources. |
Microsoft.OperationsManagement/managementConfigurations/write | To create management configurations to access Azure VM Insights. |
Microsoft.OperationsManagement/managementConfigurations/delete | During the Assessment, the Assessment application setups a VM within a new workspace ID to configure where data should be sent. During the Assessment, the VM will send data to this workspace. After the Assessment is complete, this permission is required to to roll back configurations for the VM within the workspace during the cleanup process, ensuring the system configuration remains consistent |
Microsoft.OperationsManagement/register/action | To register the operations management service required to initialize Azure Monitor or Log Analytics to start managing resources. |
Microsoft.OperationsManagement/solutions/read | To access the details of existing operation solutions in place so that the Assessment application can scope out the additional monitoring solutions it needs to implement.
|
Microsoft.OperationsManagement/solutions/write | To configure any monitoring service like LAW or DCR, an operations management service needs to be created. |
Microsoft.OperationsManagement/solutions/delete | During the Assessment, the Assessment application creates LAW/DCR services. Once the Assessment is complete, these are removed to roll back the changes and maintain configuration consistency. |
Microsoft.Resources/deployments/operations/read | To check current deployment operations triggered by the Assessment app. |
Microsoft.Resources/deployments/operationstatuses/read | To check the operational status of deployments created by the Assessment app, for example, the creation of LAW, DCR, VM Insights etc. Note this permission only allows the Assessment application to know the status of the operations and operations status such as “Succeeded” is fetched to confirm operations are proceeding smoothly |
Microsoft.Resources/deployments/read | To read current deployments (For example, LAW, DCR or enabling VM Insights created by the Assessment app on the resource group). |
Microsoft.Resources/deployments/write | To create a deployment (For example, LAW, DCR or enabling VM Insights) on a resource group. |
Microsoft.Resources/deployments/delete | During the Assessment, the Assessment application creates deployments for tasks such as LAW/DCR creation and enabling VM Insights. Once the Assessment is complete, this permission is required to remove the deployments as part of the cleanup process |
Microsoft.Resources/subscriptions/resourceGroups/read | To get the list of the resource groups present. |
Microsoft.RecoveryServices/vaults/read | To get the list of vaults that have ASR replica to check supportability |
Microsoft.RecoveryServices/vaults/replicationProtectedItems/read | To get the list of virtual machines that have ASR enabled to check supportability |
Microsoft.ContainerService/managedClusters/read | To get the list of all the AKS clusters to check supportability |
Microsoft.ContainerService/managedClusters/agentPools/machines/read | To get the list of all virtual machines of the AKS clusters to check supportability |
GCP:
Permission Name | Description |
|---|---|
compute.disks.list | To get disk IDs of all the disks present in that project. |
compute.instances.get | To get Information of each Instance, which will help Lucidity get the instance type to identify the number of disk slots available for that particular instance. |
compute.instances.list | To get the instance IDs of all the instances present in that specific project. |
compute.instances.update | After adding metadata, label. update call should be made to sync the properties. |
compute.zones.list | To get all the supported zones in that project. |
monitoring.timeSeries.list | To retrieve Ops Agent metrics, including mount point metrics, Lucidity typically fetches utilization data from the last 10 minutes and stores it. |
osconfig.osPolicyAssignments.create | To create an OS policy that includes details on Ops Agent installation, enabling support for Lucidity Assessment metrics collection. |
osconfig.osPolicyAssignments.delete | To delete the OS policy created for Lucidity Assessment once the Assessment is complete. |
osconfig.osPolicyAssignments.get | To get the details of the created OS policy and check its status. |
osconfig.osPolicyAssignments.list | To list all OS policies in the project and verify if Lucidity's policy has already been created. |
osconfig.osPolicyAssignments.update | To update the OS policy after Lucidity Assessment, modifying it to uninstall the Ops Agent. We ideally wait 10 minutes for the uninstallation to complete. |
resourcemanager.projects.get | To get project details such as project ID and organization ID to include it in the Assessment report. |
resourcemanager.projects.list | To get all the projects in that organisation. |
serviceusage.services.disable | To disable this service if Lucidity Assessment has enabled it. |
serviceusage.services.enable | To enable this service, because the Ops agent uses this service to get the utilization metrics. |
osconfig.osPolicyAssignmentReports.list | To get compliance reports showing whether a VM instance complies with the assigned OS policies. |
serviceusage.services.list | To list all the services present in that project. Lucidity checks if osconfig service is present in the list or not and enables it, if it is disabled |
resourcemanager.folders.list | To identify project hierarchy and organizational structure |
Q8) What are the most common reasons because of which utilization is not captured?
A8) In the vm_list.csv file in the Assessment report, we have a column utilization_error. In this column, we populate the reason for why utilization information was not captured. Below is a detailed view of all the reasons and scenario when this happens:
Error Message | Cloud Provider Name | Scenario | Troubleshooting Steps |
|---|---|---|---|
Metric Collection Agent not installed | AWS, Azure, GCP | SSM/RCA agent was not installed on the VM because the OS doesn't support the agent or we are not sure if the OS supports this agent or not | Kindly install/enable agent on the specified VMs |
Cloud Metric agent timed out | AWS, Azure | In case of SSM/RCA, the command output got timed out. | NA (Unexpected scenarios. Don’t have an RCA as of now) |
VM is not in running state | AWS, Azure, GCP | VM is currently in shut down or stopped state | Kindly restart the VM and rerun the Assessment so that metrics can be collected from these previously stopped instances. |
Policy was not applied to this resource due to insufficient permissions | GCP | If GCP policy was not applied on the instance, then Ops agent will not run, so we will not get utilization info | The permission policy was not applied to this VM. Please review and update the permissions for this resource and rerun the Assessment. You can find the missing permissions in the log file. |
Resource does not have an active service account | GCP | In GCP code, we need to check if the service account is attached to the VM or not. If service account is not attached, then we will not get utilization info | If a VM does not have an active service account, we cannot collect metrics for that specific VM. Please make sure that an active service account is associated with the VM. Steps to Add a Missing Service Account to a VM in GCP: 1. Open Compute Engine - Go to the Google Cloud Console: 👉 [GCP Compute Engine](https://console.cloud.google.com/compute/instances) - Select the correct Project where the VM is located. 2. Find the VM with the Missing Service Account - Navigate to the specific VM instance: 👉 [VM Details] (https://console.cloud.google.com/compute/instancesDetail/zones/us-central1-a/instances/<vmName>?project=<project>) - Replace <vmName> and <project> with your actual VM and project name. 3. Shutdown the VM - Click on the VM instance name. - Click STOP to shut down the VM before making changes. 4. Edit the VM Configuration - Click on Edit (top of the page). 5. Modify the Service Account - Scroll down to API and Identity Management. - Locate the Service Account field. - Select the default service account assigned to the project. - Typically, it follows this format: <project-number>-compute@developer.gserviceaccount.com - Under Cloud API access scopes, select Allow default access. 6. Save Changes & Restart the VM - Scroll down and click Save. - Restart the VM by clicking Start. Once the VM is running, it should now have the correct service account attached with proper access to Google Cloud APIs. |
OS doesn't support Ops Agent Installation | GCP | In GCP, we support only those VMs where Ops agent can be installed for metric collection | We support only those OS' where Ops agents can be installed. Please refer to the below link for details on which OS' support Ops agent installation: https://cloud.google.com/stackdriver/docs/solutions/agents/ops-agent#supported_operating_systems |
An error occurred. Please contact our support team | AWS, GCP | This is the default error message. Please contact your Lucidity SPOC for more details | Please contact your Lucidity SPOC for more details |