Andrew Mercer
on this page

Hybrid Server Operations

A "hybrid estate" here means one team responsible for servers that live in very different places:

Location Examples Control plane you get for free What you have to add
Azure IaaS Azure VMs, VM Scale Sets Azure Resource Manager, VM agent, Azure Update Manager (no charge for Azure VMs) Patch orchestration settings, maintenance configurations, policy
On-premises Hyper-V, VMware, KVM, bare metal Nothing Azure-native Azure Arc Connected Machine agent (azcmagent)
Third-party data centre / other cloud Colo racks, hosting providers, AWS/GCP VMs The provider's console (if any) Azure Arc agent, outbound network path to Azure

The goal is one inventory, one patch schedule model, one compliance view, regardless of where the server physically runs. Azure Arc projects non-Azure machines into Azure Resource Manager as Microsoft.HybridCompute/machines resources, which lets Azure Update Manager, Azure Policy machine configuration, Defender for Servers and Resource Graph treat them like Azure VMs.

This guide covers the platform: onboarding, scheduling, hardening. The process side lives in the sibling guides: Patch and Upgrade Management, Configuration Management and Vulnerability Remediation.


Contents

  1. Build the inventory first
  2. Onboarding non-Azure servers with Azure Arc
  3. Azure Update Manager concepts
  4. Patch orchestration modes for Azure VMs
  5. Maintenance configurations and dynamic scopes
  6. Pre- and post-maintenance events
  7. Reporting with Azure Resource Graph
  8. Cost and licensing notes
  9. Hardening: choosing a baseline
  10. Hardening Windows Server
  11. Hardening Linux
  12. Auditing baselines at scale with machine configuration
  13. Rolling hardening out without breaking things
  14. Servers that can't be Arc-enabled
  15. Operational checklist

Build the inventory first

Every other activity — patching, scanning, exception tracking — fails quietly if the asset list is wrong. A server that isn't in inventory is a server nobody patches.

Minimum fields per server:

Field Why it matters
Hostname / resource ID Join key across scanner, CMDB, Azure
Location (Azure region, DC, provider) Network paths, maintenance windows, data residency
OS and version EOL tracking, which patch tooling applies
Environment (dev/test/prod) Patch ring assignment
Patch group / wave Which maintenance configuration it belongs to
Business owner and technical owner Who approves windows, who fixes app breakage
Criticality (tier 0–3) Remediation SLA and change process
Internet-exposed (yes/no) Vulnerability prioritisation
Reboot constraints Clustered? Stateful? Needs manual failover?

Enforce the tag set with Azure Policy (Require a tag on resources / Inherit a tag from the resource group) so new Azure VMs and Arc machines can't land untagged. A suggested tag schema:

env            = prod | test | dev
patch-group    = prod-wave1 | prod-wave2 | nonprod
owner          = [email protected]
criticality    = tier0 | tier1 | tier2 | tier3
exposure       = internet | internal
reboot-policy  = auto | coordinated | manual

Reconcile three lists at least monthly: the CMDB, Azure Resource Graph (Azure VMs + Arc machines), and the vulnerability scanner's asset list. Anything that appears in only one list is either a ghost or an unmanaged server — both are findings.


Onboarding non-Azure servers with Azure Arc

Prerequisites

  • Supported OS: Windows Server 2012 R2 and later; mainstream Linux distros (RHEL, Rocky, Alma, Oracle, Ubuntu, Debian, SLES, Amazon Linux, Azure Linux). Check the current support matrix before onboarding anything unusual.
  • Outbound HTTPS (TCP 443) from the server to Azure endpoints — directly, through a proxy, or privately through an Azure Arc Private Link Scope over ExpressRoute/VPN.
  • Resource providers registered in the target subscription:
az provider register --namespace Microsoft.HybridCompute
az provider register --namespace Microsoft.GuestConfiguration
az provider register --namespace Microsoft.HybridConnectivity
az provider register --namespace Microsoft.AzureArcData
  • A service principal (or managed identity for automation) with the Azure Connected Machine Onboarding role on the target resource group — nothing broader.

Interactive onboarding (one server)

The portal (Azure Arc > Machines > Add/Create > Add a machine) generates a script. Under the hood it installs the agent and runs azcmagent connect.

At-scale onboarding

Linux:

# Install the agent (script from https://aka.ms/azcmagent; review before running)
curl -fsSL -o install_linux_azcmagent.sh https://gbl.his.arc.azure.com/azcmagent-linux
sudo bash install_linux_azcmagent.sh

sudo azcmagent connect \
  --service-principal-id     "$ARC_SP_ID" \
  --service-principal-secret "$ARC_SP_SECRET" \
  --tenant-id                "$TENANT_ID" \
  --subscription-id          "$SUBSCRIPTION_ID" \
  --resource-group           "rg-arc-servers-dc1" \
  --location                 "canadacentral" \
  --cloud                    "AzureCloud" \
  --tags "env=prod,patch-group=prod-wave2,[email protected],criticality=tier1"

Windows (elevated PowerShell):

# Download and install the agent MSI
Invoke-WebRequest -Uri "https://aka.ms/AzureConnectedMachineAgent" -OutFile "$env:TEMP\AzureConnectedMachineAgent.msi"
msiexec /i "$env:TEMP\AzureConnectedMachineAgent.msi" /qn /l*v "$env:TEMP\azcmagent-install.log"

& "$env:ProgramFiles\AzureConnectedMachineAgent\azcmagent.exe" connect `
  --service-principal-id     $env:ARC_SP_ID `
  --service-principal-secret $env:ARC_SP_SECRET `
  --tenant-id                $env:TENANT_ID `
  --subscription-id          $env:SUBSCRIPTION_ID `
  --resource-group           "rg-arc-servers-dc1" `
  --location                 "canadacentral" `
  --tags "env=prod,patch-group=prod-wave2,[email protected],criticality=tier1"

Push the same script out with whatever you already have: Ansible, Group Policy startup script, Configuration Manager, or your hosting provider's automation. Don't put the SP secret in the script file — pass it in from Ansible Vault, a CI secret, or retrieve it from Key Vault at run time, and rotate it after the onboarding campaign.

Ansible example

- name: Onboard servers to Azure Arc
  hosts: arc_onboarding
  become: true
  vars:
    arc_rg: rg-arc-servers-dc1
    arc_location: canadacentral
  tasks:
    - name: Check if already connected
      ansible.builtin.command: azcmagent show --json
      register: arc_show
      changed_when: false
      failed_when: false

    - name: Download installer
      ansible.builtin.get_url:
        url: https://gbl.his.arc.azure.com/azcmagent-linux
        dest: /tmp/install_linux_azcmagent.sh
        mode: "0700"
      when: arc_show.rc != 0

    - name: Install agent
      ansible.builtin.command: bash /tmp/install_linux_azcmagent.sh
      when: arc_show.rc != 0

    - name: Connect
      ansible.builtin.command: >
        azcmagent connect
        --service-principal-id {{ arc_sp_id }}
        --service-principal-secret {{ arc_sp_secret }}
        --tenant-id {{ tenant_id }}
        --subscription-id {{ subscription_id }}
        --resource-group {{ arc_rg }}
        --location {{ arc_location }}
        --tags "env={{ env }},patch-group={{ patch_group }},owner={{ owner }}"
      no_log: true
      when: arc_show.rc != 0 or (arc_show.stdout | from_json).status != "Connected"

Verify and troubleshoot

azcmagent show                 # status, resource ID, last heartbeat
azcmagent check --location canadacentral   # endpoint reachability test
azcmagent logs                 # bundle logs for support
azcmagent config list          # proxy, extension allow/block lists

Lock the agent down: by default Arc allows remote extension installs. On sensitive servers, restrict which extensions may run:

azcmagent config set extensions.allowlist "Microsoft.CPlat.Core/WindowsPatchExtension,Microsoft.CPlat.Core/LinuxPatchExtension,Microsoft.GuestConfiguration/ConfigurationforLinux,Microsoft.Azure.AzureDefenderForServers/MDE.Linux"
azcmagent config set guestconfiguration.enabled true

Arc for VMware vSphere / SCVMM

If on-premises is mostly vSphere or SCVMM, Arc-enabled VMware vSphere and Arc-enabled SCVMM add VM lifecycle operations (start/stop/create) on top of the guest agent. Patching and policy still go through the guest-level Connected Machine agent described above.


Azure Update Manager concepts

Azure Update Manager (AUM) replaced Automation Update Management (retired August 2024) and needs no Log Analytics workspace or Automation account. It uses the WindowsPatchExtension / LinuxPatchExtension on each machine and drives the native package manager (Windows Update Agent, apt, dnf/yum, zypper).

Concept What it does
Assessment Scans for missing updates. On-demand, or periodic assessment every 24 hours (enable via policy).
One-time update Install now, ad hoc. Good for emergency patches.
Maintenance configuration A Microsoft.Maintenance/maintenanceConfigurations resource: schedule + what to install + reboot behaviour.
Assignment Links machines to a maintenance configuration — statically (by resource ID) or via a dynamic scope (by subscription, RG, location, OS, tags).
Pre/post events Event Grid events fired before and after a schedule runs, so you can start VMs, drain load balancers, snapshot disks, run smoke tests.
Hotpatch Rebootless security updates for supported Windows Server editions (Azure Edition on Azure; Windows Server 2025 via Arc).

Turn on periodic assessment fleet-wide so the compliance view is never stale. The built-in policies are:

  • Configure periodic checking for missing system updates on azure virtual machines
  • Configure periodic checking for missing system updates on azure Arc-enabled servers

Assign both at management group scope with a DeployIfNotExists/Modify effect and a remediation task for existing machines.


Patch orchestration modes for Azure VMs

Azure VMs have a patchMode setting that decides who patches the VM. Scheduled patching through AUM only works when this is set correctly.

Windows patchMode Linux patchMode Meaning
AutomaticByPlatform AutomaticByPlatform Azure orchestrates. Combined with bypassPlatformSafetyChecksOnUserSchedule = true, this is "Customer Managed Schedules" — required for maintenance configurations. Without the bypass flag, Azure applies Automatic VM guest patching on its own safe-deployment cadence.
AutomaticByOS — Windows Update on the box decides (classic automatic updates).
Manual — Nothing automatic; you install manually or through WSUS/GPO.
— ImageDefault Whatever the distro image does (e.g. unattended-upgrades on Ubuntu).

For AUM schedules, set:

resource "azurerm_linux_virtual_machine" "app" {
  # ...
  patch_mode                                             = "AutomaticByPlatform"
  patch_assessment_mode                                  = "AutomaticByPlatform"
  bypass_platform_safety_checks_on_user_schedule_enabled = true
  reboot_setting                                         = "IfRequired"  # only valid with AutomaticByPlatform
}
az vm update -g rg-app -n vm-app-01 \
  --set osProfile.linuxConfiguration.patchSettings.patchMode=AutomaticByPlatform \
        osProfile.linuxConfiguration.patchSettings.automaticByPlatformSettings.bypassPlatformSafetyChecksOnUserSchedule=true

Arc-enabled servers don't have patchMode; they're always "customer managed" as far as AUM is concerned.

Conflict to avoid: if a Linux VM is on AUM schedules and has unattended-upgrades or dnf-automatic installing on its own timer, you get two patching systems racing each other and nonsense compliance data. Pick one per server. A common split:

  • AUM schedules for production (controlled windows, reporting, reboots coordinated).
  • Native unattended tooling for low-risk dev/test or homelab, as covered in Unattended Updates.

Maintenance configurations and dynamic scopes

Design maintenance configurations around waves, not around individual servers. A typical monthly cycle aligned to Patch Tuesday (second Tuesday of the month):

Configuration Scope (dynamic) Schedule Content
mc-nonprod env=dev,test Patch Tuesday + 1 day, 22:00 Critical, Security, Updates
mc-prod-wave1 patch-group=prod-wave1 (canaries) Patch Tuesday + 4 days, 02:00 Critical, Security
mc-prod-wave2 patch-group=prod-wave2 Patch Tuesday + 11 days, 02:00 Critical, Security
mc-prod-manual reboot-policy=manual None (one-time runs during a change) —

The recurrence syntax supports offsets from a weekday occurrence, so "four days after Patch Tuesday" is Month Second Tuesday Offset4.

Azure CLI

az maintenance configuration create \
  --resource-group rg-patching \
  --resource-name mc-prod-wave2 \
  --location canadacentral \
  --maintenance-scope InGuestPatch \
  --extension-properties InGuestPatchMode="User" \
  --maintenance-window-start-date-time "2026-12-01 02:00" \
  --maintenance-window-duration "03:55" \
  --maintenance-window-time-zone "Eastern Standard Time" \
  --maintenance-window-recur-every "Month Second Tuesday Offset11" \
  --reboot-setting IfRequired \
  --install-patches-windows-parameters classifications-to-include="Critical" "Security" "UpdateRollup" \
  --install-patches-linux-parameters classifications-to-include="Critical" "Security" package-name-masks-to-exclude="docker-ce*" "postgresql*"

Duration is the install window; AUM stops starting new installs near the end of it and reserves time for reboot. Windows needs at least 20 minutes reserved for reboot, Linux 15 — keep windows comfortably larger than your slowest server.

Terraform

resource "azurerm_maintenance_configuration" "prod_wave2" {
  name                     = "mc-prod-wave2"
  resource_group_name      = azurerm_resource_group.patching.name
  location                 = "canadacentral"
  scope                    = "InGuestPatch"
  in_guest_user_patch_mode = "User"

  window {
    start_date_time = "2026-12-01 02:00"
    duration        = "03:55"
    time_zone       = "Eastern Standard Time"
    recur_every     = "Month Second Tuesday Offset11"
  }

  install_patches {
    reboot = "IfRequired"

    windows {
      classifications_to_include = ["Critical", "Security", "UpdateRollup"]
      kb_numbers_to_exclude      = []
    }

    linux {
      classifications_to_include    = ["Critical", "Security"]
      package_names_mask_to_exclude = ["docker-ce*", "postgresql*"]
    }
  }

  tags = { owner = "[email protected]" }
}

resource "azurerm_maintenance_assignment_dynamic_scope" "prod_wave2" {
  name                         = "prod-wave2-scope"
  maintenance_configuration_id = azurerm_maintenance_configuration.prod_wave2.id

  filter {
    locations     = []
    os_types      = ["Windows", "Linux"]
    resource_types = [
      "Microsoft.Compute/virtualMachines",
      "Microsoft.HybridCompute/machines",
    ]
    tag_filter = "All"
    tags {
      tag    = "patch-group"
      values = ["prod-wave2"]
    }
  }
}

Dynamic scope assignments are made at subscription scope; create one per subscription that holds servers.

Exclusions that belong in the configuration, not in people's heads

  • Database engines with their own upgrade process (PostgreSQL, MariaDB/MySQL, SQL Server CUs).
  • Container runtimes on hosts where a restart takes down every container.
  • Kernel packages on hosts where you manage kernels separately (livepatch, vendor-certified kernels).
  • Agent software managed by its own updater (Defender for Endpoint, backup agents).

Every exclusion should map to an entry in the exception register with an owner and a reason, otherwise it becomes permanent by accident.


Pre- and post-maintenance events

Maintenance configurations publish events to an Event Grid system topic. Subscribe a webhook, Azure Function, or Automation runbook to:

Event Typical action
Microsoft.Maintenance.PreMaintenanceEvent Start deallocated VMs (AUM skips stopped machines), take disk snapshots, put nodes into maintenance in the load balancer or monitoring, disable alerting noise
Microsoft.Maintenance.PostMaintenanceEvent Run smoke tests, re-enable alerting, stop VMs that were started, post results to the team channel

Pre-events have a short timeout before the schedule proceeds, so keep handlers fast and idempotent, and make them fail closed for critical tiers: if the snapshot step fails, cancel the run for that machine rather than patching without a rollback point.

Example post-event Function logic (pseudo-code):

def on_post_maintenance(event):
    run_id = event["data"]["CorrelationId"]
    results = resource_graph(f"""
        patchinstallationresources
        | where properties.maintenanceRunId =~ '{run_id}'
        | project id, status = tostring(properties.status),
                  installed = toint(properties.installedPatchCount),
                  failed = toint(properties.failedPatchCount),
                  reboot = tostring(properties.rebootStatus)
    """)
    failed = [r for r in results if r["status"] != "Succeeded"]
    post_to_channel(summary(results), alert=bool(failed))
    for host in successful_hosts(results):
        run_smoke_test(host)

Reporting with Azure Resource Graph

Resource Graph is the single query surface for both Azure VMs and Arc machines.

Machines with pending security or critical updates:

patchassessmentresources
| where type in~ (
    "microsoft.compute/virtualmachines/patchassessmentresults",
    "microsoft.hybridcompute/machines/patchassessmentresults")
| extend p = properties
| extend machineId = tostring(split(id, "/patchAssessmentResults/")[0])
| project machineId,
          osType = tostring(p.osType),
          lastAssessed = todatetime(p.lastModifiedDateTime),
          critical = toint(p.availablePatchCountByClassification.critical),
          security = toint(p.availablePatchCountByClassification.security),
          rebootPending = tobool(p.rebootPending)
| where critical > 0 or security > 0
| order by critical desc, security desc

Machines whose assessment is stale (> 3 days):

patchassessmentresources
| where type endswith "/patchassessmentresults"
| extend lastAssessed = todatetime(properties.lastModifiedDateTime)
| where lastAssessed < ago(3d)
| project id, lastAssessed

Last installation result per machine:

patchinstallationresources
| where type endswith "/patchinstallationresults"
| extend p = properties
| summarize arg_max(todatetime(p.lastModifiedDateTime), *) by machine = tostring(split(id, "/patchInstallationResults/")[0])
| project machine, status = tostring(p.status), startedBy = tostring(p.startedBy),
          installed = toint(p.installedPatchCount), failed = toint(p.failedPatchCount)

Servers with no maintenance configuration assigned:

resources
| where type in~ ("microsoft.compute/virtualmachines", "microsoft.hybridcompute/machines")
| join kind=leftouter (
    maintenanceresources
    | where type =~ "microsoft.maintenance/configurationassignments"
    | extend machine = tolower(tostring(properties.resourceId))
    | project machine
  ) on $left.id == $right.machine
| where isempty(machine)
| project id, name, tags

(Dynamic-scope matches don't always appear as per-machine assignment rows; validate this query against your own tenant and fall back to tag-based checks.)

Pin these to an Azure Workbook or export to your own dashboard. Property names in Resource Graph occasionally change — test queries in the portal's Resource Graph Explorer before wiring them into alerts.


Cost and licensing notes

  • Azure VMs: Azure Update Manager is free.
  • Arc-enabled servers: charged per server per day of management, except when the server is covered by Microsoft Defender for Servers Plan 2, is enrolled for Extended Security Updates through Arc, or runs on Azure Local with Azure benefits. Check the current pricing page — this has changed more than once.
  • Machine configuration (guest configuration) on Arc: also billed per server unless covered by Defender for Servers / other benefits.
  • Windows Server 2016 reaches end of extended support on January 12, 2027. ESUs are available through Azure Arc (generally available since August 2026). Unlike earlier ESU programs, don't assume ESU is free on Azure VMs for 2016 — confirm licensing before you plan around it, and treat ESU as a bridge to an upgrade, not a destination. See Patch and Upgrade Management.
  • Hotpatch for Windows Server 2025 via Arc is available at no additional cost (since May 2026), but still needs a quarterly baseline reboot.

Hardening: choosing a baseline

Pick one primary baseline per OS family and document deviations. Mixing baselines without a source of truth produces endless arguments about which setting is "right."

Baseline Coverage Strengths Notes
CIS Benchmarks Windows Server, RHEL/Rocky/Alma, Ubuntu, Debian, SLES, Amazon Linux, plus apps (SQL Server, PostgreSQL, Nginx, Tomcat, Kubernetes) Industry standard; auditors recognise it; Level 1 (low impact) and Level 2 (defence in depth) profiles; separate Member Server / Domain Controller profiles Free PDFs; CIS-CAT Pro, build kits and Group Policy objects need SecureSuite membership
Microsoft Security Baselines Windows Server, Windows client, Edge, M365 apps Shipped as GPO backups in the Security Compliance Toolkit; tested by Microsoft Fewer settings than CIS; Windows only
Azure compute security baseline Windows and Linux on Azure and Arc Built into Azure Policy machine configuration; free audit on Azure VMs Microsoft-defined, close to CIS but not identical
CIS baselines via Azure Policy (OSConfig) Windows Server 2025 (audit-only initial release, May 2026), Linux distros Native audit in Azure Policy without third-party tooling Check coverage per OS before relying on it
DISA STIGs Most enterprise OSes Required in US government contexts Strict; operational impact higher than CIS L1

Current CIS versions worth knowing as of late 2026: Windows Server 2025 Benchmark v2.1.0 and Windows Server 2022 v5.1.0 (both updated September 2026), Ubuntu 24.04 LTS v2.0.0, RHEL 9 v3.0.0, plus new Ubuntu 26.04 coverage. Benchmarks update several times a year — record the exact version you implement.

Profile selection rule of thumb: CIS Level 1 everywhere; Level 2 on tier-0 systems (domain controllers, PKI, jump hosts, anything holding secrets for other systems) where you can test thoroughly.


Hardening Windows Server

Domain-joined servers

Group Policy remains the most reliable enforcement mechanism for domain members.

  1. Import the baseline GPOs (CIS build kit or Microsoft Security Compliance Toolkit) into a test OU first.
  2. Split into layered GPOs rather than one monolith: - SEC-Baseline-MemberServer (CIS L1 Member Server) - SEC-Baseline-DomainController - SEC-Role-IIS, SEC-Role-SQL, SEC-Role-FileServer (role-specific relaxations or additions) - SEC-Exceptions-<app> (documented deviations, linked to exception IDs)
  3. Use LocalGPO/LGPO.exe from the toolkit for workgroup or non-domain servers.

High-value settings that frequently show up as findings:

# SMBv1 off
Disable-WindowsOptionalFeature -Online -FeatureName SMB1Protocol -NoRestart
Set-SmbServerConfiguration -EnableSMB1Protocol $false -Force

# SMB signing required (server and client)
Set-SmbServerConfiguration -RequireSecuritySignature $true -Force
Set-SmbClientConfiguration -RequireSecuritySignature $true -Force

# Disable legacy TLS/SSL protocols in SChannel
foreach ($proto in 'SSL 2.0','SSL 3.0','TLS 1.0','TLS 1.1') {
  foreach ($side in 'Server','Client') {
    $k = "HKLM:\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\$proto\$side"
    New-Item -Path $k -Force | Out-Null
    New-ItemProperty -Path $k -Name Enabled -Value 0 -PropertyType DWord -Force | Out-Null
    New-ItemProperty -Path $k -Name DisabledByDefault -Value 1 -PropertyType DWord -Force | Out-Null
  }
}

# LSA protection (RunAsPPL)
New-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\Lsa' -Name RunAsPPL -Value 1 -PropertyType DWord -Force

# NTLMv2 only, refuse LM & NTLM
New-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\Lsa' -Name LmCompatibilityLevel -Value 5 -PropertyType DWord -Force

# Print Spooler off where not needed (CIS recommends disabling on DCs and most servers)
Set-Service -Name Spooler -StartupType Disabled; Stop-Service -Name Spooler -Force

Treat these snippets as illustrations — in production, set them through GPO or DSC so they're enforced continuously, not once.

Additional controls that baselines assume but don't configure:

  • LAPS (Windows LAPS is built in from the April 2023 updates onward) for unique local admin passwords.
  • Credential Guard and HVCI on Windows Server 2022/2025 where hardware supports it (watch for driver compatibility).
  • Defender for Endpoint with tamper protection; ASR rules in audit first.
  • Windows Firewall enabled on all profiles with inbound default deny.

Validation

  • Get-SmbServerConfiguration, Get-TlsCipherSuite, auditpol /get /category:* for spot checks.
  • Microsoft Policy Analyzer (toolkit) to diff effective GPO against baseline.
  • CIS-CAT Pro or your scanner's compliance module (Qualys Policy Compliance, Tenable audit files, Defender Vulnerability Management security baselines assessment) for scored reports.

Hardening Linux

Tooling per distro

Distro Native/vendor tooling Community automation
RHEL / Rocky / Alma / Oracle scap-security-guide + oscap (CIS and STIG profiles), RHEL Image Builder with OpenSCAP remediation at build time ansible-lockdown/RHEL9-CIS, RHEL8-CIS
Ubuntu Ubuntu Security Guide (usg) with Ubuntu Pro — CIS and DISA-STIG profiles, audit and fix ansible-lockdown/UBUNTU24-CIS, UBUNTU22-CIS
Debian OpenSCAP content is limited; mostly community roles ansible-lockdown roles, dev-sec/ansible-collection-hardening
SLES scap-security-guide profiles community roles
Azure Linux Microsoft-hardened by default; check machine configuration CIS policies —

OpenSCAP on RHEL-family

sudo dnf install -y openscap-scanner scap-security-guide
ls /usr/share/xml/scap/ssg/content/            # ssg-rhel9-ds.xml, etc.

# List profiles
oscap info /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml | grep -A1 'Profile'

# Audit against CIS Level 1 Server
sudo oscap xccdf eval \
  --profile xccdf_org.ssgproject.content_profile_cis_server_l1 \
  --results /var/tmp/cis-results.xml \
  --report  /var/tmp/cis-report.html \
  /usr/share/xml/scap/ssg/content/ssg-rhel9-ds.xml

# Generate an Ansible playbook that remediates only the failed rules
oscap xccdf generate fix --fix-type ansible \
  --result-id "" /var/tmp/cis-results.xml > cis-remediate.yml

Review generated remediation before running it — it applies everything that failed, including settings your applications may depend on.

Ubuntu Security Guide

sudo pro attach <token>
sudo pro enable usg
sudo apt install -y usg

sudo usg audit cis_level1_server            # HTML + XML report in /var/lib/usg
sudo usg generate-tailoring cis_level1_server tailor.xml   # customise rules
sudo usg audit --tailoring-file tailor.xml
sudo usg fix --tailoring-file tailor.xml    # apply (test first!)

Tailoring files are the right place to record deviations; store them in Git next to the exception register.

Settings that most often break things

Control Typical breakage Mitigation
Separate /tmp, /var/tmp with noexec Installers and some apps execute from /tmp Set TMPDIR for those apps; exception for build hosts
Disable unused filesystems (cramfs, squashfs, udf) Snap packages need squashfs Exclude rule on Ubuntu if snaps are used
SSH AllowGroups/AllowUsers Automation accounts locked out Add automation group before rollout
auditd rules with -e 2 (immutable) Changes to audit rules require reboot Fine for prod; leave mutable in dev
Firewall enabled with default deny Monitoring/backup agents blocked Inventory listening ports first (ss -tlnp)
kernel.yama.ptrace_scope Debuggers, some APM agents Exception per host
Password/PAM policy Breaks LDAP/SSSD auth stacks if applied blindly Test against your auth setup

Auditing baselines at scale with machine configuration

Azure Policy machine configuration (formerly guest configuration) audits — and optionally enforces — settings inside the OS of Azure VMs and Arc machines, and shows results per setting in the Policy compliance blade.

Built-in initiatives to start with:

  • Windows machines should meet requirements of the Azure compute security baseline
  • Linux machines should meet requirements for the Azure compute security baseline
  • CIS-based machine configuration policies for supported Linux distros and Windows Server 2025 (check the built-in definitions list for your OS versions)

Assignment pattern:

  1. Assign the prerequisite initiative Deploy prerequisites to enable Guest Configuration policies on virtual machines (adds system-assigned identity and the extension).
  2. Assign the baseline initiative in Audit mode at management group scope.
  3. Export non-compliant settings, compare against your chosen CIS profile, and record deviations as exemptions (see Exception Management).
  4. For a subset of settings you're confident about, author custom packages with ApplyAndAutoCorrect (see Configuration Management).

Query results via Resource Graph:

guestconfigurationresources
| where name contains "AzureLinuxBaseline" or name contains "AzureWindowsBaseline"
| extend compliance = tostring(properties.complianceStatus),
         machine = tostring(split(properties.targetResourceId, "/")[8])
| project machine, compliance, lastChecked = todatetime(properties.lastComplianceStatusChecked)

Defender for Cloud also surfaces OS misconfigurations through Defender Vulnerability Management security baseline assessments (Defender for Servers Plan 2), mapped to CIS. Use whichever source your compliance reporting is built on, but don't run three overlapping audits and reconcile them by hand.


Rolling hardening out without breaking things

  1. Inventory dependencies — listening ports, service accounts, scheduled tasks/cron, agents, auth methods.
  2. Audit only across the whole estate. Produce a per-setting failure count.
  3. Group settings by risk: zero-impact (banner text, log retention), low-impact (SMB signing, TLS 1.0 off), high-impact (firewall default deny, PAM changes, noexec mounts).
  4. Pilot on non-production servers that mirror production roles. Run application test suites, not just "it booted."
  5. Canary production — one server per role, monitor for a full business cycle (including month-end jobs if relevant).
  6. Wave rollout reusing the patch groups.
  7. Enforce continuously (GPO, DSC, Ansible, machine configuration auto-correct) and alert on drift.
  8. Document every deviation as an exception with an expiry date.

Bake hardening into images too: build golden images (Packer, Azure Image Builder, RHEL Image Builder) that already pass the baseline, so new servers start compliant rather than being fixed after deployment.


Servers that can't be Arc-enabled

Some servers will never run the Arc agent: air-gapped networks, appliances, vendor-locked systems, customer-owned hosts in a hosting contract.

  • Patch: WSUS/Configuration Manager for Windows; a local repository mirror (Foreman/Katello, Pulp, apt-mirror/aptly, Satellite) plus Ansible for Linux.
  • Assess: authenticated scans from Qualys/Tenable scanner appliances inside the network segment.
  • Harden: Ansible or GPO; CIS-CAT or OpenSCAP reports exported for evidence.
  • Report: push results into the same dashboard (Log Analytics custom table via the Logs Ingestion API, or your SIEM) so these servers aren't invisible.

Record why each is excluded from Arc, with an owner — it's an exception like any other.


Operational checklist

Daily - [ ] Review failed or partially failed patch runs from last night - [ ] Check machines with stale assessments (> 3 days) - [ ] Triage new critical/KEV findings (see Vulnerability Remediation)

Weekly - [ ] Reconcile Arc inventory vs CMDB vs scanner - [ ] Review Arc agents disconnected > 7 days - [ ] Review reboot-pending machines

Monthly - [ ] Patch Tuesday wave plan confirmed and communicated - [ ] Baseline compliance trend reviewed; new deviations recorded as exceptions - [ ] Arc agent version currency (azcmagent auto-upgrade or update through your package manager)

Quarterly - [ ] Re-validate baseline version against latest CIS release - [ ] Review all hardening exceptions for expiry - [ ] EOL horizon review (next 12 months) - [ ] Hotpatch baseline reboot months scheduled (Windows Server 2025 hotpatch)


References

  • Azure Update Manager documentation — Microsoft Learn
  • Azure Update Manager pricing — azure.microsoft.com/pricing/details/azure-update-management-center
  • Azure Arc-enabled servers — Microsoft Learn
  • Azure Policy machine configuration — Microsoft Learn
  • CIS Benchmarks — cisecurity.org/cis-benchmarks
  • Microsoft Security Compliance Toolkit — Microsoft Download Center
  • Ubuntu Security Guide — ubuntu.com/security/certifications/docs/usg
  • ComplianceAsCode / scap-security-guide — github.com/ComplianceAsCode/content
  • Ansible Lockdown — github.com/ansible-lockdown