Making Your Own VHD Files From Scratch

A few years ago a coworker came to me with a problem. He was in charge of a VM now considered legacy, but it was of course the special kind of legacy. The "too important to shut off but not important enough to actually spend money to maintain" kind of legacy. He hadn't built the VM, he inherited it. The host machine was going out of warranty and he was looking to move the VM to the cloud, what we call "lift and shift", as in pick the whole thing up and put it somewhere else until it's old enough to let us kill it.

Problem was, whoever had built the VM was well-meaning, but misguided. He or she had created the .VHD file as a Dynamic disk with a maximum size of 1TiB. That way, the VM would be able to just grow and grow as big as it wanted and never run out of disk space. But even though the VM was using nowhere near all of its allowed virtual disk space, it had changed owners (probably many owners) and eventually the cloud came along. In trying to move the VM into the cloud, where it could potentially stay safe indefinitely, my coworker had tried to convert the VHD from Dynamic — Azure doesn't allow Dynamic VHDs — to Fixed. But a Fixed 1TiB VHD is going to take 1TiB of space no matter what, and that was just breaking everything all to hell. His computer. The VM host server. The cloud. Everything.

So he came to me.

I'd never thought about how a VHD file works until that day, and I ended up writing some tools to help read and edit VHD disk metadata for that project. (I'd wondered if it would be possible to edit the Dynamic VHD metadata and resize it down to a smaller size. The answer to that question is "Don't ever try to do that if you value your sanity.") Years passed and eventually I decided to re-write one of those inspection tools from scratch in C# for personal use, which is easy enough, but I never really thought about writing my own VHD creating software because Mount-VHD was good enough.

Eventually I started playing with qemu-img and it vexed me with its love of creating sparse files where none should exist.

So last night, just to see if I could, I wrote up a VHD-creating tool. In PowerShell. It doesn't require an elevated PowerShell prompt to run and is merely a proof-of-concept of how simple the Fixed Disk VHD file format is.

$ErrorActionPreference = 'Stop'
Set-StrictMode -Version 2

Function Get-Checksum {
  Param([Parameter(Mandatory=$True)] [Byte[]] $Bytes)

  (0..3) | foreach {
    $Bytes[64+$_] = 0
  }

  $sum = ($Bytes | Measure-Object -Sum).Sum
  $chk = [System.BitConverter]::GetBytes( [System.Net.IPAddress]::HostToNetworkOrder([UInt32]::MaxValue - $sum) )
  Return $chk[4..7]
}

Function Set-CHS {
  Param([Parameter(Mandatory=$True)] [Double] $Sectors)

  [Double] $c  = 65535
  [Double] $h  = 16
  [Double] $s  = 255

  if ($Sectors -gt ($c * $h * $s)) {
    $Sectors = $c * $h * $s
  }
  if ($Sectors -ge ($c * $h * 63)) {
    $c = $Sectors / $s
  } else {
    $s = 17
    $c = $Sectors / $s
    $h = ($c + 1023) / 1024

    if (4 -gt $h) {
      $h = 4
    }
    if ( ($c -ge (1024 * $h)) -or (16 -lt $h) ) {
      $s = 31
      $h = 16
      $c = $Sectors / $s
    }
    if ($c -ge (1024 * $h)) {
      $s = 63
      $h = 16
      $c = $Sectors / $s
    }
  }
  $c       = [Math]::Floor($c / $h)
  $outbuf  = [System.BitConverter]::GetBytes( [System.Convert]::ToUInt16($c) )
  [Array]::Reverse($outbuf)
  $outbuf += [System.Convert]::ToByte($h)
  $outbuf += [System.Convert]::ToByte($s)
  Return $outbuf
}

Function New-CustomVHD {
  Param(
    [Parameter(Mandatory=$True)]  [String] $Name         = $null,
    [Parameter(Mandatory=$True)]  [UInt64] $SizeBytes    = $null,
    [Parameter(Mandatory=$False)] [String] $VhdDirectory = [System.IO.Path]::Combine(${Env:PUBLIC}, 'Documents', 'Hyper-V' ,'Virtual hard disks')
  )

  if (3145728 -gt $SizeBytes) {
    Write-Error "SizeBytes too low, minimum VHD size is 3145728"
    Return
  }

  $ascii         = New-Object System.Text.ASCIIEncoding
  $epoch         = New-Object System.DateTimeOffset 2000,1,1,0,0,0,0
  $file_name_tmp = [System.IO.Path]::Combine($VhdDirectory, ("{0}.tmp" -f $Name))
  $file_name_dst = [System.IO.Path]::Combine($VhdDirectory, $Name)

  $footer  = $ascii.GetBytes('conectix') # Cookie
  $footer += @(0,0,0,2) # Features
  $footer += @(0,1,0,0) # FileFormatVersion
  $footer += @(255,255,255,255,255,255,255,255) # DataOffset
  $ts      = (Get-Date).ToUniversalTime() - $epoch.DateTime
  $buf_ts  = [System.BitConverter]::GetBytes( [System.Convert]::ToUInt32([Math]::Floor($ts.TotalSeconds)) )
  [Array]::Reverse($buf_ts)
  $footer +=  $buf_ts # Timestamp
  $footer += $ascii.GetBytes('cust') # CreatorApplication
  $footer += @(0,1,0,0) # CreatorVersion
  $footer += $ascii.GetBytes('Wi2k') # CreatorHostOS
  $footer += [System.BitConverter]::GetBytes( [System.Net.IPAddress]::HostToNetworkOrder($SizeBytes) ) # OriginalSize
  $footer += [System.BitConverter]::GetBytes( [System.Net.IPAddress]::HostToNetworkOrder($SizeBytes) ) # CurrentSize
  $footer += Set-CHS -Sectors ([System.Convert]::ToDouble($SizeBytes / 512)) # DiskGeometry
  $footer += @(0,0,0,2) # DiskType
  $footer += @(0,0,0,0) # Checksum init

  $id   = New-Object byte[] 16
  $rnd  = New-Object System.Random
  $rnd.NextBytes($id)

  $footer += $id # UniqueID
  $footer += 0 # SavedState

  $checksum = Get-Checksum -Bytes $footer

  (0..3) | foreach {
    $footer[64+$_] = $checksum[$_]
  }

  $footer += New-Object byte[] (512 - $footer.Length)

  $fs = [System.IO.File]::Create($file_name_tmp)
  $fs.SetLength($SizeBytes)
  $fs.Seek(0, [System.IO.SeekOrigin]::End) | Out-Null
  $fs.Write($footer, 0, $footer.Length)
  $fs.Close()

  $attrib = (Get-ItemProperty -Path $file_name_tmp).Attributes
  if ([System.IO.FileAttributes]::SparseFile -band $attrib) {
    $item_property = @{
    'Path'  = $file_name_tmp;
    'Name'  = 'Attributes';
    'Value' = [System.IO.FileAttributes]::SparseFile -bxor $attrib;
    }
    Set-ItemProperty @item_property
  }

  Move-Item -Path $file_name_tmp -Destination $file_name_dst -Force:$True -Confirm:$False
  Return
}

All of this comes from the file format spec and with some basic testing today I've been able to attach a custom-created VHD to a Hyper-V VM and put FreeBSD on it. Your actual mileage may vary, use at your own risk.

(Resolution: I ended up making a second VHD, much smaller than the 1TiB original VHD, shrinking the 1TiB volume as tiny as it could get with either diskpart or Resize-Partition, cloning the file system to the second VHD, and then deleting and recreating the boot loader and MBR from a command prompt with a Windows install ISO. Fortunately for everyone involved, the 1TiB disk was not part of a RAID array, JBOD-whatever, encrypted, or Secure Boot-enabled. My advice to you is: wherever possible, split your OS and data volumes and keep them separate. I do this for everything in the cloud and it make upgrades much, much easier.)