2018-10-20

The Quest for a Small Windows 10 Installation

There are a number of tutorials online for installing "minimal" versions of Windows XP and Windows 7. These guides aim to create a reduced set of binaries and features for Windows that will fit in typically a few hundred megabytes of storage. There are not many tutorials for "how to install a minimal version of Windows 10" and what I did find were mostly a hodge-podge of one-liners on Reddit. We can do better.

From time to time I want to set up a Windows VM. In particular, I want to set up a short-lived Windows VM that will run some software or do some legacy task, and then I tear it down and go on my merry way. I used to use the free Windows VMs that Microsoft provides for browser testing and they're decent VMs, but they are huge. I have tried, with limited success, to shrink this .VHDX file so I can run more than two or three of them at once. Even if I only use one Edge testing VM, I have this big old 18+ GiB .VHDX file laying around that I have to keep because it has my settings on it and is already configured the way I want it. But it's huge!

I uninstalled programs. I reordered the files to the front of the virtual disk with a defragmenting application. I zeroed out the freespace with an old version of SDelete (SDelete v2.0 has substantially worse performance than v1.61). I ran the Optimize-VHD PowerShell cmdlet. Some of this made reductions here and there, but these gains were relatively minor. There's just too much "stuff" in the free VM that Microsoft offers.

The best thing for a tiny Windows install is Windows Server Nano, which is a even-more-stripped-down version of Server Core that has all the UI and graphics goo removed. Server Nano is just a kernel, a shell, and a remote management endpoint so you can assign it work. It's great! It's tiny! It's exactly what I want! But, it sucks for gaming and it's basically discontinued as of 2017! Technically, you could say that Nano Server is having its "scope limited" to just container applications, but that's the same thing in my book. Sigh. OK. I couldn't handle the price tag of buying a Windows Server licence anyway.

So I kept on using the free VMs, updating them periodically as new versions of Windows 10 were released, and wondering if there was a better way, a way to run Windows 10, for free, in a VM that doesn't eat half the space of an SSD drive that I could instead be using to store cat videos and episodes of Pushing Daisies.

And there is a better way. In fact, Microsoft has a better way and gives it away for free, kinda. They took Windows 10, an OK operating system, and pulled all the garbage out of it that people don't want: it has no Cortana integration, no Store integration, no Edge integration. It doesn't automatically try to upgrade itself to The Next Thing every six months. It's called "Windows 10 Enterprise LTSB", or "Enterprise S", or now "Enterprise LTSC", and it's Windows the way Windows was meant to be. I just wish they'd promoted this version of Windows more than "not at all whatsoever" before I chose to leave Windows World and find something better.

The following process is how I set up a Windows 10 VM from scratch. Your actual mileage may vary. My goal in performing these steps is to produce the smallest Windows 10 VM I can. There is a trade-off between small size and other system metrics, like performance and security. In this case, I don't mind an unoptimized VM if it is tiny. If I really wanted performance from my Windows 10 install, I wouldn't virtualize it in the first place.

  1. Fetch the latest Windows 10 Enterprise LTSB install ISO. This can be found on the Microsoft Evalution Center. You want the "ISO - LTSB" version; if you pick "ISO - Enterprise" it will prompt you for registration information and not give you what you want.

  2. When you have the ISO downloaded, create your VM, attach the ISO as a virtual CD/DVD, and boot. A quick way to do this in an elevated PowerShell window is like so:

    $ErrorActionPreference = 'Stop'
    Set-StrictMode -Version 2
    
    $vm_name          = 'win10-ltsb'
    $iso_path         = 'C:\Path\to\14393.0.160715-1616.RS1_RELEASE_CLIENTENTERPRISE_S_EVAL_X64FRE_EN-US.ISO'
    $proc_count       = 2
    $vhd_length       = 32   * [Math]::Pow(2,30) # GiB
    $memory_min_bytes = 4096 * [Math]::Pow(2,20) # MiB
    $memory_max_bytes = 4096 * [Math]::Pow(2,20) # MiB
    
    $vhd_file_name = "{0}-osDisk.vhd" -f ($vm_name)
    $vm_vhd_path   = [System.IO.Path]::Combine(${Env:PUBLIC}, 'Documents', 'Hyper-V', 'Virtual hard disks', $vhd_file_name)
    
    # Create a new VHD and VM:
    $vhd_properties = @{
      'Path'      = $vm_vhd_path;
      'SizeBytes' = $vhd_length;
      'Dynamic'   = $True;
    }
    New-VHD @vhd_properties | Out-Null
    
    $vm_create_properties = @{
      'Generation'         = 1;
      'Name'               = $vm_name;
      'MemoryStartupBytes' = $memory_min_bytes;
      'VHDPath'            = $vm_vhd_path;
    }
    New-VM @vm_create_properties

  3. Remove the default unconfigured virtual network adapter and virtual DVD drive and add your own. Then, start the VM:

    $vm_net_adapter = @{
      'VMName'     = $vm_name;
      'SwitchName' = (Get-VMSwitch | Select-Object -First 1 -ExpandProperty Name);
    }
    Remove-VMNetworkAdapter -VMName $vm_name
    Add-VMNetworkAdapter @vm_net_adapter
    
    $vm_dvd_properties = @{
      'VMName'             = $vm_name;
      'ControllerNumber'   = 1;
      'ControllerLocation' = 0;
      'Confirm'            = $False;
    }
    Remove-VMDvdDrive @vm_dvd_properties
    
    $vm_dvd_properties['Path'] = $iso_path
    
    Add-VMDvdDrive @vm_dvd_properties
    $vm_set_properties = @{
      'Name'               = $vm_name;
      'DynamicMemory'      = $True;
      'MemoryMinimumBytes' = $memory_min_bytes;
      'MemoryMaximumBytes' = $memory_max_bytes;
      'ProcessorCount'     = $proc_count;
      'CheckpointType'     = 'Disabled';
    }
    Set-VM @vm_set_properties
    
    Start-VM -VMName $vm_name

  4. Connect to the VM and let it boot to the installer GUI. When you get to the "Install Now" window, press Shift+F10. This will open a command prompt window that we'll use to install Windows instead of using the GUI.

  5. Prepare your virtual disk with "diskpart". This will completely destroy any data on your virtual disk, so make sure you really want to format your VHD before you continue. Don't do this on a real system that has important data on it. The commands to run are:

    X:\sources> diskpart
      DISKPART> select disk 0
      DISKPART> clean
      DISKPART> create part pri
      DISKPART> select part 1
      DISKPART> active
      DISKPART> format quick
      DISKPART> assign letter=c
      DISKPART> exit

    CAVEAT: This partition scheme is incompatible with BitLocker disk encryption. Encryption of a VM is beyond the scope of this tutorial. For this VM, speed and at-rest security are secondary to a small VM footprint. If you wish, you can typically enable BitLocker encryption on your VMs after you have completed installation, and the manage-bde.exe utility will repartition your disk for you, if it can, as needed.

    CAVEAT: This creates an MBR partition on your VHD, which restricts it to running in Hyper-V as a Gen 1 VM. If you want your VM to be Gen 2 or to support UEFI, you'd need to adjust your diskpart commands to create an EFI boot partition. Since this might impact VHD size, I am avoiding such a configuration. For the curious, try something like this:

    REM For people who want a UEFI-friendly Gen 2 VHD layout:
    X:\sources> diskpart
      DISKPART> select disk 0
      DISKPART> clean
      DISKPART> convert gpt
      DISKPART> create part efi size=128
      DISKPART> format fs=fat32 quick
      DISKPART> create part pri
      DISKPART> format fs=ntfs quick
      DISKPART> assign letter=c
      DISKPART> list vol
      DISKPART> exit

    Remember to create the VM as a Gen 2 VM in the first place.

  6. There's a trick to getting a minimal install that allows us to install the Windows OS in a transparently-compressed format. Install Windows with DISM, using the /Compact argument:

    dism.exe /Apply-Image /ImageFile:D:\sources\install.wim /Index:1 /Compact /ApplyDir:C:\

    NOTE: You can compact the Windows base OS binaries after you've installed with compact.exe. Run the first command to check if your binaries have already been compressed, and if not, run the second command:

    C:\Windows\System32\compact.exe /CompactOS:query
    C:\Windows\System32\compact.exe /CompactOS:always

    In lieu of installing to the VHD and then compressing the OS, we compress and install at the same time.

    NOTE: If we were using a Windows install ISO that contained multiple editions on it (Pro, Home, Education, et cetera) on it, we'd need to adjust the /Index:N value to match the order of the edition we wanted. The LTSB install ISO does not include multiple editions, so "/Index:1" works. If you want to inspect your install media and see which editions/indexes are available to you, run:

    dism.exe /Get-ImageInfo /ImageFile:D:\sources\install.wim

    NOTE: The name of the .WIM file may also vary from ISO to ISO depending on which version of Windows you're installing, so you can look for the biggest file in the D:\sources directory by running dir /s D:\sources | find "install.". The file name may not even end in .WIM, but it's going to be the only file that's 3+ gigs in size.

  7. Configure the bootloader:

    bcdboot C:\Windows

  8. Exit the command prompt, stop the VM, and eject the install media. I like to stop the VM after I close the command prompt by clicking "Repair your computer" and then "Turn off your PC". Most other options in the GUI just restart your machine, but this method actually halts it for real.

    If you want to make a backup of the .VHD file now, you can. This VHD is smaller than a normal Windows install and usable as-is, but it can stand to get smaller.

  9. We can open up the VHD before before we start it for the first time and make some configuration changes:

    $vhd_obj           = Mount-VHD -Path $vm_vhd_path -PassThru
    $part_obj          = Get-Partition -DiskNumber $vhd_obj.DiskNumber
    $drive_letter      = $part_obj.DriveLetter + ':'
    $vhd_system32_path = "{0}:\Windows\System32" -f ($part_obj.DriveLetter)
    $vhd_sysprep_path  = [System.IO.Path]::Combine($vhd_system32_path, 'Sysprep')
    $vhd_system_hive   = [System.IO.Path]::Combine($vhd_system32_path, 'config', 'SYSTEM')

  10. Mount the registry of the fledgling new VM and hardcode its pagefile and swapfile maximum sizes to 256 MiB:

    reg.exe load HKLM\VHDSYSTEM $vhd_system_hive
    
    reg.exe add "HKLM\VHDSYSTEM\ControlSet001\Control\Session Manager\Memory Management" /f /v PagingFiles /t REG_MULTI_SZ /d "?:\pagefile.sys 256 256"

    256 MiB is a comfortable compromise to me between making the OS image slim and making the OS unusable. Windows will set the swapfile to be at least 256MiB anyways if you try to make it smaller than that.

  11. Implement the Spectre registry key fix:

    reg.exe add "HKLM\VHDSYSTEM\ControlSet001\Control\Session Manager\Memory Management" /v FeatureSettingsOverride /t REG_DWORD /d 8 /f
    
    reg.exe add "HKLM\VHDSYSTEM\ControlSet001\Control\Session Manager\Memory Management" /v FeatureSettingsOverrideMask /t REG_DWORD /d 3 /f
    
    reg.exe unload HKLM\VHDSYSTEM

  12. Write an unattend file. This is non-trivial and could make for a series of blog posts in its own right.

    Back in September of 2001 I began a project of writing documentation for building a comprehensive unattended network installation process for Windows NT 4.0 Workstation. I took this idea on as a personal challenge: how does one take 12 IBM computers with network access and consistently format them and re-install them using only a network share, some DOS Ethernet drivers, and a floppy disk? It can be done, even with buggy closed-source tools with names like "sysdiff". It was nerdily glorious and took me all summer to get working. The bulk of the time I spent building the automated OS network deployment system was fighting with drivers and NT bugs, but a close second was drafting a thorough unattended "answer file". Back then it was an .INI file, nowadays Microsoft likes XML. It's going to turn into JSON eventually.

    A well-crafted unattend file is a really amazing read. If you want to write your own, start with a basic template and then read through the Components documentation online. For the purposes of this document, you can start with this one. It's important to note that the following file:

    (a) Uses the amd64 architecture and will not work on an x86 install. If you'd rather install an x86 VM, WHAT IS WRONG WITH YOU?

    (b) defines an Administrators-level user account and password you should change before you use it on your own VMs

    (c) disables the System Restore feature to save space

    (d) disables system hibernation

    (e) disables the Windows Update service

    (f) sets the locale and time zone to en-US and Pacific Standard Time

    (Carefully) make any modifications to this file you need to make, and then save this file as "unattend.xml":

    <?xml version="1.0" encoding="utf-8"?>
    <unattend xmlns="urn:schemas-microsoft-com:unattend">
      <settings pass="oobeSystem">
    
        <component name="Microsoft-Windows-International-Core" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
          <InputLocale>0409:00000409</InputLocale>
          <SystemLocale>en-US</SystemLocale>
          <UILanguage>en-US</UILanguage>
          <UserLocale>en-US</UserLocale>
        </component>
    
        <component name="Microsoft-Windows-Shell-Setup" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State">
          <Display>
            <ColorDepth>32</ColorDepth>
            <HorizontalResolution>1024</HorizontalResolution>
            <RefreshRate>60</RefreshRate>
            <VerticalResolution>768</VerticalResolution>
          </Display>
          <FirstLogonCommands>
            <SynchronousCommand wcm:action="add">
              <CommandLine>powercfg.exe hibernate off</CommandLine>
              <Order>1</Order>
            </SynchronousCommand>
            <SynchronousCommand wcm:action="add">
              <CommandLine>sc.exe config wuauserv start= disabled</CommandLine>
              <Order>2</Order>
            </SynchronousCommand>
            </FirstLogonCommands>
          <OOBE>
            <HideEULAPage>true</HideEULAPage>
            <HideWirelessSetupInOOBE>true</HideWirelessSetupInOOBE>
            <NetworkLocation>Work</NetworkLocation>
            <ProtectYourPC>1</ProtectYourPC>
          </OOBE>
          <RegisteredOrganization>MYORG</RegisteredOrganization>
          <RegisteredOwner>OWNER</RegisteredOwner>
          <UserAccounts>
            <LocalAccounts>
              <LocalAccount wcm:action="add">
                <Password>
                  <Value>MY_PASSWORD_HERE</Value>
                </Password>
                <Description>Administrator Account</Description>
                <DisplayName>My Account</DisplayName>
                <Group>Administrators</Group>
                <Name>MY_USERNAME_HERE</Name>
              </LocalAccount>
            </LocalAccounts>
          </UserAccounts>
          <TimeZone>Pacific Standard Time</TimeZone>
        </component>
    
        <component name="Microsoft-Windows-SystemRestore-Main" processorArchitecture="amd64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
          <DisableSR>1</DisableSR>
        </component>
    
      </settings>
    </unattend>

  13. Copy your unattend.xml to the \Windows\System32\Sysprep directory in the .VHD file:

    Copy-Item -Verbose -Path 'C:\Path\to\unattend.xml' -Destination $vhd_sysprep_path

  14. There shouldn't be a pagefile or swapfile if you haven't booted the VM yet, but accidents happen. Delete them if they exist:

    $vhd_pagefiles = @('page', 'swap') | % {"{0}:\{1}file.sys" -f ($part_obj.DriveLetter, $_)}
    Remove-Item -Verbose -Path $vhd_pagefiles -Force -Confirm:$False -ErrorAction SilentlyContinue

  15. Defrag. This isn't to actually defragment your VHD, it's using the "/X" option: "Perform free space consolidation on the specified volumes."

    Start-Process -NoNewWindow -Wait -FilePath 'defrag.exe' -ArgumentList @($drive_letter, '/V', '/H', '/X')

  16. Zero out the free space of the VHD. This requires SDelete v1.61 as mentioned previously, since SDelete v2 is broken. You can find SDelete v1.61 online pretty easily. It's also available as a Chocolatey package though expectedly you'll need to specify the exact version because it's not the latest. Verify the checksum(s) of the binary before you run it:

    MD5 = e189b5ce11618bb7880e9b09d53a588f

    SHA1 = 964f7144780aff59d48da184daa56b1704a86968

    SHA256 = 97d27e1225b472a63c88ac9cfb813019b72598b9dd2d70fe93f324f7d034fb95

    SHA512 = 292c3ea75fb957fa9dd04554c4d58b668a09c11655a88e7bc993306bf9feece8fbfefdd2934ce4e2df91947d2caff337bfab8dc990425e54bcbfe239a4d073e2

    Start-Process -NoNewWindow -Wait -FilePath 'C:\Path\to\sdelete161.exe' -ArgumentList @('-accepteula', '-z', $drive_letter)

  17. Dismount the .VHD file, then remount it with the -ReadOnly option. This allow Hyper-V to shrink its size with Optimize-VHD and all our efforts really pay off:

    $1MiB      = [Math]::Pow(2,20)
    $size_orig = (Get-Item -Path $vm_vhd_path).Length
    Dismount-VHD -Path $vm_vhd_path
    
    $vhd_obj  = Mount-VHD -Path $vm_vhd_path -PassThru -ReadOnly
    $part_obj = Get-Partition -DiskNumber $vhd_obj.DiskNumber
    
    Optimize-VHD -Path $vm_vhd_path
    Dismount-VHD -Path $vm_vhd_path
    $size_new = (Get-Item -Path $vm_vhd_path).Length
    
    $result = @()
    $result += "orig size: {0:0.0} MiB" -f ($size_orig / $1MiB)
    $result += "new  size: {0:0.0} MiB" -f ($size_new / $1MiB)
    $result += "shrinkage: {0:0.00}%" -f (100 * (1.0 - ($size_new / $size_orig)))
    Write-Host ([String]::Join("`n", $result))

Make a backup of your VHD at this point, not just for reverting back to it in case you make a mistake or hit an error in building your new VM, but because whenever you want a VM you can grab this backup VHD, attach it to your VM, and you're off to the races.

Now when you start your VM you'll have a fairly tiny Windows 10 setup. "Tiny" in this regard is comparative, as the slimmest I've been able to get this .VHD is about 3.99 GiB before I hit "Start" for the first time. That's much smaller than the free VMs you get with which to test the Edge browser, but it's enormous compared to some of the skinny Win7 images that you can create by following some of those other online tutorials.

This VM also isn't patched, so you may want to take care of that. I'd recommend it, except for circumstances where you won't ever connect this VM to the Internet. Be aware, though, that the evaluation ISO you downloaded needs to phone home in order to start your time-based 90-day trial. Otherwise, you'll find that your VM keeps shutting down every 60 minutes just to annoy you.

You can get around this by running "slmgr.vbs /rearm" from an elevated prompt and NOT rebooting when prompted to do so, but this only works one time and one time only. If you stop or restart the VM, you're back to square one.

Extra credit: Once your VM is set up, run one of the fixer scripts on Github on it. This is advanced Windows Fu, but I like to review all the settings that these scripts toggle and select the subset that's right for me. They try to minimize the telemetry stuff, but they have the side benefit of enabling a bunch of nice little customizations that I would normally end up doing by hand anyway.

I'm in the process of converging on this skinny Windows 10 LTSB image for my go-to Windows VM. I think it has potential for when I need to run a Windows-only workload (I'm looking at you, OS-dependent backup paths in SpiderOAK). I think a better way to do this, though it will yield a larger file size than what you'll get following the above method, would be to take the install ISO and slipstream a handful of current hotfixes into it with NTLite. This way I don't have to re-download and install gigs of patches every time I want to run a VM, at the cost of maintaining a custom ISO that needs to be cultivated or remade every few months.

Just adding the May 2018 servicing stack update, the October 2018 cumulative update, and the September 2018 Adobe Flash player update to the VM resulted in a final .VHD file that was 5.99 GiB. That's huge, but for a Windows VM I can't find a way to make it smaller without compromising security.

2018-10-13

Walking Calmly But Quickly Toward the Exits: On Leaving an Operating System

So I've concluded I'm done with Windows.

It's not that the operating system is too buggy, or too expensive, or too anything else. It's just become a big ugly set of hoops and I don't want to contort myself to jump through them anymore.

I came to this decision, slowly but surely, after a thousand tiny cuts. The realization came to me after the theft of the start menu in Windows 8, slipstreaming problems, the fact that Hyper-V is still really finicky and unstable, the Windows 10 telemetry thing, that one month I installed its security update and it broke all of my existing SecureStrings and the certificates installed in CurrentUser, and the spontaneous clean-slate all-or-nothing upgrades that I am obligated to endure like spoonfuls of cod liver oil just because I'm told they're good for me. These upgrades always end up breaking something, though not always the same thing twice.

I'm not mad. This is not an acrimonious divorce. It's mostly the culmination of a couple of years of conscious decisions to stop investing in the Windows platform for my personal and professional needs. It isn't even that I found something better or drank a new kind of Kool-Aid. I don't particularly like Linux all that much and the BSDs don't support all of the demands I have, whether it be software-based or hardware compatibility-restricted.

Turns out that in 2018, the operating systems field kind of sucks for consumers. There isn't a be-all, end-all OS that will scratch every itch. Even Apple's own hardware-tailored offerings leave much to be desired.

I come not to eulogize Windows. It was a fine OS for me for, let's see, twenty years? Ran every game I ever wanted (except the DOS ones (and eventually not the older CD-ROM ones after about 2007)), stopped crashing all the time around 2002, had a kick-butt media player for a while. It even evolved beyond the need to wipe and reinstall it every year like clockwork.

But times change, and wants change. I want a next-gen filesystem that I can leverage to compose a solid backup and painless restore system. I'd like to update my software in a way that won't necessarily reset everything until I manually reorganize, reinstall, or replace the things that are suddenly incompatible with a mandatory update. I want to not have a juvenile init system reinvent the wheel and insist I relearn years of sysadmin skills because it chooses to randomly re-enumerate my hardware devices or replace old, reliable tools with totally different ones under the misguided notion that if it's newer, it must be better. I want to still be able to play StarCraft once in a while. I want my hardware to be mine again.

Sure, I'll still keep a Windows install ISO around. There are still some games that haven't been ported to other OSes, but I already look with an increasingly negative eye at Windows-only and Windows-first software. Browser-based services began to break the ironclad grasp operating systems held on software and that, combined with easy virtualization and emulation tools, helps to make me feel like I have a little bit of choice again. The microcosm of "the other guys" has exploded since the era when the only UNIX software you could get was for backend networking stuff, proprietary databases, and university physics departments. Eventually, decent-looking user interfaces and WYSIWYG tools trickled into the open source ground water supply and you didn't have to spend your evenings on an IRC channel learning black magic shell script incantations through osmosis just to get a usable desktop setup.

I bought some new machines a few years back, at a time on the consumer PC metrics graph just a little to the right of the "good enough" point where the cost curve intersected the reliability curve. They just keep chugging along. Machines are durable enough now that, with a decent SSD, they can run for years without any real performance degradation or ageism penalty and satisfactorily do all but the most demanding rendering stuff in a pleasantly short amount of time. Any new hardware I buy these days is either a replacement part or a luxury item.

So really, I already bought my last Windows-dedicated hard drive at a point in the past I don't rightly recall. Windows as I mentioned finally runs great these days, until it decides to reboot and/or upgrade on you. When the time comes to replace the drive in my last Windows workstation, or the machine itself, the next drive will host something else. I'm not sure precisly what or even approximately when. It was fun while it lasted. Except for the crashing. And the spontaneous restarts. And the integrated advertising. And the fundamental UX changes imposed by corporate fiat. And the suspicious data-collection for purposes unknown. But other than that it was fun.

Next time: How to install Windows 10. No, really!

2018-10-06

Can Soup Be Evil? The Power of Editing in the Language of Film and Filmmaking

I know I'm late to the party, but I was able recently to watch season one of True Detective, an HBO crime drama from 2014 about two homicide investigators, a ton of prostitutes, and several bundles of sticks. It's an excellent character-driven story that underscores how many shades of grey there are between right and wrong and how bizarre, even contradictory, it can be to be even try to define a moral scale in an objectively neutral cosmos that doesn't love you or even notice that you exist.

While the philosophical questions that True Detective poses are positively Nietzscheian in nature, the story remains a clear, though complex, whodunnit that leaves me asking one question over and over again.

Can soup be evil?

Without going too far into spoiler territory, True Detective unambiguously delves into the seedy underbelly of a crime-laden world full of hookers and murder. HBO may bill itself as "not TV", but there are distinct limits to what it can put on the air without getting fined by the FCC. I found that the writer/director pairing of Nic Pizzolatto and Cary Joji Fukunaga executed a potent and effective method of getting under my skin without ever needing to resort to sensationalistic violence or grossly exploitative visuals.

They just did a Reverse Kuleshov. And it works.

Is a "Reverse Kuleshov" a real thing? Is it possible? Or is a Reverse Kuleshov still just a regular Kuleshov and sequence doesn't matter? I don't know. I'm not a professor of film studies.

The Kuleshov Effect is an old technique. It's been studied and put to use by filmmakers since the dawn of moving pictures. Folding Ideas has a great summary of the Kuleshov Effect I recommend you watch. Simply put, the Kuleshov Effect is an editing technique that transitions the emotional weight created by an A shot to the facial expression of a person in a B shot. The actual film experiment that Kuleshov created intercut a single shot of a man with a neutral expression on his face with other images: a plate of soup, someone laying in a coffin. Watchers of the short film inherently conveyed the feeling generated by the imagery with the man's reaction. The soup shot made him look hungry. The coffin shot made him look sad.

It was the same shot of the man's neutral expression in every instance. He wasn't really looking at food or at a dead person. The emotion invoked in the viewer by the first image was naturally and necessarily carried over into the image of the man that proceeded it. As humans, we inherently look for continuity from moment to moment, and where there is none, we create it in our own minds.

This is a simple rule of moviemaking, almost too obvious to mention, but critical in tricking an audience to understand your film as a cohesive narrative and not what it really is: an enormous collection of still frames of actors, pretending, made separately on different days over the course of several weeks and carefully revised to make it look fluid and natural. I think that a Kuleshov Effect happens every time you see a character react to something out of frame: when the teen finds a dead body in a horror movie, when Orson Welles starts clapping like a mad man at his lady love's mediocre performance in Citizen Kane, when Sam Witwicky starts muttering "no-no-no-no-no-no" and goes running ostensibly from one giant CGI robot to another, when Spock's dad Sarek sheds a tear while listening to the classical music recital in that episode of Star Trek: The Next Generation.

Perhaps I'm reading too much into it, though. Perhaps this effect is rarer and more subtle than I talk it up to be. I like to think that film has a narrative flow to it that links every shot together to tell a story and isn't just a random compilation of things the director got to point a camera at before he ran out of money. I don't know for certain if the Kuleshov Effect is a special thing that only master moviemakers use or if it's a core component of cinema that is essential to the process and language of editing. To roughly compare it to music, it is an advanced idea like augmented ninths and pentatonic scales, or it is as foundational as volume and pitch?

Back to the soup.

The Kuleshov Effect conveys a relation from a thing to a person's response to witnessing it, even though in movie-making terms that person may not actually be seeing it, or anything similar. Nowadays, when actors film their scenes, they are typically only ever going to see a guy dressed in a green suit with a tennis ball on the end of a stick taped to his back.

Then I watched True Detective and I asked myself a question. Can the opposite also be true? Can a person's reaction in shot A be conveyed into an object in shot B?

True Detective effectively uses this Reverse Kuleshov Effect, if there is such a thing, better than anything I've seen in a long time. You may recall all the scenery that Nicolas Cage chewed, wincing and flinching as he watches the alleged snuff film at the beginning of 8MM. The same thing occurs here wherein people, including a hardened homicide detective, look at images and video of bizarre, horrific abuse. The images are either not revealed or are relatively tame; tame enough to air on premium cable American television, but in order to convey to the audience that seriously fucked-up shit is being witnessed, the actors freak the hell out at what they are seeing as they see it. It is a really amazing blend of acting and editing that delivers a chillingly uncomfortable visual and an uneasy, goosebump-inducing feeling in the viewer. I compare it to the decrypted video log from the film Event Horizon only better, because the True Detective producers do much more with much less.

You never see an uncensored version of what the characters see, and you don't need to. Your comprehension of the video is informed by their reactions to it. And from that point forward, you know that the VHS cassette is, for lack of a better word, wicked. Whenever it appears in frame it has a sinister, warped feeling about it, and it's just an assembly of plastic and magnetic tape. It's generic, it's uniform, and you have a dozen tapes that look just like it in a box in a closet somewhere. It bears no unique shape or design, and it's not doing anything per se, but you, having seen not it but people's reactions to it, grow an intense feeling of disdain for it. You see it not for what it really is, but for what the True Detective cast and crew have concocted it to be: something vile, something unnatural, unholy. Mysteriously, supernaturally so.

I chalk it up to the power of visual storytelling. It's a natural human trait to see soup and think hunger. I think it takes talent to do the opposite: showing a person and then a thing, and getting an even stronger link between them. It takes real talent to do that so well that you feel physical revulsion, that sick to your stomach feeling, from an inanimate object.

2018-06-25

LUKS Encryption with ZFS Root on Void Linux

The list of non-systemd operating systems that run ZFS on the root partition is a short list, but a valued one. Today, we install Void Linux. The documentation for this OS is a litle lacking. Parts of the OS documentation are decent, especially the advanced chroot-based installation page. There are also separate pages for installing Void Linux with LUKS and installing Void Linux with a ZFS root, but not both at the same time. Let's fix that.

This is going to be very similar to how we installed Devuan jessie, only instead of using the debootstrap tool, Void Linux provides its own rootfs tarball that we will install and configure. There are a couple of gotchas that can render your zpool unbootable if you follow the wiki, but by now these steps should seem really familiar: make a LUKS container, make a zpool in the LUKS container, extract a base OS into the zpool, chroot to the base OS, install ZFS, install bootloader, cross fingers, reboot. That's really all we're doing.

Start with an Ubuntu live CD. Ubuntu 16.04+ includes the ZFS kernel modules but not the userland utilities. We start with Ubuntu because it's faster to "apt-get install" these packages than to download and build ZFS DKMS modules from scratch (twice), but if that's what you feel like doing, hey man, go for it.

Wipe the MBR, create a new partition table, and create one partition for the LUKS container. Assuming your disk is /dev/sda:

sudo su
DEVICE=/dev/sda # set this accordingly
LUKSNAME=cryptroot

wipefs --force --all ${DEVICE}
# or do this, or do both:
dd if=/dev/zero of=${DEVICE} bs=1M count=2

# Set MBR
/sbin/parted --script --align opt ${DEVICE} mklabel msdos
/sbin/parted --script --align opt ${DEVICE} mkpart pri 1MiB 100%
/sbin/parted --script --align opt ${DEVICE} set 1 boot on
/sbin/parted --script --align opt ${DEVICE} p # print

# Create LUKS container and open/mount it
cryptsetup luksFormat --hash=sha512 --key-size=512 --cipher=aes-xts-plain64 --use-urandom ${DEVICE}1
cryptsetup luksOpen ${DEVICE}1 ${LUKSNAME}

# We put this UUID into an env var to reuse later
CRYPTUUID=`blkid -o export ${DEVICE}1 | grep -E '^UUID='`

Necessary on Ubuntu: install ZFS utils in live CD session:

apt-get install -y zfsutils-linux
/sbin/modprobe zfs # May not be necessary

Create your new ZFS zpool and datasets. This example will create multiple datasets for the system root directory, /boot, /home, /var, and /var/log.

TARGET=/mnt
ZPOOLNAME=zroot
ZFSROOTBASENAME=${ZPOOLNAME}/ROOT
ZFSROOTDATASET=${ZFSROOTBASENAME}/default

/sbin/zpool create -f \
  -R ${TARGET} \
  -O mountpoint=none \
  -O atime=off \
  -O compression=lz4 \
  -O normalization=formD \
  -o ashift=12 \
  ${ZPOOLNAME} /dev/mapper/${LUKSNAME}

/sbin/zfs create -o canmount=off        ${ZFSROOTBASENAME}
/sbin/zfs create -o mountpoint=/        ${ZFSROOTDATASET}
/sbin/zfs create -o mountpoint=/boot    ${ZPOOLNAME}/boot
/sbin/zfs create -o mountpoint=/home    ${ZPOOLNAME}/home
/sbin/zfs create -o mountpoint=/var     ${ZPOOLNAME}/var
/sbin/zfs create -o mountpoint=/var/log ${ZPOOLNAME}/var/log

/sbin/zpool set bootfs=${ZFSROOTDATASET} ${ZPOOLNAME} # Do not skip this step

/sbin/zpool status -v # print zpool info

Fetch the Void Linux rootfs. Get it from any of the project's list of mirrors. Assuming your architecture is x86_64, fetching the latest Void rootfs at time of writing would look like this:

VOIDMIRROR=https://repo.voidlinux.eu/live/current
wget -N ${VOIDMIRROR}/void-x86_64-ROOTFS-20171007.tar.xz
wget -N ${VOIDMIRROR}/sha256sums.txt
wget -N ${VOIDMIRROR}/sha256sums.txt.sig

Validate the rootfs checksum. You should also fetch and verify its GPG signature, but you probably won't.

sha256sum ./void-x86_64-ROOTFS-20171007.tar.xz

Compare this checksum with the value from sha256sums.txt. If it matches, untar its contents into ${TARGET}:

tar xf ./void-x86_64-ROOTFS-20171007.tar.xz -C ${TARGET}

Create a new ${TARGET}/etc/fstab that matches your ZFS datasets. For example:

cat ~/fstab.new

/dev/mapper/cryptroot /        zfs  defaults,noatime 0 0
zroot/boot            /boot    zfs  defaults,noatime 0 0
zroot/home            /home    zfs  defaults,noatime 0 0
zroot/var             /var     zfs  defaults,noatime 0 0
zroot/var/log         /var/log zfs  defaults,noatime 0 0

chmod 0644 ~/fstab.new
mv ~/fstab.new ${TARGET}/etc/fstab

Create a LUKS key file, add it to the LUKS container, and put its info into a crypttab:

KEYDIR=${TARGET}/boot
KEYFILE=rootkey.bin

# Create key file:
dd if=/dev/urandom of=${KEYDIR}/${KEYFILE} bs=512 count=4
# or, faster:
# openssl rand -out ${KEYDIR}/${KEYFILE} 2048
chmod 0 ${KEYDIR}/${KEYFILE}

cryptsetup luksAddKey ${DEVICE}1 ${KEYDIR}/${KEYFILE} # This prompts for the LUKS container password

ln -sf /dev/mapper/${LUKSNAME} /dev

# Set crypttab:
echo "${LUKSNAME} ${CRYPTUUID} /${KEYFILE} luks" >> ${TARGET}/etc/crypttab

Mount some special mountpoints into the new FS:

for i in /dev /dev/pts /proc /sys
do
  echo -n "mount $i..."
  mount -B $i ${TARGET}$i
  echo 'done!'
done

Copy /etc/resolv.conf into the new system. You need this to resolve network endpoints.

cp -p /etc/resolv.conf ${TARGET}/etc/

chroot into ${TARGET}:

chroot /mnt

Configure the system. There are some post-installation steps documented that you can perform now with regards to setting the hostname, adding users, installing software, et cetera. At a minimum, set the root password and a locale:

passwd
echo "LANG=en_US.UTF-8" > /etc/locale.conf
echo "en_US.UTF-8 UTF-8" >> /etc/default/libc-locales
xbps-reconfigure -f glibc-locales

Update the Void Linux software package repository. You may want to pick a faster mirror first, as I've done here:

# optional:
echo 'repository=http://lug.utdallas.edu/mirror/void/current' > /etc/xbps.d/00-repository-main.conf

Then:

xbps-install -Su

The Void Linux rootfs is tiny, only about 35 MB, and very minimalistic. Install some packages that are not in the base install: a kernel, "cryptsetup" so you unlock your LUKS container, and the "GRUB" and "ZFS" packages so you can boot your system and access your zpool:

xbps-install linux cryptsetup grub zfs

A note about kernels: Void Linux has a number of kernels available. Check your mirrors for all your available options. The default kernel package is "linux", which will give you a modern kernel, but you can also select "linux-lts" which will install an older, presumably more stable, kernel. If neither of these suit you, you can review the linux kernel packages available and install the kernel that best fits you. For instance, Linux kernel version 4.17.1 was released on 2018-06-11 and had a corresponding Void package available within 3 days, so running "xbps-install linux4.xx" for any available value of "xx" is a plausible kernel you can use here. Caveat: not all kernels are created equal and for your ZFS-root Linux machine to work your kernel needs to understand ZFS, which means that a new kernel will need to compile new kernel modules. This can fail, and often does. Be careful about mixing and matching your kernels with your DKMS modules, or you may lose the ability to import your zpools. If you install a specific kernel, make sure to install the matching kernel headers or you will be unable to build your ZFS kernel modules.

Ensure that GRUB can read your ZFS root dataset:

grub-probe /

The output of this command must be "zfs". If it isn't, stop and correct your install.

Edit the dracut settings so your initrd will contain the LUKS key file.

vi /etc/kernel.d/post-install/20-dracut

Make the following change:

- dracut -q --force boot/initramfs-${VERSION}.img ${VERSION}
+ dracut -q --force --hostonly --include /boot/rootkey.bin /rootkey.bin boot/initramfs-${VERSION}.img ${VERSION}

Adjust the "/boot/rootkey.bin" and "/rootkey.bin" values as needed. These should match ${KEYDIR}/${KEYFILE} and the /${KEYFILE} value you put into ${TARGET}/etc/crypttab, respectively.

Build your new initrd. This requires you to know your exact kernel version:

xbps-reconfigure -f linux4.16

Adjust the name of your linux4.xx package accordingly. Your DKMS modules will have been built when you install the "zfs" XBPS package, but the reconfiguration step here will attempt to re-compile them if they aren't already present. Be aware: if your "spl" and "zfs" DKMS builds fail, you will not be able to boot your machine. Stop now and fix your kernel before proceeding.

Edit /etc/default/grub. You will want to edit or add the following three lines:

  • GRUB_CMDLINE_LINUX_DEFAULT # add the "boot=zfs" option
  • GRUB_CMDLINE_LINUX="cryptdevice=${CRYPTUUID}:${LUKSNAME}"
  • GRUB_ENABLE_CRYPTODISK=y

As an example, changes to your /etc/default/grub might look like this:

- GRUB_CMDLINE_LINUX_DEFAULT="loglevel=4 slub_debug=P page_poison=1"
+ GRUB_CMDLINE_LINUX_DEFAULT="loglevel=4 slub_debug=P page_poison=1 boot=zfs"
+ GRUB_CMDLINE_LINUX="cryptdevice=UUID=93a7dbeb-2ae0-48b2-bd00-c806ae9066df:cryptroot"
+ GRUB_ENABLE_CRYPTODISK=y

Install the bootloader.

mkdir -p /boot/grub
grub-mkconfig -o /boot/grub/grub.cfg
grub-install /dev/sda

Exit the chroot.

exit # leave the chroot

Unmount your mountpoints.

for i in sys proc dev/pts dev
do
  umount ${TARGET}/$i
done

Unmount your ZFS mountpoints and change their mount option to "legacy". This is a start-time mounting reliability thing that may or may not be necessary for you, but I've found that some systems that use ZoL have problems with letting ZFS automatically manage mounting their datasets.

/sbin/zfs unmount -a

for dataset in boot home var/log var
do
  /sbin/zfs set mountpoint=legacy ${ZPOOLNAME}/${dataset}
done

/sbin/zpool export -a -f

Reboot.

reboot

There are some scary-looking error messages in the init sequence that I haven't figured out how to fix, but they seem to be benign. The method given here boots a Void Linux system (seemingly) without trouble.

Final thoughts on Void Linux: I've been playing around with getting a LUKS+ZFS-on-root configuration in Void for at least a couple of months without success until recently. The OS itself is a nice example of a Linux distro that isn't a typical Debian/Ubuntu/Red Hat fork. It appears to have been created in 2008 by a (former?) NetBSD developer to showcase the XBPS package management system, which itself appears to be an ideological re-design of pkgsrc. The project lead went missing in January 2018, so Void has had to scramble to obtain in absentia control over their own project. They are, for lack of a better term, forking themselves. Since Void uses a rolling release model and there are no regularly-scheduled release milestones to be blessed by the guy in charge, this doesn't really affect you as an end user, but I thought it was worth mentioning that enough people care about Void to not let one person let it die.

2018-05-01

A "review" of The Carrier (1988)

Your touch is sweet and kind
But there's something on your mind
I can't see your eyes, I can't see your eyes....

My Blu-ray copy of The Carrier came in the mail today. I tell you this because now that I have my copy, you should get yours.

I like good movies, but I loooove bad ones. I thought The Carrier would be one of the latter and discovered it to be closer to the former. I dismissed it on its face as a conventional horror movie: low budget, no-name cast, shot over two weekends with a small town of extras who had a camera, a microphone on a stick, and a lot of "can do" enthusiasm.

Boy, was I wrong. So, so wrong. If you choose to watch The Carrier, you should not make the same mistake I did. It is not your run of the mill late-80s horror film, not by any stretch of the imagination.

The movie starts with a square dance scene and an off-brand Rebel Without a Cause/The Outsiders-lookin' young man crashing the party. A group of locals is casually talking about "the black thing" that has begun appearing on the edge of town and debating whether to fear it or doubt its existence. "Oh," I thought. "This is going to be a by-the-numbers horror movie with a not-so-subtle social message about bigotry in it." I lost interest. I let the movie play and went off into the other room to prepare dinner, occasionally poking my head in to see if the creature feature special effects would be goofy enough to make me chuckle.

Do not do this. Sit down. Watch the film. There's a lot going on in this movie that is easy to miss, even if you think you're paying close attention.

I first saw The Carrier last month, and I haven't stopped thinking about it since. I half-heartedly watched it out of the side of my eye at first and then became so engaged and so haunted by its imagery that I sat down and watched it again the next day. Then I kicked myself for missing so much of it the first time around.

Then I went out and bought it on Blu-ray.

I don't even own a Blu-ray player.

Upon first glance, I considered The Carrier to be a fun, low-budget horror flick, something MST3K-worthy in the same vein as The Bloodwaters of Dr. Z or Boggy Creek (I or II, take your pick), or even The Giant Spider Invasion. Especially The Giant Spider Invasion. All of these films are so-bad-they're-good in the "get drunk with some friends and make fun of the hillbillies" way.

I was wrong. Very, very wrong.

The Carrier is not a perfect film and it certainly contains flaws that plague every low-budget movie. If you give it a chance, you will bear witness to an intelligent story of alienation and fear. Part The Outsiders, part The Crucible, part The Purge, this film is the story of a young man estranged from his community who becomes, very literally, toxic to everyone around him. I dare not say more. Perhaps I've already said too much. Don't spoil this movie for yourself. Just push play and observe.

The film is a delight to watch. It was shot on that grainy sorta-expired film stock from the mid 1980s that makes it look like it could have been filmed in the 1970s. The acting is poor but heartfelt and earnest. The special effects are exemplary for the budget of the movie, and just when you start thinking this is going to be a fun, cheesy "guy in a rubber suit" movie, it veers you down a sharp left turn of paranoia and old-fashioned, home-grown, utterly volatile clan warfare. What?

The Hatfields and the McCoys were a spat over a Scrabble game compared to The Carrier. This movie gets bonkers. Full-on mob mentality, red scare, wrap-yourself-in-plastic-to-stay-pure, murder-your-neighbor-for-his-stuff, end-of-days mania. And it is engrossing and frightening and bizarre and wonderful. Every so often you find a gem like The Carrier, where a humble band of folks just try to make a little cinematic enterprise for funsies and end up with a secret masterpiece of dread and creeping, amorphous, "fear of the unknown" terror. The film underscores the horrifying notion that anything can kill you at any time and you can't know what or how, but you have to do something — anything — to protect yourself and your loved ones at any cost.

And in that mad rush to find your own tentative safety, what are you willing to sacrifice to reach it?

10 out of 10. Make sure your cats are safely upstairs, wrap yourself in a big plastic sheet, and watch The Carrier. This movie knocked my socks off and that was when I was only half paying attention to it. The full experience is shocking, bizarre, and will stay with you for days.

The Blu-ray contains two cuts of the movie, a director's cut and a theatrical cut. The theatrical cut contains a 2-second error at the half-way point that forces the video and audio to be unwatchably out of sync and I still bought it anyway because it's just that good. If you want to see a small town of hillbillies lose their god-damned minds over barn cats, it's on YouTube in its entirety.