You want me to deal with how many VMDKs!?

Recently I was given approximately 55 VMDKs after a cyber incident to try and identify the root cause. Ideally, these systems would be online so that we could deploy our distributed threat hunting and forensic analysis tool of choice, Velociraptor however this wasn’t a possibility in this scenario.

This means I had an interesting problem to solve; how do I get the relevant forensic artefacts that I wanted to so that I could triage the VMDKs and identify which systems that I wanted to dig deeper into.

Two different solutions

My first impression was to use the Arsenal Image Mounter command line interface to mount each image one-by-one and run my command line utilities of choice. Simple enough; write a powershell script that mounts the VMDK, gets the drive letter and identifies whether there’s a Windows operating system there, and then run tools.

Rather than re-inventing the wheel I went to Twitter and asked the community.

Mark Spencer to the rescue! Mark provided the right commands I needed, but alas *user error that was entirely my fault* I couldn’t get it working for my use case. I briefly read through the API documentation but wasn’t able to determine what the error message meant. (I later found out that the error, which said something about providing too many parameters, was just a by-product of me not running as Admin, which I thought I had done).

Promising I’d come back to this, I thanked Mark and went onto my second solution (don’t worry I’ll come back to AIM_CLI).

X-Ways CLI

I’ve always heard that X-Ways had a CLI but I never really used it. With a bit of reading I found out exactly how simple it is to create a new case and add an image (or 55).

I didn’t end up scripting this but the command is simply:

C:\program files\xways\xwforensics64.exe CreateCase:<Case location> AddImage:<Image location> AddImage: <Image location> etc etc (55 times)

Sounds easy enough right? Well, it opened up and loaded in all my images, but unfortunately it would die if I tried to explore recursively across them all (millions of items, so understandable).

The solution here was to (over the course of a few hours) recurse through each image and then flag any partition that contained a Windows partition, and save after each recurse. After that process completed I had a 20GB case, and I could recurse through all of the flagged partitions, which was slow but far more manageable.

Now, after a little waiting, I could export the specific event logs or registry hives I wanted to start my triage! Based on what I already knew from the attack, Shellbags and Remote Desktop events were going to be my friend and I was able to filter 55, down to only a handful and off to the races we go!

But wait, what about AIM?

While X-Ways worked it was still a bit of a process and I wanted to come back to the PowerShell AIM solution. The main reason is that I could kick off my process with some or all of the VMDKs in a list and then run whatever command line utility I wanted against the mounted data.

There’s really not much to the mount script (when someone is kind enough to give you the solution), and then it’s just a matter of adding a bit of looping magic.

The main requirements are to download the API DLLs for here and LibEWF from here (the one on their GitHub was old at the time, so I tried a newer version of LibEWF which is packaged in the latest version of AIM), but that didn’t fix my issue and I’m sure it’s something on my end I need to figure out. It appears that there’s something happening where the disk is getting mounted, but the volumes aren’t coming online and it’s something on the backburner to figure out.

# Function to mount a provided file and run a commandline tool against the mount point
function AIM_Mount {
    param ($filepath,$diffFile)
    Add-Type -Path .\Arsenal.ImageMounter.Devio.dll
    $adapter = [Arsenal.ImageMounter.ScsiAdapter]::new()
    if ($filepath.EndsWith(".e01")){
        [Arsenal.ImageMounter.Devio.Server.Interaction.DevioServiceFactory]::GetService($filepath, [Arsenal.ImageMounter.Devio.Server.Interaction.DevioServiceFactory+VirtualDiskAccess]::ReadOnly,[Arsenal.ImageMounter.Devio.Server.Interaction.DevioServiceFactory+ProxyType]::LibEwf)    
    if ($filepath.EndsWith(".vmdk")){
        $service = [Arsenal.ImageMounter.Devio.Server.Interaction.DevioServiceFactory]::GetService($filepath, [Arsenal.ImageMounter.Devio.Server.Interaction.DevioServiceFactory+VirtualDiskAccess]::ReadOnly,[Arsenal.ImageMounter.Devio.Server.Interaction.DevioServiceFactory+ProxyType]::DiscUtils)
    $service.WriteOverlayImageName = $diffFile
    $flags = [Arsenal.ImageMounter.DeviceFlags]::WriteOverlay -bor [Arsenal.ImageMounter.DeviceFlags]::ReadOnly
        $service.StartServiceThreadAndMount($adapter, $flags)
    catch [Exception] {
    [Arsenal.ImageMounter.IO.NativeFileIO]::EnumerateDiskVolumesMountPoints('\\?\' + $service.GetDiskDeviceName()) | ForEach-Object { 
        if (Test-Path ($_ + '\Windows')){
            $filepath + "| Windows path found at: " + $_ 
            # ---------------------------------- #
            #                                    #
            #                                    #
            # ---------------------------------- #
$rootDirectory = $PSScriptRoot
Set-Location -Path $rootDirectory
$vmdklocations = "D:\"
$vmdkfullpathList = @()
$vmdkfullpathList = Get-ChildItem $vmdklocations -Recurse | where {$_.FullName.EndsWith(".vmdk")} | %{$_.FullName}
Foreach ($vmdkFullPath in $vmdkFullPathList){
# Requires the actual VMDK descriptor file and not the -flat file (which contains all the data)
    if (!$vmdkFullPath.EndsWith("-flat.vmdk")){
        $vmdkName = $(Split-Path $vmdkFullPath -leaf)
        $diffFile = $vmdkName + '.diff'
        AIM_mount -vmdkpath $vmdkFullPath -diffFile $diffFile

Not going to pretend it’s as versatile and clean as it should be; But I can simply change the $vmdklocations variable and it will recursively find all of my vmdks to mount and run a command against. In the above code it doesn’t have anything running in there, but you can add anything you want.

I haven’t really tested the mount function against E01s, and I haven’t written the code to take a mixture of E01 and VMDKs. I imagine it shouldn’t be too difficult to add in when I need to. If you want to run this against E01’s, you’ll have to modify the looping part, but I’d suggest starting by testing out the mount module on its own before looping.

As an FYI, I haven’t gotten the EWF component to work properly for what I want to do, so it’s something I’m still working on. It will mount the drive, but in the images that I am playing with the logical volumes don’t have accessible Windows partitions (even though I know they do, and AIM works fine), so further testing is required.

Onto the investigation

Overall, the investigation has gone well with this process. Everything was pretty much done after the X-Ways case was created, but being able to get triage data out of the X number of images was a worthwhile automation exercise that I’ll be re-using in the future.

Hopefully that helps someone else; I will certainly be using it or a variation of which in the future. Even just to run standard tools because I don’t have to think about the mounting process. Not sure whether it’s quicker than doing it all through the GUI but it’s definitely good for repeated processes. Even cooler, is I can also add in a “create X-ways case” function or any other commandline functions too, so that I can save a few extra minutes at the start of a case.

Edit: Joachim has asked for clarification on the LibEWF usage:
I used the DLL that is provided with AIM GUI. This DLL is relatively recent and works within the AIM GUI. They previously provided a compiled DLL on their GitHub that was old which has since been updated. I tried both the old one and the current one without success.

I am making inquiries as to whether the issue is with the DLL, which I doubt because AIM GUI works fine, or with my inability to fix whatever needs to be fixed in the PowerShell which is much more likely. It could also be something to do with the API not onlining the volumes properly but that’s still to be determined.

In answer to the question of why didnt I use XYZ other technique. Because I didn’t 🙂 I was pushing through the second weekend of work and needed to get a system that I knew I could get to work so I could continue on with the investigation 🙂

One thought on “You want me to deal with how many VMDKs!?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s