Home / VMware Recovery

VMware Recovery VMDK, VMFS and snapshot chains

A virtual machine is a file, that file lives on a datastore and the datastore lives on an array, so a single failure can look like three different disasters at once. This page covers datastores that will not mount, snapshot chains left unconsolidated by a backup job, thin disks, deleted guests and the case nobody enjoys — a virtual machine sitting on a RAID that has failed underneath it.

Virtual machines live on arrays, so the band is from £500 + VAT, rising with the number of member disks, or £300 + VAT where the host ran a single drive. The diagnostic is free and closes two working days after the disks reach the bench.

// virtual machines, and how they go missing

Six ways a VM stops existing

Virtualisation moves the problem rather than removing it. The guest is a file, the file lives on a datastore, and the datastore lives on an array. Any of the three can fail, and the symptom on screen rarely tells you which one did.

The datastore will not mount

The host boots, the storage is visible, and the VMFS volume is either absent or shows as unformatted. The partition table, the volume header or the metadata region has been damaged. The virtual disks themselves are usually entirely intact underneath, which is why the worst thing available here is to let anything reformat or repair the datastore.

A snapshot chain that never consolidated

Snapshots were taken, a backup job left them behind, and the machine has been running on a chain of delta files for months. Break the chain and the guest reverts to a state from last spring, or refuses to power on at all. Every link in that chain matters, and the disk descriptors that join them can be rebuilt when they are lost.

The datastore filled up and everything paused

Thin-provisioned disks and forgotten snapshots grow until the datastore has no room left, at which point the guests stop dead mid-write. Databases and file systems inside them are then inconsistent. Freeing space is only the first half of the job; the guests usually need work inside them as well.

A VM deleted or removed from inventory

Removing from inventory only unregisters it and the files remain. Delete from disk actually unlinks them, and on VMFS the blocks are then free to be reused by anything else running on that datastore. Powering the host down quickly is the single biggest factor in how much comes back.

The array under the datastore failed

Two problems stacked. The RAID has to be reconstructed with the right geometry before the datastore is even readable, and only then can the guest inside it be repaired. Get the stripe order slightly wrong and the virtual disk opens and is full of nonsense, which is exactly why the layout is proved against the data.

The guest boots and its own file system is broken

The virtual disk is fine and what is inside it is not: a Windows guest with a damaged NTFS index, a Linux guest with a corrupt ext4 journal, a database that will not attach. The recovery happens inside the virtual disk, on a copy, in the same way it would on a physical machine.

// the file formats, in plain terms

What a virtual disk actually is

A VMware virtual disk is normally two files. A small text descriptor records the geometry, the disk type and, where snapshots exist, which file is the parent of which. A large flat file holds the contents, and inside that flat file is an ordinary file system — NTFS, ext4, XFS — laid out exactly as it would be on a physical disk. That is the useful part: once the flat file is readable, everything a laboratory knows about recovering a Windows or Linux volume applies to it unchanged.

Descriptors are small and easily lost, and losing one is far less serious than it looks. The geometry can be worked out from the flat file itself and the descriptor rewritten. Losing a link in a snapshot chain is the more serious case, because the delta files hold only the blocks that changed after the snapshot was taken. The chain has to be reassembled in the right order before it means anything, and a chain reassembled in the wrong order will happily produce a bootable machine containing a mixture of two different points in time.

Thin provisioning, thick disks and zeroed blocks

A thick disk reserves its full size at creation. A thin disk grows as it is written to, and it does that in grains tracked by an internal directory. Damage that directory and the mapping between where a block sits in the file and where the guest believes it sits is lost, though it can generally be rebuilt from the file system inside. The practical consequence to understand is different: on a thin disk, blocks that were never written contain nothing at all. A 2TB thin disk holding 300GB of data has 1.7TB of nothing, and no laboratory can conjure files out of space that was never used.

VMFS, and why it is worth being careful with

VMFS is a cluster file system built for large files and multiple hosts writing at once. Its metadata lives in a defined region, it uses large block sizes, and it keeps enough redundancy that a damaged volume header is frequently recoverable. What it does not survive is being written over. A host that is told to reformat the datastore, or a VMFS-6 volume created over a VMFS-5 one during an upgrade that went sideways, writes new metadata across the area that recorded where every virtual disk begins. Some of that is reconstructable by finding the virtual disks directly in the raw blocks; some of it is not.

If a datastore has stopped mounting, the correct action is to stop. Do not let the host repair it, do not create a new datastore on the same disks, and do not run consolidation or expansion in the hope it shakes something loose. Put the host in maintenance, power it down, and let the underlying disks be imaged.

Hyper-V, and the same problem in different clothing

The Microsoft equivalent uses VHD and VHDX files with their own differencing chains and their own checkpoint mechanism, sitting on NTFS or on a cluster shared volume. The names change and the failure modes do not: a chain missing a link, a dynamically expanding disk whose block allocation table is damaged, a checkpoint merged badly, a guest whose own file system is inconsistent. Those jobs run the same way and sit in the same price band.

// what survives, what does not, and what it costs

The honest list

Recovered routinely: a datastore whose metadata is damaged but whose blocks are intact; a virtual disk missing its descriptor; a snapshot chain that can be put back into the correct order; a guest whose own file system has been corrupted; a VM deleted from a datastore that was powered down soon afterwards; virtual disks sitting on an array that failed and can be reconstructed.

Recovered partially, and honestly quoted as such: a deleted VM on a busy datastore that carried on running for days, because free blocks get reused by whatever else is writing; a snapshot chain missing a delta file, which yields the state at the last intact link rather than the state at the crash; a guest that was mid-transaction when the datastore filled, where the files come back but the database inside needs its own repair.

Not recoverable, and there is no point pretending otherwise: a datastore that has been fully reformatted and written to since; thin-disk regions that were never written; blocks overwritten by other guests continuing to run after a deletion; and a snapshot deleted deliberately whose changes were never merged anywhere. Where that is what has happened, you will be told at the diagnostic stage, before any money changes hands.

How the job runs

The underlying disks are imaged first, individually, exactly as they would be for any array job. The array is then reconstructed on copies and the parameters proved by checking that the VMFS structures land where the geometry predicts. Once the datastore is readable, the virtual disks are extracted, snapshot chains are reassembled in order, and each guest is opened as a file system in its own right. Anything wrong inside the guest is repaired at that point, again on copies.

Deliverables are usually one of two things: the virtual disk files, ready to be registered on a rebuilt host, or the contents of the guest as ordinary folders and files. Say which you want early, because it changes how the last stage is done, and there is no harm in asking for both.

What to send from Leicester

The datastore disks, bare, each labelled with its bay position, with a photograph of the front of the chassis taken beforehand. Not the host, not the controller, not the enclosure. Include a note of the ESXi version, the array level if you know it, how many guests were on the datastore and which one matters most — priorities genuinely change the order of work on a job this size.

Post it tracked and insured, or drive it to the Cambridge bench and hand it over, Mon–Fri 9:00am–5:30pm. It is about 70 miles from Leicester, the M1 to J19 and then the A14 east, roughly ninety minutes, with parking outside the door two minutes from Junction 32. Nobody collects, and there is no counter in Leicester.

The price

Virtual machines nearly always live on arrays, so the band is the array one: from £500 + VAT, rising with the number of member disks. Where the host ran a single disk rather than an array, it is £300 + VAT. The diagnostic is free and closes two working days after the disks are booked in, and the figure that follows is fixed in writing. Logical work is covered by no fix, no fee; electronic and mechanical failures, chip-level work, DVR jobs and forensic jobs sit outside it, and physical work takes half upfront.

// related pages

Nearby on this site

// questions about virtual machine recovery

Common questions

No. Put the host into maintenance mode and power it down. Repair options rewrite the metadata that records where each virtual disk starts, and that metadata is usually the only thing wrong. The virtual disks underneath are generally intact, and they stay that way as long as nothing new is written to the volume.
Often, and speed decides how much. Removing from inventory leaves the files alone; deleting from disk frees the blocks, and other guests running on that datastore will reuse them. Shut the host down as soon as you realise. A datastore powered off within the hour usually gives everything back, one left running for a week rarely does.
It depends which link is missing. Delta files hold only the changes made after each snapshot, so a chain reassembled in the right order gives the machine as it was at the crash. A missing delta gives you the state at the last intact link instead, which may be weeks old. Descriptors can be rebuilt; missing data cannot.
No, it is two jobs in sequence. The RAID is reconstructed first, with the geometry proved against the data rather than trusted from the controller, and only then is the datastore readable. Getting the stripe order wrong produces a virtual disk full of rubbish, which is why the parameters are verified before anything is extracted.
It sits in the array band, so from £500 + VAT rising with the number of member disks, or £300 + VAT if the host ran a single disk. The diagnostic costs nothing and closes two working days after the disks are booked in. The quotation is fixed in writing before any work begins.

Do not let the host repair the datastore.

The virtual disks are usually intact under a damaged VMFS header. A repair pass writes over the one thing that still knows where they start.

// how the media reaches Cambridge

Sending a device — and the three exceptions

Multi-disk jobs travel as a set of bare drives in one box, and they travel by post far more often than by car. Tracked and insured is the sensible choice for a parcel holding four disks of company data, and something handed over in Leicester before the last collection is normally logged in at Cambridge the next working day.

The general rule is the drive travels and the machine stays behind — out of the laptop, out of the tower, out of the iMac, out of the recorder under the counter. This bench does not dismantle equipment, and a repair shop will do it while you wait. Three things are the other way round, and getting them wrong costs you the recovery: an external drive stays sealed in its own case, a NAS comes as a complete unit, and a WD My Passport or My Book travels whole with its cable, because on those the encryption key is held on the bridge board rather than on the disk — separate the two and the data becomes unreadable even to us. A Fusion Mac needs both of its drives, each labelled. The one thing nobody can work round is flash soldered onto the mainboard, as on Apple Silicon machines: if it will not come off, there is nothing to post.

  • A stiff box or a well-padded mailer, with enough packing that nothing moves when you shake it. Power supplies, docks and cables can stay at home unless the drive is one of the WD units above.
  • Running a RAID or a server? Send the member disks on their own, not the chassis or the controller, and write the bay order on each one — 1, 2, 3 and so on. Photograph the front of the unit before you pull anything, because that photograph occasionally saves a day of work.
  • Fill in the shipping and booking-in form (PDF) — a name, a number you actually answer, and a line on how the trouble started — and put it in the box.
  • Special Delivery is tracked and insured and is what most people use; your own courier is equally fine. Handing it over in person also works: reception at the Cambridge address takes devices across the counter, Mon–Fri 9:00am–5:30pm. What does not exist is a Leicester counter or anyone who comes to collect.
// write this on the label

Cambridge Data Recovery

Compass House
Vision Park, Chivers Way
Cambridge, CB24 9AD

↓ Print the shipping & booking-in form (PDF)

Address it to Cambridge Data Recovery. It is about seventy miles from Leicester if you fancy driving it — M1 south to Junction 19, then the A14 east — and the lab is two minutes off Junction 32 with parking at the door. Posting costs you a stamp and a day instead. Whichever you choose, you hear from us the moment it is booked in, and the free diagnostic closes two working days after that.

Not certain what belongs in the box? Ring 0800 689 0668 before you tape it up, or let the free online diagnostic ask the questions for you.