Home / Blog / RAID 5 Rebuilds and Parity
Business · blog post

What parity actually is, and why rebuilding a degraded RAID 5 is the most dangerous hour that array will ever have

Almost every RAID 5 that arrives at this bench was working perfectly until somebody did the obvious thing. The array reported a failed member, a replacement went in, the rebuild started, and part-way through it the volume disappeared for good. That sequence is so consistent it is worth explaining why it happens.

Parity is a sum, and that is the whole trick

Take three disks. Write a block of data to the first, a second block to the second, and on the third write what amounts to the running total of the other two. Lose any one of the three and you can work out what it held from the two that remain — the missing block is the total minus the block you still have. That is parity in one paragraph, and RAID 5 is that idea spread across every disk in the set so no single member carries all the totals.

Two consequences fall straight out of it, and neither is intuitive. The first is that a RAID 5 tolerates exactly one missing disk, no more. With one gone the arithmetic still closes; with two gone there are two unknowns and one equation, and no amount of software solves that. The second is that reconstructing the missing disk means reading every single block on every remaining disk. Not the used portion. Not the important folders. All of it, end to end, without a single unreadable sector anywhere in the set.

That second point is the one that ends arrays. A degraded RAID 5 is not a system with a spare wheel on. It is a system where the redundancy has already been spent, and the rebuild is the moment you ask every other disk in the box — disks of the same age, from the same batch, that have done the same hours in the same warm cupboard — to perform flawlessly for several hours in a row. That is the least likely moment in their lives for them to do it.

Why the second disk fails during the rebuild and not before

Hard disks accumulate unreadable sectors quietly. A sector that has gone bad in a region nobody has touched for two years announces nothing, because nothing has asked to read it. In normal service a four-bay array might touch a third of its surface in a month, so a slow decline on the other two thirds is completely invisible. The array reports itself healthy, because by its own lights it is.

A rebuild changes that overnight. Suddenly every block on every surviving member has to be read, including the two thirds nobody has been near since the box was commissioned. Any sector that has quietly gone bad in that time surfaces now, and on a controller that treats a read failure as a disk failure, the member is dropped. That is your second disk gone, and the array is now unrecoverable by its own arithmetic — while the actual quantity of damaged data on that drive might be a few kilobytes.

The volume of reading involved is worth stating in numbers, because it explains why this got worse rather than better as disks grew. Rebuilding a four-bay array of 8TB disks means reading roughly 24TB without a fault. Disks got much larger over the last fifteen years; their published unrecoverable-read-error rates did not improve at the same pace. The arithmetic that made RAID 5 comfortable on 500GB drives is considerably less comfortable on 8TB ones, which is why serious storage people moved to RAID 6 and its second parity block and why plenty of NAS boxes in small offices never got that memo.

What the array does not protect you from

RAID is often bought as a backup and it is not one, and the difference matters more than any argument about levels. Parity protects against one specific event: a disk dying. It does nothing whatsoever about somebody deleting the wrong folder, because the deletion is faithfully written across every member at once. It does nothing about ransomware, for exactly the same reason — every encrypted file is dutifully protected by parity in its encrypted state. It does nothing about a controller writing corruption, a power event during a write, a fire, a theft or a flood in Leicester's less fortunate riverside industrial units.

The other thing worth knowing is what happens in a NAS that was never really an array. A great many two-bay boxes sold to small businesses shipped configured as a stripe rather than a mirror, because a stripe shows the full combined capacity on the box and looks more generous in a shop. A striped pair has no redundancy at all and loses everything if either disk goes. Owners generally discover this on the day it happens. If you have a two-bay unit under a desk and nobody has checked which mode it is in, that is a five-minute job worth doing this week.

For what it is worth, RAID 1 and RAID 10 are far kinder in recovery than RAID 5, because a mirror is a whole readable copy rather than a set of simultaneous equations. Where the budget is there, that is the trade worth making in a small office. RAID recovery covers the levels side by side, and NAS recovery deals with the Synology, QNAP, Netgear and Buffalo boxes specifically.

The first hour, in the right order

When a volume disappears, the instinct is to make it come back. Resist it, because almost every route back writes to the disks. Here is the order that keeps the options open. Stop writing to the array immediately and take the applications off it — a database still trying to commit transactions to a broken volume is actively making the reconstruction harder. Do not start a rebuild. Do not initialise anything the controller offers to initialise. Do not run a file system repair against the volume, and treat CHKDSK with real suspicion here; what CHKDSK actually does is worth two minutes of anybody's time before they let it near an array.

Then photograph the front of the chassis with the disks still in it, so the bay order is recorded in a way nobody can argue about later. Label each disk with its bay number as you take it out — 1, 2, 3, 4, left to right or top to bottom, and write down which convention you used. The order matters enormously, because reconstructing an array means working out the stripe size, the parity rotation and the disk order, and knowing the order for certain removes one unknown from a problem that has several.

Write down what the controller said and when. Which disk was reported failed, on what date, and whether anybody replaced it. Whether a rebuild was started and how far it got. Whether the box has been power-cycled since. Every one of those facts changes how the reconstruction is approached, and reconstructing them from memory three weeks later is unreliable in a way that costs money.

How an array is actually recovered

Nothing is rebuilt on your hardware and nothing is written to your disks. Every member is imaged individually first, on equipment that can set its own timeouts and step past a difficult region rather than hammering at it until the drive gives up. That imaging stage is why arrays are priced by the member count: a four-bay box is four separate imaging jobs before the array work has even begun, and a failing member can take a day and a half on its own.

Once the images exist, the parameters are worked out from the data rather than trusted from the label on the box. Stripe size, block order, parity rotation, which member was stale and by how long. A member that dropped out three weeks before the others is holding an out-of-date version of every block it has, and using it blindly produces a volume that mounts and is quietly full of corruption — which is worse than one that does not mount at all. The reconstruction is then assembled virtually, from the images, and the file system is checked before anybody claims a result.

It is worth saying plainly where this ends badly. Two genuinely dead members with no overlap is the end of the road on a RAID 5, and no laboratory changes that. An array that somebody has already re-initialised, re-created with different parameters or rebuilt onto has had new structures written over the old ones, and how much survives depends entirely on how far that got. Both are far more recoverable on the day it happens than after a fortnight of attempts.

What it costs and how the disks reach the bench

Arrays, NAS boxes, SANs and servers start at £500 + VAT and rise with the member count, quoted in writing after a free diagnostic that closes 2 working days after the disks are booked in. A single drive on its own is £300 + VAT, cards and USB sticks £250 + VAT, a disk out of a CCTV or DVR recorder £400 + VAT. Logical faults run under no fix, no fee. The published exclusions are electronic and mechanical failures, chip level work, DVR and Forensic jobs, and those take 50% of the quoted figure upfront because donor parts get bought for one named drive.

Send the member disks only. Not the chassis, not the controller, not the rails. Label each one with its bay number, put the photograph of the front panel in with them, and pack them so nothing can move in the box. Post it tracked and insured to Cambridge Data Recovery, Compass House, Vision Park, Chivers Way, Cambridge CB24 9AD. From Leicester that is about seventy miles — M1 south to Junction 19 then the A14 east, roughly an hour and a half — and the lab sits two minutes off Junction 32 with parking at the door if you would rather hand them over; reception takes drop-offs Mon–Fri 9:00am–5:30pm. Nobody collects and there is no counter in Leicester.

The pages that go with this one are RAID 5 data recovery for the parity arrays specifically, server data recovery where the box is a server rather than a NAS, VMware and virtual machine recovery where the volume held datastores, and data recovery cost for every band in full. Anything you want to ask before packing, the freephone is 0800 689 0668, Mon–Fri 9:00am–5:30pm.

A degraded array has already spent its safety margin. The rebuild is not the repair — it is the single heaviest read the disks will ever be asked to perform, at the one moment they have no cover left. Copy what you can off it first, or image it before anybody touches the replacement disk.

// what people ask about this one

Common questions

Yes, though not in the way people assume. The urgency is not that the array will collapse this afternoon; it is that you are now running with no redundancy, so the next fault of any kind takes the volume. Copy the important data off to somewhere else before doing anything else at all. That copy is worth more than any decision you make about the replacement disk.
Only after you have a copy of the data somewhere else. Rebuilding forces every remaining member to be read end to end, which is precisely when quietly accumulated bad sectors surface and a second member gets dropped. Where the data is already safe, a rebuild is a reasonable thing to try. Where it is not, a rebuild is a gamble with the only copy you own.
Not automatically. It depends heavily on whether both are genuinely dead or whether one was simply ejected by the controller and is physically fine, which is a very common pattern. A member dropped for a handful of bad sectors can usually be imaged and brought back into the reconstruction. Two mechanically failed disks with no overlap between the damaged regions is a much harder proposition, and you will get an honest answer at the diagnostic rather than an optimistic one.
It makes the job longer rather than impossible. Disk order is one of the parameters that has to be established, and it can be derived from the data itself, but that takes analysis time that a photograph would have saved. If the disks are still in the unit, photograph the front now and label them as you remove them. If they are already out and jumbled, say so plainly — guessing and being wrong is worse than admitting it.
Only if it is mirrored. A great many two-bay boxes ship configured as a stripe, because that shows the full combined capacity, and a stripe has no redundancy whatsoever — either disk failing takes everything. Look in the storage manager for the volume type. If it says RAID 0, JBOD or anything about a single pooled volume across both disks, you have no protection at all and it is worth knowing that today rather than later.
RAID, NAS, SAN and server work starts at £500 + VAT and rises with the number of member disks, fixed in writing after a free diagnostic that closes 2 working days from booking in. Every member is imaged separately before any reconstruction, which is where the time goes. Logical faults run no fix, no fee; electronic and mechanical failures, chip level work, DVR and Forensic jobs are the stated exclusions and take 50% upfront.

Rather put it in front of an engineer?

Diagnosis costs nothing and finishes 2 working days from the moment Cambridge books your device in. The figure after that is written down and fixed. £250 + VAT a card or stick, £300 + VAT one drive or SSD.