[clue-talk] BAARF - Battle Against Any Raid Five (er, Four, er Free.)

[Nate Duehr]

On Oct 21, 2007, at 6:51 PM, Dan Poler wrote:

> Generally, if a RAID5 solution is properly engineered, the write
> performance hit is at least somewhat negated by a large about of fast
> cache in front of the disk. These days that's usually in the range of
> gigabytes when dealing with expensive FibreChannel storage (EMC,  
> NetApp,
> Hitachi, 3Par, et al); I've seen as high as 128 GB of battery-backed
> DRAM in front of the spindles in some solutions -- the host sees the
> write as committed when it gets to the DRAM, and the array worries  
> about
> getting it to the spindles; the battery comes into play in case the
> power goes out before DRAM is emptied. There are expensive RAID5
> solutions on the market with performance within 5-10% the speed of
> writing natively to the spindle.

I was careful to state that RAID 5 *might* be a poor choice for those  
of us implementing RAID via md in software and that the interesting  
thing is that hardware engineering HAD to be "brought to bear" a few  
years ago when engineers and sysadmins got lazy and RAID 5'ed  
*everything* without really thinking about it.

If you have the $ to buy the gear above, you've got a real production  
system and a budget, and you're in great shape.

If you're doing a small system on a budget -- big disks and LVM + md  
RAID 1 might be more appropriate than RAID 5.  Especially if you  
can't afford multiple disk controllers.

> And then there's the RAID5 hardware controllers built into PC's within
> the means of folks like us. IMO those are generally god-awful, and
> probably why RAID5 has earned such a bad name. In many cases  
> Linux's md
> software RAID facility is actually -faster- than the hardware RAID.

Agreed.  md is very good.  (And I used to bash it, but it's  
definitely gotten better over time, I finally got some time to play  
with it and test it again, and got the bad taste out of my mouth from  
previous experiences with it.)

> Bottom line is, don't assume RAID5 is evil; depends on the need and  
> the
> hardware and software driving it. If you need to deploy a storage
> solution, spend some time playing around with tools like iozone to see
> what makes the most sense for your needs -- configure a couple ways  
> and
> do performance testing that mimics the block sizes your application  
> (or
> filesystem) will be using.

Yep, but many small server setups don't have time to do that level of  
testing, and should know that RAID 5, especially done only in  
software, causes a built-in performance hit for writing.  If the  
application does more writing than reading, it *might* not be the  
best option.  Very few people building small business servers realize  
this.

At least this part came true... LOL.

>> I'm sure someone will feel strongly about this, now that I've thrown
>> out all this drek on the topic.

LVM + md is a pretty nice option for those folks "on a budget", but  
RAID 5 done only in software can *really* slow things down... that's  
all.

As to your point about cost, Dan -- I can see where RAID 01/10 would  
definitely be expensive, but with the size of even fast disk getting  
bigger and bigger... the cost just isn't that high anymore -- for  
say, four huge disks for a striped/mirrored setup.  Even if you had  
to buy slower disks than the smaller ones you might put in a  
"typical" RAID 5 array, you might find the difference in write speed  
to a striped/mirrored setup to be enough better than the bigger but  
slower (read: Cheap!) disks could be used.

Just thoughts.  It doesn't seem like you read the articles or my  
comments.  I'm not anti-RAID 5, I'm just fascinated that it seems to  
get "overused".

In the above 4 disk setup, there are also failure scenarios where two  
physical drives have failed, but it doesn't kill the array.  In a 4  
disk RAID 5 setup, two dead physical disks means a dead array.  RAID  
5 DP... that's a LOT of writing... useful but a lot of I/O for this  
type of a "small server" setup.

--
Nate Duehr
nate at natetech.com