Xserve RAID - Multiple drive failure.

Morning all,
An Xserve RAID I have been called to take a look at is behaving rather erratically! Here's the chain of events over the last 3 days:
(4 x 180GB - RAID 5)
- 2 folders (the largest in size) suddenly appear empty. The capacity and available space do not change.
- Some remaining files and folders appear corrupted / incomplete e.g. Quickbooks company file.
- Data Rescue appear to recover all data, missing files are recoverd as "orphaned" files.
- DRIVE 1 FAILS showing red light and disappearing from RAID admin.
- RAID set shows "degraded" but still accessible due to RAID 5 config.
- Replace drive 1 with 750GB Apple supplied drive.
- RAID 5 set rebuilds automatically showing green and "online"
- missing files in the 2 large folders still appears missing.
- DRIVE 2 FAILS
I believe the chances of 2 drives failing in such short succession are extremely small and I now thing that drive 1 is probably ok and that something else is at fault here.
Xserve RAID is on the latest FW 1.5.1 and no other errors are present.
Can anyone shed any light on this, could it possibly be the controller ?

The act of the rebuild could have been enough to push the second drive over the edge.
I would say... if the Xserve RAID has been running 24x7 since you purchased it, you may want to consider replacing it. The 180 GB drives were discontinued almost 5 years ago from what I recall... 5 years of constant use is really a long, long time for had drives. It's more typical in data center usage to replace drives after about 3 years of use (even if their MTBF is 500,000+ hours, those numbers are rather meaningless). Hard drives fail on a "bathtub curve," where once you get to the far end of it the odds of failure are great, and lots will fail around the same time. You can't rely on the Xserve RAID to run 24x7 for 10 years, or you'll definitely have problems. There is no RAID on the planet you can expect to run for 10 years without issues with the drives... even if you're using $1500 apiece fiber or SAS drives.

Similar Messages

Hit 5 Transfer Limit for purchased music.Multiple drive failures. Wht Now?

I've had multiple drive failures on my G=4 and I've reached
the last allowable transfer of purchased music from my
Ipod back to the new drive.
****.... Why doesn't it just recognize the computer otherwise
so that my re=loads aren't used up. Only 5 are allowed.
So now....What happens if and when this drive fails? Do I then
lose all future use of that very expensive, purchased music?
Makes me want to NEVER download from the Itunes store again,\
if this is indeed the case.
What can I do?

There is no limit on "transfer of purchased music from my Ipod back to the new drive." You can store the iTunes Store purchased music as many times as your want on as many different hard drives, and play them all, as long as the computer doing the playing is authorized with your iTunes account (the one used to purchase the songs)
There is a "five" limit, and it is based on machine. You can play music purchased with a specific iTunes Store account on up to FIVE different computers at any given time. Each one is registered with the account, and when you reach five, you cannot add any more. The machine itself is identified (by something on the motherboard) so a new hard drive should not make the machine a "new" machine. So not sure what's happening if all you did was replace the hard drive...
In any event, you can clear your iTunes Store account of all authorized machines and get a fresh start. Here's how (see section +To deauthorize all computers associated with your account+ in the linked article).
http://support.apple.com/kb/HT1420

Xserve raid (14 drive) / several problems

Hi, I have bought a used xserve RAID (the one with 14 drives). It's using 1.5.1 firmware. I have equipped the RAID with 14 WD5000-AAKB-00H8A0 500GB drives.
I first installed 4 drives (slot 1-4), created a RAID array and later on extended the RAID with 10 drives. But those 10 drives never worked without problems.
The problems we have are:
- drive-modules working in slots 8-14 are not working in slot 7
- some drive-modules don't start-up at all. Only red led showing in the front panel.
- The lower controller slot sometimes makes problems, even when swapping upper & lower controller.
Has anybody good experience with this RAID system and can give me some tips what to do?

This is all a bit odd.
You say you have extended the RAID with 10 drives. Bear in mind that you can't create a proper hardware RAID volume across the two controllers/sides of the RAID. You can make arrays over 1-7 then over 8-14.
Did the bad drives ever work? If they are new, have you been in to RAID Admin and made them available for use? Under RAID Admin > Components, click the drives radio button. Do the drives show up here as installed?

Building RAID, hard drive failure?

I just reinstalled Arch to go with disk encryption and to setup my spare drives (4 2TiB HDDs) as one RAID5. Now, I started the process with
mdadm --create /dev/md0 --level=5 --raid-devices=4 /dev/sd{b,c,d,e}1
After about 11 hours, it was going at half the speed it started out at, and then eventually it failed saying that sdb1 was bad. I've now got this in /proc/mdstat:
Personalities : [raid6] [raid5] [raid4]
md127 : inactive sdd1[2] sdc1[1] sde1[4]
5860532931 blocks super 1.2
unused devices: <none>
So, two questions:
1) I've checked the drive with hdsentinel, and it says it is fine -- in fact, it get's a 100% in health, while two other drives in the array have less than that. Should I check it with something else as well? Should I just try to add it to the array? Should I get it replaced, just in case?
2) Whenever I have a drive I want to add to this array to complete it (I wanted RAID5 so I would need 4 2TiB drives to get 6TiB of space), how do I add it? I assume the process will be the same no matter if it's the same drive, just trying again, or a brand new one. I couldn't find anything on our wiki about it, and the information I'm finding about failures isn't really that understandable to me. Could someone point me to a good resource for this?
Thanks!

1) Run 'smartctl -t long' on all the disks, ideally when they're not in use/not mounted. It will tell you when it should be done, and you can also check 'smartctl -c' for percentage of completion. But it will take a while on large drives. Afterwards have a look at 'smartctl -A ... | grep Sector' and 'smartctl -l selftest' - there should be zero or close to zero reallocated/pending sectors and the self-test should be "completed without errors".
2) man mdadm

Multiple SMART Failures on Drive Slot #11

(Using firmware 1.5.1/1.5.1c.)
I've attempted to build a RAID 5 on the lower controller twice. Each time the drive in slot # 11 returned SMART failure. Each attempt had a different drive. Set it to RAID Now when I built it. There is a hot spare spare, but attempting to copy/delete files from RAID with OK status has failed also. The build must have failed because it stopped building the array 4 hours later. Typical build times are at least 10 hours long.
Something is up with slot # 11.
RAID Admin reported -
Disk 11 Reported and Error: COMMAND 0X35 ERROR:0x10STATUS:0x51 LBA:0x2F3EF80
Suggestions appreciated.

Are these Apple Drive Modules you're replacing or are you just swapping a new drive into the carrier itself?
It's pretty common to have 3rd party drives fail to work properly. The Xserve RAID uses drives with a firmware specifically tuned and optimized for the Xserve RAID, so few 3rd party drives have the necessary settings.

Xserve Raid Not Showing all Drives Empty - No Green Lights on Xserve Raid Drives

I have an Xserve Raid. I recently set it up with my Xserve server with fiber cables and now when the Xserve Raid system boots and starts all the drives show as empty in the Raid monitor. Not sure if I have something configured wrong and I think I shut it down incorrectly before I got it hooked up right. When I first got it I was able to see the lights on all the drives but now it boots but no drives show up at all. I have fiber cables to the host Xserve machine and the Xserve Raid plugged into the network with only 1x ethernet cables (not 2 if that makes a difference). When I try to manage the Xserve Raid all drives show as empty even though all the 500GB drives are plugged in. Not seeing any green lights when the Xserve raid boots indicates something is up. I reset the controller cards to stock with the paper clip to factory settings. Does anyone have any ideas on what to try. Anyone have this issue? I haven't found any discussions except one that had a single side of drives go down because of a bad controller card. My Xserve Raid shows no drives at all. Any help is appreciated.

Expansion is a 2 step process: First, you expand the array. Second, you merge the slices. The documentation is clear on step #2, but the GUI doesn't FORCE you to do it. However, it is necessary.
If you did both steps, try rebooting your host and see if it picks up the changed size. It does work... the usual issue though is growing the file system to recognize the additional size. On Linux, I can't help you.

What to do with our Xserve RAID, moving forward - Need Advice.

We have an 3.5 TB Xserve RAID (14 drives - 250 GB each, split into two 7-drive volumes). As many others have, we've moved into more HD and tapeless workflow. The Xserver RAID was purchased really for one HD project with DVCPRO HD back in 2004 and hasn't really been used since except for backup storage.
I'd really like to move us into a SAN system, but am curious about others' perspectives on populating our Xserve RAID with 1 TB or greater drives now or get a different RAID setup. We've moved to Mac Pros, but the Xserve RAID is still dedicated to just one G5. I am doing some reading on setups such as the EVO and CalDigit, but haven't setup a SAN. It's time though.
We have three post machines - two editorial and one sound station. I could really use some insight on how to use our setup in a SAN scenario or what's the cheapest way we can get into a SAN workflow. Granted, drives are cheap these days, but it's so inefficient copying data between two machines to make sure we have a copy of the same media.
Thanks for any input folks.

+I'd really like to move us into a SAN system, but am curious about others' perspectives on populating our Xserve RAID with 1 TB or greater drives now or get a different RAID setup.+
I'll start with the fact that 750GB are the absolute max you'll be able to put into the Xserve RAID. Compatible 750 PATAs are really hard to find (and are expensive when you can find them) so that avenue makes little sense. I think you should really be thinking in terms of new hardware at this point. I know several people who run XSAN and are much happier with the Promise RAIDs they have now over the older Xserve RAIDs.
Which hardware and SAN software you should be looking at I'll leave up to someone with more modern experience in the video editing realm- I've been out of it for too many years now.
My $.02,
=Tod

Raided xserve raid boot volume, OK to run disk utility?

I have been asked to assist another site in our organization, and they have an older g5 xserve and xserve raid. They have the xserve booting off a raided boot volume on the xserve raid (2 drives in a raid 1 config)
They have started having some issues, so I thought I would check the obvious stuff first (i.e. disk util > repair drives)....Just wanted to check here first and make sure that I can SAFELY run disk util on a raided boot volume that is hosted on the xserve raid...
also, does anyone know if diskwarrior will also work in this config? (again, safety being an issue)
Thanks in advance...

Yes, you can run Disk Utility to check the disk, or Disk Warrior or TechTool Pro and any other tool that checks the HD.
That is the point of RAID, it changes nothing of what an OS understands of a HD. If there are any problems and the drive/data is somehow 'lost' it is not due to RAID.
Safety would expect that you ahve a backup of this drive already, so running a disk check would only enhance and not reduce your safety by confirming reliability.
Peter

How to connect multiple Xserve Raid for Best Performance

I like to get an idea how to connect multiple Xserve Raid to get the best performance for FCP to do multiple stream HD.

Again, for storage (and retrieval), FireWire 400 should be fast enough. If you are encoding video directly to the external drive, then FireWire 800 would probably be beneficial. But as long as the processing of the video is taking place on the fast internal SATA drive, and then you are storing files on the external drive, FireWire 400 should be fine.
Instead of speculating about whether it will work well or not, you need to set it up and try your typical work flow. That is the only way you will know for sure if performance is acceptable or not.
For Time Machine, you should use a single 1.5TB drive. It is likely that by the time your backup needs comes close to exceeding that space, you will be able to buy a 3TB (or larger) single drive for the same cost. Also, I would not trust a RAID where the interaction between the two drives is through two USB cables and a hub. If your primary storage drive fails, you need your backup to be something that is simple and reliable.
Oh, and there should be no problem with the adapter, if you already have it and it works.
Edit: If those two external drives came formatted for Windows, make sure you have use Disk Utility Partition tab to repartition and reformat the drive. When you select the drive in the Disk Utility sidebar, at the bottom of the screen +Partition Map Scheme+ should say *GUID Partition Table*. When you select the volume under the drive in the sidebar, Format should say *Mac OS Extended (Journaled)*.

Proper procedure for replacing drive in Xserve RAID RAID5 set

I've got a five-drive RAID-5 set (with a sixth hot spare) in an Xserve RAID running the 1.5/1.50f firmware. One of the drives in the RAID-5 set has an amber/orange status light on and has been getting occasional errors like to following:
Timestamp: 11/10/10 10:34:53 AM
Priority: Warning
Controller: Upper Controller
Type: 112
Event ID: 1000
Event: Disk 5 Reported An Error. COMMAND:0x35 ERROR:0x10 STATUS:0x51 LBA:0x19B80
Description: The drive reported an ATA error. This is a failure in the communication from the RAID Controller to the drive.
I have double checked the drives in RAID Admin and, as the drive is only in a warning state, the hot spare has not been pulled into the RAID set yet. As this is an old drive, I'd like to replace that particular drive first. I have a current, full backup of the data, but want to make sure I understand the process correctly.
I understand the "Installing or Replacing an Apple Drive Module" section of http://manuals.info.apple.com/en/XserveRAID_UserGuide.PDF, but it and RAID Admin's built-in help don't describe what will happen when replacing a drive in a RAID set that has a hot spare. When I pull out the drive and replace it, will it correctly use the newly inserted drive or will it use the hot spare? If it uses the hot spare, will the hot spare revert back to a hot spare once the new drive is inserted or will it permanently become a member of the RAID set and need to be moved to the original drive's slot? Or, should I just pull out the hot spare, pull out the failing drive, and pop the hot spare into the failing drive's slot?

Hello, makkintosshu, and welcome to the AppleBoards,
If you pull out the drive the RAID should/will immediately start rebuilding using the hot spare. The hot spare will become a new permanent member of the RAID and the new replacement drive will become the new hot spare. The physical slot locations of the drives don't matter you can build a RAID from any combination of drives as long as they are on the same side.
If you pull the hot spare and then the failing drive the RAID will wait for a new drive before taking action. I find it hard to recommend this course of action unless there is a really good reason for you not wanting the hot spare to become part of the RAID. Rebuilding is going to take a good long while and you want it to start as soon as possible - as long as the RAID is not rebuilt your data is at risk. Letting the RAID rebuild hang as you physically swap out the failed the drive strikes me as bad idea that needs a really good justification.
HTH,
=Tod

Xserve SSD OS drive Raid 5 array - What if SSD OS drive is corrupt or fails

Xserve SSD OS Drive option with Raid 5 array - What if SSD OS drive fails or the OS becomes corrupted? Could you just re install the os and then upload all of your server settings and it would be able to talk to the raid array and be good to go?
Additionally if the os is in the raid 5 and becomes corrupted and you are not able to get the os to boot would the raid 5 data not be able to be accessed? Or could you just reinstall the os on the array and be ok?
This would be the same if the os is inside a software mirror set?
Thanks in advance!!!

Hi
If you're asking can OSX Server be reinstalled without a reformat the answer would be no. This assumes you're using the OSX Server Installer DVD. Assuming no other hardware issues you should be able to access any/all data on any of the Drives using Target Disk Mode and another mac. This will at least allow a transfer of important data to another source. Having said that you should be keeping back-ups anyway.
You can export Server Admin's configuration as a series of property lists. You can export Users, Groups and Computer Lists from WorkGroup Manager. You can archive the whole of the LDAP database - which will contain everything except home folder information - using Server Admin.
I'm not sure if this is still the case but one of Apple's Best Practice tips was to setup and configure the Server exactly as you wanted it first. Once you're happy use any one of probably half a dozen methods to make a Bootable cloned backup. A free built-in method would be to boot from an appropriate Installer Disk and save the Server OS as a .dmg to a connected USB or Firewire disk. Or use something like CarbonCopyCloner to do regular scheduled clones to an externally attached drive.
My 2p.
Tony

How to restore a software raid mirror after a drive failure

i set up a software raid mirror with two hard drives in a mac pro. then one failed as reported by disk utility. i replaced the drive. it does not seem possible to restore this raid short of copying the files to a third location and then erasing and establishing a new raid. is there a way to simply "restore"?

Question: Do I need special software to administer the Mac Pro RAID Card or the Xserve RAID Card?
Answer: Normal administration can be carried out using the RAID Utility (found in /Application/Utilities) or by using the raidutil command. For more information refer to the User’s Guide or man raidutil.
The command-line utility should be available in Single-User mode.
To run RAID Utility, you may need to boot to an alternate source of Mac OS to be able to manipulate the Boot drive.
This article suggests using the Make Spare command:
RAID Utility 1.0 Help > If a Disk Fails
Message was edited by: Grant Bennet-Alder

Which 750GB drives used in an XServe RAID?

Could somebody with an XServe RAID with 750GB drives please tell me, which drives are used in this configuration? You can look it up in the XServe RAID admin utility, under Arrays and Drives when selecting a drive in "drive" mode.
Thanks a lot,
Floh

ST3750640 NA P
Revision 3.BTF
it is a seagate

Where to buy a xserve RAID drive module in the UK?

Hi does anyone know anywhere that still sells xserve RAID drive modules for a 2005 RAID
we have 400 GB drives but can use a 500 GB because we have the latest firmware.
If modules are unavailable can I take the failed drive out of the module and replace with a new one.
The current drive is a Hitachi Deskstar HDS724040KLAT80

Just sold our complete xsan setup (42 400Gb modules included) for 2500 euro... (so 60 euro per drive) and that included 4 XServes as well

Finding replacement xServe RAID Drive Modules for sale

I've just started a new IT job and inherited an xServe (snow leopard) and xServe RAID
I'm not an expert on either. I've some experience with OS X Server, but I've never been a full on server admin.
One drive is dead. I believe they're 500GB units in the first six bays.
I also believe this is the most recent xServe RAID hardware. Purchased in 2008 or so.
So, first, where on earth would I buy a replacement?
And assuming that's possible, what are people's suggestions for buying drive modules for the other bays, and replacement parts for the xServe itself (power suppies, etc.).
Thanks!

first of all i don't have experience with proprietary Apple products, but typically i believe you can buy any hard drive so long as the interface is compatible (SATA or SAS for example) and the size is the same. there should not be a need to buy 'special Apple xserve hard drives'. if you buy a drive larger than 500 GB then the extra space beyond 500 GB will just be ignored depending on the type of RAID you have setup. definitely do not buy a disk smaller than 500 GB. pull out the disk that is bad and see what the make/model is. my xserve uses Seagate drives so you could probably just search eBay or whatever for the model number of the drive and use that. the hard drive should be able to be removed from the caddy so you don't really need a whole new "module".
as for power supplies those are proprietary and will need to be purchased thru apple or somewhere else like eBay, just search for the part number...

Xserve RAID - Multiple drive failure.

Similar Messages

Maybe you are looking for