Best way of making redundant AFP/SMB server

Question

We have bought two XServe G5 systems in the hope of setting up a load balanced pair that would provide redundant access to AFP/SMB file shares (plus web and email). We have an XServe RAID box which we where going to attach to both servers.
What have people done to get the redundancy we want. These are the options I have looked at:
1) Put XSan on the RAID and connect both servers to it and then reshare the connection. Reading the XSan I see that XSan is not good for re-sharing via AFP/SMB. Also, it seems that I would have to setup everything on each machine seperately as the settings are stored locally.
2) Split the jobs between the two servers say email/web on one and AFP/SMB on the other. Then setup IP Failover for each server to the other, so that if one server fails the other takes over it's jobs. I presume I cannot simply use a single volume for this as mounting the same volume on each of the servers will cause problems. Is that correct?
3) Put all the services on one machine and setup IP failover to fall back to the second machine as required. This removes the issue of two machines accessing the RAID array at the same time.
Any advice would be greatly accepted.
Thanks
Ian
iMac CoreDuo   Mac OS X (10.4.6)  

Answer 1

This may help:
About IP Failover
First off, the IP Failover feature won't magically
configure the backup server to do what the primary
one was doing. In other words, with one server
configured for file services and the other for web
and email services, if the file server failed, the
other one wouldn't just start up file services by
itself!
IP Failover is best used where one server is
configured exactly as another. One server is used,
and the backup server is not used until the first one
fails. The backup server has a separate IP address
and network connection that it uses to periodically
poll the main server. If the main server leaves the
network, then the backup server enables a second
Ethernet interface which has been configured with the
main server's IP address. (You may be able to
multi-home one Ethernet connection, but I haven't
tried that. Plus, each of your Xserves has two
Ethernet ports, so that shouldn't be a problem.)
Yes, but you are not getting good value for money out of that solution.
Looking at the IP Failover it seems to work by having the secondary server monitor the primary. Once it fails it activates that address on the secondary and then enabling the required services on there. This doesn't seem to preclude each server being the master for the others secondary, as long as the services on each master are different.
Lets say mail runs on server 1 and AFP runs on server 2. Server 2 acts as the secondary for server 1 and has a script that activates mail on server 2 when it detects failure. Similarly if server 2 fails it will migrate the ip address server 1 and then startup the AFP service. Obviously each will have to be configured with matching services on each server.
Using the Xserve RAID
If you plan on storing all of your data on the Xserve
RAID, you need to know how "full" the RAID unit is
going to be and how many RAID volumes you wish to
create so that you can decide on how to connect it to
each server.
As we're migrating from an existing server to these we have a very good idea of what the sizes and volumes should be. We have a Xserve raid with four disks all on one side. Setup as a Raid 5 array that is suffient space for all our services. We can either partition that down into multiple volumes for each server/service.
What I don't have my mind around yet is how you control which server sees which volumes. We have to have each server be able to access each volume (so it can run that service in the event of failure) but I think it's going to be an issue to have two servers access the same volume at once. Is that correct? If two servers access the same volume at the same time it's going to cause trouble?
Think of the Xserve RAID as two
devices: each one has a fibre-channel port and holds
seven hard drives. Each "group of seven" can be
connected to any other device on the fibre-channel
network. Without a fibre-channel switch, you can
connect one or two other devices (one or two other
Xserves) to the Xserve RAID.
So you have these scenarios:
Using 1-7 drives and creating one large RAID volume:
you'll use one of two RAID controllers on your
Xserve RAID. You can connect the RAID to only one
Xserve or to a fibre-channel switch in order to
connect it to more devices.
Using 2-7 drives and creating two RAID volumes:
you'll use both RAID controllers (say, 3 on one side
and 3 on the other). Each RAID controller connects
to another fibre-channel device; thus, without a
switch, you can connect one to each server or both
to the same one.
Using 8 to 14 drives and creating one large RAID
volume: you'll be filling up one RAID controller and
all or part of the second, connecting both
fibre-channel connectors to the same Xserve, unless
you use a switch.
Using 8 to 14 drives and creating two RAID volumes:
You have the same options as with 2-7 drives.
See http://www.apple.com/xserve/raid/deployment.html
for some pictures, which may help.
We have the fibre switch so each server can access all drives. I just don't know to limit what sees what, or if I need to do that.
Xsan
If you're just connecting the Xserve RAID to one or
two servers and planning to reshare the volume(s)
created on the Xserve RAID, then Xsan is not
necessary, nor should it be used.
But is it possible for two servers to use the same disk, I know it is with XSan but what about without it.
I think what you want is the ability to use the
Xserve RAID as a locally-connected volume on each
server, where share points can be defined. The
server is connected to the Xserve RAID using a
fibre-channel cable, just as a FireWire, USB, or
eSATA hard drive would be connected to the server.
As far as the server is concerned, the Xserve RAID
controller or controllers represent external hard
disks. The difference is that the Xserve RAID also
has Ethernet connectivity so that you can manage it
without having to log into your server. Once
connected, addressing using fibre-channel is
automatic; the Xserve RAID gets a WWPN address and
the RAID volume appears on the desktop of the server
- no Xsan required at all. From there, it's
perfectly safe to create share points and share them
via AFP or SMB; clients would connect using AFP or
SMB protocols with IP addressing over an Ethernet
network to which both the Xserve and the client
computers are connected. Even though the Xserve
RAID may also have an Ethernet connection to that
network, no read and write commands are sent via
Ethernet to the RAID; the server sends those via its
fibre-channel connection to the Xserve RAID as it
would with any other drive when housing share
points.
As I say it's fine for one machine at a time, what about 2.
For load balancing concerns, see Apple's File
Services Guide for recommendations. On an Xserve G5
with 2GB or more of memory, you should easily
accommodate 50 simultaneously connected users
(mixture of AFP and SMB) without a performance hit.
Depending on what your users are doing (and the
speed of the Ethernet network to which your clients
and Xserve are connected), you may be able to handle
more or less users. More RAM in the Xserve and a
Gigabit Ethernet network for the server and clients
would allow you to simultaneously balance more
clients with less of a reduction in performance.
Xsan works differently, and its requirements are
somewhat different. With Xsan, your Xserve RAID is
connected to a fibre-channel network. (In the
previous direct-to-server example, the fibre-channel
network consisted of just the Xserve and the Xserve
RAID.) In this network, all clients, at least one
Xserve, and the Xserve RAID are all connected via a
fibre-channel network. The Xserve has Xsan software
installed on it and becomes a dedicated metadata
controller. Clients must have special Xsan client
software installed for the SAN volumes to appear. In
such cases, the protocol used to mount the SAN
volumes is the Xsan client protocol, not AFP or SMB.
Although you technically can reshare an Xsan volume
via AFP or SMB, performance would take a hit. Since
the Xsan volume is writeable by all other clients on
the fibre-channel network, adding an Xserve to
reshare the SAN volume via AFP or SMB would allow
clients to connect via Ethernet (Wi-Fi, etc.), but
the server wouldn't have exclusive access to the SAN
volume.
To be honest, two Xserves and an Xserve RAID simply
aren't enough to warrant a SAN in my book. Typically
SANs are used where there are a large number of "done
servers" doing computation tasks, and they all need
to be able to have access to the same "local volume."
They're also used in cases where clients need access
to large amounts of storage set up to work like a
"local disk" on a few video production computers.
Just for comparison, Gigabit Ethernet offers
throughput bursts of up to 125MB/s (megabytes per
second), and Fibre Channel offers burst of up to
200MB/s. (Apple's 400MB/s claim only somewhat makes
sense if you're using both Xserve RAID controllers
connected to the same server.) Even so, both media
sustain around 50MB/s to 75MB/s, which is quite good.
In fact, that matches local disk performance. The
local serial ATA hard disks that are used in Power
Mac G5 models are serial ATA 1.5Gb/s (gigaBITS per
second) or 187.5MB/s at maximum bursts;I typically
see performance in the 40MB/s to 50MB/s range.
Just for fun, here are throughput speeds of several
common connectors, in megabytes per second:
Serial ATA "3.0": bursts up to 375MB/s, sustains
about 75MB/s
LVD SCSI 320: bursts up to 320MB/s, sustains about
100MB/s
Fibre Channel: bursts up to 200MB/s, sustains about
60MB/s
Serial ATA "1.5": bursts up to 187.5 MB/s, sustains
about 50MB/s
SCSI-3 LVD SCSI 160: bursts up to 160MB/s, sustains
about 60MB/s
Ultra ATA/133 (PATA): bursts up to 133MB/s, sustains
about 50MB/s
Gigabit Ethernet: bursts up to 125MB/s, sustains
about 50MB/s
FireWire 800: bursts up to 100MB/s, sustains about
50MB/s
ATA/66 (PATA): bursts up to 66MB/s, sustains about
40MB/s
USB 2.0: bursts up to 60MB/s, sustains about 30MB/s
FireWire 400: bursts up to 50MB/s, sustains about
20MB/s
10/100 "Fast" Ethernet: bursts up to 12.5MB/s
Judging by your request, I think the "no Xsan"
scenario is the one you want.
--Gerrit
I would love to take XSan out of the picture and if that means that I can only use one of the servers at a time then fine I can live with that, but, I would rather have both active and working for me even if I have to split the services between the systems.