CANNOT_LOAD_BUNDLE_ERR from remote client (same local network).

Question

Hello,
We're having problems (as many other people) with remotely connecting (but on same network) Server Monitor (1.8(5)) from a 10.6.4 client to the LOM IP (or FQDN) of our Xserve (model Xserve3,1)(early 2009)(running under 10.6.4)(LOM revision 1.1.8)(Boot ROM versionXS31.0081.B06).
Although connections of Server Monitor on that Xserve via localhost work fine, connections from the client (a Macbook Pro running 10.6.4) give the infamous CANNOTLOAD_BUNDLEERR.
Note that connections from the client to our OTHER Xserve (early 2008)(model Xserve2,1)(LOM revision 1.1.2)(Boot ROM Version XS21.006C.B06) work fine (both via localhost and non localhost).
Please help!
shawn
p.s. Here's a list of what didn't help:
Our LOM Firmware does seem to be up-to-date on both the working Xserve (the one that does allow remote connections) and the not-willing-to-do-remote Xserve. http://support.apple.com/kb/HT3539 (trying to install a LOM updater tells us we're already up to date.. and anyway, the LOM Firmware update should only be for Early2008 Xserves).
This unsolved archived discussion did not help:
Server Admin/Monitor Issues (MrHoffman)
http://discussions.apple.com/thread.jspa?threadID=2325127
Nor did reading "LOM it up!" (MacTroll) or the discussion that followed.
http://www.afp548.com/article.php?story=20070211101829496&query=120
Using Network Utility > Lookup from the client machine shows that the IP address does indeed point to the correct FQDN and vice versa.
Running, from the client, dig server2.epfl.ch showed the expected IP addresses.
Running, from the client, dig -x with the returned IP address, showed the expected FQDN.
Pinging the LOM's FQDN gives me successful ACKs of .200ms.
Running changeip -checkhostname on the server shows the non-LOM ip address and correct non-LOM FQDN. How do I use changeip to check the LOM hostname and IP address (which is different) ??
We're not mixing 10.5 and 10.6; Both server and client are using 10.6.4.
Running ServerMonitor directly on the server (using localhost) works fine.
ConfigureLocalMachine on the not-working Xserve (call it Xserve2) shows LOM using Ethernet PORT2 (but switching to PORT1 doesn't help). The Subnet-mask and Router values are configured identically to the working Xserve (call it Xserve1) where the config does allow remote access. The version of ServerMonitor on both Xserves and on the clientMacs is 1.8(5). Every machine is 10.6.4.
The command "ipmitool lan print 2" on Xserve2 reveals the correct IP address and other infos. But the command "ipmitool lan print 1" says that Channel 1 is not a LAN Channel. Is this important? If so, what do I do with this info?
Ideas from reading LOM No Workie (Faby) don't help.
http://lists.apple.com/archives/Macos-x-server/2006/Dec/msg00620.html
I've also uselessly read:
Xserve User's Guide : http://www.apple.com/support/xserve/
OS X Server Manuals : http://www.apple.com/server/macosx/resources/documentation.html
HT2773 document - Xserve (Late 2006 and later): Configuring Lights-Out Management (LOM) : http://support.apple.com/kb/HT2773

Answer 1

I'm going to skim over some details of IP and IP routing.
My IP for that NIC on which the LOM is running is 128.178.100.87.
If I'm reading you correctly, I should NOT have an IP for my LOM (if you say 'correct' then I'm really confused).
Your LOM needs its own unique IP address.
This can be a little confusing if you're looking at the hardware, but it's entirely possible to have several separate IP hosts (devices) on one network jack, and each needs its own (unique) address. You're not assigning IP addresses to cables, you're assigning addresses to hosts.
And if I read you to say that yes 128.178.100.87 should be assigned to the LOM but not the NIC,.. then to what is the LOM attached/configured?
Well, if two hosts (or even two network controllers on the same host) have the same address (and the LOM is a host), then (best case) the IP traffic on the second host controller to start detects the address collision and won't allow the host to go online. Worst case, traffic gets snarled.
Finally, it seems that this setup on XserveBAD (an XServe3,1) is the same on XserveGOOD (and Xserve2,1) in terms of IPs NICs and LOMs. The only diff seems to be Xserve versions.
There was a brief kerfuffle with the Xserve3,1 boxes and the tools back when those boxes first arrived, but that all settled down with current tools and firmware.
*2. 2nd dual-port NIC*
YES indeed! You're good! From the Preferences>Network panel I see we have a PCI ethernet with two ports (both XserveBAD & XserveGOOD have this) link-aggretated. To that bond, we've assigned the IP address that people use to ssh into the machines. LOM is not there. XserveBAD's LOM is configured to say +LOM uses ethernet Port 1+ and XserveGOOD's LOM is set to +uses Network1+ (the GUI looks different on the two machines).
Two networks (also) means you'll have to consider IP routing; which NIC is used to respond to traffic. Inbound IP packet traffic might well arrive at the box, but if the box doesn't have its routing configured "correctly" then the IP packets "outbound" in response might not be sent out the expected controller; the inbound IP traffic and the IP responses don't automatically use the same controller. Those routing determinations are made separately, based on the local subnet, available local static routes, and the rest of the packets get sent to the default router to deal with.
For both XserveBAD and XserveGOOD, the Preferences>Network panel shows the IP that we configured for the LOM to be the one on 'Ethernet 1" with subnet mask of 255.255.255.0. And in both cases the IP address used for sshing (the one used on the PCI ethernet bonded ports) is on 'Ethernet 0' with subnet mask 255.255.255.0.
The subnet masks tells IP where the IP packet is going to be set. In the most general of terms, that mask tells IP routing where to send the packet, and that decision (and specifically with the 255.255.255.0 mask) says that if the first three bytes are the same, it's a local address and the messages get sent directly. If the first three bytes differ (and it's not an established route), it goes to the designated "gateway" router.
p.s. FWIW, I almost went with SadLOM and HappyLOM but I'm still pretending to be a serious mac admin-wannabe here.
Uh-huh. You'll get over that.
p.p.s. I didn't catch your drift concerning public/private subnets.. but I'm googling, I'm googling and reading here http://articles.techrepublic.com.com/5100-10878_11-6089187.html
Your network uses IP addresses in the public static IP space, and IP addresses in one of the private address spaces. (The same would apply if you had IP addresses in different subnets in any of the private IP address blocks, though the use of a public space implies there's a "public" LAN and a private LAN.)
That looks reasonable as far as it goes, albeit a little stale, and it unfortunately lacks the associated discussion of routing; of why you'd be interested in subnets, and what happens with the packets. The class stuff discussed there is replaced with what's called "CIDR" - classless routing -- and is long gone. (I've been around long enough that the class A, B and C stuff is second nature and old habit, but that "classful" scheme has been replaced with the far more flexible "classless" CIDR scheme.) And the article lists 17 years for IPv4, and the last of the available IPv4 blocks are likely to be handed out within the next few years; we're all dealing with IPv6 now, and its likely that IPv4 addresses are going to get harder to get and probably also more expensive.
when next you're in Switzerland, I owe you free beers.
Danke. Der Ratskeller in Zürich war ziemlich gut. Quite a pretty walking area around there too, and with a beautiful view of the river.