Whole workgroup (SMB) dies for Macs but not PCs

Question

Hello all,
I'm having a very frustrating problem with a mixed Mac (Snow Leopard, all latest updates) and PC (Vista/XP) home network. Normally everything works fine and I can share files and printers between the Macs and PCs (i.e. this is not a simple configuration or permissions issue).
However, when I try to do a large (~500MB or so) file transfer from any Mac to any PC (wired or wireless, doesn't matter), the WHOLE workgroup dies for all Macs on the network. When I say the whole workgroup, I mean all the Macs lose access to any machine in the SMB workgroup (i.e. WORKGROUP or whatever all the machines are set to). Specifically, this is what happens:
- All SMB shares immediately stop working on all Macs (transfers in progress are cut off)
- All Macs and all PCs can still access the internet just fine and can still ping each other, so it isn't like the network connections have physically died or the router froze, etc.
- Using Finder Go > Connect to server, none of the Macs can access any SMB share on the network, either by name (smb://pc-name) or by IP address (smb://1.2.3.4). Doing so results in an immediate "Connection failed" message (i.e. it doesn't time out or stall, it just fails instantly)
- Windows machines can continue to access all SMB shares (including Mac shares) just fine! Just not vice versa.
Here are some things I've tried so far:
- Rebooting any/all Macs does NOT clear up the issue. It's as if the whole SMB workgroup is stuck in a "corrupt" state which is not cleared by rebooting the Macs.
- Switching from wired to wireless or vice versa doesn't help.
- Turning ALL machines off, waiting a while, then turning all back on again clears the problem, but only until the next large file transfer.
- Changing the workgroup name on all machines temporarily solves it until the next large file transfer (but this requires rebooting all the PCs which is a huge hassle).
- Tried clean reinstalling Snow Leopard 10.6.2 on several of the Macs -- no change.
- Clean reinstalled Vista on several of the PCs -- no change.
- Changed the internet/WiFi router for a different brand (now using DD-WRT) -- no change.
- Changed the Ethernet switches to different brands and used new Ethernet cables to each wired machine -- no change.
I'm not a networking guru but I'm a software developer and have been running a mixed Windows/Linux/Mac home network for some time now, so I'm not a novice either. This is one of the more frustrating problems I've ever faced.
If anyone has ANY ideas, it would be greatly appreciated.
Thanks!

Answer 1

I am not a network guru either, but perhaps if you set the Macs' network ports to jumbo packets, the problem might be fixed. Information on how to do that is available in this web page http://support.apple.com/kb/TA21333?viewlocale=en_US
Also, if that doesn't help, you might need to get or borrow a fluke and start packet tracing some of the network connections. Maybe there is a bad or misconfigured hub or switch in your network fabric. Flukes are expensive though, so an alternative might be to find a packet sniffer for the Mac or Windows and start sniffing packets during one of those problematic file transfers. This might point you to an underlying hardware problem or misconfiguration in how your network is configured.