Running BizTalk on multiple non-clustered servers. Potential serious issue

Question

Hi,
I have just discovered what has potential to be a fairly large issue in our current BizTalk Architecture. We are running BizTalk Server 2006 R2 on 2 different servers, running Windows Server 2003 R2. These servers are NOT clustered. Instead, BizTalk is installed
to both, configured on 1 and then on the 2nd, they have joined the existing BizTalk environment. We have 1 host with 2 host-instances for say an Orchestration Host. (So 1 Orchestration Host and then 1 Orchestration Host Instance on each server), we also have
a host instance for Receive and Send on each server.
This is a problem that I have identified on an application currently in UAT:
We have a routine called 'Recovery Mode'. It doesn't matter what Recovery Mode is but what does matter is that only a single instance can go into recovery mode at the same time. 
Consider this:
bool inRecovery;
while (isBroken) {
if (!inRecovery){
inRecovery = true;
// Recover
inRecovery = false;
else {
// Wait, recheck until not in recovery, or fixed.
Imagine an instance on server 1 needs to go into recovery, it first checks the inRecovery flag, gets returned false so goes into recovery, let's say this takes 5 minutes to complete. During this time, an instance on server 2 needs to go into recovery, checks
the flag, but the flag on server 2 has not been set to true.... Now I have 2 instances in Recovery...
My questions are:
1. Can I cluster 2 non-clustered servers? I'd need to upgrade from Windows Server 2003 R2 Standard to Enterprise is that possible? If so, Can I simply upgrade and cluster the servers or would I need to format the disks?
2. If no to number 1, where would you store the 'inRecovery' flag'? I can't store in a DB as there will be slight cross over where instance 2 might return false in my example just as instance 1 is writing unless I can lock it on write or something.
Some advice would be greatly appreciated.
Thanks
Rod

Answer 1

To answer your specific question:
1. Yes, but you would need to upgrade to at least Enterprise.  Windows Server 2003 R2 Standard does not support clustering.  IIRC, the upgrade from Standard to Enterprise is pretty easy, perhaps just a SKU change.
2. If you absolutely cannot upgrade Windows Server, then you'd have to store the state externally, such as SQL Server.  There shouldn't be any concurrency issues if you do the check in the right sequence.  Basically, begin a transaction, attempt
to UPDATE, return weather the UPDATE was successful, meaning that call grabbed the lock.  If you check then update, yes, there is a miniscule chance both will return the same value but only one near-sumultaneous UPDATE will succeed.