Set Server Pool Master to Maintenance Mode

Hi all,
when trying to set a server pool master to maintenance, I got this: "OVM-2020 Server pool master (vm-13) can not be set to maintenance mode, please use server pool master failover policy to change its role, then try again."
I have dug through the help and docs, but can not find a method for dynamically reassigning the server pool master role.
This document: http://download.oracle.com/docs/cd/E15458_01/doc.22/e15441/server.htm#CCHIEBCE , just says "You must first reassign the Server Pool Master role to another server in the server pool.", without describing how.
This document: http://download.oracle.com/docs/cd/E15458_01/doc.22/e15441/site.htm#insertedID4 , states "You can also dynamically change the Oracle VM Server which acts as the Server Pool Master without causing any outages. See Section 3.4.1, "Editing Server Pool".", which just links back to the same section I am already in.
Now, I can just migrate all my machines and then stop ovs-agent, but I am not always the one managing this, and having a nice way through the web interface would be great.
Please just tell me I am blind and point me to the section in the manuals which tells me how to do it, or is this a feature which got dropped before release?
Thanks.

I have got the same problem, I want to switch the role of the server pool master to another server in the same pool. But after reading these answers, I have absolutly no clue how to do it!
We have a productiv environment, so any ideas like shutting down or stopping VMs are not acceptable.
So here is the situation:
"ovs#1" is hosting all VMs at the moment - Server type: Utility Server,Virtual Machine Server
"ovs#2" is free of any VMs - Server type: Server Pool Master,Utility Server,Virtual Machine Server
I want to switch the Server Pool Master from "ovs#2" to "ovs#1" without any downtime. How does this work? Thanks!
Specifications:
- Oracle VM Manager 2.2.0
- Oracle VM server release 2.2.1
Edited by: user11932329 on 15.09.2010 04:50

Similar Messages

OVM-2020 Server pool master can not be set to maintenance mode

Hi Guys.
I have 2 server in ha mode
server 1 (10.99.99.161) with oracle Manager installed (server pool master, utility server, vm server)
Server 2 (10.99.99.161) without Oracle Manager installed (utility server, vmserver)
HA enable with server1 and server 2
10.99.99.165 is the virtual server pool master
1 San connected with server 1 and server 2.
Without problem I can connect on https://10.99.99.165:4443/OVS without problem.
All seems work, but :
1 - I I try to set in maintenance mode the server 1, the VM Manager say me:
OVM-2020 Server pool master (10.99.99.162) can not be set to maintenance mode, please use server pool master failover policy to change its role, then try again.
What I shuld do ?
2 - If I turn off the server 1 the server pool master is attribuite to server 2, but I'm unable to connect on https://10.99.99.165:4443/OVS webpage. It is correct, it is a bug, or I'm doing something wrong ? Must I upgrate something ?
Many Thanks
Luca

user8857532 wrote:
1 - I I try to set in maintenance mode the server 1, the VM Manager say me:
What I shuld do ?You need to live migrate all your guests off the current Master server, then issue service ovs-agent stop on the command-line. This will cause the pool mastery to switch to another server. Once that's done, you can start the agent agent.
2 - If I turn off the server 1 the server pool master is attribuite to server 2, but I'm unable to connect on https://10.99.99.165:4443/OVS webpage. It is correct, it is a bug, or I'm doing something wrong ? Must I upgrate something ?
You need to ensure the VM that's running the Oracle VM Manager software is still running. Live Migrate it first (preferred option) and flag it HA-enabled so that it is automatically restarted in the case of a server failure.

Non-server pool master in VM Manager

So I have two vm hosts that are not HA. The first is the Server Pool Master and everything is working well there. On the second host machine (the non-master) I put my template in the seed_pool folder and I can use xm to start the template and everything is fine, but I can not seem to get Oracle VM Manager to import the Virtual Machine Templates from the second host. Any ideas on what I did wrong?
Thanks,
Wesley

Hi Wesley,
all servers in a server pool (even if not enabled for HA) have shared repositories (this was changed with OVM 2.2, before that there was the concept of local repositories, but it does not seem to be existing anymore with OVS 2.2). These can be listed with
/opt/ovs-agent-latest/utils/repos.py -l
These are the folders the agents looks into the seed_pool to import the machines.
(note repos.py does not have an option for local repository).
If the repository is not listed here, where you put you seed_pool, then it is clear why you cannot import the template.
Regards
Sebastian

OVM-2008 The Server Pool Master has been registered with some other pool

I am getting this error, while creating my first ever server-pool.
Check prerequisites to create server pool (First-Pool) failed: (OVM-2008 The Server Pool Master (172.16.0.37) has been registered with some other pool, and can not register it again.)
Any idea what can be going wrong?
regards, Yora

The only time I've seen this is when the server really has been registered with another pool. You need to clean out the local ovs-agent database at */etc/ovs-agent/db* and restart the agent.

Changing the Server Pool Master

I need to remove one of the servers of a three node cluster, but before I remove that node I need to remove it as the Server Pool Master and make another node the server pool master. When I go to edit a server, the check box is there, but it’s disable. How do I enable this or accomplish this switch?

It's actually pretty simple, if both the old and new Server Pool Master servers are online. First, make sure that you're not running any VMs on either the old (existing) or new (planned) Pool Master servers. You also need to be sure that all your VMs are either Powered Off or Running, i.e. none being migrated or suspended.
Then, here are the steps:
1. Go to the Servers tab and delete the NEW Pool Master from the pool.
2. Edit the EXISTING Pool Master and replace it's details with those of the NEW Pool Master, i.e. IP Address/Hostname. If the new pool master is also a utility server, you must ensure that you select that option in the Edit screen.
3. Click OK
4. Click the Server Pool tab and select your server pool. Then click the Restore button to update the status of all your servers/VMs in the pool.
5. Manually restart the agent on the NEW Pool Master. Login to the Dom0 of the new pool master and run:
# service ovs-agent stop
# service ovs-agent start
You now have the new Server Pool Master running and the old one has actually been removed from the pool. You can add it back as a normal server using the default process in the GUI.
I know there are guys inside Oracle working on an official whitepaper on Reassigning the Server Pool Master role in situations where the old/new servers are not online, so you should keep a look out on the Virtualization OTN site.

OVM 3.0.1 : Server Pool Master Error

am starting to see sudden errors on the console. All the vm's went offline including the server icon showing RED and in status Running (Error)
The event shows : Server Pool Master Missing Event / Server Pool Master Error.
Can anyone help me how to fix this issue? This just happened all of a sudden.

Ok I found the problem...although the locking from the rescan job seemed to disappear or timeout overnight.
It was actually the virtual machine that I had under that host that was in error. All I had to do was acknowledge the event that the virtual machine was powered off (selecting the vm, going to events, then acknowledge all). It's really annoying that VM 3 has this function because you are forced to acknowledge the event before you can continue with other activities.
It would be nice if there was maybe a way to only have that enabled for specific events. Another reason that this might have happened, was that I did not have HA enabled on the VM, so it wasn't migrated to my other server when I did the reboot.

Server Pool Master - 2 node HA setup - Need suggestion

I run OVM 2.2.1 in HA since 1 year.
Have 2 node. Guests failover when Utility Server,Virtual Machine Server crash.
But have problem with Server Pool Master.... ( My Server Pool Master it installed on my Node 1)
When my Server Pool Master crash, All HA guest move and restart on my second server. But because i can have only one Server Pool Master i can't graphically manage my cluster and my guest anymore.
When my primary server come back it doesn't resume it master role.
The only way to take back my console, is to restart my second server.
Note: I have a cluster URL to access to my console, however, But Oracle Express have no replication option then the BD of my second server is always empty.
What is you solution ?
How have you fix this situation ?
What the trick ?
Thanks
David

I have no answer on why your serverpool master does not resume its role when it restarts but I suggest you learn and use the XM commands on your oracle VM server to manage your guests.
To get a quick list of xm commands you can type "xm help" on your console. For more details and info run "man xm"
When your serverpool master comes back up, run the xm migrate command to move your guests from node2 to node1. When this completes, you can now reboot node2 with no issues and as I understand it, the serverpool master resumes its role after node2 is rebooted.

Server Pool Master fails and cannot fail over to another VM Server

Dear All,
Oracle VM 2.2.2
I have 2 VM Servers connect to Storage 6140 Array and on VM Manager I enable HA on the server pool, then on all virtual machines.
- VM Server 1 has role as Server Pool Master, Utility Server, Virtual Machine Server and has virtual machines running
- VM Server 2 has role as Utility Server, Virtual Machine Server and has virtual machines running.
I try to shutdown the VM Server 1 act as Server Pool Master but I don't see Server Pool Master fail over to another VM Server 2 and also status become to Unreachable both of 2 Servers.
Especially, All virtual machines cannot be accessible.
Please kindly give advice for this.
Thanks and regards,
Heng

Thanks Avi, I'll find and read that document. And thanks also for elaborating about the Utility Server.
After reading the followups to my original question, I tried to think of possible server "layouts" in a HA environment.
1) "N" servers in the pool, one of them is Pool Master, Utility Server AND VM Guests Server at the same time. Maybe this will be the preferred server for smaller, quicker VMs.
2) "N" servers in the pool, one is Pool Master AND Utility Server, but has no VM guests running on it
3) "N" servers in the pool, one is the Pool Master, another one is the Utility Server (none of them has VMs running on them), and finally a number of VM Guest servers
Let's take case 1. If the Pool Master & Utility server fails, given that it has VM guests running on it as well, I understand from your explanation that I'll be ANYWAY able to manually "live migrate" the guests somewhere else, using VM Manager. Is this correct?
If it's correct, then it's just a question of how much money I want to spend to have dedicated servers for different tasks, JUST FOR BETTER PERFORMANCES REASONS. Do you agree? And especially: do YOU have dedicated Pool Masters (just to figure out your "real" approach to the problem :-) )
I feel that I still miss something, the picture is not completely clear to me. The fact is, that I'm now testing on my new bladesystem, but for now I put up one single blade. Testing HA will be the next step. I was just trying to get a few things sorted out in advance, but there is still something that I'm missing, as I was saying...
Looking forward to your next reply, thanx again
Rob

Create alert when management server is in maintenance mode

has anyone setup alerting, such that, an alert is generated when a gateway or a management server is put into maintenance mode? (we are assuming there are multiple management servers and gateways)
if so, how did you achieve this?

1) You may create a time script monitor to get any server in maintenance mode. Using powershell script get-scommaintenancemode to get server in maintenance mode information.
2) How to create a powershell script monitor
http://scug.be/dieter/2014/04/24/scom-creating-a-powershell-script-monitor-with-silect-mpauthor/
http://www.code4ward.net/main/Blog/tabid/70/EntryId/51/How-to-create-a-Two-State-PowerShell-Script-Monitor-using-the-Authoring-Console-Part-1.aspx
3) For powershell get-scommaintenancemode, pls refer to
http://technet.microsoft.com/en-us/library/hh920235(v=sc.20).aspx
Roger

Creation of server pool in dhcp environment fails

Hi,
I am new to Oracle VM, please help me with my aim of setting up an "as simple to configure as possible" virtual machine. I chose the "all-in-one" approach, mentioned in the documentation, where server-pool-master, utility-server, vm-server are all on the same machine.
Now, I have a problem, which consists in the dhcp based network environment in my workgroup. How can work with fixed IP's in order to configure my server pool?
Can the server pool virtual IP be the same, as the IP of
Server Pool Master
Utility Server
Virtual Machine Server
Many thanks in advance

user8768362 wrote:
Do i understand you correctly, that it is possible to configure the Oracle VM server and the pool and the utility server and the vm guests on one machine?Yes.
Do i understand you correctly, that i should assign the pool the very same physical IP of the Oracle VM server host?In the case of a single server, the pool doesn't need an IP address.

Added server not listed in Server pool (not even on the list of servers)

Hello VM Gurus,
I'm playing with Oracle VM and I've encountered problem I can not solve of find existing solution.
I'm playing with Oracle VM version 2.1.5.
I have 3 psychical machines (let say A,B,C). On every machine I have installed WMWare workstation. I've created all together 5 VMWare virtual machines.
On Physical machine A I have 1 VMWare virtual machine named A1. This is used as Virtual Machine Server.
On Physical machine B I have 2 VMWare virtual machine named B1 and B2. B1 is used as Oracle VM Manager. B2 is used as Virtual Machine Server (also Server Pool Master and Utility Server)
On Physical machine C I have 2 VMWare virtual machine named C1 and C2. Both are used as Virtual Machine Serves.
1. I've Created Pool of servers (HA set up) and first added server B2.
2. After that I've added to pool A1 without any problems
3. After that I've added to pool C1 without any problems.
3. After that I've tried to add to pool C2. Process (wizard "Add server") finishes without any problems. But server does not appear on the list of servers in pool. It does not exist On a list of servers neither. If I try to add server again I got error OVS-2005 saying that this server is part of another Pool.
If I stop agent on C2 , delete folder /etc/ovs-agent/db and start agent, I can again add server, but results are same (it does not appear on list of servers. Another try to add server results in OVS-2005 error).
If done also this process:
1. I've stopped agent on C2, deleted folder /etc/ovs-agent/db and starte agent
2. I've removed C1 from pool
3. I've added C2 to pool without any problems
4. I've tried to add c1 to Pool. Same problems as in the past with C2 (not appearing on the list of servers).
I went though agent logs, VM manager logs. I ca not spot any problems. I've connected to database with SQL plus and there was no evidence of failed server there (so data is really not written to repository).
I run out of ideas. What could be wrong? where to look whyt is problem. How to diagnose problem. I believe that Oracle VM somehow treat this two VMWare servers as one, but I can not prove this.
Somebody else had this problem? Any hint what to diagnose further? I understand my configuration is a bit stupid, but I'm trying to get familiar with Oracle VM.
Any hint will be appreciated.
Regards
Darko

user465397 wrote:
I have 3 psychical machines (let say A,B,C). On every machine I have installed WMWare workstation. I've created all together 5 VMWare virtual machines.This is a really bad idea, btw -- if you have three physical machines, you should install Oracle VM directly on each machine. Or use one machine as the NFS server and Oracle VM Manager and the other two as Oracle VM Servers. Trying to run a Type 1 Hypervisor inside a Type 2 hypervisor is fraught with peril. I'm surprised you actually got this far, to be honest. :)

"Class instance is already in Maintenance Mode" - but it isn't

Hello,
we have a powershell script to set the servers managed by scom agent in maintenance mode (e.g. for windows updates and restart).
A colleague recognized an error during running this script. It says that instances of the server are still in maintenance mode. But if you check through SCOM it is not. And for resetting maintenance mode with 0 it says it is not in maintenance mode.
Is it possible to clear maintenance mode for all things wiht a sql statement?
Script:
#=============================================================================#
# Remote-MaintenanceModeV5.ps1 #
# Powershell Script to put a SCOM agent into maintenance mode. #
# Autor: Roman Strecker #
# Date: 14.12.2012 #
# Bekannte Fehler: Speicher auf dem Zielhost (SCOM) muss hoehergesetzt werden!#
# winrm set winrm/config/winrs `@`{MaxMemoryPerShellMB=`"512`"`} #
#=============================================================================#
param(
[Parameter(Mandatory = $false)][string]$ComputerName,
[Parameter(Mandatory = $false)][string]$Dauer
Function SetMaintenanceMode
$ok = Test-Connection 10.202.14.29 -Count 1 -Quiet
if (-not($ok)) {
$SCOMServer = "srv14v030"
else{
$SCOMServer = "srv14v029"
#Kleinster Zeitinterval bei SCOM sind 6 Minuten!
if ([int]$Dauer -eq '0') {
Write-Host "Löschen des Maintenance Modes wird durchgeführt!"
$DeleteMM = $true
else {
if ([int]$Dauer -lt '6') {
$Dauer = "6"
$startTime = [DateTime]::Now
$endTime = $startTime.AddMinutes($Dauer)
Write-Host "`nFolgende Parameter werden verwendet: `nServername: $ComputerName `nWartungs-Dauer: $Dauer Minuten `nWartungsmodus wird geprüft..."
#Get-ChildItem env:
#Erzeuge ein Credential-Objekt für die Verbindung zum SCOM:
$secpasswd = ConvertTo-SecureString "xxxxxx" -AsPlainText -Force
$Credential = New-Object System.Management.Automation.PSCredential ("SA_SCOM-ActionAcc", $secpasswd)
#Erzeuge eine Remote-Session zum SCOM:
#$Session = New-PSSession -ComputerName $SCOMServer -Authentication kerberos
$Session = New-PSSession -ComputerName $SCOMServer -Credential $Credential
$Session = get-pssession
Invoke-Command -Session $Session -ScriptBlock {
param($ComputerName, $endTime, $DeleteMM)
Import-Module OperationsManager
try {
$MonitoringObjects = Get-SCOMMonitoringObject | where {$_.DisplayName -like "$ComputerName*"}
$MObject.ViewName
catch {
Write-Host "Fehler beim Verbinden mit dem SCOM ist aufgetreten!`nBitte versuchen Sie es später noch ein Mal."
Write-Host "Vollständige Fehlermeldung: "$_.Exception.Message
if ($MonitoringObjects -eq $null) {
Write-Host "Servername konnte nicht gefunden werden: " $ComputerName
else {
foreach ($ComputerInstance in $MonitoringObjects) {
if ($ComputerInstance.InMaintenanceMode) {
if ($DeleteMM) {
#Wartung muss gelöscht werden
$ComputerInstance.StopMaintenanceMode([DateTime]::Now.ToUniversalTime(), "Recursive")
$Ausgabe = "Server aus dem Wartungsmodus entfernt."
else {
#Server ist Bereits im Wartungsmodus
$MMEntry = Get-SCOMMaintenanceMode -Instance $ComputerInstance
Set-SCOMMaintenanceMode -MaintenanceModeEntry $MMEntry -EndTime $EndTime -Comment "Wartungsmodus wird angepasst."
$Ausgabe = "Wartungsmodus für den Server " + $ComputerName + " ist angepasst bis " + ((Get-SCOMMaintenanceMode -Instance $ComputerInstance).ScheduledEndTime).ToLocalTime()
else {
if ($DeleteMM) {
#Server war nicht im Wartungsmodus und soll gelöscht werden
$Ausgabe = "Server war nicht im Wartungsmodus!"
else {
#Setzen des Servers in die Wartung
Start-SCOMMaintenanceMode -Instance $ComputerInstance -EndTime $endTime -Reason "PlannedOther" -Comment "Wartungsmodus wird gesetzt."
$Ausgabe = "Wartungsmodus für den Server " + $ComputerName + " ist gesetzt bis " + ((Get-SCOMMaintenanceMode -Instance $ComputerInstance).ScheduledEndTime).ToLocalTime()
#Ausgabe für die letzte Instanz:
Write-Host $Ausgabe
} -ArgumentList $ComputerName, $endTime, $DeleteMM
#Schliesse die Remote-Session
Remove-PSSession -Session $Session
#MAIN:
#Prüfung, ob die Parameter eingeben wurden:
if ($ComputerName -eq '') {
$ComputerName = Read-Host "Bitte den Servernamen eingeben"
$Dauer = Read-Host "Bitte die Dauer in Minuten eingeben (0 = Löschen des Wartungsfensters)"
#Wenn noch immer nichts gesetzt ist, dann abgebrochen:
if ($ComputerName -eq '' -OR $Dauer -eq '') {
Write-Host "Vorgang abgebrochen!"
else {
SetMaintenanceMode
Sleep 5
Regards.

Hi also in this post :)
Thats a nice query - thank you. But the systems with the error are not listed.
Maybe there is an error in our script:
foreach ($ComputerInstance in $MonitoringObjects) {
if ($ComputerInstance.InMaintenanceMode) {....$MMEntry = Get-SCOMMaintenanceMode -Instance $ComputerInstance

Full server pool crashes when adding new iSCSI server

Hi,
we have a Pool Group with 2 machines (1 Server Pool Master + Server Virtual Machine + Utility Server and another 1 Server Virtual Machine). Both have a iSCSI Shared Disk which builds /OVS partition.
This is working, we can use High Availability, Migrate guests etc.
But when adding new Server Virtual Machines to the pool (with guests running), current machines in the Pools get restarted.
My question is, can server virtual machines be "hot added" to the pool group while guests are running?
Thanks and regards,
Marc

Hi,
hosts file seems to be correct.
Messages logs during the crash time are the following:
Node vmserver15 = Server Pool Master, Utility Master and Server Virtual Machine
Dec 10 12:40:02 vmserver15 kernel: vlan500: port 3(vif6.0) entering disabled state
Dec 10 12:40:02 vmserver15 kernel: device vif6.0 left promiscuous mode
Dec 10 12:40:02 vmserver15 kernel: type=1700 audit(1260445202.434:16): dev=vif6.0 prom=0 old_prom=256 auid=4294967295 ses=4294967295
Dec 10 12:40:02 vmserver15 kernel: vlan500: port 3(vif6.0) entering disabled state
Dec 10 12:40:02 vmserver15 kernel: loop10: dropped 10114 extents
Dec 10 12:40:03 vmserver15 udhcpc: udhcp client (v0.9.8) started
Dec 10 12:40:03 vmserver15 udhcpc: Lease of 193.109.175.25 obtained, lease time 172800
Dec 10 12:40:04 vmserver15 kernel: device vif7.0 entered promiscuous mode
Dec 10 12:40:04 vmserver15 kernel: type=1700 audit(1260445204.774:17): dev=vif7.0 prom=256 old_prom=0 auid=4294967295 ses=4294967295
Dec 10 12:40:04 vmserver15 kernel: vlan500: topology change detected, propagating
Dec 10 12:40:04 vmserver15 kernel: vlan500: port 3(vif7.0) entering forwarding state
Dec 10 12:40:05 vmserver15 kernel: loop10: fast redirect
Dec 10 12:40:06 vmserver15 kernel: blkback: ring-ref 770, event-channel 9, protocol 1 (x86_32-abi)
Dec 10 12:53:35 vmserver15 kernel: o2net: no longer connected to node vmserver10.pic.es (num 1) at 193.109.174.110:7777
Dec 10 12:53:36 vmserver15 kernel: (4989,0):o2hb_do_disk_heartbeat:776 ERROR: Device "sdb1": another node is heartbeating in our slot!
Dec 10 12:53:37 vmserver15 kernel: o2net: accepted connection from node vmserver10.pic.es (num 1) at 193.109.174.110:7777
Dec 10 12:53:38 vmserver15 kernel: (4989,0):o2hb_do_disk_heartbeat:776 ERROR: Device "sdb1": another node is heartbeating in our slot!
Dec 10 12:53:50 vmserver15 last message repeated 6 times
Dec 10 12:53:51 vmserver15 kernel: o2net: no longer connected to node vmserver10.pic.es (num 1) at 193.109.174.110:7777
Dec 10 12:53:52 vmserver15 kernel: (4989,0):o2hb_do_disk_heartbeat:776 ERROR: Device "sdb1": another node is heartbeating in our slot!
Dec 10 12:53:53 vmserver15 kernel: o2net: accepted connection from node vmserver10.pic.es (num 1) at 193.109.174.110:7777
Dec 10 12:53:53 vmserver15 kernel: (5638,0):dlm_send_remote_convert_request:393 ERROR: dlm status = DLM_IVLOCKID+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):dlmconvert_remote:327 ERROR: dlm status = DLM_IVLOCKID+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):ocfs2_cluster_lock:1206 ERROR: DLM error DLM_IVLOCKID while calling dlmlock on resource M000000000000000001050c00000000: bad lockid+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):ocfs2_inode_lock_full:2064 ERROR: status = -22+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):ocfs2_inode_lock_atime:2193 ERROR: status = -22+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):__ocfs2_file_aio_read:2434 ERROR: status = -22+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):dlm_send_remote_convert_request:393 ERROR: dlm status = DLM_IVLOCKID+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):dlmconvert_remote:327 ERROR: dlm status = DLM_IVLOCKID+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):ocfs2_cluster_lock:1206 ERROR: DLM error DLM_IVLOCKID while calling dlmlock on resource M000000000000000001050c00000000: bad lockid+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):ocfs2_inode_lock_full:2064 ERROR: status = -22+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):ocfs2_write_begin:1845 ERROR: status = -22+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):ocfs2_file_buffered_write:2016 ERROR: status = -22+
Dec 10 12:53:53 vmserver15 kernel: (5638,0):__ocfs2_file_aio_write:2173 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):dlm_send_remote_convert_request:393 ERROR: dlm status = DLM_IVLOCKID+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):dlmconvert_remote:327 ERROR: dlm status = DLM_IVLOCKID+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):ocfs2_cluster_lock:1206 ERROR: DLM error DLM_IVLOCKID while calling dlmlock on resource M000000000000000000020744c1370e: bad lockid+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):ocfs2_inode_lock_full:2064 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):ocfs2_reserve_suballoc_bits:449 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):ocfs2_reserve_cluster_bitmap_bits:682 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):ocfs2_local_alloc_reserve_for_window:930 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):ocfs2_local_alloc_slide_window:1063 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):ocfs2_reserve_local_alloc_bits:537 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):__ocfs2_reserve_clusters:725 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):ocfs2_lock_allocators:677 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):ocfs2_write_begin_nolock:1751 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):ocfs2_write_begin:1861 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):ocfs2_file_buffered_write:2016 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: (7923,0):__ocfs2_file_aio_write:2173 ERROR: status = -22+
Dec 10 12:53:58 vmserver15 kernel: loop: Write error at byte offset 37644512256, length 4096.+
. <the above bold and cursive text is repetead few times>
Dec 10 12:58:29 vmserver15 kernel: (5638,3):dlmconvert_remote:327 ERROR: dlm status = DLM_IVLOCKID
Dec 10 12:58:29 vmserver15 kernel: (5638,3):ocfs2_cluster_lock:1206 ERROR: DLM error DLM_IVLOCKID while calling dlmlock on resource M000000000000000001050c00000000: bad lockid
Dec 10 12:58:29 vmserver15 kernel: (5638,3):ocfs2_inode_lock_full:2064 ERROR: status = -22
Dec 10 12:58:29 vmserver15 kernel: (5638,3):ocfs2_write_begin:1845 ERROR: status = -22
Dec 10 12:58:29 vmserver15 kernel: (5638,3):ocfs2_file_buffered_write:2016 ERROR: status = -22
Dec 10 12:58:29 vmserver15 kernel: (5638,3):__ocfs2_file_aio_write:2173 ERROR: status = -22
Dec 10 13:01:16 vmserver15 syslogd 1.4.1: restart.
Node vmserver10: Virtual Server Machine
Dec 10 12:53:35 vmserver10 kernel: o2net: no longer connected to node vmserver15.pic.es (num 0) at 193.109.174.115:7777
Dec 10 12:53:35 vmserver10 kernel: (5029,0):ocfs2_dlm_eviction_cb:98 device (8,17): dlm has evicted node 0
Dec 10 12:53:35 vmserver10 kernel: (20996,0):dlm_get_lock_resource:844 E3FE9E5767CA457FA697980EB637E93B:M000000000000000000022044c1370e: at least one node (0) to recover before lock mastery can begin
Dec 10 12:53:36 vmserver10 kernel: (5344,4):dlm_get_lock_resource:844 E3FE9E5767CA457FA697980EB637E93B:$RECOVERY: at least one node (0) to recover before lock mastery can begin
Dec 10 12:53:36 vmserver10 kernel: (5344,4):dlm_get_lock_resource:878 E3FE9E5767CA457FA697980EB637E93B: recovery map is not empty, but must master $RECOVERY lock now
Dec 10 12:53:36 vmserver10 kernel: (5344,4):dlm_do_recovery:524 (5344) Node 1 is the Recovery Master for the Dead Node 0 for Domain E3FE9E5767CA457FA697980EB637E93B
Dec 10 12:53:36 vmserver10 kernel: (20996,0):ocfs2_replay_journal:1183 Recovering node 0 from slot 0 on device (8,17)
Dec 10 12:53:37 vmserver10 kernel: o2net: connected to node vmserver15.pic.es (num 0) at 193.109.174.115:7777
Dec 10 12:53:38 vmserver10 kernel: (8672,1):dlm_get_lock_resource:844 ovm:$RECOVERY: at least one node (0) to recover before lock mastery can begin
Dec 10 12:53:38 vmserver10 kernel: (8672,1):dlm_get_lock_resource:878 ovm: recovery map is not empty, but must master $RECOVERY lock now
Dec 10 12:53:38 vmserver10 kernel: (8672,1):dlm_do_recovery:524 (8672) Node 1 is the Recovery Master for the Dead Node 0 for Domain ovm
Dec 10 12:53:40 vmserver10 kernel: kjournald starting. Commit interval 5 seconds
Dec 10 12:53:51 vmserver10 kernel: o2net: no longer connected to node vmserver15.pic.es (num 0) at 193.109.174.115:7777
Dec 10 12:53:53 vmserver10 kernel: o2net: connected to node vmserver15.pic.es (num 0) at 193.109.174.115:7777
Dec 10 12:53:53 vmserver10 kernel: (3761,0):dlm_convert_lock_handler:489 ERROR: did not find lock to convert on grant queue! cookie=0:92+
Dec 10 12:53:53 vmserver10 kernel: lockres: M000000000000000001050c0000000, owner=1, state=0+
Dec 10 12:53:53 vmserver10 kernel:   last used: 0, refcnt: 3, on purge list: no+
Dec 10 12:53:53 vmserver10 kernel:   on dirty list: no, on reco list: no, migrating pending: no+
Dec 10 12:53:53 vmserver10 kernel:   inflight locks: 0, asts reserved: 0+
*Dec 10 12:53:53 vmserver10 kernel:   refmap nodes: [ ], inflight=0*+
Dec 10 12:53:53 vmserver10 kernel:   granted queue:+
Dec 10 12:53:53 vmserver10 kernel:     type=5, conv=-1, node=1, cookie=1:243, ref=2, ast=(empty=y,pend=n), bast=(empty=y,pend=n), pending=(conv=n,lock=n,cancel=n,unlock=n)+
Dec 10 12:53:53 vmserver10 kernel:   converting queue:+
Dec 10 12:53:53 vmserver10 kernel:   blocked queue:+
. <the above bold and cursive text is repetead few times>
Dec 10 12:57:18 vmserver10 modprobe: FATAL: Module ocfs2_stackglue not found.
Dec 10 12:57:18 vmserver10 kernel: (3761,0):dlm_convert_lock_handler:489 ERROR: did not find lock to convert on grant queue! cookie=0:92+
Dec 10 12:57:18 vmserver10 kernel: lockres: M000000000000000001050c0000000, owner=1, state=0+
Dec 10 12:57:18 vmserver10 kernel:   last used: 0, refcnt: 3, on purge list: no+
Dec 10 12:57:18 vmserver10 kernel:   on dirty list: no, on reco list: no, migrating pending: no+
Dec 10 12:57:18 vmserver10 kernel:   inflight locks: 0, asts reserved: 0+
*Dec 10 12:57:18 vmserver10 kernel:   refmap nodes: [ ], inflight=0*+
Dec 10 12:57:18 vmserver10 kernel:   granted queue:+
Dec 10 12:57:18 vmserver10 kernel:     type=5, conv=-1, node=1, cookie=1:243, ref=2, ast=(empty=y,pend=n), bast=(empty=y,pend=n), pending=(conv=n,lock=n,cancel=n,unlock=n)+
Dec 10 12:57:18 vmserver10 kernel:   converting queue:+
Dec 10 12:57:18 vmserver10 kernel:   blocked queue:+
. <the above bold and cursive text is repetead few times>
Dec 10 12:58:32 vmserver10 kernel: (3761,0):dlm_unlock_lock_handler:511 ERROR: failed to find lock to unlock! cookie=0:1849
Dec 10 12:58:33 vmserver10 modprobe: FATAL: Module ocfs2_stackglue not found.
Dec 10 12:59:02 vmserver10 kernel: o2net: connection to node vmserver15.pic.es (num 0) at 193.109.174.115:7777 has been idle for 30.0 seconds, shutting it down.
Dec 10 12:59:02 vmserver10 kernel: (0,0):o2net_idle_timer:1503 here are some times that might help debug the situation: (tmr 1260446312.830107 now 1260446342.828243 dr 1260446312.830066 adv 1260446312.830319:1260446312.830320 func (b9f5fd13:506) 1260446312.830109:1260446312.830303)
Dec 10 12:59:02 vmserver10 kernel: o2net: no longer connected to node vmserver15.pic.es (num 0) at 193.109.174.115:7777
Dec 10 12:59:32 vmserver10 kernel: (3761,0):o2net_connect_expired:1664 ERROR: no connection established with node 0 after 30.0 seconds, giving up and returning errors.
Dec 10 13:01:42 vmserver10 kernel: o2net: connected to node vmserver15.pic.es (num 0) at 193.109.174.115:7777
Dec 10 13:01:45 vmserver10 kernel: ocfs2_dlm: Node 0 joins domain E3FE9E5767CA457FA697980EB637E93B
Dec 10 13:01:45 vmserver10 kernel: ocfs2_dlm: Nodes in domain ("E3FE9E5767CA457FA697980EB637E93B"): 0 1
Dec 10 13:01:51 vmserver10 kernel: o2net: accepted connection from node vmserver16.pic.es (num 2) at 193.109.174.116:7777
Dec 10 13:01:56 vmserver10 kernel: ocfs2_dlm: Node 2 joins domain E3FE9E5767CA457FA697980EB637E93B
Dec 10 13:01:56 vmserver10 kernel: ocfs2_dlm: Nodes in domain ("E3FE9E5767CA457FA697980EB637E93B"): 0 1 2
Dec 10 13:09:05 vmserver10 modprobe: FATAL: Module ocfs2_stackglue not found.
Dec 10 13:16:45 vmserver10 kernel: o2net: connection to node vmserver16.pic.es (num 2) at 193.109.174.116:7777 has been idle for 30.0 seconds, shutting it down.
Dec 10 13:16:45 vmserver10 kernel: (0,0):o2net_idle_timer:1503 here are some times that might help debug the situation: (tmr 1260447375.655426 now 1260447405.655712 dr 1260447375.655413 adv 1260447375.655427:1260447375.655427 func (b9f5fd13:503) 1260446516.75600:1260446516.75608)
Dec 10 13:16:45 vmserver10 kernel: o2net: no longer connected to node vmserver16.pic.es (num 2) at 193.109.174.116:7777
Dec 10 13:17:15 vmserver10 kernel: (3761,0):o2net_connect_expired:1664 ERROR: no connection established with node 2 after 30.0 seconds, giving up and returning errors.
Dec 10 13:17:19 vmserver10 kernel: (5029,0):ocfs2_dlm_eviction_cb:98 device (8,17): dlm has evicted node 2
Dec 10 13:17:20 vmserver10 kernel: (3761,0):ocfs2_dlm_eviction_cb:98 device (8,17): dlm has evicted node 2
Dec 10 13:29:05 vmserver10 kernel: o2net: no longer connected to node vmserver15.pic.es (num 0) at 193.109.174.115:7777
Dec 10 13:29:05 vmserver10 kernel: (5029,0):ocfs2_dlm_eviction_cb:98 device (8,17): dlm has evicted node 0
Dec 10 13:29:06 vmserver10 kernel: (5344,4):dlm_get_lock_resource:844 E3FE9E5767CA457FA697980EB637E93B:$RECOVERY: at least one node (0) to recover before lock mastery can begin
Dec 10 13:29:06 vmserver10 kernel: (5344,4):dlm_get_lock_resource:878 E3FE9E5767CA457FA697980EB637E93B: recovery map is not empty, but must master $RECOVERY lock now
Dec 10 13:29:06 vmserver10 kernel: (5344,4):dlm_do_recovery:524 (5344) Node 1 is the Recovery Master for the Dead Node 0 for Domain E3FE9E5767CA457FA697980EB637E93B
Dec 10 13:29:06 vmserver10 kernel: (28412,0):ocfs2_replay_journal:1183 Recovering node 0 from slot 0 on device (8,17)
Dec 10 13:29:09 vmserver10 kernel: kjournald starting. Commit interval 5 seconds
Dec 10 13:29:16 vmserver10 kernel: o2net: accepted connection from node vmserver16.pic.es (num 2) at 193.109.174.116:7777
Dec 10 13:29:20 vmserver10 kernel: ocfs2_dlm: Node 2 joins domain E3FE9E5767CA457FA697980EB637E93B
Dec 10 13:29:20 vmserver10 kernel: ocfs2_dlm: Nodes in domain ("E3FE9E5767CA457FA697980EB637E93B"): 1 2
Dec 10 13:32:08 vmserver10 kernel: o2net: connected to node vmserver15.pic.es (num 0) at 193.109.174.115:7777
Dec 10 13:32:11 vmserver10 kernel: ocfs2_dlm: Node 0 joins domain E3FE9E5767CA457FA697980EB637E93B
Dec 10 13:32:11 vmserver10 kernel: ocfs2_dlm: Nodes in domain ("E3FE9E5767CA457FA697980EB637E93B"): 0 1 2
Dec 10 13:36:10 vmserver10 shutdown[28681]: shutting down for system reboot+
I will investigate what seems to be going on and post it here.
Thanks for your help.
Edited by: Marc Caubet on 11-Dec-2009 02:05
Edited by: Marc Caubet on 11-Dec-2009 02:07

How to place multiple servers in Maintenance Mode in SCOM 2012

Hi All,
I am trying to place multiple servers in Maintenance Mode in SCOM 2012 but i could not find any PoSh script or any other solution to do it.
Could someone help me please?
Regards, Srini

Hi,
Please find the script.
need to have access and you can provide list of servers in text file.
param([int32]$TimeMin, [string]$Reason, [string]$Comment)
$api = new-object -comObject 'MOM.ScriptAPI'
Import-Module operationsmanager
New-SCOMManagementGroupConnection
$Servers = Get-Content "ServerList.txt"
$Time = (Get-Date).Addminutes($TimeMin)
Foreach ($Server in $Servers)
#Get Computer instance
$ComputerClass = Get-SCOMClass -Name Microsoft.Windows.Computer
$ComputerClassInstance = Get-SCOMClassInstance -Class $ComputerClass | Where {$_.DisplayName -eq $Server}
If ($ComputerClassInstance.InMaintenanceMode -eq $true)
#Write-Host $Server " is in maintenance mode"
Write-Host $Server" already under Maintenance Mode or not accessible by SCOM, skipped from script execution" -foregroundcolor "red"
$api.LogScriptEvent('MaintenanceModeScript.ps1', 201, 1, "$Server already found under MM, skipped from script execution")
Else
#Write-Host $Server " is NOT in maintenance mode, attempting to put in MM......."
If ($ComputerClassInstance -ne $Null)
$HealthServiceWatcherClass = Get-SCOMClass -name:Microsoft.SystemCenter.HealthServiceWatcher
#Get Health Service Watcher Class instance of the server
$HSWClass = Get-SCOMClass -Name Microsoft.SystemCenter.HealthServiceWatcher
$HSWClassIns = Get-SCOMClassInstance -Class $HSWClass | Where {$_.DisplayName -eq $Server}
#Starting the maintenance mode
Start-SCOMMaintenanceMode -Instance $HSWClassIns -EndTime $Time -Reason $Reason -Comment
$Comment
Start-SCOMMaintenanceMode -Instance $ComputerClassInstance -EndTime $Time -Reason $Reason
-Comment $Comment
Write-Host "Health Service Watcher and Agent server "$Server " kept in maintenance
mode for $TimeMin minutes" -foregroundcolor "green"
$api.LogScriptEvent('MaintenanceModeScript.ps1', 200, 0, "$Server kept in maintenance
mode for $TimeMin minutes")
Else
Write-Host $Server" not found in Domain" -foregroundcolor "red"
$api.LogScriptEvent('MaintenanceModeScript.ps1', 202, 1, "$Server not found in domain")
Regards
sridhar v

Maintenance Mode Questions

Hi All,
I have SCOM 2012 SP1 CU5 and have few questions on Maintenance Mode (MM).
1) I put my 800 agents in MM for 300 minutes and they get into "Not Monitored" state. When I pull all these 800 agents out of MM together they remain in "Not Monitored" state for about 2 to 3 hours. They get into healthy state slowly
during these 3 hours. Is this expected? Shouldn't they all get healthy quickly? Is the number of agents slows the process and so they take time to get into Healthy State?
2) When I pull my agents out of MM and after they get healthy, I get flood of alerts for them like Health service heartbeat failure, failed to connect to computer., low free disk space, etc. Why is this so as they were in MM and not healthy?
3) When the servers come out of MM, will all the workflows (rules and monitors) run again and generate alerts? even if their alerts (previously created) are already Active in SCOM ?
4) While rebooting the SCOM servers during maintenance activity, what should be the sequence among Management Servers, DB server, DW Server, Reporting Server, Gateway Servers, Web console servers ?
Thanks, S K Agrawal

1) For agent "not monitor" after MM, Try to restarting the System Center Management Health service on the agent-managed computer
2) For What to do for a not monitored state, you can refer below link
http://technet.microsoft.com/en-us/library/hh212723.aspx
3) Yes, after server come out from maintenance mode, it will generate alerts and work normal.
4) Sequence will be DB server, DW Server, Management server, Reporting server, Gateway server, web console Server.
Also you can refer below link for more info.
http://support.microsoft.com/kb/2704170/en-us
http://support.microsoft.com/kb/942866/en-us
Please remember, if you see a post that helped you please click "Vote As Helpful" and if it answered your question, please click "Mark As Answer"
Mai Ali | My blog: Technical | Twitter:
Mai Ali

Set Server Pool Master to Maintenance Mode

Similar Messages

Maybe you are looking for