Wiki won't accept OD user logins after upgrading to server 3.1.1

Question

I have an OS X server 3.x running on OS X 10.9.3. I have several WiKi users, most are OpenDirectory (neiwork) users and one is a 'local' user. Prior to the weekend I was running Server 3.0.3 and all was good. Since uograding to Server 3.1.1 none of the network/OD users can log in to their Wikis. The local user can log in just fine. The network users can log in to all their other servcies (Mail, Calendar,Contacts, SMB/AFP shares etc.) just fine. They can also log into my regular SSL web-site which requires passwords. Only the wiki login is broken.
In the collabd.dlog file I see:
May 21 19:28:20 anarion.home.thejenkinsfamily.org.uk collabd[3747] <Error>: [CSAuthService.m:316 5b04000 +0ms] Digest did not validate
May 21 19:28:20 anarion.home.thejenkinsfamily.org.uk collabd[3747] <Error>: [CSServiceDispatcher.m:260 5b04000 +0ms] Caught exception "Invalid Credentials" [CSAuthBadDigest] executing [http]Request{AuthService.validateUsernameAndPasswordDigest:remember:(<<scrubbe d>>)}:
followed by an exception stack trace.
But of course the credential are valid.
Anyone seen this? Any ideas on how to fix it? Might it be fixed in the 3.1.2 update that came out a few days ago (I have not updated to that as yet)?
Thanks,
Chris

Answer 1

Okay, this problem has re-surfaced (on server 3.1.2). Here are the symptoms:
1.    It only affects network (Open Directory) users. It affects all network users.
2.    It only affects the Wiki service; OD users can authenticate for all other services (Mail, Contacts, Calendar, VPN, Time Machine, OS X login on client machines) without problem.
3.    Stopping and restarting the Wiki Servcie and the OD servcie does not fix the problem, but rebooting the server does. Things will then be fine for a while (I have not yet quantified how long 'a while' is) and then the problems occurs again.
4.    When the problem is present, even a newly created user cannot login to the Wiki servcie.
5.    When the login failure occurs, the collabd.log file contains the following:
Jun  1 09:52:23 anarion.home.thejenkinsfamily.org.uk collabd[73366] <Info>: [CSServiceDispatcher.m:237 b904000 +10979ms] TIMER: 0ms ---> Executed [http]Request{SettingsService.authSettings()}
Jun  1 09:52:23 anarion.home.thejenkinsfamily.org.uk collabd[73366] <Info>: [CSServiceDispatcher.m:237 b904000 +0ms] TIMER: 1ms ---> Executed [http]Request{AuthService.challengeForUsername:advanced:(newtesty,1)}
Jun  1 09:52:23 anarion.home.thejenkinsfamily.org.uk collabd[73366] <Info>: [CSExecutionTimer.m:14 b904000 +0ms] TIMER: 2ms ---> Executed batch BatchRequest{(
            "[http]Request{AuthService.challengeForUsername:advanced:(newtesty,1)}"
Jun  1 09:52:23 anarion.home.thejenkinsfamily.org.uk collabd[73366] <Info>: [CSServiceDispatcher.m:237 b904000 +119ms] TIMER: 0ms ---> Executed [http]Request{SettingsService.authSettings()}
Jun  1 09:52:23 anarion.home.thejenkinsfamily.org.uk collabd[73366] <Error>: [CSAuthService.m:316 b904000 +0ms] Digest did not validate
Jun  1 09:52:23 anarion.home.thejenkinsfamily.org.uk collabd[73366] <Error>: [CSServiceDispatcher.m:260 b904000 +0ms] Caught exception "Invalid Credentials" [CSAuthBadDigest] executing [http]Request{AuthService.validateUsernameAndPasswordDigest:remember:(<<scrubbe d>>)}:
                0   CoreFoundation                      0x00007fff920e325c __exceptionPreprocess + 172
                1   libobjc.A.dylib                     0x00007fff95709e75 objc_exception_throw + 43
                2   CSService                           0x000000010897ca2e -[CSAuthService sessionForDigest:remember:] + 1583
                3   CSService                           0x000000010897c3a7 -[CSAuthService validateUsernameAndPasswordDigest:remember:] + 65
                4   CoreFoundation                      0x00007fff91fcea5c __invoking___ + 140
                5   CoreFoundation                      0x00007fff91fce8c4 -[NSInvocation invoke] + 308
                6   CSService                           0x00000001088eba40 -[CSServiceDispatcher executeRequest:asPartOfBatch:usingServiceImpl:] + 4589
                7   CSService                           0x00000001088ec4dc __43-[CSServiceDispatcher executeBatchRequest:]_block_invoke_3 + 83
                8   CSService                           0x00000001088f1df0 -[NSArray(CollabBlockMethods) map:] + 233
                9   CSService                           0x00000001088ec435 __43-[CSServiceDispatcher executeBatchRequest:]_block_invoke_2 + 160
                10  CSService                           0x00000001088f2490 +[CSExecutionTimer recordTime:ofBlock:] + 78
                11  CSService                           0x00000001088f22c2 +[CSExecutionTimer timerNamed:aroundBlock:] + 77
                12  CSService                           0x00000001088ec1dd __43-[CSServiceDispatcher executeBatchRequest:]_block_invoke + 296
                13  PostgreSQLClient                    0x0000000108855941 -[PGCConnection transactionInBlock:onError:] + 150
                14  CSService                           0x00000001088ec03b -[CSServiceDispatcher executeBatchRequest:] + 285
                15  CSService                           0x000000010896f748 +[CSServiceDispatchHTTPRouter routeServiceRequest:response:] + 999
                16  CSService                           0x00000001088f2d1f __21-[CSServiceBase init]_block_invoke_6 + 48
                17  CSService                           0x000000010896c81a __53-[CSRoutingHTTPConnection httpResponseForMethod:URI:]_block_invoke + 95
                18  CSService                           0x000000010896fd6c -[CSHTTPBackgroundResponse bounce:] + 286
                19  Foundation                          0x00007fff91a2276b __NSThread__main__ + 1318
                20  libsystem_pthread.dylib             0x00007fff98b0f899 _pthread_body + 138
                21  libsystem_pthread.dylib             0x00007fff98b0f72a _pthread_struct_init + 0
                22  libsystem_pthread.dylib             0x00007fff98b13fc9 thread_start + 13
Jun  1 09:52:23 anarion.home.thejenkinsfamily.org.uk collabd[73366] <Info>: [CSExecutionTimer.m:14 b904000 +0ms] TIMER: 3ms ---> Executed batch BatchRequest{(
            "[http]Request{AuthService.validateUsernameAndPasswordDigest:remember:(<<scrubb ed>>)}"
Anyone have any suggestions before I log a bug with Apple?