Migrate wiki from MLS to MAVS

Question

Dear Folks,
I want to provide a solution on how to migrate the Wiki from a Mountain Lion Server to a Mavericks Server. Both Servers are running but in different locations and they can't communicate directly with each other. Upgrading the MLS was no option - after what I had read in hundreds of threads. The MLS is a 10.8.5 (with the latest version of server.app up until the 15th of October 2013) and Mavericks a 10.9.1 (with the latest version of server.app up until New Years Eve).
I hadn't done it without the help of Andreas from Metalab in Vienna (lovely Hackspace) - so the grace goes to him!!
First things first - he isolated 2 issues on MLS - revolving postgresql Version AND Socket! Thank you Apple-developers you did a great job here - since this product is called „The Server for everyone“ I believe either the title is incorrect or some guys haven’t understood what the a server for everyone implies - since everyone isn’t a geek, nerd, developer etc. - think about it!
Trying to connect to psql on MLS:
bash-3.2# psql
psql: could not connect to server: No such file or directory
          Is the server running locally and accepting
          connections on Unix domain socket "/var/pgsql_socket/.s.PGSQL.5432"?
What?!
bash-3.2# serveradmin status postgres
postgres:state = "RUNNING"
What?!
So he started digging on MLS:
bash-3.2# netstat -na|grep LISTEN
nothing tangible here - nothing ist listening to 5432
Dig deeper on MLS:
bash-3.2# lsof | grep postgre
          #output omitted for clarity
postgres_ 2546      _postgres   10u     unix 0xa319efaa0af41357       0t0         /Library/Server/PostgreSQL For Server Services/Socket/.s.PGSQL.5432
postgres_ 2547      _postgres   10u     unix 0xa319efaa0af3fbe7       0t0         /Library/Server/PostgreSQL For Server Services/Socket/.s.PGSQL.5432
          #output omitted for clarity
postgres_ 2548      _postgres   10u     unix 0xa319efaa0af3fa57       0t0         /Library/Server/PostgreSQL For Server Services/Socket/.s.PGSQL.5432
          #output omitted for clarity
postgres_ 2549      _postgres   10u     unix 0xa319efaa0af3f8c7       0t0         /Library/Server/PostgreSQL For Server Services/Socket/.s.PGSQL.5432
          #output omitted for clarity
postgres_ 3102      _postgres   10u     unix 0xa319efaa0b220357       0t0         /Library/Server/PostgreSQL For Server Services/Socket/.s.PGSQL.5432
          #output omitted for clarity
Hold the horses still - that got him thinking! How does that fit to the output of the psql-command just seconds earlier (and I literally mean seconds!) Can you see it?
The truth is, the Socket is at:
/Library/Server/PostgreSQL For Server Services/Socket/.s.PGSQL.5432
but is expected by psql to be in /var/psql_socket/.s.PGSQL.5432!
Expected What?!
Verified it again on MLS:
bash-3.2# psql
psql: could not connect to server: No such file or directory
          Is the server running locally and accepting
          connections on Unix domain socket "/var/pgsql_socket/.s.PGSQL.5432"?
bash-3.2# psql --help
Yep - when postgresql was compiled obviously somebody was sitting on his fingers - during implementation nobody had told psql the path for the socket at the "new" location?! Ergo - psql is looking for the socket in the default location - where it is not.
Then, with the help, he managed it to get in.
Trying to get in on MLS:
bash-3.2# psql -p "/Library/Server/PostgreSQL For Server Services/Socket/.s.PGSQL.5432"
psql: invalid port number: "/Library/Server/PostgreSQL For Server Services/Socket/.s.PGSQL.5432"
Trying again on MLS:
bash-3.2# psql -h "/Library/Server/PostgreSQL For Server Services/Socket/.s.PGSQL.5432"
psql: could not connect to server: Not a directory
          Is the server running locally and accepting
          connections on Unix domain socket "/Library/Server/PostgreSQL For Server Services/Socket/.s.PGSQL.5432/.s.PGSQL.5432"?
And again - almost there on MLS:
bash-3.2# psql -h "/Library/Server/PostgreSQL For Server Services/Socket/" -p 5432
psql: FATAL:  role "root" does not exist
Typo:
bash-3.2# psql -h "/Library/Server/PostgreSQL For Server Services/Socket/" -p 5432 -u colab
psql: invalid option -- u
Try "psql --help" for more information.
And - tata - he was in on MLS:
bash-3.2# psql -h "/Library/Server/PostgreSQL For Server Services/Socket/" -p 5432 template1 collab
psql (9.1.9, server 9.2.4)
WARNING: psql version 9.1, server version 9.2.
         Some psql features might not work.
Type "help" for help.
template1=#
Lets find the databases and roles on MLS:
template1=# \l
                                   List of databases
       Name        |   Owner    | Encoding | Collate | Ctype |    Access privileges   
-------------------+------------+----------+---------+-------+------------------ -------
caldav            | caldav     | UTF8     | C       | C     |
collab            | collab     | UTF8     | C       | C     |
device_management | _devicemgr | UTF8     | C       | C     |
postgres          | _postgres  | UTF8     | C       | C     |
template0         | _postgres  | UTF8     | C       | C     | =c/_postgres           +
                   |            |          |         |       | _postgres=CTc/_postgres
template1         | _postgres  | UTF8     | C       | C     | _postgres=CTc/_postgres+
                   |            |          |         |       | =c/_postgres
webauth           | webauth    | UTF8     | C       | C     |
(7 rows)
And there are the roles on MLS:
template1=# \du
                              List of roles
Role name  |                   Attributes                   | Member of
------------+------------------------------------------------+-----------
_devicemgr | Create DB                                      | {}
_postgres  | Superuser, Create role, Create DB, Replication | {}
caldav     | Create DB                                      | {}
collab     | Superuser, Create role, Create DB              | {}
webauth    | Superuser, Create role, Create DB              | {}
Aaaaah - so there is a role collab and a database collab! Lets connect to the "right" database and dive into it on MLS.
template1=# \q
bash-3.2# psql -h "/Library/Server/PostgreSQL For Server Services/Socket/" -p 5432 collab collab
psql (9.1.9, server 9.2.4)
WARNING: psql version 9.1, server version 9.2.
         Some psql features might not work.
Type "help" for help.
collab=# \d
                   List of relations
Schema |            Name             | Type  | Owner 
--------+-----------------------------+-------+--------
public | blog_entity                 | table | collab
public | document_entity             | table | collab
public | entity                      | table | collab
public | entity_acls                 | table | collab
public | entity_acls_defaults        | table | collab
public | entity_attrs                | table | collab
public | entity_changesets           | table | collab
public | entity_comment              | table | collab
public | entity_lock                 | table | collab
public | entity_preview              | table | collab
public | entity_private_attrs        | table | collab
public | entity_tag                  | table | collab
public | entity_type                 | table | collab
public | file_entity                 | table | collab
public | filedata_entity             | table | collab
public | filename_reservation        | table | collab
public | global_settings             | table | collab
public | groups                      | table | collab
public | migration_entity            | table | collab
public | migration_status            | table | collab
public | migrationplaceholder_entity | table | collab
public | notification                | table | collab
public | page_entity                 | table | collab
public | podcast_entity              | table | collab
public | podcast_episode_entity      | table | collab
public | preview_queue               | table | collab
public | project_entity              | table | collab
public | relationship                | table | collab
public | savedquery_entity           | table | collab
public | search_index                | table | collab
public | search_stat                 | table | collab
public | session                     | table | collab
public | subscription                | table | collab
public | user_activity               | table | collab
public | user_entity                 | table | collab
public | user_entity_favorites       | table | collab
public | user_entity_read_status     | table | collab
public | user_entity_unread_status   | table | collab
public | user_entity_updates         | table | collab
public | user_entity_watched         | table | collab
public | user_readall_time           | table | collab
(41 rows)
That looks fantastic! Lets start dumping with some help through pg_dump --help and then issuing the following on MLS:
bash-3.2# pg_dump -h "/Library/Server/PostgreSQL For Server Services/Socket/" -p 5432 -f /Volumes/USBSTICK/wikidatabase -U collab collab
pg_dump: server version: 9.2.4; pg_dump version: 9.1.9
pg_dump: aborting because of server version mismatch
What?! How could that be?! The commands refer to an older version?!
Start digging again on MLS!
bash-3.2# which psql
/usr/bin/psql
bash-3.2# which pg_dump
/usr/bin/pg_dump
That's where I stepped in and told Andreas "Hey - ahm - I know of a path completely different…. /Applications/Server.app/.….". Thanks mate!
Dig again on MLS!
bash-3.2# /Applications/Server.app/Contents/ServerRoot/usr/bin/pg_dump --version
pg_dump (PostgreSQL) 9.2.4
Lovely - that one looks good - lets use it. All of that got us finally to the right command on MLS.
bash-3.2# /Applications/Server.app/Contents/ServerRoot/usr/bin/pg_dump -h "/Library/Server/PostgreSQL For Server Services/Socket/" -p 5432 -f /Volumes/USBSTICK/wikidatabase -U collab collab
There it is - the whole Database on the stick - finally - it took Andreas only 5 Minutes - approximately? I am just baffled.
Then I copied the Folders from the /Library/Server/Wiki/FileData of the MLS to the exact same location on the MAVS and sat permission - navigating to /Library/Server/Wiki first:
bash-3.2# chown -R _teamsserver:_teamsserver FileData/
bash-3.2# chmod -R +a "_www allow read" FileData/
I compared the the collab DB on the MLS with the MAVS collab DB to see the differences - BTW - the Socket has moved again on the MLS! Move up in this documentary and you’ll see on how to find it on MAVS.
bash-3.2# lsof | grep 5432 
          #output omitted for clarity
postgres_   382    _teamsserver    3u     unix 0x399cdd6eeea1efe1       0t0         /Library/Server/Wiki/PostgresSocket/.s.PGSQL.5432
          #output omitted for clarity
bash-3.2# psql -h "/Library/Server/Wiki/PostgresSocket/" -p 5432 collab collab
psql (9.2.4)
Type "help" for help.
collab=# \d
                   List of relations
Schema |            Name             | Type  | Owner 
--------+-----------------------------+-------+--------
public | adc_device_entity           | table | collab
public | adc_team_entity             | table | collab
public | blog_entity                 | table | collab
public | bot_entity                  | table | collab
public | botgroup_entity             | table | collab
public | botrun_entity               | table | collab
public | build_agent_registry        | table | collab
public | document_entity             | table | collab
public | email_notification          | table | collab
public | entity                      | table | collab
public | entity_acls                 | table | collab
public | entity_acls_defaults        | table | collab
public | entity_attrs                | table | collab
public | entity_auditlog             | table | collab
public | entity_changesets           | table | collab
public | entity_comment              | table | collab
public | entity_lock                 | table | collab
public | entity_preview              | table | collab
public | entity_private_attrs        | table | collab
public | entity_tag                  | table | collab
public | entity_type                 | table | collab
public | file_entity                 | table | collab
public | filedata_entity             | table | collab
public | filename_reservation        | table | collab
public | global_settings             | table | collab
public | groups                      | table | collab
public | migration_entity            | table | collab
public | migration_status            | table | collab
public | migrationplaceholder_entity | table | collab
public | page_entity                 | table | collab
public | podcast_entity              | table | collab
public | podcast_episode_entity      | table | collab
public | preview_queue               | table | collab
public | relationship                | table | collab
public | savedquery_entity           | table | collab
public | scm_commit_entity           | table | collab
public | scm_server                  | table | collab
public | scmrepogroup_entity         | table | collab
public | search_index                | table | collab
public | search_stat                 | table | collab
public | session                     | table | collab
public | subscription                | table | collab
public | timeseries                  | table | collab
public | timeseries_toc              | table | collab
public | user_activity               | table | collab
public | user_entity                 | table | collab
public | user_entity_favorites       | table | collab
public | user_entity_read_status     | table | collab
public | user_entity_unread_status   | table | collab
public | user_entity_updates         | table | collab
public | user_entity_watched         | table | collab
public | user_readall_time           | table | collab
public | visible_entity_tag          | view  | collab
public | wiki_entity                 | table | collab
public | work_queue                  | table | collab
public | work_schedule               | table | collab
public | work_schedule_recurrence    | table | collab
public | work_schedule_status        | table | collab
(58 rows)
Hmmm - 41 tables in the old, and 58 in the new. What am I going to do? To be able to roll back I dumped the MAVS DB before I would do anything else.
I had to decide between trying to take the old MLS DB and just pg_restore it into the MAVS DB, or - trying to upgrade the MLS instance to MAVS. I was afraid doing the upgrade since the MLS was a mess. At the end I would decide to go for the latter - since I had dumps. It took a while but - hey - it worked. Everything in place! Wiki running - everything else, too! The database has now 58 tables - ok - lets dump it. Notice again - no need to specify the proper version of pg_dump, but the Socket has changed from MLS to MAVS - again - you could see this when I connected with psql as well!
Please use (-F c) compression when dumping - otherwise you’ll receive a nasty error when importing it at the MAVS.
bash-3.2# pg_dump -h "/Library/Server/Wiki/PostgresSocket/" -p 5432 -F c -f /Volumes/USBSTICK/MLSafterupgrade2MAVS_collab_db_compressed.pgdump -U collab collab
I then copied the dump to the server - the Desktop of the administrativ account - I did it through screensharing which is accessible after establishing a VPN to the location the server resides in.
Unfortunately the command for pg_restore has a different syntax then the pg_dump. Use pg_dump --help to see the details. I’ll explain it quickly:
-c is to clean entries in the target tables (you must not use capital c)
-d specifies the database into which you would like to restore
-h specifies (again) where the DB is - the socket
-p the port to use
-U is the role that likes to work on the DB
you must not use -f since it’s not needed
bash-3.2# pg_restore -c -d collab -h "/Library/Server/Wiki/PostgresSocket/" -p 5432 /Volumes/OSXDATA/Users/macminiadmin/Desktop/MLScollabdb_compressed.pgdump -U collab
pg_restore: [archiver (db)] Error while PROCESSING TOC:
pg_restore: [archiver (db)] Error from TOC entry 2466; 2605 16639 CAST CAST (text[] AS public.hstore)
pg_restore: [archiver (db)] could not execute query: ERROR:  cannot drop cast from text[] to public.hstore because extension hstore requires it
HINT:  You can drop extension hstore instead.
    Command was: DROP CAST (text[] AS public.hstore);
pg_restore: [archiver (db)] Error from TOC entry 1816; 2616 16636 OPERATOR CLASS hash_hstore_ops collab
pg_restore: [archiver (db)] could not execute query: ERROR:  cannot drop operator class hash_hstore_ops for access method hash because extension hstore requires it
HINT:  You can drop extension hstore instead.
    Command was: DROP OPERATOR CLASS public.hash_hstore_ops USING hash;
          #output omitted for clarity
I saw quite a lot errors (some revolving functions etc.) - however, it worked. Wikis are present again on the MAVS. Not without the initial help of Andreas - thank you once again!
I would like to recommend to everyone joining hackerspaces in their location. I do not know them in other areas - I can only speak for the metalab in Vienna. Fantastic place with fantastic, very knowledgable people. I would also recommend reading a book like PostgreSQL 9 Admin Cookbook - this helped me quit a lot as well. It’s available through various stores - virtual or real - choose what you prefer.

Answer 1

Frustrated, not angry...
Well folks. I think this is goodbye to OS X Server - the server that I thought was much easier to use.  I've followed instructions on this forum and several others over the past 7 months.  It's cost me so much time and headache.
Migrating wikis from Mountain Lion Server to Mavericks server has yet to work
Upgrading our Mountain Lion Server to Mavericks leaves us without wiki, calendar, and other services
Apple support docs don't do the trick
Phone calls with Apple support techs haven't provided any solutions.
Our current Mountain Lion Server suddenly lost all filesharing authentication capability and Apple enterprise solutions haven't been able to figure this one out.
I'm stuck with a server with broken mountain lion server I have never been able to upgrade and/or migrate and ultimately would be left to copy/paste wikis to a fresh Mavericks server, export/import calendars from each client station, and contacts - I don't even know how that's going to work.
I believe in keeping IT folks employed, but...
The support from Apple seems so marginal (just for the server) and the plethora of problems with upgrading to Mavericks server and/or the manual fight to only potentially get it done seems very un-Apple like and just downright counter to what Apple represents.
So I'll think different...
Our server will be relegated to a client machine with simple file sharing.  Perhaps We'll buy Daylite to assume some of the responsibilites.  Truth is, even when things were working, it was always flaky in some way.
I really wanted to employ OS X Server in an efficient way, but it's been more trouble than I can accomodate.
Fortunately I am not an average client user nor a novice, so working in terminal is comfortable.
Perhaps in a few years or so, I'll reconsider, but my experience with OS X server on Mountain Lion and Mavericks has affected my trust.
Thanks for all of the help to those who provided instructions, workarounds, directions toward solutions, etc.
For those of you who had a successful migration - consider yourself lucky.
...by the way...after finishing the incredible instructions on this page (and i sincerely mean that despite them not working for me) the result was a wiki page that looked like this
Caught exception "[<CSEntityPlaceholder 0x7fb2d15d5910> valueForUndefinedKey:]: this class is not key value coding-compliant for the key externalID." [NSUnknownKeyException] executing route /app-context/wiki/:
0   CoreFoundation                      0x00007fff8a0fe25c __exceptionPreprocess + 172
1   libobjc.A.dylib                     0x00007fff8da20e75 objc_exception_throw + 43
2   CSService                           0x0000000104bc7957 -[CSLocalServiceProxy forwardInvocation:] + 1229
3   CoreFoundation                      0x00007fff8a05c1c4 ___forwarding___ + 452
4   CoreFoundation                      0x00007fff8a05bf78 _CF_forwarding_prep_0 + 120
5   CSService                           0x0000000104bf9573 __27-[CSAppContextService init]_block_invoke240 + 180
6   CSService                           0x0000000104bdf81a __53-[CSRoutingHTTPConnection httpResponseForMethod:URI:]_block_invoke + 95
7   CSService                           0x0000000104be2d6c -[CSHTTPBackgroundResponse bounce:] + 286
8   Foundation                          0x00007fff9073976b __NSThread__main__ + 1318
9   libsystem_pthread.dylib             0x00007fff8bc58899 _pthread_body + 138
10  libsystem_pthread.dylib             0x00007fff8bc5872a _pthread_struct_init + 0
11  libsystem_pthread.dylib             0x00007fff8bc5cfc9 thread_start + 13