MDM Server Enrollment Fails with Nginx Proxy

Question

Hello everyone, I'm at my wits end trying to configure nginx as a transparent proxy to my OS X Server (in virtualbox) running our mdm service. Without nginx in the mix the mdm server works properly, and I can performce remote device enrollment and management, but I host multiple servers on the same network, and need to be able to proxy to multiple ips and ports using nginx.
The setup without nginx:
I'm using external DNS for my fqdn, mdm.servername.com, which is correctly resolving to my public ip address. I have my firewall successfully forwarding ports 443, and 1640 to my mdm server at private ip, 10.0.1.60. I have the OS X server DNS disabled, and I'm also using mdm.servername.com as my OS Server's hostname. I set mdm.servername.com to resolve to 127.0.0.1 in /etc/hosts in order to build the open directory, and then I removed this entry from /etc/hosts. Don't know if this is one of the issues. I can access http://mdm.servername.com/mydevices from outside of the network, and enroll my ipad without problems. Everything is working properly until I introduce nginx.
The setup with nginx:
Again using external DNS for my fqdn, mdm.servername.com. I have nginx running on private ip 10.0.1.50, and I'm forwarding ports 443 and 1640 to 10.0.1.50. I have the OS X mdm server running on ip 10.0.1.50. The mdm server has DNS disabled, and the mdm server is also using mdm.servername.com as the OS hostname. I can still access the website portion of mdm (https://mdm.servername.com/mydevices), I can install the Trust Profile without any issues, but when I try to enroll my iPad from outside the network, it successfully "enrolls the certificate", but when it's "Installing Profile" it evenutally times out. I see the progression of api calls in the nginx access logs:
Nginx access.log
173.171.23.81 - - [16/Nov/2013:19:56:47 -0500] "GET /scep/?operation=GetCACert&message=Device%20Management%20Identity%20Certificate HTTP/1.1" 200 1126 "-" "profiled/1.0 CFNetwork/672.0.8 Darwin/14.0.0"
173.171.23.81 - - [16/Nov/2013:19:56:48 -0500] "GET /scep/?operation=GetCACaps&message=Device%20Management%20Identity%20Certificate HTTP/1.1" 200 52 "-" "profiled/1.0 CFNetwork/672.0.8 Darwin/14.0.0"
173.171.23.81 - - [16/Nov/2013:19:56:49 -0500] "POST /scep/?operation=PKIOperation HTTP/1.1" 200 3046 "-" "profiled/1.0 CFNetwork/672.0.8 Darwin/14.0.0"
173.171.23.81 - - [16/Nov/2013:19:58:20 -0500] "PUT /devicemanagement/api/device/mdm_checkin HTTP/1.1" 504 191 "-" "MDM/1.0"
The 504 error on the PUT happens about 30 seconds after the iPad already reports the profile installation failed.
Nginx error.log
2013/11/16 19:58:20 [error] 7983#0: *10 upstream timed out (110: Connection timed out) while reading response header from upstream, client: xxx.xxx.xx.xx, server: mdm.servername.com, request: "PUT /devicemanagement/api/device/mdm_checkin HTTP/1.1", upstream: "https://10.0.1.60:443/devicemanagement/api/device/mdm_checkin", host: "mdm.servername.com"
So nginx is reporting a timeout talking to the mdm server for the PUT checkin action...
On the MDM Server the Apache Logs indicate connection errors also
Apache access_log
mdm.servername.com 10.0.1.50 - - [16/Nov/2013:19:56:47 -0500] "PUT /devicemanagement/api/device/mdm_checkin HTTP/1.0" 403 - "-" "MDM/1.0"
Apache error_log
[Sat Nov 16 19:56:22 2013] [error] (61)Connection refused: proxy: HTTP: attempt to connect to 127.0.0.1:3328 (127.0.0.1) failed
[Sat Nov 16 19:56:22 2013] [error] ap_proxy_connect_backend disabling worker for (127.0.0.1)
[Sat Nov 16 19:56:22 2013] [error] (61)Connection refused: proxy: HTTP: attempt to connect to 127.0.0.1:3329 (127.0.0.1) failed
[Sat Nov 16 19:56:22 2013] [error] ap_proxy_connect_backend disabling worker for (127.0.0.1)
[Sat Nov 16 19:56:22 2013] [error] (61)Connection refused: proxy: HTTP: attempt to connect to 127.0.0.1:3327 (127.0.0.1) failed
[Sat Nov 16 19:56:22 2013] [error] ap_proxy_connect_backend disabling worker for (127.0.0.1)
[Sat Nov 16 19:56:22 2013] [error] (61)Connection refused: proxy: HTTP: attempt to connect to 127.0.0.1:3325 (127.0.0.1) failed
[Sat Nov 16 19:56:22 2013] [error] ap_proxy_connect_backend disabling worker for (127.0.0.1)
[Sat Nov 16 19:56:22 2013] [error] (61)Connection refused: proxy: HTTP: attempt to connect to 127.0.0.1:3326 (127.0.0.1) failed
[Sat Nov 16 19:56:22 2013] [error] ap_proxy_connect_backend disabling worker for (127.0.0.1)
[Sat Nov 16 19:56:22 2013] [error] (61)Connection refused: proxy: HTTP: attempt to connect to 127.0.0.1:3324 (127.0.0.1) failed
[Sat Nov 16 19:56:22 2013] [error] ap_proxy_connect_backend disabling worker for (127.0.0.1)
[Sat Nov 16 19:56:22 2013] [error] (61)Connection refused: proxy: HTTP: attempt to connect to 127.0.0.1:3322 (127.0.0.1) failed
[Sat Nov 16 19:56:22 2013] [error] ap_proxy_connect_backend disabling worker for (127.0.0.1)
[Sat Nov 16 19:56:22 2013] [error] (61)Connection refused: proxy: HTTP: attempt to connect to 127.0.0.1:3323 (127.0.0.1) failed
[Sat Nov 16 19:56:22 2013] [error] ap_proxy_connect_backend disabling worker for (127.0.0.1)
[Sat Nov 16 19:58:17 2013] [error] [client 10.0.1.50] Re-negotiation handshake failed: Not accepted by client!?
I'm assuming I have something misconfigured with nginx, but I'm not smart enough to put all these pieces together for a solution...Hopefully one you more capaple guys or gals can help me out...
Nginx Config
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
        worker_connections 768;
http {
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 65;
  types_hash_max_size 4096;
  default_type application/octet-stream;
  ssl_certificate     /etc/nginx/ssl/servername.com/_.servername.com.combined.crt;
  ssl_certificate_key /etc/nginx/ssl/servername.com/_.servername.com.key;
  access_log /var/log/nginx/access.log;
  error_log /var/log/nginx/error.log;
  gzip on;
  gzip_disable "msie6";
  server {
    listen        443 ssl;
    listen        1640;
    server_name   mdm.servername.com;
    location / {
      proxy_pass      $scheme://10.0.1.60:$server_port$request_uri;
      sendfile off;
      proxy_set_header   Host                   $host;
      proxy_set_header   X-Real-IP            $remote_addr;
      proxy_set_header   X-Forwarded-For  $proxy_add_x_forwarded_for;
      proxy_max_temp_file_size 0;
      #this is the maximum upload size
      client_max_body_size       10m;
      client_body_buffer_size    128k;
      proxy_connect_timeout      90;
      proxy_send_timeout         90;
      proxy_read_timeout         90;
      proxy_buffer_size          4k;
      proxy_buffers              4 32k;
      proxy_busy_buffers_size    64k;
      proxy_temp_file_write_size 64k;
Additional info:
I have a wildcard ssl cert from go daddy that nginx is using for ssl, and I'm also using this for the websites server on OS X Server. I'm using the intermediate cert that Server generates for signing the configuration profile.
Thanks in advance for any help you can provide.

Answer 1

Thanks for the input Stephen! I am having trouble recieving application pushes on my guest network, I'm going to try and open the ports like you suggested.
Opening ports 443, and 1640 are definitely a must on the MDM server, or enrollment from the guest network will not succeed.
I noticed a lot of action in the /Library/Logs/ProfileManager/php.log on my MDM server, whenever I bypass nginx, and successfully enroll, so I'm guessing there needs to by some fastcgi proxying in the nginx config block:
1::Nov 18 10:52:55.329 [30721]  {require_once (mdm_enroll.php:11)} vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv - POST mdm_enroll
1::Nov 18 10:53:00.499 [30721]  {GetMDMACLFromUserAgentHeader (mdm_enroll.php:40)} iOS version 7.0
1::Nov 18 10:53:06.400 [30721]  {SendFinalOutput (mdm_enroll.php:69)} Sent Final Output (10933 bytes)
1::Nov 18 10:53:06.400 [30721]  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - /devicemanagement/mdm/mdm_enroll
0::Nov 18 10:53:06.419 [30721]  {SendFinalOutput (mdm_enroll.php:69)} Completed in 11303ms | 200 OK [https://mdm.servername.com/devicemanagement/mdm/mdm_enroll]
1::Nov 18 10:53:22.673 [30722]  {LogElapsedTime (common.php:74)} Time since script start: 441us [https://mdm.servername.com/devicemanagement/api/device/mdm_checkin]
1::Nov 18 10:53:22.720 [30722]  {require_once (mdm_checkin.php:11)} vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv - PUT mdm_checkin
0::Nov 18 10:53:22.751 [30722]  checkin: 'Authenticate'
1::Nov 18 10:53:23.046 [30722]  {SendFinalOutput (mdm_checkin.php:145)} Sent Final Output (0 bytes)
1::Nov 18 10:53:23.046 [30722]  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - /devicemanagement/mdm/mdm_checkin
0::Nov 18 10:53:23.046 [30722]  {SendFinalOutput (mdm_checkin.php:145)} Completed in 373ms | 200 OK [https://mdm.servername.com/devicemanagement/api/device/mdm_checkin]
1::Nov 18 10:53:28.220 [32005]  {LogElapsedTime (common.php:74)} Time since script start: 476us [https://mdm.servername.com/devicemanagement/api/device/mdm_checkin]
1::Nov 18 10:53:28.220 [32005]  {require_once (mdm_checkin.php:11)} vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv - PUT mdm_checkin
0::Nov 18 10:53:28.220 [32005]  checkin: 'TokenUpdate'
1::Nov 18 10:53:28.225 [32005]  {Target_for_incoming_request (target.php:210)} Found target iOS: <''[10](OWNER:User[9])>1::Nov 18 10:53:28.501 [32005]  {SendFinalOutput (mdm_checkin.php:145)} Sent Final Output (0 bytes)
1::Nov 18 10:53:28.501 [32005]  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - /devicemanagement/mdm/mdm_checkin
0::Nov 18 10:53:28.501 [32005]  {SendFinalOutput (mdm_checkin.php:145)} Completed in 281ms | 200 OK [https://mdm.servername.com/devicemanagement/api/device/mdm_checkin]
I tried adding the following location section into the config, but still no luck:
  location /devicemanagement/api/device/mdm_checkin {
    fastcgi_pass 10.0.1.49:$server_port;
    fastcgi_index index.php;
    include fastcgi_params;