Consistent PageServerChildCrash

Question

Hello SAP,
We have a reporting dashboard in place that serves up reports to users on an on-demand viewing basis.
Reports are built and tested in Crystal Reports 2011 and uploaded to Crystal Server 2011 (BI 4.x platform).
The dashboard (built with the BI .NET SDK) serves up requests by querying the Crystal Reports 2011 Processing server with varied parameters as determined by the report being viewed.
Now the issue that we are noting is that Crystal Server machine is frequently throwing a PageServerChildCrash in the event viewer.
With this detail -> ( The description for Event ID 0 from source PageServerChildCrash cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer. )
This is generally followed by "A failure occurred while the server was processing report file X:XXXX (RCIRAS0568)" and
"A subprocess in the processing serer was forced to terminate. (RCIRAS0622)" by the BusinessObjects_crproc (DocumentProcessingServer)
and/or "An error occurred while creating a subprocess in the processing server. (RCIRAS0604)" by the BusinessObjects_crcache(DocumentProcessingServer)
The logs for the processing server simply show :
trc file: "cr2011proc_LMCR1.CrystalReports2011ProcessingServer_child_CRPE9_ncs.trc", trc level: 1, release: "720"
M 
M [Thr 10856] Tue Nov 19 17:12:29 2013
M  [Thr 10856] ***LOG Q0I=> NiPGetHostByName: '$SMDAgentHost' not found: getaddrinfo [ninti.c 895]
M  [Thr 10856] *** ERROR => Main agent failed to be initilized due to incorrect host setting! [ncsmtdatasen 20]
M  [Thr 10856] *** ERROR => Main agent failed to be initilized due to incorrect host setting! [ncsmtdatasen 20]
M  [Thr 10856] NCS trace timer is disabled since the interval is set to 0
M 
M [Thr 10856] Tue Nov 19 17:12:30 2013
M  [Thr 10856] NCS data timer is disabled since the interval is set to 0
M  [Thr 10856] NCS library version 2.3.9 (unicode) loaded
M  [Thr 10856] NCS_ProcInit API invoked
Searching on here suggested that this was a standard issue, although I'd like to know what's causing this as well.
What this results in is that at times the report wouldn't display and the call to load a report would load indefinitely (or finally timeout after a long while). Looking at the metrics for the Crystal Reports 2011 processing server in Central Management Console, I note that when the reports don't load (or aren't going to load), The columns User & Last Accessed remain blank (Processing Server Metrics section under Open Jobs).  The Document ID and Document Name columns are correctly filed out and the Job Lifetime begins it's count.
This issue is not bound to any one report and would load one properly for some time or a whole bunch in a steady stream and then stop for some or all.
Sometimes a PageServerChildCrash is noted in the eventviewer and the report still loads.
If an error is thrown to the user on the viewer another attempt can usually be made by simply refreshing or trying the call again.
Otherwise this is only resolved by restarting the Processing Server and then trying again.
At this point the Reporting dashboard is just unstable and I'm not sure where else to look. We have upgraded the machines processor count, clock speed and memory with some performance increase but not much in terms of this issue.
Any assistance would be very much appreciated.
Sincerely,

Answer 1

Thanks for your response Ludek.
The above function populates paramFields based on the loaded report type, Details were collapsed in for brevity, but the expanded content looks like this (reportType is actually retrieved from the request and is not technically "passed" into the function)
        /// <summary>
        /// Sets up the provided parameterfields object and saves to session
        /// </summary>
        /// <param name="paramFields">Parameter fields to be populated</param>
        /// <returns>True if succesfully assigned; false otherwise</returns>
        private bool AssignReportParameters(ParameterFields paramFields)
            bool success = ReportParams != null;
            if (success)
                if (ReportParams.ReportType == "1" || ReportParams.ReportType == "2" || ReportParams.ReportType == "4")
                    AddParamFields(paramFields, "STORE", Session["storeParam"]);
                    if (ReportParams.ReportType == "2")
                        AddParamFields(paramFields, "FromDate", ReportParams.FromDate);
                        AddParamFields(paramFields, "ToDate", ReportParams.ToDate);
                else if (ReportParams.ReportType == "3")
                    AddParamFields(paramFields, "SKU", ReportParams.SKU);
                else
                    //Unknown report type alert
                    success = false;
                Session["CRVParamFields"] = success ? paramFields : null;
            return success;
        private void AddParamFields(ParameterFields paramFields, String field, object value)
            if (paramFields != null)
                paramFields.Add(field, ParameterValueKind.StringParameter, DiscreteOrRangeKind.DiscreteValue);
                ParameterDiscreteValue param = new ParameterDiscreteValue() { Value = value };
                paramFields[field].CurrentValues.Add(param);
                paramFields[field].HasCurrentValue = true;
            else
                throw new Exception("The provided parameterfields is null");
All the parameters are passed in as string fields. This has been confirmed to work and all the reports have successfully loaded this way pre and post production, this is as the date parameters passed in are properly formatted date strings. I have noted the parameter kind exception in the past before at the start of the project and these instantly fail and the exception handler catches it.
All the same at your hinting I would update the code to pass in date objects and specify the kind to keep things proper. I know this to work as well.
Updated code >>
        /// <summary>
        /// Sets up the provided parameterfields object and saves to session
        /// </summary>
        /// <param name="paramFields">Parameter fields to be populated</param>
        /// <returns>True if succesfully assigned; false otherwise</returns>
        private bool AssignReportParameters(ParameterFields paramFields)
            bool success = ReportParams != null;
            if (success)
                if (ReportParams.ReportType == "1" || ReportParams.ReportType == "2" || ReportParams.ReportType == "4")
                    AddParamFields(paramFields, "STORE", Session["storeParam"], ParameterValueKind.StringParameter);
                    if (ReportParams.ReportType == "2")
                        AddParamFields(paramFields, "FromDate", DateTime.Parse(ReportParams.FromDate), ParameterValueKind.DateParameter);
                        AddParamFields(paramFields, "ToDate", DateTime.Parse(ReportParams.ToDate), ParameterValueKind.DateParameter);
                else if (ReportParams.ReportType == "3")
                    AddParamFields(paramFields, "SKU", ReportParams.SKU, ParameterValueKind.StringParameter);
                else
                    //Unknown report type alert
                    success = false;
                Session["CRVParamFields"] = success ? paramFields : null;
            return success;
        private void AddParamFields(ParameterFields paramFields, String field, object value, ParameterValueKind valueKind)
            if (paramFields != null)
                paramFields.Add(field, valueKind, DiscreteOrRangeKind.DiscreteValue);
                ParameterDiscreteValue param = new ParameterDiscreteValue() { Value = value };
                paramFields[field].CurrentValues.Add(param);
                paramFields[field].HasCurrentValue = true;
            else
                throw new Exception("The provided parameterfields is null");
The issue we are having is intermittent but happens often enough to be noticeable. It also occurs whether the report uses date parameters or not (in fact clients load the non date ranged reports more often).
Your assistant is much appreciated and every suggestion would be considered carefully.