Oracle BPM Process Data mart

Question

I am required to create audit reports on BPM workflows.
I am new to thid & need some guidance on configuring BPM Process Data mart. What are the pre-requisites for configuring it & what are the steps to do it.
Also, need some inputs on BAM database. What is the frequency of data upload. Is it data update or insert in BAM.

Answer 1

Hi,
You might want to check out the Administration and Configuration Guides on http://download.oracle.com/docs/cd/E13154_01/bpm/docs65/index.html.
I suspect you might find the BAM and Data Mart portions of this documentation a bit terse, so I've added the steps below that provides more detail. I wrote this for ALBPM 6.0, but believe it will still work for Oracle BPM 10g. It was created from an earlier ALBPM 5.7 document Support wrote called "ALBPM 5_7 Configuring and Troubleshooting the BAM and DataMart Updater.pdf.
You can define how often you want the contents in both databases updated (actually inserted) and how long you want to persist the contents of the BAM database during the configuration.
Here's the contents of the document:
1. Introduction
The use of BAM (Business Activity Monitoring) and Data Mart (or Warehouse) information is becoming more and more widespread in today’s BPM project implementations for the obvious benefits they bring to the management and tuning of processes.
BAM is basically composed by a collection of measurements of current processes load and execution times. This gives us an idea of how the business is doing at this moment (in a pseudo real-time fashion).
Data Mart, on the other hand, is a historical view of the processes load and execution times. And this gives us an idea of how the business has developed since the moment the projects have been put in place.
In this document we are not going to describe exhaustively all configuration aspects of the BAM and Data Mart Updater, but rather we will quickly move from one configuration step to another paying more attention to subjects that have presented some difficulties in real-life projects.
2. Creating the Service Endpoints
The databases for BAM and for Data Mart first have to be defined in the External Resources section of the BPM Process Administrator.
In this following example the service endpoint ‘BAMJ2EEWL’ is being defined. This definition is going to be used later as BAM storage. At this point nothing is created.
Add an External Resource with the name ‘BAMJ2EEWL’ and, as we use Oracle, select the Oracle driver, then click <Next>:
On the following screen, specify:
·     the hostname – here I have used ‘localhost’ as I am just setting this up to work on my laptop
·     the port for the Oracle service
·     the SID – here I have use Oracle Express so the SID is ‘XE’
·     the new user to create / use in Oracle for this database – here I have specified ‘BPMBAM’. This user, and its database, will be created later
·     the password for the user
Scroll down to the bottom of the page and click <Save>.
In addition to a standard JDBC connection that is going to be used by the Updater Service, a remote JDBC configuration needs to be added as the Engine runs in a WebLogic J2EE container. This Data Source is needed to grant the Engine access over BAM tables thru the J2EE Connection Pool instead of thru a dedicated JDBC. The following is an example of how to set this up.
Add an External Resource with the name ‘BAMremote’ and select the Oracle driver, then click <Next>
On the following screen, specify:
·     the Lookup Name that will be used subsequently in WebLogic - here I have given it the name ‘XAbamDS’
Then click <Save>.
In the next example the definition ‘DWHJ2EEWL’ is created to be used later as Data Mart storage. If you are not going to use a Data Mart storage you can skip this step.
Add an External Resource with the name ‘DWHJ2EEWL’ and select the Oracle driver, then click <Next>:
On the following screen, specify:
·     the hostname – here I have used ‘localhost’ as I am just setting this up to work on my laptop
·     the port for the Oracle service
·     the SID – here I have use Oracle Express so the SID is ‘XE’
·     the new user to create / use in Oracle for this database – here I have specified ‘BPMDWH’. This user, and its database, will be created later
·     the password for the user
3. Configuring BAM Updater Service
Once the service endpoint has been created the next step is to enable the BAM update, select the service endpoint to be used as BAM storage and configure update frequency and others. Here the “Updater Database Configuration” is the standard JDBC we configured earlier and the “Runtime Database Configuration” is the Remote JDBC as we are using the J2EE Engine.
So, here’s the example of how to set up the BAM Updater service….
Go into ‘Process Monitoring’ and select the ‘BAM’ tab and enter the relevant information (using the names created earlier – use the drop down list to select):
Note that here, to allow me to quickly test BAM reporting, I have set the update frequency to 1 minute. This would not be the production setting.
Once the data is input, click <Save>.
We now have to create the schema and related tables. For this we will open the “Manage Database” page that has appeared at the bottom of the BAM screen (you may have to re-select that Tab) and select to create the database and the data structure. The user required to perform this operation is the DB system administrator:
Text showing the successful creation of the database and data structures should appear.
Once we are done with the schema creation, we can move to the Process Data Mart configuration screen to set up the Common Updater Service parameters. Notice that the service has not been started yet… We will get to that point later.
4. Configuring Process Data Mart Updater Service
In the case that Data Mart information is not going to be used, the “Enable Automatic Update” checkbox must be left off and the “Runtime Database Configuration” empty for this service. Additionally, the rest of this section can be skipped.
In the case it is going to be used, the detail level, snapshot time and the time of update should be configured; in addition to enabling the updater and choosing the storage configuration. An example is shown below:
Still in ‘Process Monitoring’, select the ‘Process Data Mart’ tab and enter the name created earlier (use the drop down list to select).
Also, un-tick the Generate O3 Cubes (see later notes):
Then click <Save>.
Once those properties have been configured the database and the data structure have to be created. This is performed at the “Manage Database” page for which the link has appeared at the bottom of the page (as with BAM). Even when this page is identical to the one shown above (for the BAM configuration) it has been opened from the link in the “Process Data Mart” page and this makes it different.
Text showing the successful creation of the database and data structures should appear.
5. Configuring Common Updater Service Parameters
In the “Process Data Mart” tab of the Process Monitoring section -along with the parameters that are specific to the Data Mart - we will find some parameters that are common to all services. These parameters are:
• Log directory: location of the log file
• Messages logged from Data Store Updater: severity level of the Updater logs
• Language
• Generate Performance Metrics: enables performance metrics generation
• Generate Workload Metrics: enables workload metrics generation
• Generate O3 Cubes: enables O3 Cubes generation
In this document we are not going to describe in detail each parameter. But we will mention a few caveats:
a. the Log directory must be specified in order for the logs to be generated
b. the Messages logged from Data Store Updater, which indicates the level
of the logs, should be DEBUG for troubleshooting and WARNING otherwise
c. Performance and Workload Metrics need to be on for the typical BAM usage and, even when either metric might not be used on the initial project releases, it is recommended to leave them on in case they turn out to be useful in the future
d. the Generation of O3 Cubes must be off if this service is not used, otherwise the Data Mart Updater service might not work properly .
The only changes required on this screen was to de-select the ‘Generate O3 Cubes’ as shown in the last section.
6. Set up the WebLogic configuration
We need to set up the JDBC data source specified above, so go to Services / JDBC / Data Sources.
Click on <Lock and Edit> and then <New> to add a New data source.
Specify:
·     the Name – use the name you set up in the Process Administrator
·     the JNDI Name – again use the name you set up in the Process Administrator
·     the Database Type – Oracle
·     use the default Oracle Database Driver
Then click <Next>
On the next screen, click <Next>
On the next screen specify:
·     the Database Name – this is the SID – for me that is XE
·     the Host Name – as I am running on my laptop, I’ve just specified ‘localhost’
·     the Database User Name and Password – this is the BAM database user specified in the Process Administrator
Then click <Next>
On the next screen, you can test the configuration to make sure you have got it right, then click <Next>
On the next screen, select your server as the target server and click <Finish>:
Finally, click <Activate Changes>.
7. The Last Step: Starting Up and Shutting Down the Updater Service
ALBPM distribution is different depending on the Operating System. In the case of the Updater Service:
-     For Unix like Operating Systems the service is started or stopped with the albpmwarehouse.sh shell script. The command in this case is going to look like this:
$ALBPM_HOME/bin$ ./albpmwarehouse.sh start
-     For Windows Operating Systems the service is installed or uninstalled as a Windows Service with the albpmwarehouse.bat batch file. The command will look like:
%ALBPM_HOME%\bin> albpmwarehouse.bat install
After installing the service, it has to be started|stopped from the Microsoft Management Console. Note also that Windows will start automatically the installed service when the computer starts. In either case the location of the script is ALBPM_HOME/bin Where ALBPM_HOME is the ALBPM installation directory. An example will be:
C:\bea\albpm6.0\j2eewl\bin\albpmwarehouse.bat
8. Finally: Running BAM dashboards to show it is Working
Now we have finally got the BAM service running, we can run dashboards from within Workspace and see the results:
9. General BAM and Data Mart Caveats
a. The basic difference between these two collections of measurements is that BAM keeps track of current processes load and execution times while Data Mart contains a historical view of those same measurements. This is why BAM information is collected frequently (every minute) and cleared out every several hours (or every day) and why Data Mart is updated infrequently (once a day) and grows indefinitely. Moreover, BAM measurements can be though of as a minute-by-minute sequence of Engine Events snapshots, while Data Mart measurements will be a daily sequence of Engine Events snapshots.
b. BAM and Data Mart table schemas are very similar but they are not the same. Thus, it is important not to use a schema created with the Manage Database for BAM as Data Mart storage or vice-versa. If these schemas are exchanged by mistake, the updater service will run anyway but no data will be added to the tables and there will be errors in the log indicating that the schema is incorrect or that some tables could not be found.
c. BAM and Data Mart Information and Services are independent from one another. Any of them can be configured and running without the other one. The information is extracted directly from the Engine Database (PPROCINSTEVENT table is the main source of info) for both of them.
d. So far there has not been a mention of engines, projects or processes in any of the BAM or Data Mart configurations. This is because the metrics of all projects published under the current Process Administrator (or, more precisely, FDI Directory) are going to be collected.
e. It is also important to note that only activities for which events are generated are going to be measured (and therefore, shown in the metrics). The project default is to generate events only for Interactive activities. This can be changed for any particular activity and for the whole process (where the activity setting, when specified, overrides the process setting). Unfortunately, there is no project setting for events generation so far; thus, remember to edit the level of event generation for every new process that is added to the project.
f. BAM and Data Mart metrics are usually enriched with Business Variables. These variables are a special type of External Variables. An External Variable is a process variable with the scope of an Instance and whose value is stored on a separate column in the Engine Instances table. This allows the creation of views and filters based on this variable. A Business Variable, then, shares all the properties of an External Variable plus the fact that its value is collected in all BAM and Data Mart measurements (in some cases the value is shown as it is for a particular instance and in others the value is aggregated).
The caveat here is that there is a maximum number of 256 Business Variables per FDI. Therefore, when publishing several projects into a single FDI directory it is recommendable to reuse business variables. This is achieved by mapping similar Business Variables of different projects with a unique real Variable (on the variable mapping performed at publish time).
g. Configuring the Updater Service Log
In section 5. Configuring Common Updater Service Parameters we have seen that there are two common Updater properties related to logging. These properties are “Log directory” and “Messages logged from Data Store Updater”, and they specify the location and level of these two files:
- dwupdater.log: which is the log for the Data Mart updater service
- bam-dwupdater.log: which is the log for the BAM updater service
In addition to these two properties, there is a configuration file called ‘WarehouseService.conf’ that allows us to modify these other properties:
- wrapper.console.loglevel: level for the updater service log
- wrapper.logfile.loglevel: level for the updater service log
- wrapper.java.additional.n: additional argument to the service JVM
- wrapper.logfile.maxsize: maximum size of the updater service log files
- wrapper.logfile.maxfiles: maximum number of updater service log files
- wrapper.logfile: updater service log file name (the default value is dwupdater-service.log)
9.1. Updater Service Log Configuration Caveats
a. The first three parameters listed above have to be modified when increasing the log level to DEBUG (since the default is WARNING). The loglevel parameters have to be set to DEBUG and a java.additional.n (where n is a consecutive integer to the already used ones) has to be set to –ea to enable asserts, since without this option no DEBUG message is going to be generated.
b. Of the other arguments, maxfiles might need to be increased to hold a few more days of data when the log level is set to DEBUG (with the default value up to two days are stored).
c. The updater service has to be stopped, uninstalled, installed and then started for any of these changes to take effect.
Hope this helps,
Dan