Approach for Oracle's TDE encryption.

Hello experts,
We are in the process of encrypting our HCM Databaseu2026
We are looking to encrypt such details as Social insurance number, First name, Last name, Date of birth, Base
salary, Email addressu2026.etc
Question is, do we encrypt at the table/column level (multitude of tables) or at the
package level?
Please suggest?
Thanks in advance.
Regards,
Iqbal

Hi ,
You may refer SAP Note 974876 - Transparent Data Encryption (TDE)
It contains all information about TDE and pointers to configuration.
Regards,
Deepak Kori

Similar Messages

Oracle user'd encrypted password (Oracle 10g)

Hi Guys,
Which table can I check for oracle user's encrypted password. I need to use the same password on the application's configuration file for connection to the database.
Thank you.

SQL> desc user$
Name                            Null?    Type
USER#                            NOT NULL NUMBER
NAME                            NOT NULL VARCHAR2(30)
TYPE#                            NOT NULL NUMBER
PASSWORD                             VARCHAR2(30)
DATATS#                       NOT NULL NUMBER
TEMPTS#                       NOT NULL NUMBER
CTIME                            NOT NULL DATE
PTIME                                  DATE
EXPTIME                             DATE
LTIME                                  DATE
RESOURCE$                       NOT NULL NUMBER
AUDIT$                              VARCHAR2(38)
DEFROLE                       NOT NULL NUMBER
DEFGRP#                             NUMBER
DEFGRP_SEQ#                             NUMBER
ASTATUS                       NOT NULL NUMBER
LCOUNT                        NOT NULL NUMBER
DEFSCHCLASS                             VARCHAR2(30)
EXT_USERNAME                             VARCHAR2(4000)
SPARE1                              NUMBER
SPARE2                              NUMBER
SPARE3                              NUMBER
SPARE4                              VARCHAR2(1000)
SPARE5                              VARCHAR2(1000)
SPARE6                              DATE

What's the best approach for handeling about 1300 connections in Oracle.

What's the best approach for handling about 1300 connections in Oracle 9i/10g through a Java application?
1.Using separate schema s for various type users(We can store only relevant data with a particular schema. Then No. of records per table can be reduced by replicating tables but we have to maintain all data with a another schema Then we need update two schema s for a given session.Because we maintain separate scheama for a one user and another schema for all data and then there may be Updating problems)
OR
2. Using single schema for all users.
Note: All users may access the same tables and there may be lot of records than previous case.
What is the Best case.
Please give Your valuable ideas

It is a true but i want a solution from you all.I want you to tell me how to fix my friends car.

About TDE encryption feature of oracle

Hi All,
I am facing problems while checking this feature of Encryption,
I created in database1, table1 with user1, coulum1 in the table1 was with encrypt keyword as follows.
create table table1(ID number(3), PHONE number(11) encrypt);
Inserted some rows and selected those rows, it shows them successfully, Ideally it should show the actual data for user1.
gave the select permission to user2 from user1 for table1 as follows:
grant select on table1 to user2;
Now logged in with user2 to same database1.
now when I query the database for table1 as follows:
select * from database1.table1;
It shows all the rows successfully, Ideally it should NOT show the actual data for user2.
Can someone please tell me the solution or explain me the procedure to test it properly.
Thanks in advance,
Nikhil Joshi.

TDE encrypts data when it is on disk "at rest". If a user has SELECT access to the data in question, TDE will transparently (hence the T) decrypt the data. TDE will never present any user with encrypted data-- either the user does not have privileges to select the data in the first place or they do and TDE will decrypt it.
If you want to limit what data a particular user can access, you'd either want to do that conventionally by, for example, creating a view on the table that does not have the phone number and only granting access on that view to user2. Or, assuming you're using 11.2, you could create a VPD policy that limits what columns a particular user can see. A VPD policy would generally make sense if you wanted to allow user2 to see the phone number for some rows but not others.
Justin

Step-by-step approach for "manual" Oracle Database Migration

Hi All,
I'm looking for a step-by-step approach for "manual" Oracle Database Migration. Basically, the minutes followed in a real-time production DB implementation.
I assume the steps may vary for different setups. However any scenario-based docs will be of great help!

The
Oracle Upgrade Guide
that comes with your download as far as I know is very detailed.

Comparing performance between TDE encryption and no encryption

Hi all,
How can i check, how much database resource (%CPU, Time elapsed) increased when using TDE encryption.
Thank you!
Dan.
Edited by: Dan on Jul 10, 2011 10:13 PM

The performance implications of using TDE are going to depend on a number of factors including
- The version of Oracle
- The hardware available (in particular whether hardware acceleration is available for encryption)
- Whether you are using tablespace encryption or column-level encryption
- If you are using column-level encryption how many columns you are encrypting
- What sort of workload your system is doing.
- Where your system bottlenecks today without encryption
Without knowing those things, it's hard to narrow down the answer to somewhere between 0 and 50% which is, obviously, far too large a range to be meaningful.
On the one hand, the worst case is probably represented by this test case where you're using column-level encryption of one column of a two column table in 10.2 and doing single-row inserts and deletes. Those operations are already heavily CPU bound and, since you're using column-level encryption, the data has to be encrypted and decrypted every time it goes into or out of the SGA. If you were using tablespace-level encryption, the data would only need to be encrypted and decrypted when it is read from or written to the disk which would be far faster in for this test case. Later versions of Oracle also tend to be more efficient.
On the other hand, if you're using 11.2 with the most recent patches and you've got hardware acceleration, Oracle is happy to trumpet the [urlhttp://www.oracle.com/technetwork/database/options/advanced-security/index-099011.html]near-zero performance impact of TDE.
Most people live somewhere between these two extremes but it's hard to guess where your particular application falls. I would guess that most people would see something like a 10-15% increase in CPU consumption but that's just a wild guess based on a relatively small sample of systems.
Justin

Goldengate 11.1.1.1 Oracle 11.2 Encrypted tablespace

Please advise why this error is showing and help us on this situation
Line: -----
Windows 2008 x64
Oracle 11.2.0.1 x64
Oracle Goldengate 11.1.1.1 x64
++++++++++++SOURCE++++++++++++++
sql>conn / as sysdba
**************CREATE TABLESPACE AND TABLE********************
CREATE TABLESPACE encrypted_ts
DATAFILE 'd:\oradata\test\encrypted_ts01.dbf' SIZE 128K
SIZE 15M
AUTOEXTEND ON NEXT 20M
MAXSIZE UNLIMITED
LOGGING
ENCRYPTION USING 'AES192'
EXTENT MANAGEMENT LOCAL
SEGMENT SPACE MANAGEMENT AUTO
DEFAULT NOCOMPRESS STORAGE(ENCRYPT);
CREATE TABLE TESTA
ID NUMBER,
NAME VARCHAR2(16 CHAR)
LOGGING
NOCOMPRESS
NOCACHE
NOPARALLEL
MONITORING;
cmd> mkstore -wrl d:\test\wallet -createEntry ORACLE.SECURITY.CL.ENCRYPTION.ORACLEGG
Enter <secret> : test
cmd>mkstore -wrl d:\test\wallet -viewEntry ORACLE.SECURITY.CL.ENCRYPTION.ORACLEGG
ORACLE.SECURITY.CL.ENCRYPTION.ORACLEGG = test
sql>ALTER SYSTEM SET ENCRYPTION WALLET CLOSE IDENTIFIED BY 'welcome1';
sql>ALTER SYSTEM SET ENCRYPTION WALLET OPEN IDENTIFIED BY 'welcome1';
Compile dbms_internal_clkm package.
sql>@prvtclkm.plb
sql>GRANT EXECUTE ON SYS.DBMS_INTERNAL_CLKM TO GGS_OWNER;
**********************************GOLDENGATE******************************************
ggsci>dblogin userid ggs_owner, ggs_owner
ggsci>add extract ext2ts, tranlog, begin now
ggsci>add rmttrail D:\ggs\dirdat\ts, extract ext2ts
ggsci>ENCRYPT PASSWORD test ENCRYPTKEY DEFAULT
Using default key...
Encrypted password: AACAAAAAAAAAAAEAZAMEFCPGPEOHYDHE
ggsci>edit params ext2ts
EXTRACT ext2ts
USERID ggs_owner, PASSWORD ggs_owner
RMTHOST 192.168.0.2, MGRPORT 7809
RMTTRAIL D:\ggs\dirdat\ts
DBOPTIONS DECRYPTPASSWORD AACAAAAAAAAAAAEAZAMEFCPGPEOHYDHE ENCRYPTKEY DEFAULT -- ???
TABLE TESTA;
+++++++++++++TARGET++++++++++++++++
sql> conn / as sysdba
**************CREATE TABLESPACE AND TABLE********************
cmd>
CREATE TABLESPACE ts
LOGGING DATAFILE 'd:\oradata\test\ts.dbf'
SIZE 15M
AUTOEXTEND ON NEXT 20M
EXTENT MANAGEMENT
LOCAL SEGMENT SPACE
MANAGEMENT AUTO;
CREATE TABLE TESTA
ID NUMBER,
NAME VARCHAR2(16 CHAR)
LOGGING
NOCOMPRESS
NOCACHE
NOPARALLEL
MONITORING;
************************************GOLDENGATE*****************************************
ggsci>DBLOGIN USERID ggs_owner, PASSWORD ggs_owner
ggsci>ADD CHECKPOINTTABLE GGS_OWNER.CHKPTAB
ggsci>ADD REPLICAT rep1, EXTTRAIL D:\ggs\dirdat\ts, CHECKPOINTTABLE GGS_OWNER.CHKPTAB
ggsci>EDIT PARAMS rep1
REPLICAT rep1
ASSUMETARGETDEFS
USERID ggs_owner, PASSWORD ggs_owner
MAP GGS_OWNER.TESTA, TARGET GGS_OWNER.TESTA;
+++++++++++++TARGET+++++++++++++++
ggsci>start replicat rep1
+++++++++++++SOURCE+++++++++++++++
ggsci>start extract ext2ts
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!EXTRACT - REPORT!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
2011-07-05 11:46:50 ERROR OGG-01028 ORA-06521: PL/SQL: Error mapping function
ORA-06512: at "SYS.DBMS_INTERNAL_CLKM", line 3
ORA-06512: at line 1.
2011-07-05 11:46:50 ERROR OGG-01668 PROCESS ABENDING.
Edited by: Oyunbold on Jul 7, 2011 3:25 AM
Edited by: Oyunbold on Jul 19, 2011 6:09 PM

Yesterday, I met the same problem. How do you solve it ? Plz, help me.
OS : AIX 5.3
DB : Oracle 11.1.0.7
OGG : 11.1.1.1
2011-09-06 15:26:56 INFO OGG-01515 Oracle GoldenGate Capture for Oracle, eora01.prm: Positioning to begin time Sep 2, 2011 6:50:49 PM.
2011-09-06 15:26:56 ERROR OGG-01028 Oracle GoldenGate Capture for Oracle, eora01.prm: ORA-12427: invalid input value for FLAGS parameter ORA-06512: at "SYS.DBMS_INTERNAL_CLKM", line 3 ORA-06512: at line 1.
2011-09-06 15:26:56 ERROR OGG-01668 Oracle GoldenGate Capture for Oracle, eora01.prm: PROCESS ABENDING.
Our Extract configuration is below :
EXTRACT EORA01
USERID ogg, PASSWORD AACAAAAAAAAAAADAHBMDHBECUCZBPIYE, encryptkey XKEY
DBOPTIONS DECRYPTPASSWORD AACAAAAAAAAAAAIARIXFKCQBMFIGFARA ENCRYPTKEY DEFAULT
ENCRYPTTRAIL
exttrail ./dirdat/ea
EOFDELAYCSECS 50
FLUSHCSECS 50
DISCARDFILE ./dirrpt/EORA01.DSC, APPEND, MEGABYTES 500
DISCARDROLLOVER AT 07:30
TABLE TVANBT.TLF_BUYS_MISS;
...

"In-Memory Database Cache" option for Oracle 10g Enterprise Edition

Hi,
In one of our applications, we are using TimesTen 5.1.24 and Oracle 9i
databases (platform - Solaris 9i).
TimesTen holds application information which needs to be accessed quickly
and Oracle 9i is a master application database.
Now we are looking at an option of migrating from Oracle 9i to Oracle 10g
database. While exploring about Oracle 10g features, came to know about
"In-Memory Database Cache" option for Oracle Enterprise Edition. This made
me to think about using Oracle 10g Enterprise Edition with "In-Memory
Database Cache" option for our application.
Following are the advantages that I could visualize by adopting the
above-mentioned approach:
1. Data reconciliation between Oracle and TimesTen is not required (i.e.
data can be maintained only in Oracle tables and for caching "In-Memory
Database Cache" can be used)
2. Data maintenance is easy and gives one view access to data
I have following queries regarding the above-mentioned solution:
1. What is the difference between "TimesTen In-Memory Database" and
"In-Memory Database Cache" in terms of features and licensing model?
2. Is "In-Memory Database Cache" option integrated with Oracle 10g
installable or a separate installable (i.e. TimesTen installable with only
cache feature)?
3. Is "In-Memory Database Cache" option same as that of "TimesTen Cache
Connect to Oracle" option in TimesTen In-Memory Database?
4. After integrating "In-Memory Database Cache" option with Oracle 10g, data
access will happen only through Oracle sqlplus or OCI calls. Am I right here
in making this statement?
5. Is it possible to cache the result set of a join query in "In-Memory
Database Cache"?
In "Options and Packs" chapter in Oracle documentation
(http://download.oracle.com/docs/cd/B19306_01/license.102/b14199/options.htm
#CIHJJBGA), I encountered the following statement:
"For the purposes of licensing Oracle In-Memory Database Cache, only the
processors on which the TimesTen In-Memory Database component of the
In-Memory Database Cache software is installed and/or running are counted
for the purpose of determining the number of licenses required."
We have servers with the following configuration. Is there a way to get the
count of processors on which the Cache software could be installed and/or
running? Please assist.
Production box with 12 core 2 duo processors (24 cores)
Pre-production box with 8 core 2 duo processors (16 cores)
Development and test box with 2 single chip processors
Development and test box with 4 single chip processors
Development and test box with 6 single chip processors
Thanks & Regards,
Vijay

Hi Vijay,
regarding your questions:
1. What is the difference between "TimesTen In-Memory Database" and
"In-Memory Database Cache" in terms of features and licensing model?
==> Product has just been renamed and integrated better with the Oracle database - Times-Ten == In-Memory-Cache-Database
2. Is "In-Memory Database Cache" option integrated with Oracle 10g
installable or a separate installable (i.e. TimesTen installable with only
cache feature)?
==> Seperate Installation
3. Is "In-Memory Database Cache" option same as that of "TimesTen Cache
Connect to Oracle" option in TimesTen In-Memory Database?
==> Please have a look here: http://www.oracle.com/technology/products/timesten/quickstart/cc_qs_index.html
This explains the differences.
4. After integrating "In-Memory Database Cache" option with Oracle 10g, data
access will happen only through Oracle sqlplus or OCI calls. Am I right here
in making this statement?
==> Please see above mentioned papers
5. Is it possible to cache the result set of a join query in "In-Memory
Database Cache"?
==> Again ... ;-)
Kind regards
Mike

Installing ArcSDE 9.2 for Oracle version 10G R2 on Windows 2003 x64

We submitted this question on an ESRI user forum, but we would also like to submit it elsewhere to ensure an answer.
We are trying to install ArcSDE 9.2 for Oracle. The version of Oracle is 10G R2 (10.2.0.1). The server is a Windows 2003 R2 Enterprise x64 edition with SP1. There are now 2 instances of Oracle on the server, but this problem originally occurred when there was only one instance. The issue: ArcSDE 9.2 apparently installs just fine. There are no error messages during the run of the installation process. However, when the Post-Installation process is called, a critical error alert pops up reporting, "The application failed to start because OCI.dll was not found. Reinstalling the application might fix this problem." ArcSDE has been uninstalled and reinstalled several times. Oracle has been uninstalled and reinstalled several times. Various searches have been made, concerning this error message. On Friday the only ESRI Article we could find mentioning this error was Article ID: 25388 which stated that the "ORA81/bin" directory needed to be added to the PATH environment variable. However, today we found a new ESRI forum entry dated 6/2/2007 with the querystring "c=2&f=1717&t=223961" in which someone reported the need to install the 32 bit Oracle_HOME. However the ESRI Support Center article he referenced, (dated November 14, 2006), stated that the 32 bit Oracle Client needed to be installed to a different Oracle_HOME and the service name created for it, is to be used in the Post-Installation process. And finally our Oracle DBA reported seeing another Support Center Article which stated that the 32 bit versions of the Oracle libraries are installed by default in a different location, and it was only necessary to add that location to the PATH environment variable. I have also seen information on Oracle forums stating that this message means that the Oracle client needed to be installed, but our Oracle DBA reports that this is installed by default with Oracle DB server. My question is of course which if any of these solutions is correct?
E.R. Joell MCSD MCDBA

I have managed an ORacle/ArcSDE environment for many years where multiple SIDs were running.
If it doesn't know which SID then it would indicate that it is picking up the wrong ORACLE_SID environment setting. To be honest, UNIX is an easier environment to operate software in as one can set environment variables to suit software very directly. Now I have not run ArcSDE on a Windows environment before so any suggestions I have may not work as I write them.
Still, I would suggest you consider writing a CMD/batch script that:
1. Set all appropriate Oracle environment variables DIRECTLY eg
SET ORACLE_BASE=c:\oracle\product
SET ORACLE_HOME=c:\oracle\product\10.2.01
SET ORACLE_SID=...
2. Set the ArcSDE environment
3. Sets the Path to include all Oracle, ArcSDE and required Windows executables
4. Start the ArcSDE service:
NET START ESRARCSDE (or whatever this thing is called)
I suspect if you can get this going you will never go back to any other method of starting and managing ArcSDE (this approach works and works well on UNIX so you will be all skilled up to migrate across to Linux/Solaris etc if you see fit).
Hope this helps.
PS I have run ArcSDE against non-supported versions quite successfully over the years so I would say try it against 10.2.0.1 and see what happens.
regards
Smon

What are the best approaches for mapping re-start in OWB?

What are the best approaches for mapping re-start in OWB?
We are using OWB repository 10.2.0.1.0 and OWB client 10.2.0.1.31. The Oracle version is 10 G (10.2.0.3.0). OWB is installed on Linux.
We have number of mappings. We built process flows for mappings as well.
I like to know, what are the best approches to incorportate re-start options in our process. ie a failure of mapping in process flow.
How do we re-cycle failed rows?
Are there any builtin features/best approaches in OWB to implement the above?
Does runtime audit tables help us to build re-start process?
If not, do we need to maintain our own tables (custom) to maintain such data?
How did our forum members handled above situations?
Any idea ?
Thanks in advance.
RI

Hi RI,
How many mappings (range) do you have in a process flows?Several hundreds (100-300 mappings).
If we have three mappings (eg m1, m2, m3) in process flow. What will happen if m2 fails?Suppose mappings connected sequentially (m1 -> m2 -> m3). When m2 fails then processflow is suspended (transition to m3 will not be performed). You should obviate cause of error (modify mapping and redeploy, correct data, etc) and then repeat m2 mapping execution from Workflow monitor - open diagram with processflow, select mapping m2 and click button Expedite, choose option Repeat.
In re-start, will it run m1 again and m2 son on, or will it re-start at row1 of m2?You can specify restart point. "at row1 of m2" - I don't understand what you mean (all mappings run in Set based mode, so in case of error all table updates will rollback,
but there are several exception - for example multiple target tables in mapping without corelated commit, or error in post-mapping - you must carefully analyze results of error).
What will happen if m3 fails?Process is suspended and you can restart execution from m3.
By having without failover and with max.number of errors=0, you achieve re-cycle failed rows to zero (0).This settings guarantee existence only two return result of mapping - SUCCSES or ERROR.
What is the impact, if we have large volume of data?In my opinion for large volume Set based mode is the prefered processing mode of data processing.
With this mode you have full range enterprise features of Oracle database - parallel query, parallel DML, nologging, etc.
Oleg

Entity Frame work for Oracle 11g -how to build a database from Model

Hi ,
I m trying to build a data access layer for using both Sql Server and Oracle(not at the same time). I used Model-first approach for creating the entities and then generate Databse from it.
I came to know from different blogs that oracle and Sql Server generation workflow and DDL generation template are different. Right now my application is supporting for Sql Server, I need to support it for Oracle too.
http://www.oracle.com/technetwork/issue-archive/2011/11-sep/o51odt-453447.html i m unable to get the Oracle DDl generation template and workflow.
please Let me know,if i missed some thing and what are the changes I need to follow and exact procedure that is reuqired to support Oracle.
Thanks,
ilyas.

Try the Model-First walkthrough in the Oracle EF OBE:
http://download.oracle.com/oll/obe/EntityFrameworkOBE/EntityFrameworkOBE.htm#t4
It provides a more detailed step by step tutorial in generating Oracle DDL from EF.
If you're still having problems, provide a detailed description on what errors you see and what steps you executed. Right now, it's hard to diagnose what the problem could be with so little detail.

Does XML approach for passing parameters in the query make the query slow?

Hi,
I am using XML approach for passing parameters in a query. This is running very slow but when I pass comma separated values in parameter, it runs very fast.
So it concludes that we should not use XML approach for passing parameters. Please confirm me that Am I right or wrong?
I have also googled to clear my doubt but not get solution till now. please help me.
Regards,
Sachin

914014 wrote:
Hi,
I am using XML approach for passing parameters in a query. This is running very slow but when I pass comma separated values in parameter, it runs very fast.
So it concludes that we should not use XML approach for passing parameters. Please confirm me that Am I right or wrong?
I have also googled to clear my doubt but not get solution till now. please help me.
Regards,
SachinShow us what you are doing, create a simple yet complete example we can run on our Oracle instances.
Then we will know exactly what you mean, and can comment appropriately.
Cheers,

Interview questions for oracle dba

Hi I am trying to find teh interview questions for oracle dba .
What will be your answer to the following questions. Can somebody shed some light...
. What is the day to day duties of oracle dba?
·     Did you use online or off-line backups?
·     What version of Oracle were you running?
·     Haw many databases and what sizes?
·     If you have to advise a backup strategy for a new application, how would you approach it and what questions will you ask?
·     If a customer calls you about a hanging database session, what will you do to resolve it?
·     Compare Oracle to any other database that you know. Why would you prefer to work on one and not on the other?
Thx
KK

. What is the day to day duties of oracle dba?
I have got a checklist, designed for my daily, weekly, fortnightly and monthly tasks. For example, in the daily checklist tasks, I check for the alert log file, competion and correction of my exports and operating system backups. DB space situation, Antivirus status, standby database status, space on disks, any rogue queries etc.
· Did you use online or off-line backups?
Both.
· What version of Oracle were you running?
8i(not any more), 9i, 10g
· Haw many databases and what sizes?
8 databases, ranging from 1 TB to 10 GB. I have segregated them into OLTP production, testing, cm, and development and a Datawarehouse.
· If you have to advise a backup strategy for a new application, how would you approach it and what questions will you ask?
How much they could afford to lose data or they want zero data loss and also I will get their requirements about down time and up time of database.
· If a customer calls you about a hanging database session, what will you do to resolve it?
First I will identify it, then see to it what exactly it is doing, capture that situation and then either try to release the locks(if its a blocking or blocked session), or kill the session, if its hanging on a runaway query.
· Compare Oracle to any other database that you know. Why would you prefer to work on one and not on the other?
Oracle passes the ACID rule with distinction as compared to the other databases. Moreover, the multiversion read consistency, recovery, flexibility and challenges of Oracle are matchless.
Cheers
fahdoracle.blogspot.com

Job opening for Oracle ADF with Java

Guys,
I am in need of 30 candidates who have experience on Oracle ADF framework along with Java. The position will be Wisconsin and the project will be contract based. The duration will be more than 6 months and rates are open.
If you or your friends are interested in this offer, please send your resumes to [email protected] with a subject "Resume for Oracle ADF position".
Thanks
Srini

Dear Sir,
With reference to the website of your company I came to know that there always are the vacancies for the suitable candidates and hence I wish to apply for the same.Currently working in hydraulic environment I am approaching you through this application with great enthusiasm, that I have scope and chance to show my ability and contribute my efforts in well direction. I am enclosing here with my complete Curriculum Vitae .
I am currently working with M/s Minar Hydrosystems PVT. LTD. in Nagpur which is one of the largest manufacturers of hydraulic/pneumatic cylinders, jacks etc. I am In charge for quality department having complete knowledge of instruments with level II in non destructive and automated non destructive techniques. Also handling SAP module in current company with effective administration.
If you need further information regarding my candidature please do write/call me.
Thanks & Regards
Rushikesh P. Deogade
Nagpur, India
Ph.:- +91 9923858523
Attachments:
B.E. Mechanical with 3 years exp as a QC Incharge.doc ‏25 KB

Performance of the LOD approach for network analisys

Hi to all,
I open new thread for speak about performance of LOD approach for Network analysis using Java API as continuing from this thread:
Partitioning of the Network for using LOD API
I remember you that:
- I'm using Oracle 11g R2;
- my network consist of 7.817.372 of links and 6.662.079 of nodes (big network);
- LINK_LEVEL of links table is setted as NULL;
- I have partitioned the network with these procedure:
partition:
EXEC sdo_net.spatial_partition('ITALIA', 'ITALIA_PART$', 10000, 'WORK_DIR_ITALIA', 'ITALIA_PART.log', 'a', 1);
and for partition blob:
EXEC sdo_net.generate_partition_blobs('ITALIA', 1, 'ITALIA_PBLOB$', true, true, 'WORK_DIR_ITALIA', 'ITALIA_PBLOB.log', 'a');
- I'm using LOD Java API for network analysis with Netbeans IDE; I took the code from NDM_tutorial of Hillsborough network.
My first analysis is the compute of the shortest path between two extreme nodes (about 1.500 km of distance) and between two near nodes (about 80 km of distance) using Dijkstra and AStar algorithm.
I did the following tests on execution times of the compute of the shortest path:
1) with partition of maximum 10000 nodes for partition (from log file I read that were been generated 1024 partitions with 1 link level):
- between two extreme nodes, about 1 minute and 50 seconds.
- between two near nodes, about 20 seconds.
then I have re-executed the two previous procedures where I changed the value of maximum nodes for partition and I did other tests:
2) with partition of maximum 15000 nodes for partition (from log file I read that were been generated 512 partitions with 1 link level):
- between two extreme nodes, about 1 minute and 50 seconds and sometimes it goes out of memory
- between two near nodes, about 20 seconds.
3) with partition of maximum 5000 nodes for partition (from log file I read that were been generated 2048 partitions with 1 link level):
- between two extreme nodes, about 1 minute and 50 seconds and sometimes it goes out of memory
- between two near nodes, about 20 seconds.
4) with partition of maximum 2000 nodes for partition (from log file I read that were been generated 4096 partitions with 1 link level):
- between two extreme nodes, about 1 minute and 50 seconds and sometimes it goes out of memory
- between two near nodes, about 15 seconds.
I think that there is any problem because I expect execution times more lower (maximum 5-6 seconds for extreme nodes).
Even changing the maximum number of nodes to partition the execution times do not change much.
I remember you that with in-memory approach on Oracle 10gR2 the execution times for every computation between two nodes was of about 4 minutes and sometimes it goes out of memory. With LOD on Oracle 11gR2 the execution times are reduced but are too long for me.
Now, my questions are:
- @Jack Wang: (if you're reading me) do you know what are the execution times for the compute of shortest path between two nodes for USA network (about 1500 km of distance)? I remember that you have used LOD for USA (56 Millions of link and 20 Millions of nodes).
- Do you think that I wrong anything? How can I do for reduce the execution times of network analysis?
If you need any more information just ask.
Thank you to all in advance very much.

Jack Wang wrote:
Are the 12/13 seconds computation times for the near node pair?Yes.
You can look at an example for computing hierarchical shortest path with a 2-level network(NAVTEQ_SF) under ndm_tutorial
~ \ndm_tutorial\examples\java\src\lod\SpWithMultiLinkLevels.java.But I think that yesterday I had not yet used the LINK_LEVEL = 2 in network analysis. Infact, I used the code into ShortestPathAnalysis.java where linkLevel is always setted = 1.
Now, I'm using the example SpWithMultiLinkLevels.java and I'm seeing that linkLevel is setted = 2 before the execution of Dijkstra and AStar algorithm. This code works with Hillsborough_network. Instead with my network (note that I'm using a region of Italy with 700.000 link and 600.000 nodes called ITAI11_METERS for to test a network with 2 LINK_LEVEL), I have a problem during readPartitionFromBlob.
This is the code:
package calcolopercorsolod;
import java.io.*;
import java.sql.*;
import java.text.*;
import java.util.*;
import oracle.jdbc.OracleConnection;
import oracle.jdbc.pool.OracleDataSource;
import oracle.spatial.util.Logger;
import oracle.spatial.network.UserDataMetadata;
import oracle.spatial.network.lod.*;
import oracle.spatial.network.lod.config.*;
import oracle.spatial.network.lod.util.PrintUtility;
public class SpWithMultiLinkLevels
private static NetworkAnalyst analyst;
private static NetworkIO networkIO;
private static void setLogLevel(String logLevel)
    if("FATAL".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_FATAL);
    else if("ERROR".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_ERROR);
    else if("WARN".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_WARN);
    else if("INFO".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_INFO);
    else if("DEBUG".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_DEBUG);
    else if("FINEST".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_FINEST);
    else //default: set to ERROR
        Logger.setGlobalLevel(Logger.LEVEL_ERROR);
public static void main(String[] args) throws Exception
        String configXmlFile = "LODConfigs.xml";
        String logLevel    =    "DEBUG";
        String dbUrl       = "jdbc:oracle:thin:@oracle:1521:mySID";
        String dbUser      = "myUser";
        String dbPassword = "myPass";
        String networkName = "ITAI11_METERS";
        long startNodeId = 15323;
        long endNodeId   = 431593;
    int linkLevel      = 1;
    double costThreshold = 1550;
    int numHighLevelNeighbors = 8;
    double costMultiplier = 1.5;
    Connection conn    = null;
    //get input parameters
    for(int i=0; i<args.length; i++)
        if(args.equalsIgnoreCase("-dbUrl"))
dbUrl = args[i+1];
else if(args[i].equalsIgnoreCase("-dbUser"))
dbUser = args[i+1];
else if(args[i].equalsIgnoreCase("-dbPassword"))
dbPassword = args[i+1];
else if(args[i].equalsIgnoreCase("-networkName") && args[i+1]!=null)
networkName = args[i+1].toUpperCase();
else if(args[i].equalsIgnoreCase("-linkLevel"))
linkLevel = Integer.parseInt(args[i+1]);
else if(args[i].equalsIgnoreCase("-configXmlFile"))
configXmlFile = args[i+1];
else if(args[i].equalsIgnoreCase("-logLevel"))
logLevel = args[i+1];
// opening connection
conn = LODNetworkManager.getConnection(dbUrl, dbUser, dbPassword);
System.out.println("Network analysis for "+networkName);
setLogLevel(logLevel);
//load user specified LOD configuration (optional),
//otherwise default configuration will be used
InputStream config = ClassLoader.getSystemResourceAsStream(configXmlFile);
LODNetworkManager.getConfigManager().loadConfig(config);
//LODConfig c = LODNetworkManager.getConfigManager().getConfig(networkName);
//get network input/output object
networkIO = LODNetworkManager.getCachedNetworkIO(
conn, networkName, networkName, null);
//get network analyst
analyst = LODNetworkManager.getNetworkAnalyst(networkIO);
double[] costThresholds = {costThreshold};
try
System.out.println("*****Begin: Shortest Path with Multiple Link Levels");
System.out.println("*****Shortest Path Using Dijkstra");
String algorithm = "DIJKSTRA";
linkLevel = 2;
costThreshold = 5000;
LogicalSubPath subPath = analyst.shortestPathDijkstra(new PointOnNet(startNodeId),
new PointOnNet(endNodeId),linkLevel, null);
PrintUtility.print(System.out, subPath, true, 10000, 0);
System.out.println("*****End: Shortest path using Dijkstra");
System.out.println("*****Shortest Path using Astar");
HeuristicCostFunction costFunction = new GeodeticCostFunction(0,-1, 0, -2);
LinkLevelSelector lls = new DynamicLinkLevelSelector(
analyst, 2, costFunction, costThresholds,
numHighLevelNeighbors, costMultiplier, null);
subPath = analyst.shortestPathAStar(
new PointOnNet(startNodeId), new PointOnNet(endNodeId), null, costFunction, lls);
PrintUtility.print(System.out, subPath, true, 10000, 0);
System.out.println("*****End: Shortest Path Using Astar");
System.out.println("*****End: Shortest Path with Multiple Link Levels");
catch (Exception e)
e.printStackTrace();
if(conn!=null)
try{conn.close();} catch(Exception ignore){}
and this is the output with the error:...
[LODNetworkAdaptorSDO::isNetworkPartitioned, DEBUG] Query String: SELECT p.PARTITION_ID FROM PROVA.ITAI11_PART$ p WHERE p.LINK_LEVEL = ? AND ROWNUM = 1 [2]
[QueryUtility::prepareIDListStatement, DEBUG] Query String: SELECT NODE_ID, PARTITION_ID FROM PROVA.ITAI11_PART$ p WHERE p.NODE_ID IN ( SELECT column_value FROM table(:varray) ) AND LINK_LEVEL = ?
[LODNetworkAdaptorSDO::readNodePartitionIds, DEBUG] Query linkLevel = 2
[NetworkIOImpl::readLogicalPartition, DEBUG] Read partition from blob table: partition 4, level 2
[LODNetworkAdaptorSDO::readPartitionBlobEntry, DEBUG] Query String: SELECT BLOB, NUM_INODES, NUM_ENODES, NUM_ILINKS, NUM_ELINKS, NUM_INLINKS, NUM_OUTLINKS, USER_DATA_INCLUDED FROM PROVA.ITAI11_PBLOB$ WHERE PARTITION_ID = ? AND LINK_LEVEL = ? [4,2]
[QueryUtility::prepareIDListStatement, DEBUG] Query String: SELECT NODE_ID, PARTITION_ID FROM PROVA.ITAI11_PART$ p WHERE p.NODE_ID IN ( SELECT column_value FROM table(:varray) ) AND LINK_LEVEL = ?
[LODNetworkAdaptorSDO::readNodePartitionIds, DEBUG] Query linkLevel = 1
[NetworkIOImpl::readLogicalPartition, DEBUG] Read partition from blob table: partition 91, level 1
[LODNetworkAdaptorSDO::readPartitionBlobEntry, DEBUG] Query String: SELECT BLOB, NUM_INODES, NUM_ENODES, NUM_ILINKS, NUM_ELINKS, NUM_INLINKS, NUM_OUTLINKS, USER_DATA_INCLUDED FROM PROVA.ITAI11_PBLOB$ WHERE PARTITION_ID = ? AND LINK_LEVEL = ? [91,1]
[NetworkIOImpl::readLogicalPartition, DEBUG] Read partition from blob table: partition 91, level 2
[LODNetworkAdaptorSDO::readPartitionBlobEntry, DEBUG] Query String: SELECT BLOB, NUM_INODES, NUM_ENODES, NUM_ILINKS, NUM_ELINKS, NUM_INLINKS, NUM_OUTLINKS, USER_DATA_INCLUDED FROM PROVA.ITAI11_PBLOB$ WHERE PARTITION_ID = ? AND LINK_LEVEL = ? [91,2]
oracle.spatial.network.lod.LODNetworkException: java.lang.NullPointerException
at oracle.spatial.network.lod.NetworkIOImpl.readPartitionFromBlob(NetworkIOImpl.java:549)
at oracle.spatial.network.lod.NetworkIOImpl.readLogicalPartition(NetworkIOImpl.java:436)
at oracle.spatial.network.lod.CachedNetworkIOImpl.readLogicalPartition(CachedNetworkIOImpl.java:114)
at oracle.spatial.network.lod.CachedNetworkIOImpl.readLogicalPartition(CachedNetworkIOImpl.java:105)
at oracle.spatial.network.lod.NetworkExplorer.getPartition(NetworkExplorer.java:335)
at oracle.spatial.network.lod.LabelSettingAlgorithm.getElementPartition(LabelSettingAlgorithm.java:520)
at oracle.spatial.network.lod.LabelSettingAlgorithm.expand(LabelSettingAlgorithm.java:561)
at oracle.spatial.network.lod.LabelSettingAlgorithm.shortestPath(LabelSettingAlgorithm.java:1362)
at oracle.spatial.network.lod.NetworkAnalyst.shortestPathHierarchical(NetworkAnalyst.java:2523)
at oracle.spatial.network.lod.NetworkAnalyst.shortestPathDijkstra(NetworkAnalyst.java:2291)
at oracle.spatial.network.lod.NetworkAnalyst.shortestPathDijkstra(NetworkAnalyst.java:2268)
at oracle.spatial.network.lod.NetworkAnalyst.shortestPathDijkstra(NetworkAnalyst.java:2249)
at calcolopercorsolod.SpWithMultiLinkLevels.main(SpWithMultiLinkLevels.java:135)
Caused by: java.lang.NullPointerException
at oracle.spatial.network.lod.NetworkIOImpl.readPartitionFromBlob(NetworkIOImpl.java:542)
... 12 more
I don't understand why the analysis does a query with PARTITION_ID=91 and LINK_LEVEL=2:
Query String: SELECT BLOB, NUM_INODES, NUM_ENODES, NUM_ILINKS, NUM_ELINKS, NUM_INLINKS, NUM_OUTLINKS, USER_DATA_INCLUDED FROM PROVA.ITAI11_PBLOB$ WHERE PARTITION_ID = ? AND LINK_LEVEL = ? [91,2]
Infact I have 512 PARTITION_ID associated to LINK_LEVEL=1 and only 16 PARTITION_ID associated to LINK_LEVEL=2. Why it search PARTITION_ID=91 with LINK_LEVEL=2? This correspondance doesn't exist.
Where I wrong?
Note that if I set LINK_LEVEL=1 it works.
Thank you very much

Approach for Oracle's TDE encryption.

Similar Messages

Maybe you are looking for