Performance of the LOD approach for network analisys

Question

Hi to all,
I open new thread for speak about performance of LOD approach for Network analysis using Java API as continuing from this thread:
Partitioning of the Network for using LOD API
I remember you that:
- I'm using Oracle 11g R2;
- my network consist of 7.817.372 of links and 6.662.079 of nodes (big network);
- LINK_LEVEL of links table is setted as NULL;
- I have partitioned the network with these procedure:
partition:
EXEC sdo_net.spatial_partition('ITALIA', 'ITALIA_PART$', 10000, 'WORK_DIR_ITALIA', 'ITALIA_PART.log', 'a', 1);
and for partition blob:
EXEC sdo_net.generate_partition_blobs('ITALIA', 1, 'ITALIA_PBLOB$', true, true, 'WORK_DIR_ITALIA', 'ITALIA_PBLOB.log', 'a');
- I'm using LOD Java API for network analysis with Netbeans IDE; I took the code from NDM_tutorial of Hillsborough network.
My first analysis is the compute of the shortest path between two extreme nodes (about 1.500 km of distance) and between two near nodes (about 80 km of distance) using Dijkstra and AStar algorithm.
I did the following tests on execution times of the compute of the shortest path:
1) with partition of maximum 10000 nodes for partition (from log file I read that were been generated 1024 partitions with 1 link level):
- between two extreme nodes, about 1 minute and 50 seconds.
- between two near nodes, about 20 seconds.
then I have re-executed the two previous procedures where I changed the value of maximum nodes for partition and I did other tests:
2) with partition of maximum 15000 nodes for partition (from log file I read that were been generated 512 partitions with 1 link level):
- between two extreme nodes, about 1 minute and 50 seconds and sometimes it goes out of memory
- between two near nodes, about 20 seconds.
3) with partition of maximum 5000 nodes for partition (from log file I read that were been generated 2048 partitions with 1 link level):
- between two extreme nodes, about 1 minute and 50 seconds and sometimes it goes out of memory
- between two near nodes, about 20 seconds.
4) with partition of maximum 2000 nodes for partition (from log file I read that were been generated 4096 partitions with 1 link level):
- between two extreme nodes, about 1 minute and 50 seconds and sometimes it goes out of memory
- between two near nodes, about 15 seconds.
I think that there is any problem because I expect execution times more lower (maximum 5-6 seconds for extreme nodes).
Even changing the maximum number of nodes to partition the execution times do not change much.
I remember you that with in-memory approach on Oracle 10gR2 the execution times for every computation between two nodes was of about 4 minutes and sometimes it goes out of memory. With LOD on Oracle 11gR2 the execution times are reduced but are too long for me.
Now, my questions are:
- @Jack Wang: (if you're reading me) do you know what are the execution times for the compute of shortest path between two nodes for USA network (about 1500 km of distance)? I remember that you have used LOD for USA (56 Millions of link and 20 Millions of nodes).
- Do you think that I wrong anything? How can I do for reduce the execution times of network analysis?
If you need any more information just ask.
Thank you to all in advance very much.

Answer 1

Jack Wang wrote:
Are the 12/13 seconds computation times for the near node pair?Yes.
You can look at an example for computing hierarchical shortest path with a 2-level network(NAVTEQ_SF) under ndm_tutorial
~ \ndm_tutorial\examples\java\src\lod\SpWithMultiLinkLevels.java.But I think that yesterday I had not yet used the LINK_LEVEL = 2 in network analysis. Infact, I used the code into ShortestPathAnalysis.java where linkLevel is always setted = 1.
Now, I'm using the example SpWithMultiLinkLevels.java and I'm seeing that linkLevel is setted = 2 before the execution of Dijkstra and AStar algorithm. This code works with Hillsborough_network. Instead with my network (note that I'm using a region of Italy with 700.000 link and 600.000 nodes called ITAI11_METERS for to test a network with 2 LINK_LEVEL), I have a problem during readPartitionFromBlob.
This is the code:
package calcolopercorsolod;
import java.io.*;
import java.sql.*;
import java.text.*;
import java.util.*;
import oracle.jdbc.OracleConnection;
import oracle.jdbc.pool.OracleDataSource;
import oracle.spatial.util.Logger;
import oracle.spatial.network.UserDataMetadata;
import oracle.spatial.network.lod.*;
import oracle.spatial.network.lod.config.*;
import oracle.spatial.network.lod.util.PrintUtility;
public class SpWithMultiLinkLevels
  private static NetworkAnalyst analyst;
  private static NetworkIO networkIO;
  private static void setLogLevel(String logLevel)
    if("FATAL".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_FATAL);
    else if("ERROR".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_ERROR);
    else if("WARN".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_WARN);
    else if("INFO".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_INFO);
    else if("DEBUG".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_DEBUG);
    else if("FINEST".equalsIgnoreCase(logLevel))
        Logger.setGlobalLevel(Logger.LEVEL_FINEST);
    else  //default: set to ERROR
        Logger.setGlobalLevel(Logger.LEVEL_ERROR);
  public static void main(String[] args) throws Exception
        String configXmlFile = "LODConfigs.xml";
        String logLevel    =    "DEBUG";
        String dbUrl       = "jdbc:oracle:thin:@oracle:1521:mySID";
        String dbUser      = "myUser";
        String dbPassword  = "myPass";
        String networkName = "ITAI11_METERS";
        long startNodeId = 15323;
        long endNodeId   = 431593;
    int linkLevel      = 1;
    double costThreshold = 1550;
    int numHighLevelNeighbors = 8;
    double costMultiplier = 1.5;
    Connection conn    = null;
    //get input parameters
    for(int i=0; i<args.length; i++)
        if(args.equalsIgnoreCase("-dbUrl"))
dbUrl = args[i+1];
else if(args[i].equalsIgnoreCase("-dbUser"))
dbUser = args[i+1];
else if(args[i].equalsIgnoreCase("-dbPassword"))
dbPassword = args[i+1];
else if(args[i].equalsIgnoreCase("-networkName") && args[i+1]!=null)
networkName = args[i+1].toUpperCase();
else if(args[i].equalsIgnoreCase("-linkLevel"))
linkLevel = Integer.parseInt(args[i+1]);
else if(args[i].equalsIgnoreCase("-configXmlFile"))
configXmlFile = args[i+1];
else if(args[i].equalsIgnoreCase("-logLevel"))
logLevel = args[i+1];
// opening connection
conn = LODNetworkManager.getConnection(dbUrl, dbUser, dbPassword);
System.out.println("Network analysis for "+networkName);
setLogLevel(logLevel);
//load user specified LOD configuration (optional),
//otherwise default configuration will be used
InputStream config = ClassLoader.getSystemResourceAsStream(configXmlFile);
LODNetworkManager.getConfigManager().loadConfig(config);
//LODConfig c = LODNetworkManager.getConfigManager().getConfig(networkName);
//get network input/output object
networkIO = LODNetworkManager.getCachedNetworkIO(
conn, networkName, networkName, null);
//get network analyst
analyst = LODNetworkManager.getNetworkAnalyst(networkIO);
double[] costThresholds = {costThreshold};
try
System.out.println("*****Begin: Shortest Path with Multiple Link Levels");
System.out.println("*****Shortest Path Using Dijkstra");
String algorithm = "DIJKSTRA";
linkLevel = 2;
costThreshold = 5000;
LogicalSubPath subPath = analyst.shortestPathDijkstra(new PointOnNet(startNodeId),
new PointOnNet(endNodeId),linkLevel, null);
PrintUtility.print(System.out, subPath, true, 10000, 0);
System.out.println("*****End: Shortest path using Dijkstra");
System.out.println("*****Shortest Path using Astar");
HeuristicCostFunction costFunction = new GeodeticCostFunction(0,-1, 0, -2);
LinkLevelSelector lls = new DynamicLinkLevelSelector(
analyst, 2, costFunction, costThresholds,
numHighLevelNeighbors, costMultiplier, null);
subPath = analyst.shortestPathAStar(
new PointOnNet(startNodeId), new PointOnNet(endNodeId), null, costFunction, lls);
PrintUtility.print(System.out, subPath, true, 10000, 0);
System.out.println("*****End: Shortest Path Using Astar");
System.out.println("*****End: Shortest Path with Multiple Link Levels");
catch (Exception e)
e.printStackTrace();
if(conn!=null)
try{conn.close();} catch(Exception ignore){}
and this is the output with the error:...
[LODNetworkAdaptorSDO::isNetworkPartitioned, DEBUG] Query String: SELECT p.PARTITION_ID FROM PROVA.ITAI11_PART$ p WHERE p.LINK_LEVEL = ? AND ROWNUM = 1 [2]
[QueryUtility::prepareIDListStatement, DEBUG] Query String: SELECT NODE_ID, PARTITION_ID FROM PROVA.ITAI11_PART$ p WHERE p.NODE_ID IN ( SELECT column_value FROM table(:varray) ) AND LINK_LEVEL = ?
[LODNetworkAdaptorSDO::readNodePartitionIds, DEBUG] Query linkLevel = 2
[NetworkIOImpl::readLogicalPartition, DEBUG] Read partition from blob table: partition 4, level 2
[LODNetworkAdaptorSDO::readPartitionBlobEntry, DEBUG] Query String: SELECT BLOB, NUM_INODES, NUM_ENODES, NUM_ILINKS, NUM_ELINKS, NUM_INLINKS, NUM_OUTLINKS, USER_DATA_INCLUDED FROM PROVA.ITAI11_PBLOB$ WHERE PARTITION_ID = ? AND LINK_LEVEL = ? [4,2]
[QueryUtility::prepareIDListStatement, DEBUG] Query String: SELECT NODE_ID, PARTITION_ID FROM PROVA.ITAI11_PART$ p WHERE p.NODE_ID IN ( SELECT column_value FROM table(:varray) ) AND LINK_LEVEL = ?
[LODNetworkAdaptorSDO::readNodePartitionIds, DEBUG] Query linkLevel = 1
[NetworkIOImpl::readLogicalPartition, DEBUG] Read partition from blob table: partition 91, level 1
[LODNetworkAdaptorSDO::readPartitionBlobEntry, DEBUG] Query String: SELECT BLOB, NUM_INODES, NUM_ENODES, NUM_ILINKS, NUM_ELINKS, NUM_INLINKS, NUM_OUTLINKS, USER_DATA_INCLUDED FROM PROVA.ITAI11_PBLOB$ WHERE PARTITION_ID = ? AND LINK_LEVEL = ? [91,1]
[NetworkIOImpl::readLogicalPartition, DEBUG] Read partition from blob table: partition 91, level 2
[LODNetworkAdaptorSDO::readPartitionBlobEntry, DEBUG] Query String: SELECT BLOB, NUM_INODES, NUM_ENODES, NUM_ILINKS, NUM_ELINKS, NUM_INLINKS, NUM_OUTLINKS, USER_DATA_INCLUDED FROM PROVA.ITAI11_PBLOB$ WHERE PARTITION_ID = ? AND LINK_LEVEL = ? [91,2]
oracle.spatial.network.lod.LODNetworkException: java.lang.NullPointerException
at oracle.spatial.network.lod.NetworkIOImpl.readPartitionFromBlob(NetworkIOImpl.java:549)
at oracle.spatial.network.lod.NetworkIOImpl.readLogicalPartition(NetworkIOImpl.java:436)
at oracle.spatial.network.lod.CachedNetworkIOImpl.readLogicalPartition(CachedNetworkIOImpl.java:114)
at oracle.spatial.network.lod.CachedNetworkIOImpl.readLogicalPartition(CachedNetworkIOImpl.java:105)
at oracle.spatial.network.lod.NetworkExplorer.getPartition(NetworkExplorer.java:335)
at oracle.spatial.network.lod.LabelSettingAlgorithm.getElementPartition(LabelSettingAlgorithm.java:520)
at oracle.spatial.network.lod.LabelSettingAlgorithm.expand(LabelSettingAlgorithm.java:561)
at oracle.spatial.network.lod.LabelSettingAlgorithm.shortestPath(LabelSettingAlgorithm.java:1362)
at oracle.spatial.network.lod.NetworkAnalyst.shortestPathHierarchical(NetworkAnalyst.java:2523)
at oracle.spatial.network.lod.NetworkAnalyst.shortestPathDijkstra(NetworkAnalyst.java:2291)
at oracle.spatial.network.lod.NetworkAnalyst.shortestPathDijkstra(NetworkAnalyst.java:2268)
at oracle.spatial.network.lod.NetworkAnalyst.shortestPathDijkstra(NetworkAnalyst.java:2249)
at calcolopercorsolod.SpWithMultiLinkLevels.main(SpWithMultiLinkLevels.java:135)
Caused by: java.lang.NullPointerException
at oracle.spatial.network.lod.NetworkIOImpl.readPartitionFromBlob(NetworkIOImpl.java:542)
... 12 more
I don't understand why the analysis does a query with PARTITION_ID=91 and LINK_LEVEL=2:
Query String: SELECT BLOB, NUM_INODES, NUM_ENODES, NUM_ILINKS, NUM_ELINKS, NUM_INLINKS, NUM_OUTLINKS, USER_DATA_INCLUDED  FROM PROVA.ITAI11_PBLOB$ WHERE PARTITION_ID = ? AND LINK_LEVEL = ? [91,2]
Infact I have 512 PARTITION_ID  associated to LINK_LEVEL=1 and only 16 PARTITION_ID associated to LINK_LEVEL=2. Why it search PARTITION_ID=91 with LINK_LEVEL=2? This correspondance doesn't exist.
Where I wrong?
Note that if I set LINK_LEVEL=1 it works.
Thank you very much