Performance of XPath

Hi. I am thinking of using XPath but am worried if XPath's performance is not good. Anyone can give some comments? Is there any implementation that incur lower than O(n) time when locating a child by looking at one of its attributes?
Thanks.

I don't know if it replies to your question but :
From my tests : JXPath from apache is the slower one, followed by
jaxen and by JXP which has on my tests the best results.
1. http://jakarta.apache.org/commons/jxpath/
2. http://www.jaxen.org
3. http://www.japisoft.com/jxpath
Note that I include a benchmark tester in JXP over 150 expressions.
I find a 25% better result on JXP rather than on jaxen.
Hope it helps,
A.Brillant

Similar Messages

Poor Performance of XPath in 1.5.0 JDK?

Dear all
I just started using the 1.5.0 java JDR/JRE. I am loading an XML file using javax.xml.parsers.DocumentBuilderFactory and navigating through the document using javax.xml.xpath.XPath.
Unfortunately the performance seems to be horendous even when I do simple navigation over the nodes. I have refactored the code so that XPath expressions are compiled only once and re-used. However I am not getting much performance improvement. I used a profiler and can basically narrow the problem to XPath as there is nothing else that is significant that happens in that function. Fairly simple code that invokes about 7 XPath expressions seems to take about .5 of a second on average. The XPath expressions are simple like "attribute::unit", "value/attribute::value" and "code/attribute::code".
So what gives? Is there a way i can improve the performance? Am I missing something?
Cheers, Martin.

I experiencing the same thing. Did you solve the problem in the end? And how?

XPath processing with namespaces does not work. Please review my code..

Dear all,
I am working on with the IMS Group's Enterprise XML standard (http://www.imsproject.org/enterprise/) and need to retrieve member information using XPath.
After significant fault finding, I have narrowed down the problem to the XML file containing multiple namespaces in the root element. I have checked out some forum postings here and on Google (programmer's best friend) for possible solutions... namely the implementation of the my own NamespaceContext object. I have implemented and yet I am still not able to retrieve the data I want.
Could someone advise where I am going wrong?
When I remove the namespaces from the root element I can find out the number of member elements in the XML (just for testing atm). I could write a regular expression to remove the namespaces from my XML, but that's just dodgy..
Here's a sample XML:
<enterprise xmlns="http://www.imsproject.org/xsd/imsep_rootv1p01" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:webct="http://www.webct.com/IMS">
<properties>...</properties>
<group>...</group>
<membership>
<member>...</member>
</membership>
</enterprise>
My implementation of the a NamespaceContext object:
public class IMSProjectNamespaceContext implements NamespaceContext {
public String getNamespaceURI(String prefix) {
if(prefix == null) throw new NullPointerException("Null prefix");
else if("xsi".equals(prefix)) return "http://www.w3.org/2001/XMLSchema-instance";
else if("webct".equals(prefix)) return "http://www.webct.com/IMS";
else if("xml".equals(prefix)) return XMLConstants.XML_NS_URI;
return XMLConstants.NULL_NS_URI;
}// end of overriding getNamespaceURI method
// This method is not necessary for XPath processing
public String getPrefix(String uri) {
throw new UnsupportedOperationException();
}// end of overriding getPrefix method
// This method is not necessary for XPath processing
public java.util.Iterator getPrefixes(String uri) {
throw new UnsupportedOperationException();
}// end of overriding getPrefixes method
}// end of IMSProjectNamespaceContext class
My method attempting to use XPath... but not successful (it should print out the number of member elements if XPath works correctly with my XML file, but only prints out 0 each time):
private void getSectionMembers(String dataXML) throws Exception {
try {
InputSource inputSource = new InputSource(new StringReader(dataXML));
XPath xPath = XPathFactory.newInstance().newXPath();
xPath.setNamespaceContext(new IMSProjectNamespaceContext());
XPathExpression xPathExpression = xPath.compile("/enterprise/membership/member");
NodeList nodes = (NodeList)xPathExpression.evaluate(inputSource, XPathConstants.NODESET);
System.out.println(nodes.getLength()); // Debugging
} catch(Exception ex) {
ex.printStackTrace();
throw new Exception("XPath Querying Failed.");
}// end of xPathQuery method

Dear dvohra09,
Thank you for your quick reply. Much appreciated. I have implemented a NamespaceContext object that is able to set more than one prefix for a URI as you suggested.
It works when I remove the first namespace that is generated in the IMSProject Enterprise XML:
xmlns="http://www.imsproject.org/xsd/imsep_rootv1p01"
When I hack the XML and set a prefix for this ('default') namespace it works... For example:
xmlns:ims="http://www.imsproject.org/xsd/imsep_rootv1p01"
Is the NamespaceContext class supposed to handle for so called "default" namespaces (without needing to touch the XML)?
My method is performing the XPath query is below:
private void getSectionMembers(String dataXML) throws Exception {
try {
InputSource inputSource = new InputSource(new StringReader(dataXML));
IMSProjectNamespaceContext imsproject = new IMSProjectNamespaceContext();
imsproject.setNamespace("xsi", "http://www.w3.org/2001/XMLSchema-instance");
imsproject.setNamespace("webct", "http://www.webct.com/IMS");
imsproject.setNamespace("ims", "http://www.imsproject.org/xsd/imsep_rootv1p01lme");
XPath xPath = XPathFactory.newInstance().newXPath();
xPath.setNamespaceContext(imsproject);
XPathExpression xPathExpression = xPath.compile("//ims:enterprise/@xmlns/text()");
NodeList nodes = (NodeList)xPathExpression.evaluate(inputSource, XPathConstants.NODESET);
System.out.println(nodes.getLength()); // Debugging
System.out.println(nodes.item(0).getNodeValue()); // Debugging
} catch(Exception ex) {
ex.printStackTrace();
throw new Exception("XPath Querying Failed.");
}// end of xPathQuery method
Regards
Zig

StreamSource, Singleton, Transformation and performance

Maybe my issue is more of design issue (disclaimer: yes, I am new to Java).
I am sure this problem has been solved several times ... so I hope someone can offer their sagely advice.
I have a tomcat (5.5) app that I am working on. I am currently using xsl to filter/do my queries (and maybe I need a different approach there, so I'm open). This is really about a query not taking 5 minutes. I thought th XSL would be the better way to do that.
I have implemented a singleton to hold the XML StreamSource in memory and that seems to be working. I verified the singleton is loading and staying, but it's still taking the same amount of time to execute the query (apply the xsl).
Is it because the StreamSource still has to Stream (i.e. I/O operation)? What is a more efficient memory (RAM is not an issue in this case) of loading and holding an XML to perform queries (XPath/XSL) against?
Below is the Singleton Implemenation:
public class Lookup {
     private static Lookup instance = null;
     private static StreamSource theXML = null;
    private static String xmldoc_filename = "/path/to/17MBPlusOf.xml";
    protected Lookup ()
       loadXML(xmldoc_filename);
    public static synchronized Lookup getInstance() {
        if( instance == null ) {
            instance = new Lookup();
        return instance;
    private void loadXML(String filename)
        if( theXML == null ) {
               theXML = new StreamSource(xmldoc_filename);
   public String xslTransform( String query ) throws TransformerException
          String xslFile = "/path/to/XSLFile.xsl";
          String xmlString;
          TransformerFactory tFactory = TransformerFactory.newInstance();
          // do the transformation here
          StreamSource theXSL = new StreamSource(xslFile);
          Transformer xslTrans = tFactory.newTransformer(theXSL);
          xslTrans.setParameter("qryText", query);
          ByteArrayOutputStream byteStream = new ByteArrayOutputStream();
          Result res = new StreamResult(byteStream);
          xslTrans.transform(theXML, res);
          xmlString = byteStream.toString();
          return xmlString;
     }

java5on wrote:
Thanks .... will try it tomorrow ... but I don't see how caching a 30-50 line xsl chunk is going to speed up the process in comparison to caching the 34+K lines of xml.
In Java, what would be the overall preferred method to load/cache a large amount of data (XML) for processing (parsing/querying) by multiple clients/queries? oh, i didn't completely follow what you were doing there. You should probably parse the xml into a Document and use a DOMSource for your transform call. I don't know if Documents are thread-safe, so you may need a similar ThreadLocal caching scheme here as well.

Fast XPath API

JXP is a Fast XPath 1.0 compliant API.
Download :
http://www.japisoft.com/jxpath/download/unregistered.zip
Home :
http://www.japisoft.com/jxpath
Forum :
http://www.japisoft.com/forum/index.php
News :
Cache support for the navigator (by default enabled)
Fixed bug for simple expressions like 1+1
Fixed bug for // usage
A DOM document can be used rather than an Element
Main features :
* High performance full XPath 1.0 solution,
* 25% faster than jaxen,
* SAX - like event system,
* Inner expression tree for fast multiple document evaluation,
* Syntax error location and cause,
* DOM support,
* FastParser lightweight node support,
* Customizable API with navigator, function libraries,
* Plugin for each node type system,
* Lower/Uppercase feature
* Full samples with DOM,
* Tested on 150 XPath expressions,
* Benchmark tests included,
* Package of 85 Ko
* Open source for the registered version,
* JDK1.1 and later compliant (tested on JDK1.1.8 and JDK1.4.1)
A.Brillant
http://www.japisoft.com

Mainly :
- Speed,
- Compact : 100Ko,
- Pluggable kit for custom node usage,
- JDK 1.1,
- Featuring like lowercase/uppercase for HTML.
Hope it replies,
A.Brillant

Pro grammatically generate build in document ID at the footer of Word document?

Hi All,
i am trying to added Document ID at the footer of Word file programatically using the following code. Following code is working only when i am uploading the document first time and its adding the ID at the footer. But problem is when i am uploading
same file into same document library, document Id is just going away. i have tried to use ItemUpdated method but it doesn't work. what i need to modify into code, so that same document ID will be remains at the footer of document no matter how many times uploaded
document. any help will greatly appreciated.
using System;
using System.Security.Permissions;
using System.Runtime.InteropServices;
using Microsoft.SharePoint;
using System.IO;
using System.IO.Packaging;
using DocumentFormat.OpenXml.Packaging;
using System.Xml;
using System.Collections.Generic;
using Microsoft.SharePoint.Security;
using Microsoft.SharePoint.Utilities;
using Microsoft.SharePoint.Workflow;
namespace AddHeaderFooterReceiver.ItemAddedEventReceiver
/// <summary>
/// List Item Events
/// </summary>
public class ItemAddedEventReceiver : SPItemEventReceiver
/// <summary>
/// An item was added.
/// </summary>
public string GetFooter()
string footerVal = "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\" ?><w:ftr xmlns:ve=\"http://schemas.openxmlformats.org/markup-compatibility/2006\"
xmlns:o=\"urn:schemas-microsoft-com:office:office\" xmlns:r=\"http://schemas.openxmlformats.org/officeDocument/2006/relationships\" xmlns:m=\"http://schemas.openxmlformats.org/officeDocument/2006/math\" xmlns:v=\"urn:schemas-microsoft-com:vml\"
xmlns:wp=\"http://schemas.openxmlformats.org/drawingml/2006/wordprocessingDrawing\" xmlns:w10=\"urn:schemas-microsoft-com:office:word\" xmlns:w=\"http://schemas.openxmlformats.org/wordprocessingml/2006/main\" xmlns:wne=\"http://schemas.microsoft.com/office/word/2006/wordml\"><w:p
w:rsidR=\"00C24C70\" w:rsidRDefault=\"00C24C70\"><w:pPr><w:pStyle w:val=\"Footer\" /></w:pPr><w:r><w:t>Hi</w:t></w:r></w:p><w:p w:rsidR=\"00C24C70\" w:rsidRDefault=\"00C24C70\"><w:pPr><w:pStyle
w:val=\"Footer\" /></w:pPr></w:p></w:ftr>";
return footerVal;
public void WDAddFooter(Stream footerContent, Stream fileContent)
// Given a document name, and a stream containing valid footer content,
// add the stream content as a footer in the document.
const string documentRelationshipType = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument";
const string wordmlNamespace = "http://schemas.openxmlformats.org/wordprocessingml/2006/main";
const string footerContentType = "application/vnd.openxmlformats-officedocument.wordprocessingml.footer+xml";
const string footerRelationshipType = "http://schemas.openxmlformats.org/officeDocument/2006/relationships/footer";
const string relationshipNamespace = "http://schemas.openxmlformats.org/officeDocument/2006/relationships";
PackagePart documentPart = null;
using (Package wdPackage = Package.Open(fileContent, FileMode.Open, FileAccess.ReadWrite))
// Get the main document part (document.xml).
foreach (System.IO.Packaging.PackageRelationship relationship in wdPackage.GetRelationshipsByType(documentRelationshipType))
Uri documentUri = PackUriHelper.ResolvePartUri(new Uri("/", UriKind.Relative), relationship.TargetUri);
documentPart = wdPackage.GetPart(documentUri);
// There is only one officeDocument.
break;
Uri uriFooter = new Uri("/word/footer1.xml", UriKind.Relative);
if (wdPackage.PartExists(uriFooter))
// Although you can delete the relationship
// to the existing node, the next time you save
// the document after making changes, Word
// will delete the relationship.
wdPackage.DeletePart(uriFooter);
// Create the footer part.
PackagePart footerPart = wdPackage.CreatePart(uriFooter, footerContentType);
// Load the content from the input stream.
// This may seem redundant, but you must read it at some point.
// If you ever need to analyze the contents of the footer,
// at least it is already in an XmlDocument.
// This code uses the XmlDocument object only as
// a "pass-through" -- giving it a place to hold as
// it moves from the input stream to the output stream.
// The code could read each byte from the input stream, and
// write each byte to the output stream, but this seems
// simpler...
XmlDocument footerDoc = new XmlDocument();
footerContent.Position = 0;
footerDoc.Load(footerContent);
// Write the footer out to its part.
footerDoc.Save(footerPart.GetStream());
// Create the document's relationship to the new part.
PackageRelationship rel = documentPart.CreateRelationship(uriFooter, TargetMode.Internal, footerRelationshipType);
string relID = rel.Id;
// Manage namespaces to perform Xml XPath queries.
NameTable nt = new NameTable();
XmlNamespaceManager nsManager = new XmlNamespaceManager(nt);
nsManager.AddNamespace("w", wordmlNamespace);
// Get the document part from the package.
// Load the XML in the part into an XmlDocument instance.
XmlDocument xdoc = new XmlDocument(nt);
xdoc.Load(documentPart.GetStream());
// Find the node containing the document layout.
XmlNode targetNode = xdoc.SelectSingleNode("//w:sectPr", nsManager);
if (targetNode != null)
// Delete any existing references to footers.
//XmlNodeList footerNodes = targetNode.SelectNodes("./w:footerReference", nsManager);
//foreach (System.Xml.XmlNode footerNode in footerNodes)
// targetNode.RemoveChild(footerNode);
// Create the new footer reference node.
XmlElement node = xdoc.CreateElement("w:footerReference", wordmlNamespace);
XmlAttribute attr = node.Attributes.Append(xdoc.CreateAttribute("r:id", relationshipNamespace));
attr.Value = relID;
node.Attributes.Append(attr);
targetNode.InsertBefore(node, targetNode.FirstChild);
// Save the document XML back to its part.
xdoc.Save(documentPart.GetStream(FileMode.Create, FileAccess.Write));
public override void ItemAdded(SPItemEventProperties properties)
string extension = properties.ListItem.Url.Substring(properties.ListItem.Url.LastIndexOf(".") + 1);
if (extension == "docx")
//string headerContent = GetHeader().Replace("hello", properties.ListItem["Name"].ToString());
//string footerContent = GetFooter().Replace("Hi", properties.ListItem["Modified"].ToString());
//string footerContent = GetFooter().Replace("Hi", properties.ListItem["_dlc_DocId"].ToString() + " V : " +properties.ListItem["_UIVersionString"].ToString());
string footerContent = GetFooter().Replace("Hi", properties.ListItem["_dlc_DocId"].ToString());
//string footerContent1 = GetFooter().Replace("Hi", properties.ListItem["_UIVersionString"].ToString());
//Stream headerStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(headerContent));
//Stream footerStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(footerContent));
Stream footerStream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(footerContent));
//Stream footerStream1 = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(footerContent1));
MemoryStream fileStream = new MemoryStream();
fileStream.Write(properties.ListItem.File.OpenBinary(), 0, (int)properties.ListItem.File.TotalLength);
//WDAddHeader(headerStream, fileStream);
//WDAddFooter(footerStream, fileStream);
WDAddFooter(footerStream, fileStream);
//WDAddFooter(footerStream1, fileStream);
properties.ListItem.File.SaveBinary(fileStream);

Instead of using Event Receiver approach have you tried looking on adding labels instead?
Here's a helpful article on how to add labels on your document
http://blog.isaacblum.com/2011/02/28/add-document-id-to-word-document-automatically/
Hope this helps
Artificial intelligence can never beat natural stupidity.

Using dynamic select (xsl:value-of) in XSLT (ABAP)

Hi folks,
I did a quite extensive search in the archive but couldn't find an answer on my question. If it has been answered before please just provide me with the link :).
I need to make a dynamic select in a xsl-value-of in an ABAP XSLT. I do not know the structure of the input message (can be of any kind) and I want to perform a xpath query on the input message. The result of the transformation should be the value (if any) of the xpath query. As the query is not known in before I want to pass the query at runtime using a parameter in XSLT. If I pass my query as a string parameter the value-of just returns the string and not the value of the xpath query. Creating the parameter as a node-set doesn't work either for me.
So is there any way to use a parameter as input to an xpath query?
Or is my last resort using a JAVA XSLT processor. This is running inside a ABAP proxy, so a call to JAVA would be a greater hassle.
Kind regards
Joscha

What DB version?
The following SQL
select xmltransform(XMLTYPE('<song>
<title>Isn''t she lovely?</title>
</song>'), XMLTYPE('<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output method="xml" encoding="UTF-8"></xsl:output>
<xsl:template match="//song">
    <out>
      <xsl:value-of select="title" disable-output-escaping="yes"></xsl:value-of>
    </out>
</xsl:template>
</xsl:stylesheet>'))
from dual;produces this on 11.1.0.6
<?xml version="1.0" encoding="UTF-8"?>
<out>Isn't she lovely?</out>and this on 10.2.0.4
<out>Isn't she lovely?</out>I found a couple of previous discussions regarding this at
{thread:id=679397}
{thread:id=879301}
The last discussion provides a workaround if you need it.

XSLT with large file size

Hello all,
I am very new to XSL. I have been reading a lot on it. I have a small transformation program which takes an xml file, an xsl file and performs the transformation. Everything works fine when the file size is in terms of KB. When I tried the same program with the file size of around 118MB, it is giving me out of memory exception. I would appreciate any comments to make my program work for bigger file size. I am posting my java code and xsl file
public static void xsl(String inFilename, String outFilename, String xslFilename) {
try {
// Create transformer factory
TransformerFactory factory = TransformerFactory.newInstance();
// Use the factory to create a template containing the xsl file
Templates template = factory.newTemplates(new StreamSource(
new FileInputStream(xslFilename)));
// Use the template to create a transformer
Transformer xformer = template.newTransformer();
// Prepare the input and output files
Source source = new StreamSource(new FileInputStream(inFilename));
Result result = new StreamResult(new FileOutputStream(outFilename));
// Apply the xsl file to the source file and write the result to the output file
xformer.transform(source, result);
} catch (FileNotFoundException e) {
System.out.println("Exception " + e);
} catch (TransformerConfigurationException e) {
// An error occurred in the XSL file
System.out.println("Exception " + e);
} catch (TransformerException e) {
// An error occurred while applying the XSL file
// Get location of error in input file
SourceLocator locator = e.getLocator();
int col = locator.getColumnNumber();
int line = locator.getLineNumber();
String publicId = locator.getPublicId();
String systemId = locator.getSystemId();
System.out.println("Exception " + e);
System.out.println("locator " + locator.toString());
System.out.println("line : " + line);
System.out.println("col : " + col);
System.out.println("No Exception");
xsl file :
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output method="xml" indent="yes"/>
<xsl:template match="/">
<xsl:element name="Hosts">
<xsl:apply-templates select="//maps/map/hosts"/>
</xsl:element>
</xsl:template>
<xsl:template match="//maps/map/hosts">
<xsl:for-each select="*">
<xsl:element name="host">
<xsl:element name="ip"><xsl:value-of select="./@ip"/></xsl:element>
<xsl:element name="known"><xsl:value-of select="./known/@v"/></xsl:element>
<xsl:element name="targeted"><xsl:value-of select="./targeted/@v"/></xsl:element>
<xsl:element name="asn"><xsl:value-of select="./asn/@v"/></xsl:element>
<xsl:element name="reverse_dns"><xsl:value-of select="./reverse_dns/@v"/></xsl:element>
</xsl:element>
</xsl:for-each>
</xsl:template>
Thanks,
Namrata
</xsl:stylesheet>

One thing you could try to do is avoid using xpath like ".//" and "*".
I had many problems in terms of memory consuptiom and performance with xpaths like above.
Altrought you have a little more work to code your xslt it performs better, and probably you will write once and run many times.

Embedding XML in XML for writing to database

I am creating a BPEL process in Oracle BPEL Manager 10.1.2.0.2 which will read an XML file using a file adapter, perform some XPath operations on the document to obtain attribute values from its content, then write those values and the document itself to a database table using a database adapter. The attribute to receive the document is a CLOB of type SQL.XMLType.
Given that use of a database adapter typically involves using a stylesheet to map the contents of a working document to a document reflecting the table schema, how do I go about mapping the XML document (as an unparsed string, with all tags intact) to the element of type string that represents the value to write to the XMLType CLOB? Every technique I've tried thus far strips away the tags.
Thanks for your time,
Paul Camann

Hi Rauno,
that sample was added in 10.1.3.1 but the idea should work in 10.1.2.0.2.
I am copying and pasting the key parts of that README.txt:
<copy>
<from expression="ora:getContentAsString(bpws:getVariableData('Invoke_1_readMovieSelect_title_param_OutputVariable','MoviesCollection'))"/>
<to variable="Invoke_2_merge_InputVariable"
part="MoviesAsClobsCollection"
query="/ns6:MoviesAsClobsCollection/ns6:MoviesAsClobs/ns6:xml"/>
</copy>
Notice the use of the ora:getContentAsString custom function.
The assign looks like this (in .bpel)
<assign name="Assign_4">
<copy>
<from expression="ora:parseEscapedXML(bpws:getVariableData('Invoke_3_readMovieAsClobSelect_title_param_OutputVariable','MoviesAsClobsCollection','/ns5:MoviesAsClobsCollection/ns5:MoviesAsClobs/ns5:xml'))"/>
<to variable="outputVariable" part="payload"
query="/ns4:MoviesCollection"/>
</copy>
</assign>
Those are the two xpaths you need for converting an xml to a CLOB/string and vice versa.
Thanks
Steve

Scraping data from Wikipedia using Htmlagilitypack

I am trying to scrape data from a table on the Wikipedia website and so far I have managed to find the nodes that I need to reference. There are a large number of entries in the table on Wikipedia, however, when I run the app I only get twelve results
and they are all identical. All results returned are duplicates of the first entry in the table.
Any ideas on how to fix?
protected async override void OnNavigatedTo(NavigationEventArgs e)
base.OnNavigatedTo(e);
string htmlPage = "";
htmlPage = await client.GetStringAsync("http://en.wikipedia.org/wiki/List_of_Games_with_Gold_games");
HtmlDocument htmlDocument = new HtmlDocument();
htmlDocument.LoadHtml(htmlPage);
foreach (var div in htmlDocument.DocumentNode.SelectNodes(".//h2"))
GameHistory newGameHistory = new GameHistory();
newGameHistory.historyTitle = div.SelectSingleNode("//i//a").InnerText.Trim();
newGameHistory.historyAdded = div.SelectSingleNode("//span[starts-with(@style, 'white')]").InnerText.Trim();
newGameHistory.historyRemoved = div.SelectSingleNode("(//span[starts-with(@style, 'white')])[2]").InnerText.Trim();
gameHistory.Add(newGameHistory);
lstGameHistory.ItemsSource = gameHistory;

@WSH_ALT the code you quote is HTMLAgilityPack syntax for performing an XPath query.
@Keith Kerswell
Without seeing more of your code, I can't be sure of your exact problem. However, the following code will successfully parse the tables on that page into two lists of objects (one for each table). Hope this helps.
public class GameHistoryItem
public string historyTitle { get; set; }
public string historyAdded { get; set; }
public string historyRemoved { get; set; }
public static async void parseWikiTables()
var client = (App.Current as App).client;
// Create a list to hold our items
List<GameHistoryItem> xbox360GameHistory = new List<GameHistoryItem>();
List<GameHistoryItem> xboxOneGameHistory = new List<GameHistoryItem>();
// Get the contents of the page into a string
var rawData = await client.GetStringAsync("http://en.wikipedia.org/wiki/List_of_Games_with_Gold_games");
// Create an HtmlDocument
var doc = new HtmlAgilityPack.HtmlDocument();
// Parse the contents into the document (e.g. load the DOM)
doc.LoadHtml(rawData);
// Now, there are two tables on this page we care about: "Xbox 360" and "Xbox One"
// It turns out that these tables both have a class of "wikitable", so...
var xboxTables = doc.DocumentNode.SelectNodes(".//table[contains(concat(' ', normalize-space(@class), ' '), ' wikitable ')]");
// Let's find out if we got exactly two tables:
if (xboxTables.Count == 2)
Debug.WriteLine("Life Is grand, we have 2 tables!");
bool is360 = true;
foreach (var xbTable in xboxTables)
try
// Get all the rows in this table
var rows = xbTable.SelectNodes(".//tr");
if (rows != null)
// The first row is the header row, just skip it
for (int x = 1; x < rows.Count; x++) {
var row = rows[x];
var cells = row.SelectNodes(".//td");
var ghi = new GameHistoryItem();
ghi.historyTitle = cells[1].InnerText;
ghi.historyAdded = cells[2].InnerText;
ghi.historyRemoved = cells[3].InnerText;
if (is360)
{ xbox360GameHistory.Add(ghi); }
else
{ xboxOneGameHistory.Add(ghi); }
catch (Exception e) { Debug.WriteLine("Error: " + e.Message); }
is360 = false;
int x360Count = xbox360GameHistory.Count;
Debug.WriteLine("There are {0} Xbox360 items, the last of which is: {1}",
x360Count,
xbox360GameHistory[x360Count - 1].historyTitle);
int xOneCount = xboxOneGameHistory.Count;
Debug.WriteLine("There are {0} Xbox One items, the last of which is: {1}",
xOneCount,
xboxOneGameHistory[xOneCount - 1].historyTitle);
The debug console ouput of this method is:
Life Is grand, we have 2 tables!
There are 42 Xbox360 items, the last of which is: Sniper Elite V2
There are 11 Xbox One items, the last of which is: #IDARB (It Draws a Red Box)

Query in XMLType

I have stored a typical Purchase Order List as XMLType in
Oracle. The structure looks roughly as -
<polist>
<po>
<customer>
<lineitems>
<lineitem>
<item>
<price>
<quantity>
So my polist has multipe po's and each po's has multipe items.
Now, I want to perform following xpath query -
- /polist/po/lineitems/lineitem/item[price="$100"]
In other words, find me an "item with specific price" from list
of all the items in all the po's in my polist.
From what I understand, predicates are not support by XPATH in
Oracle. How can I achieve this using SQL & Extract/exist ?
Any thoughts ??
--Naren

Steve:
Thanks for your response. Your solution works !! BTW, your book
has been very handy in my work.
Now, I am taking the same PO and trying to store it into
seperate tables. My document structure is similar to Example 12-
20 in your book (Oracle XML Applications) - department with
several employees. In my case, I have a purchase order with
several lineitems.
Essentially, I followed the example and got the code (written by
you) from Oreilly website to make it work with my example. Now
the problem is when I use putXML to save, the first level gets
saved with no problem.
So, as shown in the snippet below -
>>>>
<?xml version="1.0" encoding='UTF-8'?>
<ROWSET>
<ROW>
<PO_ID>10</PO_ID>
<LINEITEMS>
<LINEITEM>
<ID>7782</ID>
<PRICE>100</PRICE>
<TAXRATE>24</TAXRATE>
<QUANTITY>20</QUANTITY>
<DISCOUNT>10</DISCOUNT>
</LINEITEM>
>>>>
If I don't include the "LINEITEM" data, the instead of trigger
fires properly and the PO_ID makes it to the po_table when I
use "putXML" utility.
BTW, I modeled everything like example 12-20 - po_table,
lineitem_table, lineitem_type, lineitem_list, instead of
trigger, po_view, etc...
Now, when I try to use the "putXML" to insert data at both
levels - po and lineitem as shown in above snippet of input xml
doc, I get the following exception -
java OracleXML putXML -user "scott/tiger" -fileName bar.xml po_view
oracle.xml.sql.OracleXMLSQLException: 'java.sql.SQLException:
Non supported character set: oracle-ch
aracter-set-178' encountered during processing ROW element 0.
All prior XML row changes were rolled
back. in the XML document.
at oracle.xml.sql.dml.OracleXMLSave.saveXML
(OracleXMLSave.java:2298)
at oracle.xml.sql.dml.OracleXMLSave.saveXML
(OracleXMLSave.java:2175)
at oracle.xml.sql.dml.OracleXMLSave.insertXML
(OracleXMLSave.java:1264)
at OracleXML.Put_XML(OracleXML.java:467)
at OracleXML.ExecutePutXML(OracleXML.java:389)
at OracleXML.main(OracleXML.java:177)
I tried several variation by changing the encoding in my XML
document to - UTF-8, Unicode, etc..
I am using Oracle 9i (9.0.1.1) on windows 2000.
If you need additional information, please let me know !!
--Naren

Iterate through all documents

Hi,
We have a container which contains *1 million* documents, each of which has size 5K. Container type is wholedocument. We need to iterate through all document regualrly using container.getAllDocuments(containerTxn, DBXML_LAZY_DOCS | DBXML_NO_INDEX_NODES). For each document, we will perform a xpath query in xml document in memory.
The problem here is about whether we should use transaction. If transaction is used, getAllDocuments will report "Lock table is out of available lock entries" error when it scan about 9383 ducuments. Here are lock setting:
envp->set_lk_max_lockers(6000);
envp->set_lk_max_locks(6000);
envp->set_lk_max_objects(8000);
We have 1 million documents, it is impossible to set lock object to 1 million.
My questions:
(1) Is this normal? If so, why performing a dbxml query on the container doesn't cause such an error. I believe dbxml query has to iterate all of documents too.
(2) Is it possible to use a seek position for getAllDocuments? Thus we can commit transaction every iterating 1000 documents, and then continue the iteration from the last seek position.
(3) if we don't use transaction to call getAllDocuments, what possible problems will happen? Other threads are using transaction to read/update the documents.
Thanks.
-Yuan

Yuan,
The quickest thing to try is to add the flag DB_READ_COMMITTED to getAllDocuments(). That may reduce the number of locks you need. While you may not be able to set the various lock parameters to 1 million, you should set them much larger than you have now (e.g. 50k - 100k) for this sort of operation.
Not using transactions at all can lead to inconsistent data mostly related to index consistency with document content. Depending on the nature of your updates it may be safe. You could get stray exceptions but nothing should crash. E.g. if you are only adding/removing documents you are probably safe without transactions; although you might iterate to a document that's been removed so you'll see an exception which would have to be ignored. I'm not 100% certain this will work well but if you can test it in a high-update environment and see no issues that will tell you. The read side will not interfere with the updates.
As for "seeking" if you look under the covers getAllDocuments() is really just a wrapper for an index lookup on the name index. The XmlIndexLookup API is quite flexible in terms of returning ranges. If you want to look into returning ranges of documents, do this:
1. look at the code in XmlContainer.cpp in the function "getDocs()". It uses XmlIndexLookup to perform the operation.
2. look at the XmlIndexLookup API to figure out how to turn the lookup into a range lookup vs looking up ALL items in the index
Using XmlIndexLookup and tracking the name of your last-used document you can easily ask for all documents "higher" than that one. Further, if you know how your documents are named lexicographically you can give it a maximum as well if you wanted, but given that you can set your XmlQueryContext to be lazy you can simply iterate N times to get the next N documents without paying a penalty retrieving any documents you don't need.
For the sort of thing you are doing I highly recommend using XmlIndexLookup directly (with or without transactions) to get the flexibility you need. XmlContainer::getAllDocuments() is merely a "convenience" layer on that class.
Regards,
George

Error in applying xsl

Dear friends,
I met a problem when I try to use the OracleJSP JML taglib.
I use Window2000, Oracle8.1.7, Apache1.3.9, JServ1.x. And then, I use the sample code "Hello.jsp" and "Hello.xsl", which are provided by oracle. I also follow their instruction to configure. However, I got the following result:
JSP Error
Exception:
javax.servlet.jsp.JspException: Error applying stylesheet Hello.xsl
void ots.hello1._jspService(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse)
void oracle.jsp.runtime.HttpJsp.service(javax.servlet.ServletRequest,javax.servlet.ServletResponse)
void oracle.jsp.app.JspApplication.dispatchRequest(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse)
void oracle.jsp.JspServlet.doDispatch(oracle.jsp.app.JspRequestContext)
void oracle.jsp.JspServlet.internalService(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse)
void javax.servlet.http.HttpServlet.service(javax.servlet.ServletRequest,javax.servlet.ServletResponse)
void org.apache.jserv.JServConnection.processRequest()
void org.apache.jserv.JServConnection.run()
void java.lang.thread.run()
Looking for your suggestions! Thanks a lot in advance!
xiangnan

One thing you could try to do is avoid using xpath like ".//" and "*".
I had many problems in terms of memory consuptiom and performance with xpaths like above.
Altrought you have a little more work to code your xslt it performs better, and probably you will write once and run many times.

Reverse tele cine file size

Quick question here. Is it normal to have a file that was put through reverse telecine to be smaller then its original. I had a .mov file that was captured in final cut that was shot at 24p )(not pa) and had ro reverse it and it looks great, runs great but the file size is roughly off the top of my head 10% smaller. Also if anyone wants to take a crack at me cinema tools problem that I posted not long please be my quest. I see alot of views but no comments yet, Thank you
Nathan

One thing you could try to do is avoid using xpath like ".//" and "*".
I had many problems in terms of memory consuptiom and performance with xpaths like above.
Altrought you have a little more work to code your xslt it performs better, and probably you will write once and run many times.

Java XML Digital Signature API, how to sign different files

Hello,
I need to sign several files: binary and/or xml (in some cases just part of xml), and to implement digitla signatures in xAdes standard. So I'm looking to use Java XML Digital signature API, but can't find any examples, that would cover issues I encountered:
How to sign binary file?
Just to sign some simple "aaa.png" file and have it's signature in XML. How in right way to create referece?
(should it be something like: Reference ref = fac.newReference("aaa.png", fac.newDigestMethod(DigestMethod.SHA1, null), null, null, null); )
And how to pass file for signing? what to add/change to this code:
Document doc = dbf.newDocumentBuilder().parse(new FileInputStream("aaa.png"));
DOMSignContext dsc = new DOMSignContext(keyEntry.getPrivateKey(), doc.getDocumentElement());
(I have only found some information about needing to "dereference" or so - but no examples, how to make things work.)
How to sing several different files?
As I wrote before, several files needs to be signed, but in all examples, it's only one Document object (and only one file), how/where to add more files and if API will be capable to deal with such thing?
In one of examples what I have to achive was such code:
<Reference URI="aaa.png" xmlns="http://www.w3.org/2000/09/xmldsig#">
<DigestMethod Algorithm="http://www.w3.org/2000/09/xmldsig#sha1" />
<DigestValue>8rl/xzjAnE4yQQ2LTBvFTU2JH+c=</DigestValue>
</Reference>
If I do write code like: "fac.newReference("aaa.png", <...> );
I'll get an error during signing: signature.sign(dsc);
*"java.net.MalformedURLException: no protocol: aaa.png"*
How to avoid this?
Also, from exmaple (what to reach) above:
<Reference URI="aaa.png" xmlns="http://www.w3.org/2000/09/xmldsig#">
There is additional attribute "xmlns=<...>" - the question is if it is possible to add it by XMLSignatureFactory.newReference ?
Java API adds a lot of prefixes "ds:" , like:
<...>
<ds:Reference URI="file:/D:/try5/SignableMetadata0.xml">
<ds:Transforms>
<ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
</ds:Transforms>
<...>
Is it possible to avoid them?
Any help on any of these questions would be very appreciated

Hi,
I would like to sign a specific part of a xml message [Only the contents under the <Buyer> tag]. I have also pasted the code which i used to do this. I am getting an output xml after the xml is signed, but when I validate the xml , the xml is valid even after I change the xml contents. Could you pls tell me what I am doing wrong here. I want to know whether the xpath implementation which I have done is correct.
<?xml version="1.0" encoding="UTF-8"?>
<PurchaseOrder>
<Item number="130046593231">
<Description>Video Game</Description>
<Price>10.29</Price>
</Item>
*<Buyer id="8492340">*
*<Name>My Name</Name>*
*<Address>*
*<Street>One Network Drive</Street>*
*<Town>Burlington</Town>*
*<State>MA</State>*
*<Country>United States</Country>*
*<PostalCode>01803</PostalCode>*
*</Address>*
*</Buyer>*</PurchaseOrder>
// The code which i have used to perform the xpath transformation.
          XMLSignatureFactory fac = XMLSignatureFactory.getInstance("DOM");
     XPathFilterParameterSpec xpathFilter = new XPathFilterParameterSpec("PurchaseOrder/Buyer");
          javax.xml.crypto.dsig.Reference ref = fac.newReference
          ("", fac.newDigestMethod(DigestMethod.SHA1, null),
          Collections.singletonList
          (fac.newTransform
          (Transform.XPATH, xpathFilter)),
          null, null);
          SignedInfo si = fac.newSignedInfo
          (fac.newCanonicalizationMethod
          (CanonicalizationMethod.INCLUSIVE,
          (C14NMethodParameterSpec) null),
          fac.newSignatureMethod(SignatureMethod.DSA_SHA1, null),
Collections.singletonList(ref));
// Load the KeyStore and get the signing key and certificate.
     KeyStore ks = KeyStore.getInstance("JKS");
     char[] password = "changeme".toCharArray();
     ks.load(new FileInputStream("c:\\KeyStore"), password);
     KeyStore.PrivateKeyEntry keyEntry =
     (KeyStore.PrivateKeyEntry) ks.getEntry
     ("EISKeys", new KeyStore.PasswordProtection(password));
     X509Certificate cert = (X509Certificate) keyEntry.getCertificate();
     // System.out.println("X509Certificate:"+cert);
     // Create the KeyInfo containing the X509Data.
     KeyInfoFactory kif = fac.getKeyInfoFactory();
     List x509Content = new ArrayList();
     x509Content.add(cert.getSubjectX500Principal().getName());
     x509Content.add(cert);
     X509Data xd = kif.newX509Data(x509Content);
     KeyInfo ki = kif.newKeyInfo(Collections.singletonList(xd));
     // Instantiate the document to be signed.
     DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
     dbf.setNamespaceAware(true);
     Document doc = dbf.newDocumentBuilder().parse
     (new FileInputStream("C:\\Life2012\\DigSign\\ACORD_Request.xml"));
     NodeList rootChildList = doc.getDocumentElement().getChildNodes();
     Node bodyNode = null;
     for(int i=0;i<rootChildList.getLength();i++){
          if("Buyer".equalsIgnoreCase(rootChildList.item(i).getLocalName())){
               bodyNode = rootChildList.item(i);
               System.out.println("Body Node is obtained"+bodyNode);
               break;
     // Create a DOMSignContext and specify the RSA PrivateKey and
     // location of the resulting XMLSignature's parent element.
     //DOMSignContext dsc = new DOMSignContext
     // (keyEntry.getPrivateKey(), doc.getDocumentElement());
          // Sign only the body node
     DOMSignContext dsc = new DOMSignContext
     (keyEntry.getPrivateKey(), bodyNode);
     // Create the XMLSignature, but don't sign it yet.
     XMLSignature signature = fac.newXMLSignature(si, ki);
     // Marshal, generate, and sign the enveloped signature.
     signature.sign(dsc);

Performance of XPath

Similar Messages

Maybe you are looking for