Scraping data from Wikipedia using Htmlagilitypack

Question

I am trying to scrape data from a table on the Wikipedia website and so far I have managed to find the nodes that I need to reference. There are a large number of entries in the table on Wikipedia, however, when I run the app I only get twelve results
and they are all identical. All results returned are duplicates of the first entry in the table.
Any ideas on how to fix?
protected async override void OnNavigatedTo(NavigationEventArgs e)
base.OnNavigatedTo(e);
string htmlPage = "";
htmlPage = await client.GetStringAsync("http://en.wikipedia.org/wiki/List_of_Games_with_Gold_games");
HtmlDocument htmlDocument = new HtmlDocument();
htmlDocument.LoadHtml(htmlPage);
foreach (var div in htmlDocument.DocumentNode.SelectNodes(".//h2"))
GameHistory newGameHistory = new GameHistory();
newGameHistory.historyTitle = div.SelectSingleNode("//i//a").InnerText.Trim();
newGameHistory.historyAdded = div.SelectSingleNode("//span[starts-with(@style, 'white')]").InnerText.Trim();
newGameHistory.historyRemoved = div.SelectSingleNode("(//span[starts-with(@style, 'white')])[2]").InnerText.Trim();
gameHistory.Add(newGameHistory);
lstGameHistory.ItemsSource = gameHistory;

Answer 1

@WSH_ALT  the code you quote is HTMLAgilityPack syntax for performing an XPath query.
@Keith Kerswell
Without seeing more of your code, I can't be sure of your exact problem.  However, the following code will successfully parse the tables on that page into two lists of objects (one for each table).  Hope this helps.
public class GameHistoryItem
public string historyTitle { get; set; }
public string historyAdded { get; set; }
public string historyRemoved { get; set; }
public static async void parseWikiTables()
var client = (App.Current as App).client;
// Create a list to hold our items
List<GameHistoryItem> xbox360GameHistory = new List<GameHistoryItem>();
List<GameHistoryItem> xboxOneGameHistory = new List<GameHistoryItem>();
// Get the contents of the page into a string
var rawData = await client.GetStringAsync("http://en.wikipedia.org/wiki/List_of_Games_with_Gold_games");
// Create an HtmlDocument
var doc = new HtmlAgilityPack.HtmlDocument();
// Parse the contents into the document (e.g. load the DOM)
doc.LoadHtml(rawData);
// Now, there are two tables on this page we care about: "Xbox 360" and "Xbox One"
// It turns out that these tables both have a class of "wikitable", so...
var xboxTables = doc.DocumentNode.SelectNodes(".//table[contains(concat(' ', normalize-space(@class), ' '), ' wikitable ')]");
// Let's find out if we got exactly two tables:
if (xboxTables.Count == 2)
Debug.WriteLine("Life Is grand, we have 2 tables!");
bool is360 = true;
foreach (var xbTable in xboxTables)
try
// Get all the rows in this table
var rows = xbTable.SelectNodes(".//tr");
if (rows != null)
// The first row is the header row, just skip it
for (int x = 1; x < rows.Count; x++) {
var row = rows[x];
var cells = row.SelectNodes(".//td");
var ghi = new GameHistoryItem();
ghi.historyTitle = cells[1].InnerText;
ghi.historyAdded = cells[2].InnerText;
ghi.historyRemoved = cells[3].InnerText;
if (is360)
{ xbox360GameHistory.Add(ghi); }
else
{ xboxOneGameHistory.Add(ghi); }
catch (Exception e) { Debug.WriteLine("Error: " + e.Message); }
is360 = false;
int x360Count = xbox360GameHistory.Count;
Debug.WriteLine("There are {0} Xbox360 items, the last of which is: {1}",
x360Count,
xbox360GameHistory[x360Count - 1].historyTitle);
int xOneCount = xboxOneGameHistory.Count;
Debug.WriteLine("There are {0} Xbox One items, the last of which is: {1}",
xOneCount,
xboxOneGameHistory[xOneCount - 1].historyTitle);
The debug console ouput of this method is:
Life Is grand, we have 2 tables!
There are 42 Xbox360 items, the last of which is: Sniper Elite V2
There are 11 Xbox One items, the last of which is: #IDARB (It Draws a Red Box)