How to handle "The specified resource does not exist" exception while using entity group transactions to purge WADLogs table

Question

Hi,
We have a requirement to purge the Azure WADLogs table on a periodic basis. We are achieving this by using Entity group transactions to delete the
records older than 15 days. The logic is like this.
bool recordDoesNotExistExceptionOccured = false;
CloudTable wadLogsTable = tableClient.GetTableReference(WADLogsTableName);
partitionKey = "0" + DateTime.UtcNow.AddDays(noOfDays).Ticks;
TableQuery<WadLogsEntity> buildQuery = new TableQuery<WadLogsEntity>().Where(
TableQuery.GenerateFilterCondition("PartitionKey",
QueryComparisons.LessThanOrEqual, partitionKey));
while (!recordDoesNotExistExceptionOccured)
IEnumerable<WadLogsEntity> result = wadLogsTable.ExecuteQuery(buildQuery).Take(1000);
//// Batch entity delete.
if (result != null && result.Count() > 0)
Dictionary<string, TableBatchOperation> batches = new Dictionary<string, TableBatchOperation>();
foreach (var entity in result)
TableOperation tableOperation = TableOperation.Delete(entity);
if (!batches.ContainsKey(entity.PartitionKey))
batches.Add(entity.PartitionKey, new TableBatchOperation());
// A Batch Operation allows a maximum 100 entities in the batch which must share the same PartitionKey.
if (batches[entity.PartitionKey].Count < 100)
batches[entity.PartitionKey].Add(tableOperation);
// Execute batches.
foreach (var batch in batches.Values)
try
await wadLogsTable.ExecuteBatchAsync(batch);
catch (Exception exception)
// Log exception here.
// Set flag.
if (exception.Message.Contains(ResourceDoesNotExist))
recordDoesNotExistExceptionOccured = true;
break;
else
break;
My questions are:
Is this an efficient way to purge the WADLogs table? If not, what can make this better?
Is this the correct way to handle the "Specified resource does not exist exception"? If not, how can I make this better?
Would this logic fail in any particular case?
How would this approach change if this code is in a worker which has multiple instances deployed?
I have come up with this code by referencing the solution given
here by Keith Murray.

Answer 1

Hi Nikhil,
Thanks for your posting!
I tested your and Keith's code on my side, every thing worked fine. And when result is null or "result.count()<0", the While() loop is break. I found you code had some logic to handle the error "ResourceDoesNotExist" .
It seems that the code worked fine. If you always occurred this error, I suggest you could debug your code and find which line of code throw the exception.   
>> Is this an efficient way to purge the WADLogs table? If not, what can make this better?
Base on my experience, we could use code (like the above logic code) and using the third party tool to delete the entities manually. In my opinion, I think the code is every efficient, it could be auto-run and save our workload.
 >>Is this the correct way to handle the "Specified resource does not exist exception"? If not, how can I make this better?
In you code, you used the "recordDoesNotExistExceptionOccured " as a flag to check whether the entity is null. It is a good choice. I had tried to deleted the log table entities, but I used the flag to check the result number.
For example, I planed the query result count is 100, if the number is lower than 100, I will set the flag as false, and break the while loop. 
>>Would this logic fail in any particular case?
I think it shouldn't fail. But if the result is "0", your while loop will always run. It will never stop. I think you could add "recordDoesNotExistExceptionOccured
= true;" into your "else" block.
>>How would this approach change if this code is in a worker which has multiple instances deployed?
You don't change anything expect the "else" block. It would work fine on the worker role.
If any question about this issue, please let me know free.
Regards,
Will
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click
HERE to participate the survey.