CAML Query Benchmark

CAML query performance benchmark
Hi All,
My company has recently moved to MS technology and our portal has been developed using .Net and SharePoint 2010 to replace our old system that is based on Oracle 11g and Oracle forms.
The main observation that we have is that CAML queries are much slower than their equivalent Oracle query or even a direct SQL (just for testing).
To investigate CAML query performance we choose a list of CAML queries that we execute in our portal and (just for testing) wrote the equivalent direct SQL statements and tried both in similar network conditions to discover that CAML is at least 10 times
slower than the direct SQL!
We are suspecting that our CAML queries are not efficient (especially when join is needed) but some sample queries are direct selects with the same ration 10:1!
We need an expert advice to know whether this is the normal benchmark and what to do about it:
move part of the schema to a relational database (SQL or Oracle)
replace the CAML with SharePoint Library Query
replace the CAML with direct SQL (I read several articles about how this is considered a taboo but the current performance is not acceptable at all especially when there is a system that works with older technology that is performing much
faster)
One of the examples:
CAML:
<Where><Contains><FieldRef Name="ECM_F_BarcodeNo" /><Value Type="Text">-01-</Value></Contains></Where>
SQL:
Select *
From UserData
As Doc where
Doc.nvarchar10
like '%-01-%'
Many thanks

Hi,
Please check those links
http://charliedigital.com/2013/12/05/a-simple-way-to-improve-caml-query-performance/
http://apmblog.compuware.com/2010/03/18/how-to-avoid-the-top-5-sharepoint-performance-mistakes/
https://social.msdn.microsoft.com/Forums/sharepoint/en-US/909b8855-efea-4ee9-b63a-d021b3d416a7/sharepoint-2010-caml-query-performance-list-layout-and-design
http://sharepoint.stackexchange.com/questions/43885/caml-vs-linq-vs-sql
Tool to generate CAML queries
https://spcamlqueryhelper.codeplex.com/
https://camlex.codeplex.com/
Kind Regards,
John Naguib
Technical Consultant/Architect
MCITP, MCPD, MCTS, MCT, TOGAF 9 Foundation
Please remember to mark your question as answered if this solves your problem

Similar Messages

Retrieve All records and display in Report using CAML query in Report Builder if Parameter value is blank

Hello Experts,
i have created a report where i have one parameter field where user will pass parameter(e.g. EmpId). As per parameter record will fetched to report if no parameter is passed then it will display all records. I have done it by taking SqlServer Database as datasource.
by using Following method
Now i would like to do it by taking Sharepoint List as Datasource. For that what would be the CAML Query.
Here is my existing CAML query.
<RSSharePointList xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<ListName>Employees</ListName>
<ViewFields>
<FieldRef Name="Title" />
<FieldRef Name="FirstName" />
<FieldRef Name="LastName" />
<FieldRef Name="FullName" />
<FieldRef Name="UserName" />
<FieldRef Name="Company" />
</ViewFields>
<Query>
<Where>
<Eq>
<FieldRef Name="Title" />
<Value Type="Text">
<Parameter Name="EmployeeId"/>
</Value>
</Eq>
</Where>
</Query>
</RSSharePointList>
The above code is working if i am passing an employeeId to Parameter. If nothing is passed, it is Not retrieving any record.
Please suggest
Thank you
saroj
saroj

Your problem follows the well-established pattern of using an "All" parameter filter in SSRS. There are a few approaches depending on the size of your data. The easiest one is to return all data from CAML and then filter it within SSRS, the following thread
provides some examples,
http://stackoverflow.com/questions/18203317/show-all-records-some-records-based-on-parameter-value.
Other options include passing all of the possible values within the CAML query when "All" is selected, using multiple Report Files to distinguish between both CAML queries, or to use multiple datasets with some logic to show/hide the correct one.
Dimitri Ayrapetov (MCSE: SharePoint)

Sum and group by using CAML query

Is there any way to get the SUM of values with group by ID using CAML query?
I have a custom list, in that I have ID and Value columns as below
      ID         Value1      Value2
      1             10             4
      2              5              3
      1             15             2
      3             20             1
I want to get result:
     ID          Value1        Value2
     1             25                6
     2             5                  3
     3             20                1
I used CAML query:
<View>
<Query>
<Where>
<IsNotNull>
<FieldRef Name="ID" />
</IsNotNull>
</Where>
<GroupBy Collapse="FALSE">
<FieldRef Name="ID" />
</GroupBy>
</Query>
<ViewFields>
</ViewFields>
<Aggregations Value="On">
<FieldRef Name="Value1" Type="SUM" />
<FieldRef Name="Value2" Type="SUM" />
</Aggregations>
</View>
But
this query returns all the records
that satisfy the condition <Where>.
I
do not correctly?

You need to work with current view based aggregation. Please check below threads for your reference.
http://social.msdn.microsoft.com/Forums/sharepoint/en-US/dda5735a-fecf-403f-9495-1b63617d2fbf/question-on-a-caml-query?forum=sharepointdevelopmentlegacy
http://social.msdn.microsoft.com/Forums/sharepoint/en-US/24e88d6a-ee15-4d81-a5fe-504c7bd14e46/how-to-sum-a-column-value-using-caml-query?forum=sharepointdevelopment
Hope this helps.
My Blog- http://www.sharepoint-journey.com|
If a post answers your question, please click Mark As Answer on that post and Vote as Helpful
I've seen these
issues and articles and do them, but
does not work. I use object mozhel
Sharepoint (javascript).

Value does not fall within the expected range while passing a spfieldlookupvalue in a caml QUERY

i have to pass a column from master list -single line of text - and this will be the lookup of another column in another list
splist1---> column1 [ free text]
splist2--> column2 [ lookup column of the above list -splist1]
now am writing a caml query :
   objDisciNodeQuery.Query =
                                      string.Format(
                                   "<OrderBy>" +
                                     "<FieldRef Name='ID' />"
+
                                  "</OrderBy>" +
                                   "<Where>" +
                                    "<And>" +
                                      "<Eq>" +
                                         "<FieldRef
Name='somecolumn' />" +
                                         "<Value
Type='Text'>{0}</Value>" +
                                      "</Eq>" +
                                      "<Eq>" +
                                         "<FieldRef
Name='column2' LookupId='TRUE' />" +
                                         "<Value
Type='Lookup'>{1}</Value>" +
                                      "</Eq>" +
                                     "</And>" +
                                   "</Where>", valueofsomecolumn, lookupidvalueofcolumn2);
                                    SPListItemCollection splistItemAssocCollec
= null;
                                    splistItemAssocCollec = splist2.GetItems(objDisciNodeQuery);
here it throws Value does not fall within the expected range
Anyone has idea why i am getting this error, .
any ideas are appreciated.

We can use lookup column in caml query link this
<Where>
<Eq>
<FieldRef Name=’Departments’ LookupId=’TRUE’ />
<Value Type=’Lookup’>10</Value>
</Eq>
</Where>
Multi lookup column reference
http://naimmurati.wordpress.com/2013/12/03/multi-lookup-fields-in-caml-queries-eq-vs-contains/

Join two list with condition using caml query in SharePoint 2013 with client object model

Hi,
Want to join two list to get all fields from both list.
Am new to sharepoint and sharepoint 2013. Am working in sharepoint 2013 online apps. Am using context.executeQueryasync to load list and get items from list. Am able to get items from single list with caml query, but not able to get both list field values
with joins. I did lot of surfing..but not..
Below is my code..
ListName1 : "AssignedTasks"
ListName2 : "Tasks"
var assignedQueryTest = "<View><Joins><Join Type='INNER' ListAlias='Tasks'><Eq><FieldRef Name='TaskId' RefType='Id'/><FieldRef List='Tasks' Name='ID' /></Eq></Join></Joins>"
+ "<ViewFields><FieldRef Name='TitleValue' /><FieldRef Name='ActionItemsValue' /></ViewFields>"
+ "<ProjectedFields>"
+ "<Field Name='TitleValue' Type='Lookup' List='Tasks' ShowField='Title' /><Field Name='ActionItemsValue' Type='Lookup' List='Tasks' ShowField='ActionItems' />"
+ "</ProjectedFields>"
+ "</View>";
var web = context.get_web();
var list = web.get_lists().getByTitle("AssingedTasks");
var myQuery = new SP.CamlQuery();
myQuery.set_viewXml(assignedQueryTest);
var myItems = list.getItems(myQuery, "Include(TitleValue,ActionItemsValue)");
context.load(myItems);
context.executeQueryAsync(function () { if(myItems.get_count()>0){....}
}, errorCallback);
Here am able to get "AssignedTasks" list field values but not able to get "Tasks" list field values.
Can you please help me to resolve the issue. Or new idea for join. I have add the condition also in the query.
If anybody have good sample, please provide.
Thanks,
Pariventhan
Pariventhan.S

Hi Pariventhan,
I don't know about join but I have a workaround of this problem.
Declare one variable (itemcollection) globally. Load all the items "AssignedTasks" using context.load.
In the success method call another CAML query using <IN> tag of the ID of the items from the second list (Assuming you have look-up column of the ID of the first list to the second list).
If this is not clear to you then please let me know. If possible then I can provide code sample.
Thanks,
Aniruddha

How to check all items are not null in CAML Query?

HI All,
In a CAML Query, i got three items. In all three items, a particular column(Closure Date) should not be null. How to check in all three items, where 'Closure date' is not null?
Thanks in advance!

Hi,
Try this:
"<Where><IsNull><FieldRef Name='Closure Date' /></IsNull></Where>"
Hope it hepls!
Thanks,
Avni Bhatt
If this helped you resolve your issue, please mark it Answered

Retrieve all items in a list by caml query then write it to a word file page by page

Dears,
Greetings
I have some code which will retrieve all items of a list with caml query.
Now i want to generate a word file with all these retrieved data. I am using foreach here to get all the items in a SPListItemCollection. Now my question is how i can write all the retrieved items inside the word file?
see the below code which is writing only the last row item of the SPListItemCollection inside word file. I want to write all the retrieved rows inside this word file.
SPQuery qry = new SPQuery();
qry.Query = "<Where><Eq><FieldRef Name='Department' /><Value Type='Text'>HR DEPARTMENT</Value></Eq></Where>";
SPListItemCollection listItems = KPILIst.GetItems(qry);
foreach (SPListItem item in listItems)
lblBadgeNo.Text = item["Badge No"].ToString();
lblName.Text = item["Name"].ToString();
lblPosition.Text = item["Position"].ToString();
lblDept.Text = item["Department"].ToString();
lblHireDate.Text = item["Hire Date"].ToString();
lblGrade.Text = item["Grade"].ToString();
lblCurStatus.Text = item["Status"].ToString();
string FinalOut = "";
FinalOut = "<table cellpadding=0 cellspacing=0><tr><td style='height:30px;'></td></tr><tr><td style='height:22px;'></td></tr>";
FinalOut = FinalOut + "<tr><td style='text-align:justify;font-size:22px;'>" + lblBadgeNo.Text + "</td></tr>";
FinalOut = FinalOut + "<tr><td style='text-align:justify;font-size:22px;'>" + lblName.Text + "</td></tr>";
FinalOut = FinalOut + "<tr><td style='text-align:justify;font-size:22px;'>" + lblDept.Text + "</td></tr>";
FinalOut = FinalOut + "<tr><td style='text-align:justify;font-size:22px;'>" + lblHireDate.Text + "</td></tr>";
FinalOut = FinalOut + "</table>";
System.Text.StringBuilder strBody = new System.Text.StringBuilder("");
strBody.Append("<html " + "xmlns:o='urn:schemas-microsoft-com:office:office' " + "xmlns:w='urn:schemas-microsoft-com:office:word'" + "xmlns='http://www.w3.org/TR/REC-html40'>" + "<head><title>EPCCO : HR Comapny Letter</title>");
strBody.Append("");
strBody.Append("<style>" + "" + "</style></head>");
strBody.Append("<body dir=rtl lang=EN-US style='tab-interval:.15in'>" + "<div class=Section1>" + FinalOut.ToString() + "</div></body></html>");
Response.AppendHeader("Content-Type", "application/msword");
Response.AppendHeader("Content-disposition", "attachment; filename=Eval_" + lblBadgeNo.Text + ".doc");
Response.Write(strBody);
Regards
Shaji
I am new to SharePoint

Hello,
you can try with OpenXML SDK to create word file. I found one link for your reference:
http://sharepointplace.blogspot.in/2009/12/programmatically-creating-word.html
Another way is , use Microsoft Word Object Library.
http://social.msdn.microsoft.com/Forums/vstudio/en-US/5b82c0b5-ecaf-40f2-a68a-c7c17c85414f/create-word-documents-by-c?forum=csharpgeneral
Hope it could help
Hemendra:Yesterday is just a memory,Tomorrow we may never see
Please remember to mark the replies as answers if they help and unmark them if they provide no help

Get managed metadata column name in CAML Query

Hi All,
In my pages library, I have a managed metadatda column named "MyProduct".
When I try to retrieve this in my Caml query, the column data is displayed in a different format like : System.Collections.Generic.Dictionary`2[System.String,System.Object]
I just want to display the name of that Managed Metadata.
This is the query which I am using :
<ViewFields>
<FieldRef Name='Product' />
</ViewFields>
Please suggest.
Thanks

I have gone through the article.
var prod= listEnumerator.get_current().get_item("MyProduct")["Label"];doesnt work.The internal name for myproduct - qmdfvar prod= listEnumerator.get_current().get_item("qmdf")["Label"];Even this doesnt work
Please suggest!

CAML Query In Clause Problem

Hi,
I'm using CAML query to retrieve data from custom list in SharePoint 2010 using In clause. I'm creating the <Value></Value> tag dynamically and its working fine.
But sometimes I get a weird, no data fetch, error. I investigated on this and get to know that CAML query In clause only allow you to add 500 <Value></Value> in <Values> tag. The moment you'll entered 501 value it will not return any
data.
I need to resolve this problem and I can't send fetch request to SharePoint list if someone is of the view that I should break the clause and fetch first 500 then again 500 and so on.
Any help will be appreciated.
Thank you

Hello Kashif,
Can't you create extra <OR> blocks when you have more than 500 values?
<OR>
<OR>
<First block 500 values>
<Second block 500 values>
</OR>
<Third block rest of the values>
</OR>
If this is not possible, maybe you can add more of your CAML query.
Jeroen Molenaar

Caml query to check current logged in user - SharePoint designer

Hi,
I need to create a caml query in SharePoint designer to satisfy the below conditions.
If the logged in user (current user) is a member of a specific group. e.g. Group Name = ISS Owners, ID=5
(OR)
If the logged in user (current user) exists in assigned to field. e.g. Field Name = Assigned To
Reference to the below link is not working for my solutions.
Membership Element (Query)
CAML Query – Membership attribute
Kindly help in resolving caml query issue.
Anandhan.S Remember to 'mark or propose as answer' or 'vote as helpful' as appropriate.

The second condition is pretty straightforward
<Eq>
<FieldRef Name="AssignedTo"/>
<Value Type="Integer">
<UserID />
</Value>
</Eq>
but I don't think you can do the former. The <Membership> condition can only be applied to a <FieldRef>, not a generic <Value> argument.
But what would be the purpose of the first condition in the first place? It's invariant, not related to specific list items. Or is the requirement to return all items if the user is a member of a group (Administrators?), but only assigned-To items if not:
If so, is there some way you can refactor the check for the former condition into some higher level logic?

CAML Query Using CSOM in PowerShell

Hi,
I am trying to retrieve list items based on certain conditions using CAML query. I am using CAML query in Powershell using CSOM. The issue is no matter what condition i provide, it returns all list items. Any help would be highly appreciated. The script is:
# Location of DLL's
$loc ="C:\SharePoint\ClientDLL"
Write-Host "Date Range"
Set-Location $loc
Add-Type -Path (Resolve-Path Microsoft.SharePoint.Client.dll)
Add-Type -Path (Resolve-Path Microsoft.SharePoint.Client.Runtime.dll)
#Write the path of SharePoint Portal within double braces
$siteUrl = ""
#Login ID
$loginname = ""
#$listname=""
Write-Host "Please enter password for $($siteUrl):"
$pwd = Read-Host -AsSecureString
$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl)
$ctx.Credentials = New-Object System.Net.NetworkCredential($loginname, $pwd)
$web = $ctx.Web
$list=$Web.Lists.GetByTitle($listname)
$count = $list.ItemCount
Write-Host $count
$newline = [environment]::newline
$($list.Title)
Write-Host -NoNewline "Deleting list items from : $($listname)" -foregroundcolor black -backgroundcolor yellow
#$caml="<View><Query><Where><Eq><FieldRef Name='Title'/><Value Type='Text'>achint</Value></Eq></Where></Query></View>"
$caml="<View><Query><Where><Eq><FieldRef Name='Created'/><Value Type='DateTime' IncludeTimeValue='FALSE'>11/9/2014</Value></Eq></Where></Query></View>"
$cquery = New-Object Microsoft.SharePoint.Client.CamlQuery
$cquery.ViewXml=$caml
$listItems = $list.GetItems($cquery)
$ctx.Load($listItems)
$ctx.ExecuteQuery()
$newline
Write-Host $listItems.Count
if ($listItems.Count -gt 0)
#$listItems | % {$list.GetItemById($_.Id).DeleteObject()}
#$ctx.ExecuteQuery()
Write-Host "All list items deleted from List." -foregroundcolor black -backgroundcolor green
else
Write-Host "No Item in list: $($listname) to delete" -foregroundcolor black -backgroundcolor red
Regards,
Sudheer
Thanks & Regards, Sudheer

HI?
TRY to use query like this :
$query = "<query>
<where>
<eq>
<FieldRef Name="ID"
<Value Type="Number">$IDNum</value>
</eq>
</where>
</query>"

URL data type in CAML Query with Client side object model

hi,
How do I write a CAML query to filter list items based on the URL field using it's Description?
Same with REST API. How do I construct a REST query so that filter list items based on the URL data type using it's Description.

Hi Cooltechie,
Thanks for posting your query, Below are the example that you can use in your CAML query
<Where><Eq><FieldRef Name="FileRef"/><Value Type="Url">sites/SiteCollection/SubSite/Site Documents/Excel Report.xls</Value></Eq></Where>
Note: Do not include the server name or beginning /.
The following examples assuming you have a list or library setup with a URL column named "My Document".
The link is to a document that is hosted on the sharepoint server (do not need server name):
<Where><Eq><FieldRef Name="My_x0020_Document"/><Value Type="URL">/sites/subsite/Site%20Documents/Excel%20Report.xls</Value></Eq></Where>
The link is an absolute URL to something not on the server... for example http://www.google.com:
<Where><Eq><FieldRef Name="My_x0020_Document"/><Value Type="URL">http://www.google.com</Value></Eq></Where>
I hope this is helpful to you, mark it as Helpful.
If this works, Please mark it as Answered.
Regards,
Dharmendra Singh (MCPD-EA | MCTS)
Blog : http://sharepoint-community.net/profile/DharmendraSingh

My Library Column does not exist when I execute a Caml Query

Hello,
I am new to SharePoint programming. I am trying to understand how I can access the documents in a Library Folder and obtain the field values for each document in the container. With this in mind I have pieced together the following code from
various posts that demonstrate how to to this:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Microsoft.SharePoint.Client;
namespace ConsoleListItemsInFolder
class Program
static void Main(string[] args)
ClientContext ctx = new ClientContext("http://mwp_lenovo");
List DocumentsList = ctx.Web.Lists.GetByTitle("Claims Documents");
CamlQuery camlQuery = new CamlQuery();
camlQuery = new CamlQuery();
camlQuery.ViewXml = "<View Scope=\"RecursiveAll\"> " +
"<Query>" +
"<Where>" +
"<Eq>" +
"<FieldRef Name=\"FileDirRef\" />" +
"<Value Type=\"Text\">/Claims Documents/11111111 Stuart Little</Value>"
+
"</Eq>" +
"</Where>" +
"</Query>" +
"</View>";
ListItemCollection listItems = DocumentsList.GetItems(camlQuery);
ctx.Load(
listItems,
items => items
.Include(
item => item["Title"],
item => item["Claim Number"],
item => item["Policy Number"],
item => item["Policyholder Name"],
item => item["Document Type"]));
ctx.ExecuteQuery();
foreach (ListItem listItem in listItems)
Console.WriteLine("Title: {0}", listItem["Title"]);
Console.WriteLine("Claim NUmber: {0}", listItem["Claim Number"]);
Console.ReadLine();
When I execute the code with the debugger ctx.ExecuteQuery(); throws the following error:
Microsoft.SharePoint.Client.ServerException was unhandled
Message=Column 'Claim Number' does not exist. It may have been deleted by another user. /Claims Documents
Source=Microsoft.SharePoint.Client.Runtime
ServerErrorCode=-2147024809
ServerErrorTypeName=System.ArgumentException
ServerStackTrace=""
StackTrace:
at Microsoft.SharePoint.Client.ClientRequest.ProcessResponseStream(Stream responseStream)
at Microsoft.SharePoint.Client.ClientRequest.ProcessResponse()
at Microsoft.SharePoint.Client.ClientContext.ExecuteQuery()
at ConsoleListItemsInFolder.Program.Main(String[] args) in c:\Users\matt.paisley\Documents\Visual Studio 2012\Projects\ConsoleListItemsInFolder\ConsoleListItemsInFolder\Program.cs:line 37
at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
InnerException:
These columns do exist in the Library as verified in Library Settings. Many of the Library column names contain two words divided by a space. Is this causing the problem? Does SharePoint maintain a less offensive representation of the field
name internally? If so, how can I get it?
If I run this code with only the title, then it runs fine.
I Thank All in advance for any assistance that you can provided.
Regards,
Matt Paisley
Matthew Paisley

Hello
In addition to Geetanjali Arora answer, change also those other columns
item => item["Claim Number"],
item => item["Policy Number"],
item => item["Policyholder Name"],
item => item["Document Type"]
If you don't know the internal name of a column,
go to the list containing the columns
go to "list settings"
scoll down to the "columns" section
click on one of your columns et take a look a the url of the new page, you should found parameter "&Field=" => What is after is your field internal name (the one you should use in your CAML query)
Best regards, Christopher.
Blog |
Mail
Please remember to click "Mark As Answer" if a post solves your problem or
"Vote As Helpful" if it was useful.
Why mark as answer?

Filter SharePoint list items using CAML query as same as Like operator in SQL Server.

Hi ,
I have filtered SharePoint list items based on Name using CAML query <Contains> . Now I have a new requirement is to filter list items using Like operator in SQL. But Like operator is not in CAML.
How do I filter list items using CAML as same as Like operator in SQL.
Please let me know.
Thanks in Advance.

Did you try using <Contains>?
http://social.technet.microsoft.com/Forums/sharepoint/en-US/15766fd5-50d5-4884-82a1-29a1d5e38610/caml-query-like-operator?forum=sharepointdevelopmentlegacy
--Cheers

Using CAML Query in SharePoint Hosted app

Hi,
I am trying to execute a CAML Query in a SharePoint Hosted App, below is the JS code.
executor.executeAsync({
url: appwebUrl + "/_api/SP.AppContextSite(@target)/web/lists/getbytitle('Documents')/Items?/getitems?@target='" + hostweburl + "'",
method: "POST",
body: { 'query' : {'__metadata': { 'type': 'SP.CamlQuery' }, "ViewXml": "<View><Query><Where><Eq><FieldRef Name='Title'/><Value Type='TEXT'>value</Value></Eq></Where></Query></View>"
headers: {
"accept": "application/json; odata=verbose",
"content-type": "application/json; odata=verbose"
success: successHandler,
error: errorHandler
getting an error: Uncaught Error: "Invalid field or parameter requestInfo.body"

Are you using rest api?
Please check this
http://tech.bool.se/using-caml-sharepoint-rest-api/
http://msdn.microsoft.com/en-us/library/office/dn531433(v=office.15).aspx

CAML Query Benchmark

Similar Messages

Maybe you are looking for