SSAS Config: Increasing Query Concurrency

Currently stress testing a server and trying to increase the "Processing pool busy I/O job threads". While simulating a load of 50 concurrent queries, "Processing pool busy I/O job threads" is pegged
at 20 and the "Processing pool I/O job queue length" is anywhere from 40-160 jobs.
I've been tweaking the following properties:
ThreadPool \ Process \ MaxThreads (Default 0, Current -4)
ThreadPool \ Query \ MaxThreads (Default 0, Current -4)
etc
...and since I'm trying to simulate a highly concurrent system, I've also changed the following properties:
CoordinatorQueryBalancingFactor (Default -1, Current 1)
CoordinatorQueryBoostPriorityLevel (Default 3, Current 0)
...but I can't seem to get the "Processing pool busy I/O job threads" above 20. When I look in the msmdsrv log, I see the following:
Are the highlighted parts ("and a concurrency of 20") ...
hard wired to the number of physical cores?
related to the "Processing pool busy I/O job threads"?
How can I make the concurrency of the "Processing pool busy I/O job threads" counter go up? I've been all over the interwebs for this including the SSAS-ops/perf guide and am stumped at this point.  
Virtual Server Specs:
20 physical cores (40 logical)
128 GB memory
2 NUMA nodes
Update: after a bit of reading (SQL Server 2008 Analysis Services Unleashed), I found what appears to be a reference to this "concurrency"...
"...because by default each thread requires 1MB of memory for the stack, the system must allocate 0.5GB of memory to support 512 threads. Fortunately, this will not happen because Analysis Services limits the number of active threads by value of
the "Concurrency" property."
Is this "Concurrency" property the 20 I'm seeing in the msmdsrv log file (highlighted above)? 
Is it the [IOProcess\Concurrency], [Query\Concurrency], [Process\Concurrency] properties in the msmdsrv.ini config file? The ones that default to "2" and are
"An advanced property that you should not change, except under the guidance of Microsoft support"
according to documentation (here) ?
Update2: after more reading (on I/O completion ports), I think this is something for which I need to engage the storage admins to get confirmation on the network config between the actual server
and disks. 
BI Developer and lover of data (Blog |
Twitter)

Hi Bill,
Thank you for your question. 
I am trying to involve someone more familiar with this topic for a further look at this issue. Sometime delay might be expected from the job transferring. Your patience is greatly appreciated. 
Thank you for your understanding and support.
Regards,
Charlie Liao
TechNet Community Support

Similar Messages

  • Report burst:To increase query performance in xcelsius

    Is there anyway to increase query performance in xcelsius by using report bursting

    Fremlin,
    Report bursting is only for distributing your reports to your end users.
    You can improve performance only by following the [Best practices|https://www.sdn.sap.com/irj/boc/index?rid=/library/uuid/a084a11c-6564-2b10-79ac-cc1eb3f017ac] in xcelsius.
    -Anil

  • SSAS Tabular : MDX query goes OutOfMemory for a larger dataset

    Hello all,
    I am using SSAS 2012 Tabular to build the cube to support the organizational reporting requirements. Right now the server is Windows 2008 x64 with 16GB of Ram installed. I have the following MDX query. What this query does is get the member caption of the
    “OrderGroupNumber” non-key attribute as a measure where order group numbers pertain to a specific day and which occurs in specific seconds of a day. As I want to find in which second I have order group numbers, I cross the time dimension’s members with a specific
    day and filter the tuples using the transaction count. The transaction count is a non-zero value if an Order Group Number occurs within a specific second of a selected day.
    At present “TransactionsInflight].[OrderGroupNumber].[OrderGroupNumber]” has 170+ million members (Potentially this could grow rapidly) and time dimension has 86400 members.
    WITH
    MEMBER [Measures].[OrderGroupNumber]
    AS IIF([Measures].[Transaction Count] > 0, [TransactionsInflight].[OrderGroupNumber].CURRENTMEMBER.MEMBER_CAPTION,
    NULL)
    SELECT
    NON EMPTY{[TransactionsInflight].[OrderGroupNumber].[OrderGroupNumber].MEMBERS}
    ON COLUMNS,
    {FILTER(([Date].[Calendar Hierarchy].[Date].&[2012-07-05T00:00:00], [Time].[Time].[Time].MEMBERS),
    [Measures].[Transaction Count] > 0) } ON
    ROWS
    FROM [OrgDataCube]
    WHERE [Measures].[OrderGroupNumber]
    After I run this query it reaches to a dead-end and freezes the server (Sometimes SSAS server throws OutOfMemory exception but sometimes it does not). Even though I have 16GB of memory it uses all the memory and doing nothing. I have to do a hard-reset against
    the server to get the server online. Even I limit the time members using the “:” range operator still the machine freeze. I have run out of solutions to fine-tune the design. Could you guys provide me some guidelines to optimize this query? I am willing to
    do a design change if it is necessary.
    Thanks and best regards,
    Chandima

    Hi Greg,
    Finally I found the problem why the query goes out of memory in tabular mode. I guess this information will helpful for others and I am posting my findings.
    Some of the non-key attribute columns in the tabular model tables (mainly the tables which form dimensions) do not contain pretty names. So for the non-key attribute columns which I need to provide pretty names I renamed the columns to something else.
    For an example, in my date dimension there is a non-key attribute named “DateAltKey”. This is the date column which I am using. As this is not pretty to the client tools I renamed this column as “Date” inside the designer (Dimension
    design screen). I deployed the cube, processed the cube and no problem.
    Now here comes the fun part. For every table, inside the Tables node (Tabular SSAS Database > Tables) you can view the partition details. You have single partition per dimension table if you do not create extra partitions. I opened the partitions screen
    and clicked on the “Edit” icon and performed a Syntax Check. Surprisingly it failed. It complains about the renamed column. It complained “Date” cannot be found in source. So I realized that I cannot simply rename the columns like that.
    After that I created calculated columns (with a pretty name) for all the columns which complained and all the source columns to the calculated columns were hid from the client tools. I deployed the cube, processed the cube and performed a
    syntax check. No errors and everything were perfect.
    I ran the query which gave me trouble and guess what... it executed within 5 seconds. My problem is solved. I really do not know who did this improve the performance but the trick worked for me.
    Thanks a lot for your support.
    Chandima

  • How to increase query time out

    I have a report which is generated using a huge SQL, it is erroring while displaying results, due to time out error:
    State: HY000. Code: 10058. [NQODBC] [SQL_STATE: HY000] [nQSError: 10058] A general error has occurred. [nQSError: 17001] Oracle Error code: 1013, message: ORA-01013: user requested cancel of current operation at OCI call OCIStmtExecute
    How to increase this time so that the connection persists for a longer duration?
    thanks
    Rishabh

    I looked in the NQSConfig.ini file for you. In here you find the following:
    [ CACHE ]
    ENABLE     =     YES;
    // A comma separated list of <directory maxSize> pair(s)
    // e.g. DATA_STORAGE_PATHS = "d:\OracleBIData\nQSCache" 500 MB;
    DATA_STORAGE_PATHS     =     "/oracle/oaobi01/data/ontwbif1013/OracleBIdata/cache" 500 MB;
    MAX_ROWS_PER_CACHE_ENTRY = 100000; // 0 is unlimited size
    MAX_CACHE_ENTRY_SIZE = 1 MB;
    MAX_CACHE_ENTRIES = 1000;
    I think you can do something with this...
    You can find the nqsconfig.ini in this directory:<Install Drive>\OracleBI\server\Config
    Otherwise have a look in the Server Administration Guide, chapter 11

  • Increase Query timeout in Dashboard

    Hi,
                        When we add  a query through query browser in dashboard after certain time the connection gets timed out if the system is idle . Normally we can increase time out for webi reports but where could we increase the time for the Bex Queries to get timed out.
    Thanks&Regards,
    Krishna

    Hi Hari,
    I hope its not possible, Nothing can be done on query browser timeout. Please a raise a discussion on this in the below link.
    SAP BusinessObjects Dashboards: Ideas
    --SumanT

  • SSAS DRILLTHROUGH - Specified query is too complex error.

    Hi Guys,
    I'm working on an SSAS 2012 OLAP cube with around 100 dimensions and a single measures group. When I try to run a simple DRILLTHROUGH query such as:
    DRILLTHROUGH
    MAXROWS 1000 SELECT
    FROM [Cube] WHERE ([Measures].[Fact_Count])
    I get an error as below.
    The specified query is too complex to be evaluated as a single statement.
    Does anyone have any experience of this error and how to work around it? If I remove the majority of the dimensions from the cube I can run this query successfully. Looking at the limits set for analysis services I should be well within these.
    Thanks in advance 
    Sean

    Hi Scaneyw,
    Based on my research, this is a SQL Server Analysis Services (SSAS) design limitation. In SSAS, Drillthrough” generates internal CrossJoin query that contains all attribute hierarchies in measure group. So the error appear when MDX parser receives statement
    with> 1000 nodes.
    Here are some tips for you to avoid this error.
    Reduce number of dimension attribute hierarchies in measure group below 1000.
    Delete unneeded attributes and disable Hierarchy for attributes that are only used as properties.
    Consider creating smaller Measures Groups with less dimensions + attribute hierarchies.
    If you have any concerns about this feature, you can submit a feedback at
    http://connect.microsoft.com/SQLServer/Feedback and hope the complete “drillthrough” feature is will be improved in future SQL Server Versions.
    Regards,
    Charlie Liao
    If you have any feedback on our support, please click
    here.
    Charlie Liao
    TechNet Community Support

  • Increase Query Designer Performance?

    Hi together,
    Is it possible to increase the performance of the Query Designer? I have a query which is based on a MultiProvider with several InfoCubes. If i work some minutes with the Query Designer, it becomes slower and slower. Then it takes from one up to ten seconds to see the digit in my formula, which i pressed on the keyboard just before. This is very awful! At least the half of the time i work with the Query Designer i have to wait, that this tool stops calculating something or displaying the hourglass. Very inefficient.
    Thanks for hints in advance!

    >
    Ricardo Rosa wrote:
    > Hi Timo,
    >
    > Usually this kind of issue should be solved with frontend patch upgrade, have you tried to reproduce this with latest FEP?
    >
    > Other suggestion which should help is to search for RSZ tables inconsistencies with report ANALYZE_RSZ_TABLES, this can search for inconsistencies which can decrease the performance in the query definition and also suggest a fix for that.
    >
    > Kind regards,
    >
    > Ricardo
    Hi Ricardo,
    Yes, i've already upgraded to latest FEP version.
    But thank you for this report. It looks very helpful!
    >
    Arun Varadarajan wrote:
    > Shikha,
    > The question was regarding Query designer performance and not query performance...
    >
    > We faced similar issue earlier - even opening of Query designer took a huge amount of time but then on upgrading upgrading my system to 1 GB ram most of the performance issue went away... check the RAM usage in the task manager and CPU utilization. This might make query designer faster.
    Hi Arun,
    At this point there is no more hope for me, as my system has already 2 GB RAM and a dual core cpu with 2.2 GHz.

  • How do I increase the concurrency of AQ PLSQL notifications

    When we create any number of queues in our system (11.1.0.7) which have pl/sql notifications, one
    job is spawned to handle all the notifications. The job is called aq$_plsql_ntfn_NNNN, and it runs
    the job_action sys.dbms_aqadm_sys.register_driver. This job calls the pl/sql procedure registered
    in the registration. While that procedure is running, this job is essentially blocked. I can
    create more jobs manually in dbms_scheduler which also have the aq$_plsql_ntfn prefix and the same
    job_action, and if I create n such jobs, I can have n + 1 callbacks running concurrently (because
    AQ spawns a job whether or not I create one.)
    In addition, once these jobs finish processing notification AND a certain idle time has passed,
    these jobs exit. When each time the job exits, a new job must be spawned on the next enqueue to a
    registered queue. When the job is spawning, processing of the dequeue is much slower than when the
    job is already running and waiting to time out (second scale vs. millisecond scale).
    My questions are:
    1) Is there any way to tune the number of concurrent notification jobs? If not, is there a real
    danger to starting additional jobs myself?
    2) Is it possible to tune the idle timeout of these jobs?
    3) Am I missing some "approved" way to make notifications run in parallel, or is this lack of
    concurrency by design?
    btw, it occurs to me that I could, in my callback function, use the dbms_scheduler functionality to
    make my actual processing occur in parallel, but this seems like a "hack" to me.

    Hi,
    I would be interested to know if you have found a satisfactory way of having concurrent AQ notification with PL/SQL?
    I have opened a thread today on a very similar subject. (19-Feb-2009: Performance issues when using AQ notification with one consumer )
    Thanks in advance
    Philippe

  • Need to increase query time

    HI,
    i need to extend qeury  buffiring time in BEX'S bw.....

    hi murali
    can u tell me the process..
    i deleted the cache memory by RSRcache t-code
    now i need to increase the time of buffring

  • Query Analysis Services DMV (DISCOVER_LOCKS) with OPENQUERY SQL Server 2012

    Hi, I'm having trouble with a specific DMV, $SYSTEM.DISCOVER_LOCKS, while querying it using T-SQL OPENQUERY.
    For example, this query will run just fine:
    select * from openquery(OLAP_SERVER, 'SELECT * FROM $SYSTEM.DISCOVER_CONNECTIONS')
    But when I try:
    select * from openquery(OLAP_SERVER, 'SELECT * FROM $SYSTEM.DISCOVER_LOCKS')
    I get the following error:
    OLE DB provider "MSOLAP" for linked server "OLAP_SERVER" returned message "Internal error: An unexpected error occurred in conversion.".
    OLE DB provider "MSOLAP" for linked server "OLAP_SERVER" returned message "An error occurred while parsing the 'C5' element at line , column ('urn:schemas-microsoft-com:xml-analysis:rowset' namespace) under Envelope/Body/ExecuteResponse/return/root/row/(any).".
    Msg 7320, Level 16, State 2, Line 3
    Cannot execute the query "SELECT * FROM $SYSTEM.DISCOVER_LOCKS" against OLE DB provider "MSOLAP" for linked server "OLAP_SERVER".
    Similar problems [*] were present on SQL Server 2008, but the author obviously managed to execute queries to all DMV-s. Could the linked server properties play a role in this?
    [*] I am not able to post links or images at the moment until my account is verified, but I was referring to an article by Vidas Matelis, "SSAS 2008 DMVs – querying from the
    SQL Server and database diagrams"

    Hi Frodik,
    Generally, the reason for the error “Cannot execute the query "XX" against OLE DB provider "XX" for linked server "XX"” can be lack of sufficient memory on server. Also possibly running into an SSAS issue which could
    be bringing lots of data into cache. So in your scenario, if you server is X86 server, you can migrate it to X64 server which should help. If you server is X64 server, you can avoid this issue by increasing available memory.
    Hope this helps.
    Regards,
    Charlie Liao
    TechNet Community Support

  • Query Timeout on stored procedure executed from Access 2010

    I am trying to delete old jobs from a SQL 2008 R2 database. I created a stored procedure on the server that deletes all Jobs over 1 year old. The delete can take several minutes to run as it deletes record on several related tables. I am getting a error
    "2147217871 - [Microsoft][ODBC SQL Server Driver]Query timeout expired". The query runs fine on the Server. I have tried setting Client timeout to 300 sec but it times out way before 5 min. I am not sure what else I
    can do to fix this problem. I am hoping someone has seen this and figured out a fix.
    Here is there code
    Public Sub Cleanup_Database()
    On Error GoTo CleanUp_Err
        Dim cmd As ADODB.Command
        Set cmd = New ADODB.Command
        ODBC_conn = "ODBC;Description=testbox2;DRIVER=SQL Server;" & _
                    "SERVER=O2GMSAPPTEST\SQL122DEVL;Trusted_Connection=Yes;" & _
                    "APP=Microsoft Office 2010;DATABASE=IMB_TraceData;StatsLog_On=Yes"
        cmd.ActiveConnection = ODBC_conn
        cmd.CommandType = adCmdStoredProc
        cmd.CommandText = "DataBase_Cleanup"
        cmd.Execute
    CleanUp_Err:
        Dim i As Long
        Dim str As String
        str = ""
        For i = 0 To Errors.Count - 1
            str = str & Errors(i).Number & "-" & Errors(i).Description & " " & vbNewLine
        Next i
        If str = "" Then
            str = Err.Number & " - " & Err.Description
        End If
        MsgBox str, , "Trace Update"
    End Sub
      

    You didn't say how you were setting the client timeout, but this has worked for me in an adp.
    'Temporarily increase query timeout, which is an application-wide setting
    'CurrentProject.Connection.CommandTimeout = 60 'Too late to change this setting- no effect
    Const cstrTimeoutOptionName As String = "OLE/DDE Timeout (Sec)"
    Const clngTimeoutSecondsForQuery As Long = 300
    strQueryTimeOutOriginal = Application.GetOption(OptionName:=cstrTimeoutOptionName)
    Application.SetOption cstrTimeoutOptionName, CStr(clngTimeoutSecondsForQuery)
    Paul

  • Join hint ignored - very bad query plan results

    I'm trying to get a query that's being generated by SSAS to perform acceptably. Because this query is (at least partially) generated by SSAS, I don't have complete control over the query text.
    The problem - The query looks roughly like this:
    select
    -- a bunch of columns
    from
    T1 -- table with ~40,000,000 rows
    inner loop join T2 on T2.t2id = T1.t2id -- table with ~16,000 rows
    inner loop join T3 on T3.t3id = T1.t3id -- table with ~200,000 rows inner loop join T4 on T4.t4id = T1.t4id -- table with ~200,000 rows
    left loop join T5 on T5.t3id = T1.t3id and T5.t6id = T4.t6id -- table with 0 rows
    where
    -- some uninteresting conditions
    T1 is a Fact (or Measure) table, T2, T3 and T4 are Dimension tables, T5 and T6 are involved in the filtering of the query.  Every row of T1 WILL match exactly one row in each of T1, T2 and T3.
    You'll note that I've hinted all of the joins - according to the documentation, using join hints forces join order (which is consistent with the plan that's produced).  There's no mention that join hints can be transparently ignored by the optimizer,
    but that seems to be precisely what's happening.
    In the plan that results, the join to T4 is done as a hash join, with T1*T2*T3 (40,000,000 rows) on the "top", so it ends up trying to build a hash table with 40,000,000 rows, resulting in very high tempdb activity and very poor performance (I
    don't know how poor - I've never let it finish).
    I can see part of the reason why it's making this join choice - the estimate of T1*T2*T3 is only 35,000 rows, even though T1 has 40,000,000 rows and the join will hit on every row.
    Questions:
    1. What can I do to the query or the database to improve the estimate on the join to T3?
    2. What can I do to the query to force the optimizer to use a loop join like I asked? 
    Version is SQL Server 2008 R2 SP2 Developer Edition X64.
    OS is Windows 2008 R2
    Machine is dual-quad-hyper-threaded CPU with 96Gb of RAM and several Tb of disk spread over 8 spindles and SSDs.
    I've seen this query perform well before - I've been tuning this query for going on 7 years now and I've never seen it perform this badly that I can recall.  Not sure if it's something that's changed in SP2, or something about the distribution
    of data in this particular database that's changed, but something's sure changed.
    -cd Mark the best replies as answers!

    That would indicate that there are no foreign-key constraints, or if they, they are not trusted. (Assuming here that there WHERE clause includes no filter on T1.)
    That, or the statistics on T1 are giving a completely wrong estimates for the number of NULL values in t2id and t3id.
    Erland Sommarskog, SQL Server MVP, [email protected]
    Thanks, Erland.
    There are foreign key constraints, but they're NOCHECK.  I've tried creating and updating statistics on various columns of the tables involved - I'll re-check that I've got up to date statistics on all of the columns involved in these joins.
    What about the join-hint being ignored?  Is there anything I can do, or has something changed here recently?  Interestingly, when I run this query from SSMS, I get loop-joins all around, but when SSAS runs the query, the one join always comes out
    a hash join. Before I added the hints, all joins were hash joins - the hints worked on all joins but the one.
    -cd Mark the best replies as answers!

  • Query timeout on 10g thin driver

    I'm using the Statement.setQueryTimeout(...) method, but it doesn't seem to work when the server is shutdown.
    I do the following:
    1) create the statement and set the query timeout to 30 seconds
    2) run a query that gets stuck waiting for a lock (ie select for update...)
    Now, if I let it run normally I do see the following exception:
    ORA-01013: user requested cancel of current operation
    However, if I run the same query, but shutdown the database server before releasing the lock I see the client application hang indefinitely. The 30 second timeout never occurs.
    If I take a thread dump I see my client is stuck on a socket read, which will never come because the db server was shutdown.
    Why isn't the jdbc driver honoring the 30 second timeout if it can't communicate with the server? Does this mean that the timeout is actually controlled on the server-side?
    any info is appreciated,
    Brett

    You didn't say how you were setting the client timeout, but this has worked for me in an adp.
    'Temporarily increase query timeout, which is an application-wide setting
    'CurrentProject.Connection.CommandTimeout = 60 'Too late to change this setting- no effect
    Const cstrTimeoutOptionName As String = "OLE/DDE Timeout (Sec)"
    Const clngTimeoutSecondsForQuery As Long = 300
    strQueryTimeOutOriginal = Application.GetOption(OptionName:=cstrTimeoutOptionName)
    Application.SetOption cstrTimeoutOptionName, CStr(clngTimeoutSecondsForQuery)
    Paul

  • How to Programmatically generate .asdatabase from SSAS database from SSAS server?

    Is there a way to  Programmatically generate .asdatabase file and other ssas config files from SSAS database? Not by devenv .dwproj. 

    Thank you very much for your answer. 
    I know how to generate the asdatabase: (I can do it by devenv.exe  project.dwproj
    or build the ssas project manually).  How ever I want to programmatically generate
    .asdatabase by C#.
    I
    programmatically did
    some modifications on the SSAS database(drop some dimension and measures),  then I want to generate .asdatabase so that i can deploy it to the server. I know i can do it by xmla as well.  But I really like deploying the cube by .asdatabase, not xmla.
    Any thoughts?
    Thanks,
    Jackie

  • Concurrent requests limit in JRC

    Hi,
    I understand that there is a limit of five concurrent requests in CR 2008 JRC. Is there any way to increase this limit? I am trying to generate PDF in multi threaded environment and would like to see the options for better scalability.
    Thanks,
    Ashok

    The CRJ objects aren't thread-safe, so you'd have to ensure each thread has independent ReportClientDocument instances.
    That being stated, there's currently no way to increase the Concurrent Process Licensing (CPL) limit with CRJ.
    The CR4E isn't meant for volume batch processing of reports - BusinessObjects Enterprise or Crystal Reports Server Embedded (OEM/Partner solutions) are.
    Sincerely,
    Ted Ueda

Maybe you are looking for