MS SQL Server + ATG tuning

Hi All
I am working with MS SQL Server 2008 R2 with ATG10.0.2.
I would like to know from the experienced guys who have used this combination as to what would be the best practises involved in the tuning and tweaks related to MS SQL server 2008 with ATG
Right now, I m getting a lot of issues with the deploying assets greater than 2K.
I get transaction deadlocks and transaction timeouts.
TIA

Hi TIA,
As with many topics, there is quite a lot of information about this in the Oracle Knowledge Base.
Log in to My Oracle Support (MOS) support.oracle.com and use the search box at the top right.
If you enter for example
mssql atg
you'll find a number of useful articles.
For this topic for example start with
How to improve the performance of ATG applications using Microsoft SQL Server (Doc ID 1037890.1)
but there are many others.
Oracle ATG technical support engineers are adding knowledge articles on a daily basis, so always use the knowledge base for all topics.
HTH,
Sherryl

Similar Messages

SQL Server database Tuning Advisor not working SQL 2014 CU3

Any idea?
Problem signature:
Problem Event Name: APPCRASH
Application Name: DTAEngine.exe
Application Version: 2014.120.2000.8
Application Timestamp: 5306c816
Fault Module Name: StackHash_560c
Fault Module Version: 6.3.9600.17114
Fault Module Timestamp: 53648f36
Exception Code: c0000374
Exception Offset: PCH_DC_FROM_ntdll+0x0003D2EC
OS Version: 6.3.9600.2.0.0.400.8
Locale ID: 1033
Additional Information 1: 560c
Additional Information 2: 560c82bf94e4e2b42d9b3a1a3a0f9a27
Additional Information 3: 966f
Additional Information 4: 966f4f60943cf9e4691aec84523dade7
Read our privacy statement online:
http://go.microsoft.com/fwlink/?linkid=280262
If the online privacy statement is not available, please read our privacy statement offline:
C:\Windows\system32\en-US\erofflps.txt
SkyRiver

Hello,
Are you using the “Save to file” option? Try reducing the “Set maximum file size (MB)”. Disable file rollover or use it
for creating 2-3 files only.
Please report this issue on Microsoft Connect.
https://connect.microsoft.com/SQLServer
Hope this helps.
Regards,
Alberto Morillo
SQLCoffee.com

SQL Server Business Intelligence Development Studio 2008, download and install

Hello,
I have been looking for a while now on where to download and how to install BIDS 2008 on top of my current SQL Server 2008 installation but no luck yet. Would you mind helping me out please.
FYI, here is my current configuration:
- SQL Server 2008 and SQL Server Management Studio installed
- Visual Studio 2008 installed before the above.
I've read that I could find a new kind of project (BIDS one) in visual studio but I can't. And from other sources, I should find a link in "All Programms > Microsoft SQL Server 2008 > Business Intelligence Development Studio" but cannot.
Thank you in advance for pointing me to the download page or any kind of post that could help me out getting this.
Regards,
J.

Which edition of SQL Server 2008 you had installed? Is it SQL 2008 Express? If yes, then make sure you had installed it with Advanced Services:
http://www.microsoft.com/downloads/en/details.aspx?FamilyId=B5D1B8C3-FDA5-4508-B0D0-1311D670E336&displaylang=en
If you had installed SQL Server 2008 from a CD or Media, then you can run the installation media, and make sure you had selected BIDS as an option, along with Management Tools.
SQL Server Management tools include the following components:
SQL Server Management Studio
SQL Server Profiler
SQL Server Database Tuning Advisor
Command prompt tools, such as sqlcmd.exe and osql.exe.
Business Intelligence Development Studio add-ins to Microsoft Visual Studio
Note that Business Intelligence Development Studio is a separate option during SQL Server installation.
Hope this helps...
Chaitanya (http://twitter.com/chmediko)

Need clear steps for doing performance tuning on SQL Server 2008 R2 (DB Engine, Reporting Services and Integration Services)

We have to inverstigate about a reporting solution where things are getting slow (may be material, database design, network matters).
I have red a lot in MSDN and some books about performance tuning on SQL Server 2008 R2 (or other) but frankly, I feel a little lost in all that stuff
I'am looking for practical steps in order to do the tuning. Someone had like a recipe for that : a success story...
My (brain storm) Methodology should follow these steps:
Resource bottlenecks: CPU, memory, and I/O bottlenecks
tempdb bottlenecks
A slow-running user query : Missing indexes, statistics,...
Use performance counters : there are many, can one give us the list of the most important
how to do fine tuning about SQL Server configuration
SSRS, SSIS configuration ?
And do the recommandations.
Thanks
"there is no Royal Road to Mathematics, in other words, that I have only a very small head and must live with it..."
Edsger W. Dijkstra

Hello,
There is no clear defined step which can be categorized as step by step to performance tuning.Your first goal is to find out cause or drill down to factor causing slowness of SQL server it can be poorly written query ,missing indexes,outdated stats.RAM crunch
CPU crunch so on and so forth.
I generally refer to below doc for SQL server tuning
http://technet.microsoft.com/en-us/library/dd672789(v=sql.100).aspx
For SSIS tuning i refer below doc.
http://technet.microsoft.com/library/Cc966529#ECAA
http://msdn.microsoft.com/en-us/library/ms137622(v=sql.105).aspx
When I face issue i generally look at wait stats ,wait stats give you idea about on what resource query was waiting.
--By Jonathan KehayiasSELECT TOP 10
wait_type ,
max_wait_time_ms wait_time_ms ,
signal_wait_time_ms ,
wait_time_ms - signal_wait_time_ms AS resource_wait_time_ms ,
100.0 * wait_time_ms / SUM(wait_time_ms) OVER ( )
AS percent_total_waits ,
100.0 * signal_wait_time_ms / SUM(signal_wait_time_ms) OVER ( )
AS percent_total_signal_waits ,
100.0 * ( wait_time_ms - signal_wait_time_ms )
/ SUM(wait_time_ms) OVER ( ) AS percent_total_resource_waits
FROM sys.dm_os_wait_stats
WHERE wait_time_ms > 0 -- remove zero wait_time
AND wait_type NOT IN -- filter out additional irrelevant waits
( 'SLEEP_TASK', 'BROKER_TASK_STOP', 'BROKER_TO_FLUSH',
'SQLTRACE_BUFFER_FLUSH','CLR_AUTO_EVENT', 'CLR_MANUAL_EVENT',
'LAZYWRITER_SLEEP', 'SLEEP_SYSTEMTASK', 'SLEEP_BPOOL_FLUSH',
'BROKER_EVENTHANDLER', 'XE_DISPATCHER_WAIT', 'FT_IFTSHC_MUTEX',
'CHECKPOINT_QUEUE', 'FT_IFTS_SCHEDULER_IDLE_WAIT',
'BROKER_TRANSMITTER', 'FT_IFTSHC_MUTEX', 'KSOURCE_WAKEUP',
'LAZYWRITER_SLEEP', 'LOGMGR_QUEUE', 'ONDEMAND_TASK_QUEUE',
'REQUEST_FOR_DEADLOCK_SEARCH', 'XE_TIMER_EVENT', 'BAD_PAGE_PROCESS',
'DBMIRROR_EVENTS_QUEUE', 'BROKER_RECEIVE_WAITFOR',
'PREEMPTIVE_OS_GETPROCADDRESS', 'PREEMPTIVE_OS_AUTHENTICATIONOPS',
'WAITFOR', 'DISPATCHER_QUEUE_SEMAPHORE', 'XE_DISPATCHER_JOIN',
'RESOURCE_QUEUE' )
ORDER BY wait_time_ms DESC
use below link to analyze wait stats
http://www.sqlskills.com/blogs/paul/wait-statistics-or-please-tell-me-where-it-hurts/
HTH
PS: for reporting services you can post in SSRS forum
Please mark this reply as the answer or vote as helpful, as appropriate, to make it useful for other readers

Monitoring/ tuning tool for sql server and os environment

What is a good tool for monitoring/tuning OS and SQL layers.

OS and SQL are different things, and I am not sure what you mean under layers.
To further expand my question, what OS build, what SQL Server edition?
All these have enormous impact on strategy.
Then the question is for what? OLTP need one tuning, DW another.
Monitoring is also a question of budget, how much allowed to spend?
Secondly, this forum is not dedicated to these kinds of questions.
Arthur My Blog

Increase Performance and ROI for SQL Server Environments

May 2015
Explore
The Buzz from Microsoft Ignite 2015
NetApp was in full force at the recent Microsoft Ignite show in Chicago, talking about solutions for hybrid cloud, and our proven solutions for Microsoft SQL Server and other Microsoft applications.
Hot topics at the NetApp booth included:
OnCommand® Shift. A revolutionary technology that lets you move virtual machines back and forth between VMware and Hyper-V environments in minutes.
Azure Site Recovery to NetApp Private Storage. Replicate on-premises SAN-based applications to NPS for disaster recovery in the Azure cloud.
These tools give you greater flexibility for managing and protecting important business applications.
Chris Lemmons
Director, EIS Technical Marketing, NetApp
If your organization runs databases such as Microsoft SQL Server and Oracle DB, you probably know that these vendors primarily license their products on a "per-core" basis. Microsoft recently switched to "per-core" rather than "per-socket" licensing for SQL Server 2012 and 2014. This change can have a big impact on the total cost of operating a database, especially as core counts on new servers continue to climb. It turns out that the right storage infrastructure can drive down database costs, increase productivity, and put your infrastructure back in balance.
In many customer environments, NetApp has noticed that server CPU utilization is low—often on the order of just 20%. This is usually the result of I/O bottlenecks. Server cores have to sit and wait for I/O from hard disk drives (HDDs). We've been closely studying the impact of all-flash storage on SQL Server environments that use HDD-based storage systems. NetApp® All Flash FAS platform delivers world-class performance for SQL Server plus the storage efficiency, application integration, nondisruptive operations, and data protection of clustered Data ONTAP®, making it ideal for SQL Server environments.
Tests show that All Flash FAS can drive up IOPS and database server CPU utilization by as much as 4x. And with a 95% reduction in latency, you can achieve this level of performance with half as many servers. This reduces the number of servers you need and the number of cores you have to license, driving down costs by 50% or more and paying back your investment in flash in as little as six months.
Figure 1) NetApp All Flash FAS increases CPU utilization on your SQL Server database servers, lowering costs.
Source: NetApp, 2015
Whether you're running one of the newer versions of SQL Server or facing an upgrade of an earlier version, you can't afford not to take a second look at your storage environment.
End of Support for Microsoft SQL Server 2005 is Rapidly Approaching
Microsoft has set the end of extended support for SQL Server 2005 for April 2016—less than a year away. With support for Microsoft Windows 2003 ending in July 2015, time may already be running short.
If you're running Windows Server 2003, new server hardware is almost certainly needed when you upgrade SQL Server. Evaluate your server and storage options now to get costs under control.
Test Methodology
To test the impact of flash on SQL Server performance, we replaced a legacy HDD-based storage system with an All Flash FAS AFF8080 EX. The legacy system was configured with almost 150 HDDs, a typical configuration for HDD storage supporting SQL Server. The AFF8080 EX used just 48 SSDs.
Table 1) Components used in testing.
Test Configuration Components
Details
SQL Server 2014 servers
Fujitsu RX300
Server operating system
Microsoft Windows 2012 R2 Standard Edition
SQL Server database version
Microsoft SQL Server 2014 Enterprise Edition
Processors per server
2 6-core Xeon E5-2630 at 2.30 GHz
Fibre channel network
8Gb FC with multipathing
Storage controller
AFF8080 EX
Data ONTAP version
Clustered Data ONTAP® 8.3.1
Drive number and type
48 SSD
Source: NetApp, 2015
The test configuration consisted of 10 database servers connected through fibre channel to both the legacy storage system and the AFF8080 EX. Each of the 10 servers ran SQL Server 2014 Enterprise Edition.
The publicly available HammerDB workload generator was used to drive an OLTP-like workload simultaneously from each of the 10 database servers to storage. We first directed the workload to the legacy storage array to establish a baseline, increasing the load to the point where read latency consistently exceeded 20ms.
That workload was then directed at the AFF8080 EX. The change in storage resulted in an overall 20x reduction in read latency, a greater than 4x improvement in IOPS, and a greater than 4x improvement in database server CPU utilization.
Figure 2) NetApp All Flash FAS increases IOPS and server CPU utilization and lowers latency.
Source: NetApp, 2015
In other words, the database servers are able to process four times as many IOPS with dramatically lower latency. CPU utilization goes up accordingly because the servers are processing 4x the work per unit time.
The All Flash FAS system still had additional headroom under this load.
Calculating the Savings
Let's look at what this performance improvement means for the total cost of running SQL Server 2014 over a 3-year period. To do the analysis we used NetApp Realize, a storage modeling and financial analysis tool designed to help quantify the value of NetApp solutions and products. NetApp sales teams and partners use this tool to assist with return on investment (ROI) calculations.
The calculation includes the cost of the AFF8080 EX, eliminates the costs associated with the existing storage system, and cuts the total number of database servers from 10 to five. This reduces SQL Server licensing costs by 50%. The same workload was run with five servers and achieved the same results. ROI analysis is summarized in Table 2.
Table 2) ROI from replacing an HDD-based storage system with All Flash FAS, thereby cutting server and licensing costs in half.
Value
Analysis Results
ROI
65%
Net present value (NPV)
$950,000
Payback period
six months
Total cost reduction
More than $1 million saved over a 3-year analysis period compared to the legacy storage system
Savings on power, space, and administration
$40,000
Additional savings due to nondisruptive operations benefits (not included in ROI)
$90,000
Source: NetApp, 2015
The takeaway here is that you can replace your existing storage with All Flash FAS and get a big performance bump while substantially reducing your costs, with the majority of the savings derived from the reduction in SQL Server licensing costs.
Replace your existing storage with All Flash FAS and get a big performance bump while substantially reducing your costs.
Maximum SQL Server 2014 Performance
In addition to the ROI analysis, we also measured the maximum performance of the AFF8080 EX with SQL Server 2014. A load-generation tool was used to simulate an industry-standard TPC-E OLTP workload against an SQL Server 2014 test configuration.
A two-node AFF8080 EX achieved a maximum throughput of 322K IOPS at just over 1ms latency. For all points other than the maximum load point, latency was consistently under 1ms and remained under 0.8ms up to 180K IOPS.
Data Reduction and Storage Efficiency
In addition to performance testing, we looked at the overall storage efficiency savings of our SQL Server database implementation. The degree of compression that can be achieved is dependent on the actual data that is written and stored in the database. For this environment, inline compression was effective. Deduplication, as is often the case in database environments, provided little additional storage savings and was not enabled.
For the test data used in the maximum performance test, we measured a compression ratio of 1.5:1. We also tested inline compression on a production SQL Server 2014 data set to further validate these results and saw a 1.8:1 compression ratio.
Space-efficient NetApp Snapshot® copies provide additional storage efficiency benefits for database environments. Unlike snapshot methods that use copy-on-write, there is no performance penalty; unlike full mirror copies, NetApp Snapshot copies use storage space sparingly. Snapshot copies only consume a small amount of storage space for metadata and additional incremental space is consumed as block-level changes occur. In a typical real-world SQL Server deployment on NetApp storage, database volume Snapshot copies are made every two hours.
First introduced more than 10 years ago, NetApp FlexClone® technology also plays an important role in SQL Server environments. Clones are fully writable, and, similar to Snapshot copies, only consume incremental storage capacity. With FlexClone, you can create as many copies of production data as you need for development and test, reporting, and so on. Cloning is a great way to support the development and test work needed when upgrading from an earlier version of SQL Server. You'll sometimes see these types of capabilities referred to as "copy data management."
A Better Way to Run Enterprise Applications
The performance benefits that all-flash storage can deliver for database environments are significant: more IOPS, lower latency, and an end to near-constant performance tuning.
If you think the performance acceleration that comes with all-flash storage is cost prohibitive, think again. All Flash FAS doesn't just deliver a performance boost, it changes the economics of your operations, paying for itself with thousands in savings on licensing and server costs. In terms of dollars per IOPS, All Flash FAS is extremely economical relative to HDD.
And, because All Flash FAS runs NetApp clustered Data ONTAP, it delivers the most complete environment to support SQL Server and all your enterprise applications with capabilities that include comprehensive storage efficiency, integrated data protection, and deep integration for your applications.
For complete details on this testing look for NetApp TR-4303, which will be available in a few weeks. Stay tuned to Tech OnTap for more information as NetApp continues to run benchmarks with important server workloads including Oracle DB and server virtualization.
Learn more about NetApp solutions for SQL Server and NetApp All-flash solutions.
Quick Links
Tech OnTap Community
Archive
PDF

May 2015
Explore
The Buzz from Microsoft Ignite 2015
NetApp was in full force at the recent Microsoft Ignite show in Chicago, talking about solutions for hybrid cloud, and our proven solutions for Microsoft SQL Server and other Microsoft applications.
Hot topics at the NetApp booth included:
OnCommand® Shift. A revolutionary technology that lets you move virtual machines back and forth between VMware and Hyper-V environments in minutes.
Azure Site Recovery to NetApp Private Storage. Replicate on-premises SAN-based applications to NPS for disaster recovery in the Azure cloud.
These tools give you greater flexibility for managing and protecting important business applications.
Chris Lemmons
Director, EIS Technical Marketing, NetApp
If your organization runs databases such as Microsoft SQL Server and Oracle DB, you probably know that these vendors primarily license their products on a "per-core" basis. Microsoft recently switched to "per-core" rather than "per-socket" licensing for SQL Server 2012 and 2014. This change can have a big impact on the total cost of operating a database, especially as core counts on new servers continue to climb. It turns out that the right storage infrastructure can drive down database costs, increase productivity, and put your infrastructure back in balance.
In many customer environments, NetApp has noticed that server CPU utilization is low—often on the order of just 20%. This is usually the result of I/O bottlenecks. Server cores have to sit and wait for I/O from hard disk drives (HDDs). We've been closely studying the impact of all-flash storage on SQL Server environments that use HDD-based storage systems. NetApp® All Flash FAS platform delivers world-class performance for SQL Server plus the storage efficiency, application integration, nondisruptive operations, and data protection of clustered Data ONTAP®, making it ideal for SQL Server environments.
Tests show that All Flash FAS can drive up IOPS and database server CPU utilization by as much as 4x. And with a 95% reduction in latency, you can achieve this level of performance with half as many servers. This reduces the number of servers you need and the number of cores you have to license, driving down costs by 50% or more and paying back your investment in flash in as little as six months.
Figure 1) NetApp All Flash FAS increases CPU utilization on your SQL Server database servers, lowering costs.
Source: NetApp, 2015
Whether you're running one of the newer versions of SQL Server or facing an upgrade of an earlier version, you can't afford not to take a second look at your storage environment.
End of Support for Microsoft SQL Server 2005 is Rapidly Approaching
Microsoft has set the end of extended support for SQL Server 2005 for April 2016—less than a year away. With support for Microsoft Windows 2003 ending in July 2015, time may already be running short.
If you're running Windows Server 2003, new server hardware is almost certainly needed when you upgrade SQL Server. Evaluate your server and storage options now to get costs under control.
Test Methodology
To test the impact of flash on SQL Server performance, we replaced a legacy HDD-based storage system with an All Flash FAS AFF8080 EX. The legacy system was configured with almost 150 HDDs, a typical configuration for HDD storage supporting SQL Server. The AFF8080 EX used just 48 SSDs.
Table 1) Components used in testing.
Test Configuration Components
Details
SQL Server 2014 servers
Fujitsu RX300
Server operating system
Microsoft Windows 2012 R2 Standard Edition
SQL Server database version
Microsoft SQL Server 2014 Enterprise Edition
Processors per server
2 6-core Xeon E5-2630 at 2.30 GHz
Fibre channel network
8Gb FC with multipathing
Storage controller
AFF8080 EX
Data ONTAP version
Clustered Data ONTAP® 8.3.1
Drive number and type
48 SSD
Source: NetApp, 2015
The test configuration consisted of 10 database servers connected through fibre channel to both the legacy storage system and the AFF8080 EX. Each of the 10 servers ran SQL Server 2014 Enterprise Edition.
The publicly available HammerDB workload generator was used to drive an OLTP-like workload simultaneously from each of the 10 database servers to storage. We first directed the workload to the legacy storage array to establish a baseline, increasing the load to the point where read latency consistently exceeded 20ms.
That workload was then directed at the AFF8080 EX. The change in storage resulted in an overall 20x reduction in read latency, a greater than 4x improvement in IOPS, and a greater than 4x improvement in database server CPU utilization.
Figure 2) NetApp All Flash FAS increases IOPS and server CPU utilization and lowers latency.
Source: NetApp, 2015
In other words, the database servers are able to process four times as many IOPS with dramatically lower latency. CPU utilization goes up accordingly because the servers are processing 4x the work per unit time.
The All Flash FAS system still had additional headroom under this load.
Calculating the Savings
Let's look at what this performance improvement means for the total cost of running SQL Server 2014 over a 3-year period. To do the analysis we used NetApp Realize, a storage modeling and financial analysis tool designed to help quantify the value of NetApp solutions and products. NetApp sales teams and partners use this tool to assist with return on investment (ROI) calculations.
The calculation includes the cost of the AFF8080 EX, eliminates the costs associated with the existing storage system, and cuts the total number of database servers from 10 to five. This reduces SQL Server licensing costs by 50%. The same workload was run with five servers and achieved the same results. ROI analysis is summarized in Table 2.
Table 2) ROI from replacing an HDD-based storage system with All Flash FAS, thereby cutting server and licensing costs in half.
Value
Analysis Results
ROI
65%
Net present value (NPV)
$950,000
Payback period
six months
Total cost reduction
More than $1 million saved over a 3-year analysis period compared to the legacy storage system
Savings on power, space, and administration
$40,000
Additional savings due to nondisruptive operations benefits (not included in ROI)
$90,000
Source: NetApp, 2015
The takeaway here is that you can replace your existing storage with All Flash FAS and get a big performance bump while substantially reducing your costs, with the majority of the savings derived from the reduction in SQL Server licensing costs.
Replace your existing storage with All Flash FAS and get a big performance bump while substantially reducing your costs.
Maximum SQL Server 2014 Performance
In addition to the ROI analysis, we also measured the maximum performance of the AFF8080 EX with SQL Server 2014. A load-generation tool was used to simulate an industry-standard TPC-E OLTP workload against an SQL Server 2014 test configuration.
A two-node AFF8080 EX achieved a maximum throughput of 322K IOPS at just over 1ms latency. For all points other than the maximum load point, latency was consistently under 1ms and remained under 0.8ms up to 180K IOPS.
Data Reduction and Storage Efficiency
In addition to performance testing, we looked at the overall storage efficiency savings of our SQL Server database implementation. The degree of compression that can be achieved is dependent on the actual data that is written and stored in the database. For this environment, inline compression was effective. Deduplication, as is often the case in database environments, provided little additional storage savings and was not enabled.
For the test data used in the maximum performance test, we measured a compression ratio of 1.5:1. We also tested inline compression on a production SQL Server 2014 data set to further validate these results and saw a 1.8:1 compression ratio.
Space-efficient NetApp Snapshot® copies provide additional storage efficiency benefits for database environments. Unlike snapshot methods that use copy-on-write, there is no performance penalty; unlike full mirror copies, NetApp Snapshot copies use storage space sparingly. Snapshot copies only consume a small amount of storage space for metadata and additional incremental space is consumed as block-level changes occur. In a typical real-world SQL Server deployment on NetApp storage, database volume Snapshot copies are made every two hours.
First introduced more than 10 years ago, NetApp FlexClone® technology also plays an important role in SQL Server environments. Clones are fully writable, and, similar to Snapshot copies, only consume incremental storage capacity. With FlexClone, you can create as many copies of production data as you need for development and test, reporting, and so on. Cloning is a great way to support the development and test work needed when upgrading from an earlier version of SQL Server. You'll sometimes see these types of capabilities referred to as "copy data management."
A Better Way to Run Enterprise Applications
The performance benefits that all-flash storage can deliver for database environments are significant: more IOPS, lower latency, and an end to near-constant performance tuning.
If you think the performance acceleration that comes with all-flash storage is cost prohibitive, think again. All Flash FAS doesn't just deliver a performance boost, it changes the economics of your operations, paying for itself with thousands in savings on licensing and server costs. In terms of dollars per IOPS, All Flash FAS is extremely economical relative to HDD.
And, because All Flash FAS runs NetApp clustered Data ONTAP, it delivers the most complete environment to support SQL Server and all your enterprise applications with capabilities that include comprehensive storage efficiency, integrated data protection, and deep integration for your applications.
For complete details on this testing look for NetApp TR-4303, which will be available in a few weeks. Stay tuned to Tech OnTap for more information as NetApp continues to run benchmarks with important server workloads including Oracle DB and server virtualization.
Learn more about NetApp solutions for SQL Server and NetApp All-flash solutions.
Quick Links
Tech OnTap Community
Archive
PDF

How can I architect my data layer to yield query result pages to the application as SQL Server prepares them?

I tried to make the question as explicit as possible.
Refer to Sql Server Management Studio's Results view. Depending upon the structure of the execution plan, the Results pane may begin displaying results while the query is still executing. Can someone point me in a direction for architecting a
data layer (I am tech and framework agnostic for this task. Any solution will suffice) that will begin receiving pages of the set before SQL Server has completed the entire query?
The call from the data layer to SQL Server will obviously have to be asynchronous, but is there any additional ceremony that I need to be aware of when issuing OPTION (FAST x) to the query optimizer?

Thanks for the reply. (I actually meant to put this in the SQL Data Access forum, not the T-SQL forum)
"Generally the last step is ORDER BY in a
query, so nothing can start before that executes."
I would imagine you cannot ORDER BY and yield results as they are fetched because of the execution plan that would be generated. For the purposes of this post, please assume that sorting will be done purely client side
"Can you post your query?"
For purposes of discussion, let's assume that the query is
select *
from information_schema.columns
and also assume that you have "lots" of columns to display.
This was an exploratory question to see what would be necessary to replicate the behavior of Management Studio's Query Result view in a custom application.
I would imagine that there's going to be a lot of analysis of the execution plans that get generated in order for the OPTION (FAST x) optimizer hint to do any good, but apart from general tuning concerns that would allow SQL SERVER to yield a page of data
"fast", I was wondering if there was anything else required of the calling client to force it to yield return its first page.
After thinking about this (and phrasing it the way I did in the last sentence) perhaps this is the incorrect forum for this question. I imagine that my concerns are better addressed in forums dedicated to the technology of the calling client (which
would be a .NET assembly)
Be that as it may, if there is any ceremony that SQL Server imposes on clients in order to yield return, I would expect that my question would be in the scope of SQL Server discussions (even though I intended this to be in a different SQL Server forum)

Frequenet dead locks in SQL Server 2008 R2 SP2

Hi,
We are experiencing frequent dead locks in our application. We are using MSSQL Server 2008 R2 SP2 version. When our application is configured for 5-6 app servers, this issue is occurring frequently.
But, when the same application is used with the MSSQL Server 2008 R2 or SQL Server 2012, we don't see the dead lock issue. From the error lock and sql trace, the error message is thrown for the database table JobLock. We have a stored procedure to insert/update
for the above table when the job moves from one service to other. The same procedure works fine when used with the 2008 R2 and SQL Server 2012 Version.
Is the above issue related to the hotfix from the below url?
http://support.microsoft.com/kb/2703275
Following error message is seen frequently in the log file.
INFO : 03/24/2014 10:26:30:290 PM: [00007900:00005932] [Xerox.ISP.Workflow.ManagedActivity.PersistInTransaction] System.Data.SqlClient.SqlException (0x80131904): Transaction (Process ID 62) was deadlocked on lock resources with another process and has been
chosen as the deadlock victim. Rerun the transaction.
   at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
   at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
   at System.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
   at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
   at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async, Int32 timeout, Task& task, Boolean asyncWrite, SqlDataReader ds)
   at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, String methodName, Boolean sendToPipe, Int32 timeout, Boolean asyncWrite)
   at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
   at Microsoft.Practices.EnterpriseLibrary.Data.Database.DoExecuteNonQuery(DbCommand command)
   at Microsoft.Practices.EnterpriseLibrary.Data.Database.ExecuteNonQuery(DbCommand command, DbTransaction transaction)
   at Xerox.ISP.DataAccess.Data.Utility.ExecuteNonQuery(TransactionManager transactionManager, DbCommand dbCommand)
   at Xerox.ISP.DataAccess.Data.SqlClient.SqlActivityProviderBase.ActivityReady(TransactionManager transactionManager, Int32 start, Int32 pageLength, Nullable`1 ActivityID, Nullable`1 JobId, String ContentUrl, Nullable`1 PrevWorkStep, Nullable`1
CurrentWorkStep, String Principal, Nullable`1 Status, Nullable`1 ServerID, String HostName, Nullable`1 LockUserID, Nullable`1& ErrorCode, Byte[]& Activity_TS)
   at Xerox.ISP.DataAccess.Domain.ActivityBase.ActivityReady(Nullable`1 ActivityID, Nullable`1 JobId, String ContentUrl, Nullable`1 PrevWorkStep, Nullable`1 CurrentWorkStep, String Principal, Nullable`1 Status, Nullable`1 ServerID, String HostName,
Nullable`1 LockUserID, Nullable`1& ErrorCode, Byte[]& Activity_TS, Int32 start, Int32 pageLength)
   at Xerox.ISP.DataAccess.Domain.ActivityBase.ActivityReady(Nullable`1 ActivityID, Nullable`1 JobId, String ContentUrl, Nullable`1 PrevWorkStep, Nullable`1 CurrentWorkStep, String Principal, Nullable`1 Status, Nullable`1 ServerID, String HostName,
Nullable`1 LockUserID, Nullable`1& ErrorCode, Byte[]& Activity_TS)
   at Xerox.ISP.Workflow.ManagedActivity.<>c__DisplayClass2f.<ActivityReady>b__2d()
   at Xerox.ISP.Workflow.ManagedActivity.PersistInTransaction(Boolean createNew, PersistMethod persist)
ClientConnectionId:9e44a64f-5014-4634-9cee-4581e1b9c299
I look forward to the suggestions to get the issue resolved. Your input is much appreciated.
Thanks,
Keshava.

If you are having deadlock trouble in your SQL Server instance, this recipe demonstrates how to make sure deadlocks are logged to the SQL ServerManagement Studio SQL log appropriately using
the DBCC TRACEON, DBCC TRACEOFF, and DBCC TRACESTATUS commands. These functions enable, disable, and check the status of trace flags.
To determine the cause of a deadlock, we need to know
the resources involved and the types of locks acquired and requested. For this kind of information, SQL Server provides
Trace Flag 1222 (this flag supersedes 1204, which was frequently used in earlier versions of SQL Server.)
DBCCTRACEON(1222,
-1);
GO
With this flag enabled, SQL Server will provide output in the form of a deadlock graph, showing the executing statements
for each session, at the time of the deadlock; these are the statements that were blocked and so formed the conflict or cycle that led to the deadlock.
Be aware that it is rarely possible to guarantee that deadlocks will never occur. Tuning for deadlocks
primarily involves minimizing the likelihood of their occurrence. Most of the techniques for minimizing the occurrence of deadlocks are similar to the general techniques for minimizing blocking problems.

Improve the performance in stored procedure using sql server 2008 - esp where clause in very big table - Urgent

Hi,
I am looking for inputs in tuning stored procedure using sql server 2008. l am new to performance tuning in sql,plsql and oracle. currently facing issue in stored procedure - need to increase the performance by code optmization/filtering the records using where clause in larger table., the requirement is Stored procedure generate Audit Report which is accessed by approx. 10 Admin Users typically 2-3 times a day by each Admin users.
It has got CTE ( common table expression ) which is referred 2 time within SP. This CTE is very big and fetches records from several tables without where clause. This causes several records to be fetched from DB and then needed processing. This stored procedure is running in pre prod server which has 6gb of memory and built on virtual server and the same proc ran good in prod server which has 64gb of ram with physical server (40sec). and the execution time in pre prod is 1min 9seconds which needs to be reduced upto 10secs or so will be the solution. and also the exec time differs from time to time. sometimes it is 50sec and sometimes 1min 9seconds..
Pl provide what is the best option/practise to use where clause to filter the records and tool to be used to tune the procedure like execution plan, sql profiler?? I am using toad for sqlserver 5.7. Here I see execution plan tab available while running the SP. but when i run it throws an error. Pl help and provide inputs.
Thanks,
Viji

You've asked a SQL Server question in an Oracle forum. I'm expecting that this will get locked momentarily when a moderator drops by.
Microsoft has its own forums for SQL Server, you'll have more luck over there. When you do go there, however, you'll almost certainly get more help if you can pare down the problem (or at least better explain what your code is doing). Very few people want to read hundreds of lines of code, guess what's it's supposed to do, guess what is slow, and then guess at how to improve things. Posting query plans, the results of profiling, cutting out any code that is unnecessary to the performance problem, etc. will get you much better answers.
Justin

Is there a way to convert Crystal Reports to Microsoft SQL Server 2005 reporting services?

Post Author: Lourdes Hernandez
CA Forum: General
Our department has always worked with and we love Crystal Reports. Unfortunately, certain reports now need to be converted to Microsoft SQL Server 2005 reporting services which from what I read I think they are *.rdl reports. Does Crystal reports have any way around this? I would have to learn this other tool when I'm already so happy with Crystal. I was thinking If I could just code the report in crystal and export it in a specific format that would be compatible Microsoft, the problem would be solved.
Any ideas would be greatly appreciated.

Post Author: bettername
CA Forum: General
Ditto!
There are some software packages to convert from CR to RS, but none work 100% AFAIK, and will require manual tuning.
Seems like a big hassle and learning curve just to get reports published via another method - I know I'd put up huge resistance if they tried to take my CR away from me!

(new?) performance problem using jDriver after a Sql Server 6.5 to 2000 conversion

Hi,
This is similar - yet different - to a few of the old postings about performance
problems with using jdbc drivers against Sql Server 7 & 2000.
Here's the situation:
I am running a standalone java application on a Solaris box using BEA's jdbc driver
to connect to a Sql Server database on another network. The application retrieves
data from the database through joins on several tables for approximately 40,000
unique ids. It then processes all of this data and produces a file. We tuned
the app so that the execution time for a single run through the application was
24 minutes running against Sql Server 6.5 with BEA's jdbc driver. After performing
a DBMS conversion to upgrade it to Sql Server 2000 I switched the jDriver to the
Sql Server 2000 version. I ran the app and got an alarming execution time of
5hrs 32 min. After some research, I found the problem with unicode and nvarchar/varchar
and set the "useVarChars" property to "true" on the driver. The execution time
for a single run through the application is now 56 minutes.
56 minutes compared to 5 1/2 hrs is an amazing improvement. However, it is still
over twice the execution time that I was seeing against the 6.5 database. Theoretically,
I should be able to switch out my jdbc driver and the DBMS conversion should be
invisible to my application. That would also mean that I should be seeing the
same execution times with both versions of the DBMS. Has anybody else seen a
simlar situation? Are there any other settings or fixes that I can put into place
to get my performance back down to what I was seeing with 6.5? I would rather
not have to go through and perform another round of performance tuning after having
already done this when the app was originally built.
thanks,
mike

Mike wrote:
Joe,
This was actually my next step. I replaced the BEA driver with
the MS driver and let it run through with out making any
configuration changes, just to see what happened. I got an
execution time of about 7 1/2 hrs (which was shocking). So,
(comparing apples to apples) while leaving the default unicode
property on, BEA ran faster than MS, 5 1/2 hrs to 7 1/2 hrs.
I then set the 'SendStringParametersAsUnicode' to 'false' on the
MS driver and ran another test. This time the application
executed in just over 24 minutes. The actual runtime was 24 min
16 sec, which is still ever so slightly above the actual runtime
against SS 6.5 which was 23 min 35 sec, but is twice as fast as the
56 minutes that BEA's driver was giving me.
I think that this is very interesting. I checked to make sure that
there were no outside factors that may have been influencing the
runtimes in either case, and there were none. Just to make sure,
I ran each driver again and got the same results. It sounds like
there are no known issues regarding this?
We have people looking into things on the DBMS side and I'm still
looking into things on my end, but so far none of us have found
anything. We'd like to continue using BEA's driver for the
support and the fact that we use Weblogic Server for all of our
online applications, but this new data might mean that I have to
switch drivers for this particular application.Thanks. No, there is no known issue, and if you put a packet sniffer
between the client and DBMS, you will probably not see any appreciable
difference in the content of the SQL sent be either driver. My suspicion is
that it involves the historical backward compatibility built in to the DBMS.
It must still handle several iterations of older applications, speaking obsolete
versions of the DBMS protocol, and expecting different DBMS behavior!
Our driver presents itself as a SQL7-level application, and may well be treated
differently than a newer one. This may include different query processing.
Because our driver is deprecated, it is unlikely that it will be changed in
future. We will certainly support you using the MS driver, and if you look
in the MS JDBC newsgroup, you'll see more answers from BEA folks than
from MS people!
Joe
>
>
Mike
The next test you should do, to isolate the issue, is to try another
JDBC driver.
MS provides a type-4 driver now, for free. If it is significantly faster,
it would be
interesting. However, it would still not isolate the problem, because
we still would
need to know what query plan is created by the DBMS, and why.
Joe Weinstein at BEA
PS: I can only tell you that our driver has not changed in it's semantic
function.
It essentially send SQL to the DBMS. It doesn't alter it.

SQL Server Express Performance Limitations With OGC Methods on Geometry Instances

I will front load my question. Specifically, I am wondering if any of the feature restrictions with SQL Server Express cause performance limitations/reductions with OGC methods on geometry instances, e.g., STIntersects? I have spent time reading
various documents about the different editions of SQL Server, including the Features Supported by the Editions of SQL Server 2014, but nothing is jumping out at me. The
limited information on spatial features in the aforementioned document implies spatial is the same across all editions. I am hoping this is wrong.
The situation.... I have roughly 200,000 tax parcels within 175 taxing districts. As part of a consistency check between what is stored in tax records for taxing district and what is identified spatially, I set up a basic point-in-polygon query
to identify the taxing district spatially and then count the number of parcels within in taxing district. Surprisingly, the query took 66 minutes to run. As I pointed out, this is being run on a test machine with SQL Server Express.
Some specifics.... I wrote the query a few different ways and compared the execution plans, and the optimizer always choose the same plan, which is good I guess since it means it is doing its job. The execution plans show a 'Clustered Index Seek
(Spatial)' being used and only costing 1%. Coming in at 75% cost is a Filter, which appears to be connected to the STIntersects predicate. I brute forced alternate execution plans using HINTS, but they only turned out worse, which I guess is also
good since it means the optimizer did choose a good plan. I experimented some with changing the spatial index parameters, but the impact of the options I tried was never that much. I ended up going with "Geometry Auto Grid" with 16 cells
per object.
So, why do I think 66 minutes is excessive? The reason is that I loaded the same data sets into PostgreSQL/PostGIS, used a default spatial index, and the same query ran in 5 minutes. Same machine, same data, SQL Server Express is 13x slower than
PostgreSQL. That is why I think 66 minutes is excessive.
Our organization is mostly an Oracle and SQL Server shop. Since more of my background and experience are with MS databases, I prefer to work with SQL Server. I really do want to understand what is happening here. Is there something I can
do different to get more performance out of SQL Server? Does spatial run slower on Express versus Standard or Enterprise? Given I did so little tuning in PostgreSQL, I still can't understand the results I am seeing.
I may or may not be able to strip the data down enough to be able to send it to someone.

Tessalating the polygons (tax districts) is the answer!
Since my use of SQL Server Express was brought up as possibly contributing to the slow runtime, the first thing I did was download an evaluation version of Enterprise Edition. The runtime on Enterprise Edition dropped from 66 minutes to 57.5 minutes.
A reduction of 13% isn't anything to scoff at, but total runtime was still 11x longer than in PostgreSQL. Although Enterprise Edition had 4 cores available to it, it never really spun up more than 1 when executing the query, so it doesn't seem
to have been parallelizing the query much, if at all.
You asked about polygon complexity. Overall, a majority are fairly simple but there are some complex ones with one really complex polygon. Using the complexity index discussed in the reference thread, the tax districts had an average complexity
of 4.6 and a median of 2.7. One polygon had a complexity index of 120, which was skewing the average, as well as increasing the runtime I suspect. Below is a complexity index breakdown:
Index
NUM_TAX_DIST
1
6
<2
49
<3
44
<4
23
<5
11
<6
9
<7
9
<8
4
<9
1
<10
4
>=10
14
Before trying tessellation, I tweaked the spatial indexes in several different ways, but the runtimes never changed by more than a minute or two. I reset the spatial indexes to "geometry auto grid @ 32" and tried out your tessellation functions
using the default of 5000 vertices. Total runtime 2.3 minutes, a 96% reduction and twice as fast as PostgresSQL! Now that is more what I was expecting before i started.
I tried using different thresholds, 3,000 and 10,000 vertices but the runtimes were slightly slower, 3.5 and 3.3 minutes respectively. A threshold of 5000 definitely seems to be a sweet spot for the dataset I am using. As the thread you referenced
discussed, SQL Server spatial functions like STIntersect appear to be sensitive to the number of vertices of polygons.
After reading your comment, it reminded me of some discussions with Esri staff about ArcGIS doing the same thing in certain circumstances, but I didn't go as far as thinking to apply it here. So, thanks for the suggestion and code from another post.
Once I realized the SRID was hard coded to 0 in tvf_QuarterPolygon, I was able to update the code to set it to the same as the input shape, and then everything came together nicely.

Generic connectivity to SQL server: what kind of performance can I expect?

I have set up a generic connection to a SQL server 2k db, with Oracle on another win 2k box. Both databases are in their "out of the box" configuration, so no particular tuning has been done: but the generic connectivity shows awful performance. In fact I have tried various kind of queries, and all of them show abysmal performance. A query with an "insert - select" that transferred 80K lines executed for one hour, with an average transfer rate of 1KB per second!
I don't think this is the kind of performance I can expect from this tool, is it? Is there anyway particular I can do in order to tune the dblink over the generic connectivity?

I don't think this is the kind of performance I can
expect from this tool, is it? Is there anyway
particular I can do in order to tune the dblink over
the generic connectivity?Nevermind: apparently there are network communication problems between the two machines. Trying to execute the same tests on two co-located dbs shows performances of about 700ms in order to transfer 70000 3-column lines from a sql server db to oracle with a "insert into ... select * from" query.

Taking snapshot of oracle tables to sql server using transactional replication is taking a long time

Hi All,
I am trying to replicate around 200 oracle tables onto sql server using transaction replication and it taking a long time i.e the initial snapshot is taking more than 24 hrs and it still going on.
Is there any way to replicate those these tables faster?
Kindly help me out..
Thanks

Hi,
According to the description, I know the replication is working fine. But it is very slow.
1. Check the CPU usage on Oracle publisher and SQL Server. This issue may due to slow client processing (Oracle performance) or Network performance issues.
2. Based on SQL Server 2008 Books Online ‘Performance Tuning for Oracle Publishers’ (http://msdn.microsoft.com/en-us/library/ms151179(SQL.100).aspx). You can enable the transaction
job set and follow the instructions based on
http://msdn.microsoft.com/en-us/library/ms147884(v=sql.100).aspx.
2. You can enable replication agent logging to check the replication behavior. You may follow these steps to collect them:
To enable Distribution Agent verbose logging. Please follow these steps:
a. Open SQL Server Agent on the distribution server.
b. Under Jobs folder, find out the Distribution Agent.
c. Right click the job and choose Properties.
d. Select Steps tap, it should be like this:
e. Click Run agent and click Edit button, add following scripts by the end of scripts in the command box:
-Output C:\Temp\OUTPUTFILE.txt -Outputverboselevel 2
f. Exit the dialogs
For more information about the steps, please refer to:
http://support.microsoft.com/kb/312292
Hope the information helps.
Tracy Cai
TechNet Community Support

Error [hyt00] [microsoft][odbc sql server driver] query timeout expired

I have the below network setup:-
1. Its a simple network at my father's office at a small town called Ichalkaranji (District - Kolhapur, Maharashtra).
2. We are using private network range 192.168.1.xxx with two Windows Server 2003 Enterprise Edition with SP2 licensed copies and 15 local Windows 7 clients who are only using Server A.
3. The network is having a TP-Link Braodband Router Connected to internet with the IP 192.168.1.1.
4. Both there Windows Server 2003 Enterprise Edition with SP2 are running separate SQL Server 2005 Express with Advanced Services, you can treat them as Server A (Problematic Server with IP of 192.168.1.2)
and Server B (this is not having any issue with IP of 192.168.1.3).
5. Server A is also being used by 6 Remote users from our Kolkata office using DDNS facility through the NO IP client software which installed separately on both the servers. Kolkata remote users
do not use OR access the Server B.
6. Server B is being used by only 2 Remote users from our Erode office (Under Salem District, Tamilnadu) using DDNS facility through the NO IP client software which installed separately on both
the servers. Erode remote users do not use OR access the Server A.
7. The front end application which running separately on both the servers have been developed in VB by a local vendor at Ichalkaranji (District - Kolhapur, Maharashtra).
8. Both Servers are having the same database structure in terms of design and tables format. Only difference is that both the servers are being used separately.
9. This error OR problem is only related to Server A, where on the clients we get the message "error [hyt00] [microsoft][odbc sql server driver] query timeout expired" every now and then.
10. We have to frequently reboot the server whenever we get this message on the client machines. May be after rebooting every thing works perfectly for 2 hours / 5 Hours / may be one full day but
the the error will come back for sure.
11. Current Database back up size on Server A is around 35 GB and take around 1 hour 15 minutes daily to take the back up.
12. Current Database back up size on Server B is around 3 GB and take around 5 to 10 minutes daily to take the back up.
13. One thing I have noticed is that when ever we reboot Server A, for some time sqlsrvr.exe file will show memory usage of 200 to 300 MBs but it will start using slowly, what i understand is that
this is the way SQL Server works.
14. Both the Servers are also running Quick heal Antivirus Server Edition separate licensed copies also.
15. Server B is also running Tally ERP 9 Licenses copy which is being used locally by 15 users of Ichalkaranji (District - Kolhapur, Maharashtra) same users
Can any one help to resolve this issue. Any help will be highly appreciated.

The error message "query timeout expired" occurs, because by default many APIs, including ODBC only waits for 30 seconds for SQL Server to send any output. If no data has been seen for this period of time, they tell SQL Server to cancel execution
and return this error to the caller.
This timeout could be seen as a token that the query is taking too long time to execute and this needs to be fixed. But it can also be a pain in the rear parts, if you are content with a report taking five minutes, because you only run it once a day.
The simplest way to get rid of the error is to set the timeout to zero, which means "wait forever". This is something your vendor would have to do. This may, however, not resolve the problem, as the users may just find that the application is hanging.
To wit, there are two reasons why the query takes more than 30 seconds to complete. One is that there is simply that much work to do. This can be reduced by adding indexes or by doing other tuning, if the execution time is not acceptable. The other possibility
is blocking. That is, there is a process blocking this query from completing. This is much more likely to require attention.
It is not clear to me, whether the vendor has developed the database part as well. In this case, you should probably call the vendor's support desk to have them to sort this out.
Finally, I am little puzzled. You say that you are using Express Edition, but one of the databases is 35 GB in size. 35 GB is far above the limit for Express Edition.
Erland Sommarskog, SQL Server MVP, [email protected]

MS SQL Server + ATG tuning

Similar Messages

Maybe you are looking for