How to improve performance for Azure Table Storage bulk loads

Hello all,
Would appreciate your help as we are facing a challenge.
We are tried to bulk load Azure table storage. We have a file that contains nearly 2 million rows.
We would need to reach a point where we could bulk load 100000-150000 entries per minute. Currently, it takes more than 10 hours to process the file..
We have tried Parallel.Foreach but it doesn't help. Today I discovered Partitioning in PLINQ. Would that be the way to go??
Any ideas? I have spent nearly two days in trying to optimize it using PLINQ, but still I am not sure what is the best thing to do.
Kindly, note that we shouldn't be using SQL/Azure SQL for this.
I would really appreciate your help.
Thanks

I'd think you're just pooling the parallel connections to Azure, if you do it on one system. You'd also have a bottleneck of round trip time from you, through the internet to Azure and back again.
You could speed it up by moving the data file to the cloud and process it with a Cloud worker role. That way you'd be in the datacenter (which is a much faster, more optimized network.)
Or, if that's not fast enough - if you can split the data so multiple WorkerRoles could each process part of the file, you can use the VM's scale to put enough machines to it that it gets done quickly.
Darin R.

Similar Messages

How to select data from AZure table storage without row key and partition key

Hi
I need to select a data from azure table storage without rowkey and partition key. how in azure storage emulator click query it display all data from that table.
thanks
rajesh

Hi rajesh,
It seems that you didn't click query data using storage emulator. But I recommend you could use the azure server explore in your VS to view your data and query data. Please see this document (http://msdn.microsoft.com/en-us/library/azure/ff683677.aspx).
And base on my experience, you may need input the command on Azure storage emulator, such as this page(http://msdn.microsoft.com/en-us/library/azure/gg433005.aspx).
Regards,
Will
We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
Click
HERE to participate the survey.

How to improve performance for Custom Extractor in BI..

HI all,
I am new to BI and started working on BI for couple of weeks.. I created a Custom Extractor(Data View) in the Source system and when i pull data takes lot of time.. Can any one respond to this, suggesting how to improve the performance of my custom Extractor.. Please do the needfull..
Thanks and Regards,
Venugopal..

Dear Venugopal,
use transaction ST05 to check if your SQL statements are optimal and that you do not have redundant database calls. You should use as much as possible "bulking", which means to fetch the required data with one request to database and not with multiple requests to database.
Use transaction SE30 to check if you are wasting time in loops and if yes, optimize the algorithm.
Best Regards,
Sylvia

How to improve performance for bulk data load in Dynamics CRM 2013 Online

Hi all,
We need to bulk update (or create) contacts into Dynamics CRM 2013 online every night due to data updated from another external data source. The data size is around 100,000 and the data loading duration was around 6 hours.
We are already using ExecuteMultiple web services to handle the integration, however, the 6 hours integraton duration is still not acceptable and we are seeking for any advise for further improvement.
Any help is highly appreciated. Many thanks.
Gary

I think Andrii's referring to running multiple threads in parallel (see
http://www.mscrmuk.blogspot.co.uk/2012/02/data-migration-performance-to-crm.html - it's a bit dated, but should still be relevant).
Microsoft do have some throttling limits applied in Crm Online, and it is worth contacting them to see if you can get those raised.
100 000 records per night seems a large number. Are all these records new or updated records, or are there some that are unchanged, in which case you could filter them out before uploading ? Or are there useful ways to summarise the data before loading
Microsoft CRM MVP - http://mscrmuk.blogspot.com/ http://www.excitation.co.uk

How to improve Performance for Select statement

Hi Friends,
Can you please help me in improving the performance of the following query:
SELECT SINGLE MAX( policyterm ) startterm INTO (lv_term, lv_cal_date) FROM zu1cd_policyterm WHERE gpart = gv_part GROUP BY startterm.
Thanks and Regards,
Johny

long lists can not be produced with a SELECT SINGLE, there is also nothing to group.
But I guess the SINGLE is a bug
SELECT MAX( policyterm ) startterm
              INTO (lv_term, lv_cal_date)
              FROM zu1cd_policyterm
              WHERE gpart = gv_part
              GROUP BY startterm.
How many records are in zu1cd_policyterm ?
Is there an index starting with gpart?
If first answer is 'large' and second 'no'   =>   slow
What is the meaning of gpart? How many different values can it assume?
If many different values then an index makes sense, if you are allowed to create
an index.
Otherwise you must be patient.
Siegfried

How to improve performance for this code

Hi,
LOOP AT lt_element INTO ls_element.
READ TABLE lt_element_ident INTO ls_element_ident
WITH KEY element_id = ls_element-element_id BINARY SEARCH.
IF sy-subrc EQ 0.
MOVE ls_element_ident-value TO lv_guid.
SELECT * FROM zcm_valuation_at
APPENDING CORRESPONDING FIELDS OF TABLE lt_caseattributes
WHERE case_guid = lv_guid.
ENDIF.
ENDLOOP.
LOOP AT lt_caseattributes INTO ls_caseattributes.
IF ls_caseattributes-ext_key IS INITIAL.
SELECT SINGLE ext_key
INTO CORRESPONDING FIELDS OF ls_caseattributes
FROM scmg_t_case_attr
WHERE case_guid = ls_caseattributes-case_guid.
ENDIF.
*To get the Status description of the Case
SELECT SINGLE stat_ordno_descr
INTO ls_caseattributes-status
FROM scmgstatprofst AS a
INNER JOIN scmg_t_case_attr AS b
ON aprofile_id = bprofile_id
AND astat_orderno = bstat_orderno
WHERE case_guid = ls_caseattributes-case_guid.
MODIFY lt_caseattributes FROM ls_caseattributes INDEX sy-tabix TRANSPORTING status ext_key.
ENDLOOP.
READ TABLE lt_caseattributes INTO ls_caseattributes INDEX 1.
Regards,
Maruti

Hi,
try this kind of code:
==================================
start new
DATA:
lt_scmgstatprofst LIKE scmgstatprofst OCCURS 0 WITH HEADER LINE,
wa_scmg_t_case_attr LIKE scmg_t_case_attr.
SELECT * FROM scmgstatprofst INTO TABLE lt_scmgstatprofst.
SORT lt_scmgstatprofst BY profile_id stat_orderno.
end new
LOOP AT lt_element INTO ls_element.
READ TABLE lt_element_ident INTO ls_element_ident
WITH KEY element_id = ls_element-element_id BINARY SEARCH.
IF sy-subrc EQ 0.
    MOVE ls_element_ident-value TO lv_guid.
    SELECT * FROM zcm_valuation_at
    APPENDING CORRESPONDING FIELDS OF TABLE lt_caseattributes
    WHERE case_guid = lv_guid.
ENDIF.
ENDLOOP.
LOOP AT lt_caseattributes INTO ls_caseattributes.
IF ls_caseattributes-ext_key IS INITIAL.
    SELECT SINGLE ext_key
    INTO CORRESPONDING FIELDS OF ls_caseattributes
    FROM scmg_t_case_attr
    WHERE case_guid = ls_caseattributes-case_guid.
ENDIF.
*To get the Status description of the Case
start deletion
SELECT SINGLE stat_ordno_descr
INTO ls_caseattributes-status
FROM scmgstatprofst AS a
INNER JOIN scmg_t_case_attr AS b
ON aprofile_id = bprofile_id
AND astat_orderno = bstat_orderno
WHERE case_guid = ls_caseattributes-case_guid.
end deletion
start new
CLEAR wa_scmg_t_case_attr.
SELECT SINGLE * FROM scmg_t_case_attr INTO wa_scmg_t_case_attr
    WHERE case_guid = ls_caseattributes-case_guid.
READ TABLE lt_scmgstatprofst WITH KEY
    profile_id   = wa_scmg_t_case_attr-profile_id
    stat_orderno = wa_scmg_t_case_attr-stat_orderno
    BINARY SEARCH.
IF sy-subrc IS INITIAL.
    ls_caseattributes-status = lt_scmgstatprofst-stat_ordno_descr.
ENDIF.
end new
MODIFY lt_caseattributes FROM ls_caseattributes INDEX sy-tabix
TRANSPORTING status ext_key.
ENDLOOP.
READ TABLE lt_caseattributes INTO ls_caseattributes INDEX 1.
==================================
Regards
Walter Habich
Edited by: Walter Habich on Jun 17, 2008 8:41 AM

Azure table storage design for simple social networking

What is the best table design if I want to use Azure Table Service for a simple social networking website?
The website could have millions of users.
Users need to be able to view a list of all other users in the system sorted by the number of mutual connections.
Users must be able to view a list of their connections
User must be able to view content posted by themselves and their connections.
One major design constraint is that Azure table service queries are generally limited to the partition key and row key when there are a large number of records or else they get really slow. Another constraint is that query results are only sorted by the
partition key and then the row key.

For your scenario, I think making use of the SQL Azure makes more senses than usage of azure table storage service offering, nature of the data looks more relational in this particular context which is not recommended for table storage model design.
You can get started with SQL Azure at -
http://azure.microsoft.com/en-us/services/sql-database/
Bhushan | Blog |
LinkedIn | Twitter

How to improve performace for this program

Hi,
I want to improve performance for this program.Pls help me.Here is the code.
Thanks
kumar
--# SCCS: @(#)nscdprpt_rpt_fam_fweq_detail_aggr_dml.sql     2.1 08/17/05 12:16:11
--# Location: /appl/dpdm/src/sql/SCCS/s.nscdprpt_rpt_fam_fweq_detail_aggr_dml.sql
-- =============================================================================
-- Purpose:
-- DML Script to calculate aggregate rows from table NSCDPRPT.RPT_FAM_FWEQ_DETAIL
-- inserts one (1) monthly total row for each comparison type (6 of them)
-- Modification History
-- Date Version Who What
-- 28-Jun-05 V1.00 J. Myers Initial Version
-- 17-Aug-05 gzuerc Replaced UNION ALL code with INSERT's to avoid
-- Oracle error out of undo tablespace.
-- ============================================================================
-- NOTES:
-- This process pulls data from RPT_FAM_FWEQ_DETAIL table.
-- The target table RPT_FAM_FWEQ_DETAIL_AGGR is truncated by plan_ctry and loaded daily;
-- it is the data used in the Forecast Accuracy Measurements Report.
-- The input data is 'wide' (6 quantities), and this script outputs a single
-- row for each quantity as a 'thin' table (single column F1_QTY and F2_QTY shared by all quantities), and
-- with a ROW_TYPE = 'TOTAL'
-- Two (2) other scripts add additional rows to this table as follows:
-- 1) nscdprpt_rpt_fam_FWEQ_detail_aggr_sesn_avg_dml.sql => calculates the
-- total quantities F1_QTY, F2_QTY and AE for each LEVEL_VALUE
-- 2) nscdprpt_rpt_fam_FWEQ_detail_aggr_pct_swg_dml.sql => calculates the
-- percent swing in quantities between DISP_EVENTS for each grouping
-- of PLAN_CTRY, DIVISION, SEASON and TRG_EVENT_NUM for only LEVEL_TYPE = 'SUB_CAT'
-- =============================================================================
-- The result set from joining RPT_FAM_FWEQ_DETAIL rows with comparison types
-- is inserted into the table RPT_FAM_FWEQ_DETAIL_AGGR
INSERT INTO rpt_fam_FWEQ_detail_aggr
SELECT plan_ctry,
division,
season,
monthly_seq,
     bucket_month_date,
tier_num,
level_num,
tier_volume,
trg_event_num,
disp_event_num,
max_event_num,
compare_name,
first_display_event_num,
level_type,
level_value,
'TOTAL' AS row_type,
material,
key_material,
sap_cat_cd,
sap_cat_desc,
sap_sub_cat_cd,
sap_sub_cat_desc,
sty_colr_sdesc,
sty_grp_nbr,
sty_grp_desc,
matl_typ,
sap_prod_typ_grp, -- 'Type Group' in Brio
typ_grp_desc, -- 'Type Group' in Brio
curr_prod_i2_life_cyc_cd,
NULL AS promo_ind, -- for future use
-- Each RPT_FAM_FWEQ_DETAIL row's eight (8) quantity columns are broken down
-- into F1_QTY from DISP_aaaa_QTY, F2_QTY from TRG_aaaa_QTY and
     -- AE from aaaa_AE where 'aaaa' is equal to one of the COMPARE_TYPEs below:
-- F1_QTY
CASE WHEN compare_type = 'DELIVERY_PLAN' THEN
disp_dlvry_plan_qty
WHEN compare_type = 'NET' THEN
disp_net_qty
WHEN compare_type = 'NET_AO' THEN
disp_ao_qty
WHEN compare_type = 'NET_FTRS' THEN
disp_futr_qty
WHEN compare_type = 'NET_REPLENS' THEN
disp_replen_qty
WHEN compare_type = 'AUTH_FTRS' THEN
disp_auth_futr_qty
END as f1_qty,
-- F2_QTY
CASE WHEN compare_type = 'DELIVERY_PLAN' THEN
trg_dlvry_plan_qty
WHEN compare_type = 'NET' THEN
trg_net_qty
WHEN compare_type = 'NET_AO' THEN
trg_ao_qty
WHEN compare_type = 'NET_FTRS' THEN
trg_futr_qty
WHEN compare_type = 'NET_REPLENS' THEN
trg_replen_qty
WHEN compare_type = 'AUTH_FTRS' THEN
trg_auth_futures_qty
END as f2_qty,
-- AE
CASE WHEN compare_type = 'DELIVERY_PLAN' THEN
abs(disp_dlvry_plan_qty - trg_dlvry_plan_qty)
WHEN compare_type = 'NET' THEN
abs(disp_net_qty - trg_net_qty)
WHEN compare_type = 'NET_AO' THEN
abs(disp_ao_qty - trg_ao_qty)
WHEN compare_type = 'NET_FTRS' THEN
abs(disp_futr_qty - trg_futr_qty)
WHEN compare_type = 'NET_REPLENS' THEN
abs(disp_replen_qty - trg_replen_qty)
WHEN compare_type = 'AUTH_FTRS' THEN
abs(disp_auth_futr_qty - trg_auth_futures_qty)
END as ae,
SYSDATE AS zz_insert_tmst
FROM
-- The following in-line view provides three (3) result sets from the RPT_FAM_FWEQ_DETAIL table
     -- This in-line view returns only LEVEL_NUM = 1 or LEVEL_TYPE = 'SUB_CAT' data
     SELECT plan_ctry,
     division,
     season,
     monthly_seq,
          bucket_month_date,
     tier_num,
     level_num,
     tier_volume,
     trg_event_num,
     trg_event_type,
     disp_event_num,
     max_event_num,
     first_display_event_num,
     level_type,
     level_value,
     -- The following NULL'd columns' values cannot be saved due to aggregation
     NULL AS material,
     NULL AS key_material,
     sap_cat_cd,
     sap_cat_desc,
     sap_sub_cat_cd,
     sap_sub_cat_desc,
     NULL AS sty_colr_sdesc,
     NULL AS sty_grp_nbr,
     NULL AS sty_grp_desc,
     NULL AS matl_typ,
     sap_prod_typ_grp, -- 'Type Group' in Brio
     typ_grp_desc, -- 'Type Group' in Brio
     NULL AS curr_prod_i2_life_cyc_cd,
     sum(disp_net_qty) AS disp_net_qty,
          sum(trg_net_qty) AS trg_net_qty,
     -- ABS(sum(trg_net_qty) - sum(disp_net_qty)) AS net_AE,
     sum(disp_dlvry_plan_qty) AS disp_dlvry_plan_qty,
          sum(trg_dlvry_plan_qty) AS trg_dlvry_plan_qty,
     -- ABS(sum(trg_dlvry_plan_qty) - sum(disp_dlvry_plan_qty)) AS dlvry_plan_AE,
     sum(disp_futr_qty) AS disp_futr_qty,
          sum(trg_futr_qty) AS trg_futr_qty,
     -- ABS(sum(trg_futr_qty) - sum(disp_futr_qty)) AS futr_AE,
     sum(disp_ao_qty) AS disp_ao_qty,
          sum(trg_ao_qty) AS trg_ao_qty,
     -- ABS(sum(trg_ao_qty) - sum(disp_ao_qty)) AS ao_AE,
     sum(disp_replen_qty) AS disp_replen_qty,
          sum(trg_replen_qty) AS trg_replen_qty,
     -- ABS(sum(trg_replen_qty) - sum(disp_replen_qty)) AS replen_AE,
     sum(disp_futr_qty) AS disp_auth_futr_qty,
          sum(trg_auth_futures_qty) AS trg_auth_futures_qty --,
     -- ABS(sum(trg_auth_futures_qty) - sum(disp_futr_qty)) AS auth_futures_AE
     FROM rpt_fam_FWEQ_detail
     WHERE plan_ctry &where_plan_ctry
     and level_num = 1 -- 'SUB-CAT'
     -- AND (promo_ind <> 'Y' OR promo_ind IS null)
     GROUP BY plan_ctry,
     division,
     season,
     monthly_seq,
          bucket_month_date,
     tier_num,
     level_num,
     tier_volume,
     trg_event_num,
     trg_event_type,
     disp_event_num,
     max_event_num,
     first_display_event_num,
     level_type,
     level_value,
     sap_cat_cd,
     sap_cat_desc,
     sap_sub_cat_cd,
     sap_sub_cat_desc,
     sap_prod_typ_grp, -- 'Type Group' in Brio
     typ_grp_desc -- 'Type Group' in Brio
     ) dtl,
     -- The following in-line view returns all of the different combinations
     -- of comparison types in the RPT_FAM_FWEQ_DETAIL table
     -- This select returns the pairing of all forecast types
     SELECT event_type, compare_type, compare_name
     from
     (SELECT event_type, compare_type, compare_type_description || ' to ' || compare_type_description AS compare_name
     FROM rpt_fam_compare_types
     WHERE event_type = 'FCST'
     UNION
     -- This select returns the pairing of all bookings types with forecast types
     SELECT bkng.event_type, fcst.compare_type, bkng.compare_type_description || ' to ' || fcst.compare_type_description AS compare_name
     FROM rpt_fam_compare_types fcst,
     rpt_fam_compare_types bkng
     WHERE fcst.event_type = 'FCST'
     AND bkng.event_type = 'BKNG'
     AND fcst.compare_type = bkng.compare_type
     WHERE event_type || ' ' || compare_type <> 'FCST AUTH_FTRS'
     AND compare_type NOT IN ('NET_SHIP', 'SHIP_NET_FTRS', 'NET_ETS', 'GROSS_FTRS')
     ) cmpr
-- The two (2) in-line views are joined by EVENT_TYPE (i.e. 'FCST' and 'BKNG')
-- to form a product of all RPT_FAM_FWEQ_DETAIL rows with comparison types
WHERE dtl.trg_event_type = cmpr.event_type
ORDER BY plan_ctry,
division,
season,
monthly_seq,
tier_num,
level_num,
tier_volume,
trg_event_num,
disp_event_num,
max_event_num,
first_display_event_num,
level_type,
level_value
COMMIT
INSERT INTO rpt_fam_FWEQ_detail_aggr
SELECT plan_ctry,
division,
season,
monthly_seq,
     bucket_month_date,
tier_num,
level_num,
tier_volume,
trg_event_num,
disp_event_num,
max_event_num,
compare_name,
first_display_event_num,
level_type,
level_value,
'TOTAL' AS row_type,
material,
key_material,
sap_cat_cd,
sap_cat_desc,
sap_sub_cat_cd,
sap_sub_cat_desc,
sty_colr_sdesc,
sty_grp_nbr,
sty_grp_desc,
matl_typ,
sap_prod_typ_grp, -- 'Type Group' in Brio
typ_grp_desc, -- 'Type Group' in Brio
curr_prod_i2_life_cyc_cd,
NULL AS promo_ind, -- for future use
-- Each RPT_FAM_FWEQ_DETAIL row's eight (8) quantity columns are broken down
-- into F1_QTY from DISP_aaaa_QTY, F2_QTY from TRG_aaaa_QTY and
     -- AE from aaaa_AE where 'aaaa' is equal to one of the COMPARE_TYPEs below:
-- F1_QTY
CASE WHEN compare_type = 'DELIVERY_PLAN' THEN
disp_dlvry_plan_qty
WHEN compare_type = 'NET' THEN
disp_net_qty
WHEN compare_type = 'NET_AO' THEN
disp_ao_qty
WHEN compare_type = 'NET_FTRS' THEN
disp_futr_qty
WHEN compare_type = 'NET_REPLENS' THEN
disp_replen_qty
WHEN compare_type = 'AUTH_FTRS' THEN
disp_auth_futr_qty
END as f1_qty,
-- F2_QTY
CASE WHEN compare_type = 'DELIVERY_PLAN' THEN
trg_dlvry_plan_qty
WHEN compare_type = 'NET' THEN
trg_net_qty
WHEN compare_type = 'NET_AO' THEN
trg_ao_qty
WHEN compare_type = 'NET_FTRS' THEN
trg_futr_qty
WHEN compare_type = 'NET_REPLENS' THEN
trg_replen_qty
WHEN compare_type = 'AUTH_FTRS' THEN
trg_auth_futures_qty
END as f2_qty,
-- AE
CASE WHEN compare_type = 'DELIVERY_PLAN' THEN
abs(disp_dlvry_plan_qty - trg_dlvry_plan_qty)
WHEN compare_type = 'NET' THEN
abs(disp_net_qty - trg_net_qty)
WHEN compare_type = 'NET_AO' THEN
abs(disp_ao_qty - trg_ao_qty)
WHEN compare_type = 'NET_FTRS' THEN
abs(disp_futr_qty - trg_futr_qty)
WHEN compare_type = 'NET_REPLENS' THEN
abs(disp_replen_qty - trg_replen_qty)
WHEN compare_type = 'AUTH_FTRS' THEN
abs(disp_auth_futr_qty - trg_auth_futures_qty)
END as ae,
SYSDATE AS zz_insert_tmst
FROM
     -- This in-line view returns only LEVEL_NUM = 2 or LEVEL_TYPE = 'STYLE_GROUP' data
     (SELECT plan_ctry,
     division,
     season,
     monthly_seq,
          bucket_month_date,
     tier_num,
     level_num,
     tier_volume,
     trg_event_num,
     trg_event_type,
     disp_event_num,
     max_event_num,
     first_display_event_num,
     level_type,
     level_value,
     -- The following NULL'd columns' values cannot be saved due to aggregation
     NULL AS material,
     NULL AS key_material,
     sap_cat_cd,
     sap_cat_desc,
     sap_sub_cat_cd,
     sap_sub_cat_desc,
     NULL AS sty_colr_sdesc,
     sty_grp_nbr,
     sty_grp_desc,
     NULL AS matl_typ,
     sap_prod_typ_grp, -- 'Type Group' in Brio
     typ_grp_desc, -- 'Type Group' in Brio
     NULL AS curr_prod_i2_life_cyc_cd,
     sum(disp_net_qty) AS disp_net_qty,
          sum(trg_net_qty) AS trg_net_qty,
     -- ABS(sum(trg_net_qty) - sum(disp_net_qty)) AS net_AE,
     sum(disp_dlvry_plan_qty) AS disp_dlvry_plan_qty,
          sum(trg_dlvry_plan_qty) AS trg_dlvry_plan_qty,
     -- ABS(sum(trg_dlvry_plan_qty) - sum(disp_dlvry_plan_qty)) AS dlvry_plan_AE,
     sum(disp_futr_qty) AS disp_futr_qty,
          sum(trg_futr_qty) AS trg_futr_qty,
     -- ABS(sum(trg_futr_qty) - sum(disp_futr_qty)) AS futr_AE,
     sum(disp_ao_qty) AS disp_ao_qty,
          sum(trg_ao_qty) AS trg_ao_qty,
     -- ABS(sum(trg_ao_qty) - sum(disp_ao_qty)) AS ao_AE,
     sum(disp_replen_qty) AS disp_replen_qty,
          sum(trg_replen_qty) AS trg_replen_qty,
     -- ABS(sum(trg_replen_qty) - sum(disp_replen_qty)) AS replen_AE,
     sum(disp_futr_qty) AS disp_auth_futr_qty,
          sum(trg_auth_futures_qty) AS trg_auth_futures_qty --,
     -- ABS(sum(trg_auth_futures_qty) - sum(disp_futr_qty)) AS auth_futures_AE
     FROM rpt_fam_FWEQ_detail
     WHERE plan_ctry &where_plan_ctry
     and level_num = 2 -- 'STYLE-GRP' or 'STYLE'
     -- AND (promo_ind <> 'Y' OR promo_ind IS null)
     GROUP BY plan_ctry,
     division,
     season,
     monthly_seq,
          bucket_month_date,
     tier_num,
     level_num,
     tier_volume,
     trg_event_num,
     trg_event_type,
     disp_event_num,
     max_event_num,
     first_display_event_num,
     level_type,
     level_value,
     sap_cat_cd,
     sap_cat_desc,
     sap_sub_cat_cd,
     sap_sub_cat_desc,
     sty_grp_nbr,
     sty_grp_desc,
     sap_prod_typ_grp, -- 'Type Group' in Brio
     typ_grp_desc -- 'Type Group' in Brio
     ) dtl,
     -- The following in-line view returns all of the different combinations
     -- of comparison types in the RPT_FAM_FWEQ_DETAIL table
     -- This select returns the pairing of all forecast types
     SELECT event_type, compare_type, compare_name
     from
     (SELECT event_type, compare_type, compare_type_description || ' to ' || compare_type_description AS compare_name
     FROM rpt_fam_compare_types
     WHERE event_type = 'FCST'
     UNION
     -- This select returns the pairing of all bookings types with forecast types
     SELECT bkng.event_type, fcst.compare_type, bkng.compare_type_description || ' to ' || fcst.compare_type_description AS compare_name
     FROM rpt_fam_compare_types fcst,
     rpt_fam_compare_types bkng
     WHERE fcst.event_type = 'FCST'
     AND bkng.event_type = 'BKNG'
     AND fcst.compare_type = bkng.compare_type
     WHERE event_type || ' ' || compare_type <> 'FCST AUTH_FTRS'
     AND compare_type NOT IN ('NET_SHIP', 'SHIP_NET_FTRS', 'NET_ETS', 'GROSS_FTRS')
     ) cmpr
-- The two (2) in-line views are joined by EVENT_TYPE (i.e. 'FCST' and 'BKNG')
-- to form a product of all RPT_FAM_FWEQ_DETAIL rows with comparison types
WHERE dtl.trg_event_type = cmpr.event_type
ORDER BY plan_ctry,
division,
season,
monthly_seq,
tier_num,
level_num,
tier_volume,
trg_event_num,
disp_event_num,
max_event_num,
first_display_event_num,
level_type,
level_value
COMMIT
INSERT INTO rpt_fam_FWEQ_detail_aggr
SELECT plan_ctry,
division,
season,
monthly_seq,
     bucket_month_date,
tier_num,
level_num,
tier_volume,
trg_event_num,
disp_event_num,
max_event_num,
compare_name,
first_display_event_num,
level_type,
level_value,
'TOTAL' AS row_type,
material,
key_material,
sap_cat_cd,
sap_cat_desc,
sap_sub_cat_cd,
sap_sub_cat_desc,
sty_colr_sdesc,
sty_grp_nbr,
sty_grp_desc,
matl_typ,
sap_prod_typ_grp, -- 'Type Group' in Brio
typ_grp_desc, -- 'Type Group' in Brio
curr_prod_i2_life_cyc_cd,
NULL AS promo_ind, -- for future use
-- Each RPT_FAM_FWEQ_DETAIL row's eight (8) quantity columns are broken down
-- into F1_QTY from DISP_aaaa_QTY, F2_QTY from TRG_aaaa_QTY and
     -- AE from aaaa_AE where 'aaaa' is equal to one of the COMPARE_TYPEs below:
-- F1_QTY
CASE WHEN compare_type = 'DELIVERY_PLAN' THEN
disp_dlvry_plan_qty
WHEN compare_type = 'NET' THEN
disp_net_qty
WHEN compare_type = 'NET_AO' THEN
disp_ao_qty
WHEN compare_type = 'NET_FTRS' THEN
disp_futr_qty
WHEN compare_type = 'NET_REPLENS' THEN
disp_replen_qty
WHEN compare_type = 'AUTH_FTRS' THEN
disp_auth_futr_qty
END as f1_qty,
-- F2_QTY
CASE WHEN compare_type = 'DELIVERY_PLAN' THEN
trg_dlvry_plan_qty
WHEN compare_type = 'NET' THEN
trg_net_qty
WHEN compare_type = 'NET_AO' THEN
trg_ao_qty
WHEN compare_type = 'NET_FTRS' THEN
trg_futr_qty
WHEN compare_type = 'NET_REPLENS' THEN
trg_replen_qty
WHEN compare_type = 'AUTH_FTRS' THEN
trg_auth_futures_qty
END as f2_qty,
-- AE
CASE WHEN compare_type = 'DELIVERY_PLAN' THEN
abs(disp_dlvry_plan_qty - trg_dlvry_plan_qty)
WHEN compare_type = 'NET' THEN
abs(disp_net_qty - trg_net_qty)
WHEN compare_type = 'NET_AO' THEN
abs(disp_ao_qty - trg_ao_qty)
WHEN compare_type = 'NET_FTRS' THEN
abs(disp_futr_qty - trg_futr_qty)
WHEN compare_type = 'NET_REPLENS' THEN
abs(disp_replen_qty - trg_replen_qty)
WHEN compare_type = 'AUTH_FTRS' THEN
abs(disp_auth_futr_qty - trg_auth_futures_qty)
END as ae,
SYSDATE AS zz_insert_tmst
FROM
     -- This in-line view returns only LEVEL_NUM = 3 or LEVEL_TYPE = 'MATL' data
     (SELECT plan_ctry,
     division,
     season,
     monthly_seq,
          bucket_month_date,
     tier_num,
     level_num,
     tier_volume,
     trg_event_num,
     trg_event_type,
     disp_event_num,
     max_event_num,
     first_display_event_num,
     level_type,
     level_value,
     material,
     key_material,
     sap_cat_cd,
     sap_cat_desc,
     sap_sub_cat_cd,
     sap_sub_cat_desc,
     sty_colr_sdesc,
     sty_grp_nbr,
     sty_grp_desc,
     matl_typ,
     sap_prod_typ_grp, -- 'Type Group' in Brio
     typ_grp_desc, -- 'Type Group' in Brio
     curr_prod_i2_life_cyc_cd,
     sum(disp_net_qty) AS disp_net_qty,
          sum(trg_net_qty) AS trg_net_qty,
     -- ABS(sum(trg_net_qty) - sum(disp_net_qty)) AS net_AE,
     sum(disp_dlvry_plan_qty) AS disp_dlvry_plan_qty,
          sum(trg_dlvry_plan_qty) AS trg_dlvry_plan_qty,
     -- ABS(sum(trg_dlvry_plan_qty) - sum(disp_dlvry_plan_qty)) AS dlvry_plan_AE,
     sum(disp_futr_qty) AS disp_futr_qty,
          sum(trg_futr_qty) AS trg_futr_qty,
     -- ABS(sum(trg_futr_qty) - sum(disp_futr_qty)) AS futr_AE,
     sum(disp_ao_qty) AS disp_ao_qty,
          sum(trg_ao_qty) AS trg_ao_qty,
     -- ABS(sum(trg_ao_qty) - sum(disp_ao_qty)) AS ao_AE,
     sum(disp_replen_qty) AS disp_replen_qty,
          sum(trg_replen_qty) AS trg_replen_qty,
     -- ABS(sum(trg_replen_qty) - sum(disp_replen_qty)) AS replen_AE,
     sum(disp_futr_qty) AS disp_auth_futr_qty,
          sum(trg_auth_futures_qty) AS trg_auth_futures_qty --,
     -- ABS(sum(trg_auth_futures_qty) - sum(disp_futr_qty)) AS auth_futures_AE
     FROM rpt_fam_FWEQ_detail
     WHERE plan_ctry &where_plan_ctry
     and level_num = 3 -- 'MATERIAL'
     -- AND promo_ind <> 'Y'
     GROUP BY plan_ctry,
     division,
     season,
     monthly_seq,
          bucket_month_date,
     tier_num,
     level_num,
     tier_volume,
     trg_event_num,
     trg_event_type,
     disp_event_num,
     max_event_num,
     first_display_event_num,
     level_type,
     level_value,
     material,
     key_material,
     sap_cat_cd,
     sap_cat_desc,
     sap_sub_cat_cd,
     sap_sub_cat_desc,
     sty_colr_sdesc,
     sty_grp_nbr,
     sty_grp_desc,
     matl_typ,
     sap_prod_typ_grp, -- 'Type Group' in Brio
     typ_grp_desc, -- 'Type Group' in Brio
     curr_prod_i2_life_cyc_cd
     ) dtl,
     -- The following in-line view returns all of the different combinations
     -- of comparison types in the RPT_FAM_FWEQ_DETAIL table
     -- This select returns the pairing of all forecast types
     SELECT event_type, compare_type, compare_name
     from
     (SELECT event_type, compare_type, compare_type_description || ' to ' || compare_type_description AS compare_name
     FROM rpt_fam_compare_types
     WHERE event_type = 'FCST'
     UNION
     -- This select returns the pairing of all bookings types with forecast types
     SELECT bkng.event_type, fcst.compare_type, bkng.compare_type_description || ' to ' || fcst.compare_type_description AS compare_name
     FROM rpt_fam_compare_types fcst,
     rpt_fam_compare_types bkng
     WHERE fcst.event_type = 'FCST'
     AND bkng.event_type = 'BKNG'
     AND fcst.compare_type = bkng.compare_type
     WHERE event_type || ' ' || compare_type <> 'FCST AUTH_FTRS'
     AND compare_type NOT IN ('NET_SHIP', 'SHIP_NET_FTRS', 'NET_ETS', 'GROSS_FTRS')
     ) cmpr
-- The two (2) in-line views are joined by EVENT_TYPE (i.e. 'FCST' and 'BKNG')
-- to form a product of all RPT_FAM_FWEQ_DETAIL rows with comparison types
WHERE dtl.trg_event_type = cmpr.event_type
ORDER BY plan_ctry,
division,
season,
monthly_seq,
tier_num,
level_num,
tier_volume,
trg_event_num,
disp_event_num,
max_event_num,
first_display_event_num,
level_type,
level_value
COMMIT
/

I agree.
SELECT ticket_no,name_of_the_passenger..
FROM ticket_master
WHERE ticket_no NOT IN
(SELECT Ticket_no
FROM ticket_cancellations
WHERE trip_id = my_trip_id);
This involves creating a little temp table for each record in ticket_no. Then full scanning it.
Change it to
SELECT ticket_no,name_of_the_passenger..
FROM ticket_master B
WHERE ticket_no NOT EXISTS
(SELECT null
FROM ticket_cancellations A
WHERE A.ticket_no = B.ticket_no
AND trip_id = my_trip_id);
Then you get an index hit in both cases.

How to improve performance of MediaPlayer?

I tried to use the MediaPlayer with a On2 VP6 flv movie.
Showing a video with a resolution of 1024x768 works.
Showing a video with a resolution of 1280x720 and a average bitrate of 1700 kb/s leads to a delay of the video signal behind the audio signal of a couple of seconds. VLC, Media Player Classic and a couple of other players have no problem with the video. Only the FX MediaPlayer shows a poor performance.
Additionally mouse events in a second stage (the first stage is used for the video) are not processed in 2 of 3 cases. If the MediaPlayer is switched off, the mouse events work reliable.
Does somebody know a solution for this problems?
Cheers
masim

duplicate thread..
How to improve performance of attached query

How to improve performance of the attached query

Hi,
How to improve performance of the below query, Please help. also attached explain plan -
SELECT Camp.Id,
rCam.AccountKey,
Camp.Id,
CamBilling.Cpm,
CamBilling.Cpc,
CamBilling.FlatRate,
Camp.CampaignKey,
Camp.AccountKey,
CamBilling.billoncontractedamount,
(SUM(rCam.Impressions) * 0.001 + SUM(rCam.Clickthrus)) AS GR,
rCam.AccountKey as AccountKey
FROM Campaign Camp, rCamSit rCam, CamBilling, Site xSite
WHERE Camp.AccountKey = rCam.AccountKey
AND Camp.AvCampaignKey = rCam.AvCampaignKey
AND Camp.AccountKey = CamBilling.AccountKey
AND Camp.CampaignKey = CamBilling.CampaignKey
AND rCam.AccountKey = xSite.AccountKey
AND rCam.AvSiteKey = xSite.AvSiteKey
AND rCam.RmWhen BETWEEN to_date('01-01-2009', 'DD-MM-YYYY') and
to_date('01-01-2011', 'DD-MM-YYYY')
GROUP By rCam.AccountKey,
Camp.Id,
CamBilling.Cpm,
CamBilling.Cpc,
CamBilling.FlatRate,
Camp.CampaignKey,
Camp.AccountKey,
CamBilling.billoncontractedamount
Explain Plan :-
Description Object_owner Object_name Cost Cardinality Bytes
SELECT STATEMENT, GOAL = ALL_ROWS 14 1 13
SORT AGGREGATE 1 13
VIEW GEMINI_REPORTING 14 1 13
HASH GROUP BY 14 1 103
NESTED LOOPS 13 1 103
HASH JOIN 12 1 85
TABLE ACCESS BY INDEX ROWID GEMINI_REPORTING RCAMSIT 2 4 100
NESTED LOOPS 9 5 325
HASH JOIN 7 1 40
SORT UNIQUE 2 1 18
TABLE ACCESS BY INDEX ROWID GEMINI_PRIMARY SITE 2 1 18
INDEX RANGE SCAN GEMINI_PRIMARY SITE_I0 1 1
TABLE ACCESS FULL GEMINI_PRIMARY SITE 3 27 594
INDEX RANGE SCAN GEMINI_REPORTING RCAMSIT_I 1 1 5
TABLE ACCESS FULL GEMINI_PRIMARY CAMPAIGN 3 127 2540
TABLE ACCESS BY INDEX ROWID GEMINI_PRIMARY CAMBILLING 1 1 18
INDEX UNIQUE SCAN GEMINI_PRIMARY CAMBILLING_U1 0 1

duplicate thread..
How to improve performance of attached query

How to improve performance of attached query

Hi,
How to improve performance of the below query, Please help. also attached explain plan -
SELECT Camp.Id,
rCam.AccountKey,
Camp.Id,
CamBilling.Cpm,
CamBilling.Cpc,
CamBilling.FlatRate,
Camp.CampaignKey,
Camp.AccountKey,
CamBilling.billoncontractedamount,
(SUM(rCam.Impressions) * 0.001 + SUM(rCam.Clickthrus)) AS GR,
rCam.AccountKey as AccountKey
FROM Campaign Camp, rCamSit rCam, CamBilling, Site xSite
WHERE Camp.AccountKey = rCam.AccountKey
AND Camp.AvCampaignKey = rCam.AvCampaignKey
AND Camp.AccountKey = CamBilling.AccountKey
AND Camp.CampaignKey = CamBilling.CampaignKey
AND rCam.AccountKey = xSite.AccountKey
AND rCam.AvSiteKey = xSite.AvSiteKey
AND rCam.RmWhen BETWEEN to_date('01-01-2009', 'DD-MM-YYYY') and
to_date('01-01-2011', 'DD-MM-YYYY')
GROUP By rCam.AccountKey,
Camp.Id,
CamBilling.Cpm,
CamBilling.Cpc,
CamBilling.FlatRate,
Camp.CampaignKey,
Camp.AccountKey,
CamBilling.billoncontractedamount
Explain Plan :-
Description Object_owner Object_name Cost Cardinality Bytes
SELECT STATEMENT, GOAL = ALL_ROWS 14 1 13
SORT AGGREGATE 1 13
VIEW GEMINI_REPORTING 14 1 13
HASH GROUP BY 14 1 103
NESTED LOOPS 13 1 103
HASH JOIN 12 1 85
TABLE ACCESS BY INDEX ROWID GEMINI_REPORTING RCAMSIT 2 4 100
NESTED LOOPS 9 5 325
HASH JOIN 7 1 40
SORT UNIQUE 2 1 18
TABLE ACCESS BY INDEX ROWID GEMINI_PRIMARY SITE 2 1 18
INDEX RANGE SCAN GEMINI_PRIMARY SITE_I0 1 1
TABLE ACCESS FULL GEMINI_PRIMARY SITE 3 27 594
INDEX RANGE SCAN GEMINI_REPORTING RCAMSIT_I 1 1 5
TABLE ACCESS FULL GEMINI_PRIMARY CAMPAIGN 3 127 2540
TABLE ACCESS BY INDEX ROWID GEMINI_PRIMARY CAMBILLING 1 1 18
INDEX UNIQUE SCAN GEMINI_PRIMARY CAMBILLING_U1 0 1

duplicate thread..
How to improve performance of attached query

How to improve performance of query

Hi all,
How to improve performance of query.
please send :
[email protected]
thanks in advance
bhaskar

hi
go through the following links for performance
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/cccad390-0201-0010-5093-fd9ec8157802
http://www.asug.com/client_files/Calendar/Upload/ASUG%205-mar-2004%20BW%20Performance%20PDF.pdf
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/1955ba90-0201-0010-d3aa-8b2a4ef6bbb2

To improve performance for report

Hi Expert,
i have generated the opensales order report which is fetching data from VBAK it is taking more time exectuing in the forground itself.
it is going in to dump in foreground and i have executed in the background also but it is going in to dump.
SELECT vbeln
           auart
           submi
           vkorg
           vtweg
           spart
           knumv
           vdatu
           vprgr
           ihrez
           bname
           kunnr
    FROM vbak
    APPENDING TABLE itab_vbak_vbap
    FOR ALL ENTRIES IN l_itab_temp
*BEGIN OF change 17/Oct/2008.
    WHERE erdat IN s_erdat              AND
         submi = l_itab_temp-submi     AND
*End of Changes 17/Oct/2008.
          auart = l_itab_temp-auart     AND
*BEGIN OF change 17/Oct/2008.
          submi = l_itab_temp-submi     AND
*End of Changes 17/Oct/2008.
          vkorg = l_itab_temp-vkorg     AND
          vtweg = l_itab_temp-vtweg     AND
          spart = l_itab_temp-spart     AND
          vdatu = l_itab_temp-vdatu     AND
          vprgr = l_itab_temp-vprgr     AND
          ihrez = l_itab_temp-ihrez     AND
          bname = l_itab_temp-bname     AND
          kunnr = l_itab_temp-sap_kunnr.
    DELETE itab_temp FROM l_v_from_rec TO l_v_to_rec.
ENDDO.
Please give me suggession for improving performance for the programmes.

hi,
you try like this
DATA:BEGIN OF itab1 OCCURS 0,
     vbeln LIKE vbak-vbeln,
     END OF itab1.
DATA: BEGIN OF itab2 OCCURS 0,
      vbeln LIKE vbap-vbeln,
      posnr LIKE vbap-posnr,
      matnr LIKE vbap-matnr,
      END OF itab2.
DATA: BEGIN OF itab3 OCCURS 0,
      vbeln TYPE vbeln_va,
      posnr TYPE posnr_va,
      matnr TYPE matnr,
      END OF itab3.
SELECT-OPTIONS: s_vbeln FOR vbak-vbeln.
START-OF-SELECTION.
SELECT vbeln FROM vbak INTO TABLE itab1
WHERE vbeln IN s_vbeln.
IF itab1[] IS NOT INITIAL.
    SELECT vbeln posnr matnr FROM vbap INTO TABLE itab2
    FOR ALL ENTRIES IN itab1
    WHERE vbeln = itab1-vbeln.
ENDIF.

How to improve performance of insert statement

Hi all,
How to improve performance of insert statement
I am inserting 1lac records into table it takes around 20 min..
Plz help.
Thanx In Advance.

I tried :
SQL> create table test as select * from dba_objects;
Table created.
SQL> delete from test;
3635 rows deleted.
SQL> commit;
Commit complete.
SQL> select count(*) from dba_extents where segment_name='TEST';
COUNT(*)
4
SQL> insert /*+ APPEND */ into test select * from dba_objects;
3635 rows created.
SQL> commit;
Commit complete.
SQL> select count(*) from dba_extents where segment_name='TEST';
COUNT(*)
6
Cheers, Bhupinder

Performance for ALTER TABLE statements

Hi,
I'd like to improve performance for scripts running several ALTER TABLE statements. I have two questions regarding this.
This is the original code:
ALTER TABLE CUSTOMER_ORDER_DELIVERY_TAB ADD (
QTY_TO_INVOICE NUMBER NULL );
ALTER TABLE CUSTOMER_ORDER_DELIVERY_TAB ADD (
QTY_INVOICED NUMBER NULL );
1. Would I gain any performance by making the following changes?
ALTER TABLE CUSTOMER_ORDER_DELIVERY_TAB ADD (
QTY_TO_INVOICE NUMBER NULL,
QTY_INVOICED NUMBER NULL );
These columns are later on filled with values and then made NOT NULL.
2. Would I gain anything by make these columns NOT NULL with a DEFAULT value in the first statement and then later on insert the values?
/Roland Bali
null

1. It wud definitely increase the performance.
2. You can only have NOT NULL columns added to an existing table if the table is empty.
<BLOCKQUOTE><font size="1" face="Verdana, Arial, Helvetica">quote:</font><HR>Originally posted by Roland Bali ([email protected]):
Hi,
I'd like to improve performance for scripts running several ALTER TABLE statements. I have two questions regarding this.
This is the original code:
ALTER TABLE CUSTOMER_ORDER_DELIVERY_TAB ADD (
QTY_TO_INVOICE NUMBER NULL );
ALTER TABLE CUSTOMER_ORDER_DELIVERY_TAB ADD (
QTY_INVOICED NUMBER NULL );
1. Would I gain any performance by making the following changes?
ALTER TABLE CUSTOMER_ORDER_DELIVERY_TAB ADD (
QTY_TO_INVOICE NUMBER NULL,
QTY_INVOICED NUMBER NULL );
These columns are later on filled with values and then made NOT NULL.
2. Would I gain anything by make these columns NOT NULL with a DEFAULT value in the first statement and then later on insert the values?
/Roland Bali<HR></BLOCKQUOTE>
Naveen

How to improve performance for Azure Table Storage bulk loads

Similar Messages

Maybe you are looking for