SCHEDULING & MONITORING PROGRAMS (TO PROCESS 100 TO 5000 FILES A DAY)

We have the end to end set of programs for scheduling & monitoring any ETL programs. In big enterprises , the system to system data access always happens through ascii files. if there are systems which needs to recieve/load files & produce handoff files periodically and if the number of files handled in day are quite large (eg:atleast
100 to 5000 files ) you can make use of our programs.The features of the scheduler are
The ETL Program Scheduling
- Incoming files are grouped together ( eg: system_name + country + branch combination , Not necessary ), Similarly outgoing files as well.
- Files will get loaded only when the whole set is recieved
- More priority can be given to certain sets depending upon the need
- Loading set to Loading set dependency (very rare cases)
- Output set to loading set dependency (The output set will be generated when all(if any) dependent loading sets are loaded.
- Order of execution of files in a set (for both loading / output )
- Statistical information about everyday's loading files (Recived time , Start time & End time of a particular file on anyday)
- 5 Parrellel queues for loading and 5 for output generation ( Multi threading )
- Output files are automatically ftp'd if required
- Esay query mechanism to know which set failed , Which file/program failed in
- Status of execution of any file by ETL program from multiple sources ( Database , File Location in diff. directories , From log files )
.. and more
- Dynamic renaming of files & execution
Monitoring Program
- For the Data centre / Back office processing people
- For a incoming set, when crossed the deadline time, this monitor log shows the contact details
- When anything gets errord out the monitor log gives the set & contact details
- Holiday maintenance for any group can be maintained.
End of day programs
- To take care of purging & archiving of files, database & logfiles etc.
Tested with 2 ETL tools ( Constellar hub from DataMirror & Oraclewarehouse builder) and this scheduler is live in an International bank (confidential) in 3 regions (Asia Pacific , Europe & North America) "PROCESSING 1000+ FILES A DAY" at each of the 3 regional data centres for the last 10 years ( constellar ) .
Technology Specifications
Operating System > HP Unix
Database > Oracle
Software > Couple of Unix shell scripts & Pro*C code
IF INTERESTED PLS MAIL ME BACK AT [email protected] (i.e; ranjeesh007 at yahoo dot com)
WITH YOUR REQUIREMENT DETAILS
Cheers
Ranjeesh K Ramakrishnan
Mumbai
0091-98216 41777

Hi,
Sounds like you have a cache problem, it could be a file corruption. I would first suggest you to clear out everything in the cache directory and do a re-sync and see if issue persists.
Also, please check on your server or OneDrive to see the total amount of data that being stored. Check if you have any large files stored, like videos.
More reference about Office Document Cache settings:
http://office.microsoft.com/en-us/word-help/office-document-cache-settings-HA010388664.aspx
Hope this helps.
Thanks,
Ethan Hua CHN
TechNet Community Support

Similar Messages

Scheduling an ABAP Program Using Process Chains

hi experts,
     I have an ABAP program which downloads ODS data to a flat file
     i am trying to schedule this using process chains
     can some body give me some hints on this
     as when i try to schedule this using the process type ABAP program
     the chain is activated and starts ok but it is completed without
     downloading the file.
     the process gets finished without any errors. kindly help me
thanks

Hi dear,
maybe you already read this doc...
https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/3507aa90-0201-0010-6891-d7df8c4722f7
Maybe it can be useful to check your procedure...
Bye,
Roberto

Process chain design/schedule/monitoring/debugging

hi friends
pls help me on the issue of process chain design/schedule/monitoring/debugging
with one of real time scenario, with help of t-codes and detail navigation
thanks in advance
venkatp

I have listed below the major informatin neede for your requirement...
1.Go to RSPC to view all the chains.
2.There click on any chain which you want to run/view.
3.Click on the log view(shift+F1) to view all the previous runs of the chain and whether they have failed or successful(here you can select 1 day,week ,month etc in the display range)
4.If you want to modify the chain /add a process type /change the variant then click (Ctrl+F9) and do the changes.
Even you can change the start variant in change mode for manual scheduling>right click on start process typeselect display variant--now select start Direct Scheduling/ API or Metachain
under direct scheuling option you can trigger it by setting start immediately /Date-time/After job/event whatever you want to do
Else you can make it part of a Metachain consisting of different such small local process chains.
Now Click on Schedule (F8) to run this chain manually.
5.For Monitoring use Tcodes SM37/RSPCM/SM50
Also you can use BWCCMS.
I usually prefer sm37 to view all the background jobs but SM50 is also useful.
6.debug loop can be used to debug a process chain.(you have to specify here the gap between one event been triggered and the next subsequent process starting)
to capture process in debugger go to SM37(job overview) or to SM50(process overview)

How can i schedule concurrent program for every one hour from back end

Hi ,
I want to schedule concurrent program for every one hour from back end .
Example
1) xyz is the concurrent program that should run for every one hour with a parameter 111 and the SAME concurrent program that
should run every 2 hours with a different parameter like 222.
I mean Conc prog should run for different parametrs with a different scheduling..
Please guide me to solve the issue.
Thanks in advance...
Regards
Narender B

Hi ,
I have used following code for scheduling the concurrent program from backend.
declare
l_request_id NUMBER;
l_return_code boolean := FALSE;
BEGIN
fnd_global.apps_initialize(62991,54477,20003);
l_return_code := FND_REQUEST.SET_OPTIONS ('YES');
l_return_code :=fnd_request.set_repeat_options('16:36:00','','DAYS','START','','Y');
l_request_id:=fnd_request.submit_request(application => 'xbol',
program => 'NAPP_START_GENERATE_CHART_DATA',
description => 'Processing chart ',
start_time => SYSDATE,
sub_request => FALSE,
argument1 =>4000130957231588,
argument2 => null
COMMIT;
dbms_output.put_line('Program has been submited and request id is '||l_request_id);
END;
Here the issue was concurrent program is completing with a warning like
Resubmission of request 75588551 has been cancelled.
FND_RESUB_PRIVATE.PROCESS_INCREMENT EXCEPTION: ORA-01403: no data found
Resubmission of request 75588551 has been cancelled.
FND_RESUB_PRIVATE.PROCESS_INCREMENT EXCEPTION: ORA-01403: no data found
so i could not find the solution for this issue,please anybody guide me to solve this issue.
Regards
Narender B

How to integrate an ABAP program in process chain

Hello Experts :
I need to call an ABAP program in PROCESS CHAIN from a different system, using Process type "ABAP program", option "scheduled program".
Sombebody have the step by step or some examples
Regards,
Diana

If you search this forum for RSPC_ABAP_FINISH you will find some good answers such as:
Start an ABAP program in R/3 from a process chain in BW

Log from ABAP program in Process chain

Hello.
I have some problem. I wrote the ABAP program and insert this program in process chain. This program a correctly work, but I canu2019t view the steps of work this program. In help:
You can write logs by outputting success messages. (Error messages terminate the task!)
Example: MESSAGE s051(rsar) WITH 'Process successfully ended'.
You can see these logs on the tab page Batch in the detail monitors of the process chain.
I append this code to my program, but nothing in process chain.
Help me, please - what code I need to insert in my program so that view this message in a process chain logs?

We have an ABAP that outputs a message. The output from the following code is visible when you click on a process chain step icon, and select the middle tab that says "Backg" for background.
IF sy-subrc = 0.
    CONCATENATE text-005 p_date INTO l_msg SEPARATED BY space.
    MESSAGE i162(00) WITH l_msg.
ELSE.
    CONCATENATE text-003 p_date INTO l_msg SEPARATED BY space.
    MESSAGE e162(00) WITH l_msg.
ENDIF.

IDoc getting scheduled in stead of processed due to no resources available

I know this is supposed to go into Basis-forum, but I can't find applicable one. If you can redirect me please do so.
Setup:
Solution Manager, CUA & ECC6
Issue:
Sometimes when triggering changes from CUA to an ECC client several IDocs are fired towards that client.
(User Master Record changes, unlocks, role assignments, etc.).
Looking in BD87 I sometimes see that of these IDocs, some are processed directly (transferred to application),
However some of them remain in Status 64 (IDoc ready to be transferred to application).
If I Right-Click -> Process, they are handled perfectly so why did they have to be waiting?
The reason SAP throws for these IDocs is:
Immediate Processing in Job '0000000000005010 '
Message no. EA187
Diagnosis
At the time of IDoc generation, there were no resources for immediate processing and your system is configured so that if this happens, IDoc processing should be started in a job. Therefore, the job '0000000000005010' was started.
Procedure
If the IDoc still has status 64, look for the cause of the error in the log for that job.
Question:
We have via WE20 profiles set to process all IDocs immediately, then why do some of them still get
scheduled in jobs? (Due to lack of resources for immediate processing?).
Where can I check these 'resources' ?
Edited by: Mujo on Jan 4, 2012 2:41 PM

Hey Alex,
Yes, I have been looking into both available worker processes all seem fine. Also I've been checking to pick up
those IDocs that are in wait-status to be picked up by a (to be) scheduled job.
However a few questions:
- is it common that CUA changes fire multiple IDocs? (in my case 3, only for something trivial as removing 1 composite role
on a CUA-child system with 1 end user
- what I don't really understand yet is if this is fired in batches of (in this case) only 3 IDocs, how come the first is processed and
the other 2 are not? What is blocking that? Is 1 worker process used for processing each IDoc or 1 worker process for all 3 IDocs? To jump into your comment of "when a lot of IDocs are fired", as said: they're only 3.
- can it be that the 'program" that processes the 1st IDoc change to the user master record is then 'occupying' the change mode and that therefore the other 2 are not able to being processed or can it be that the changes just come too fast after one another?
- I would think that when CUA is installed and RFCs and IDoc types are correctly configured this resource problem
should not act up with standard SAP ECC6 settings, for firing something trivial/simple as a composite role change
- last question: if I should schedule a job to pick up the IDocs that are in waiting-to-be-processed status, what is common
interval for this? (every minute? every 5 minutes?) and what's the program to schedule?
EDIT: found at least answer to this question:
RBDAPP01. Processes inbound IDocs (status 64) that have been buffered to support mass
processing. The use of RBDAPP01 in the process flow is described in Chapter 4, "The Inbound EDI
Process." Steps to execute the RBDAPP01 program are listed in Chapter 10, "Testing the EDI
Interface."
Looks strange to me as then the point of CUA making direct changes is somewhat undermined. In this case our CUA is being fed from a Sun IDM system, and the changes on SAP landscape will be 'less near-realtime', if I express that correctly.
Your thoughts are highly appreciated.
Edited by: Mujo on Jan 5, 2012 9:47 AM
Edited by: Mujo on Jan 5, 2012 10:21 AM

How to schedule another program by giving report name and variant name?

I want to create a program for Scheduling another program using function 'FASU_V_CREATE_VARIANT_RFC'
(FASU_V_CREATE_VARIANT_RFC : This function used for creating the variant ) by giving
1) Report name
2) varaint name
Could anybody please help me.??

Scheduling Background Jobs
Use
You can define and schedule background jobs in two ways from the Job Overview:
Directly from Transaction SM36. This is best for users already familiar with background job scheduling.
The Job Scheduling Wizard. This is best for users unfamiliar with SAP background job scheduling. To use the Job Wizard, start from Transaction SM36, and either select Goto ® Wizard version or simply use the Job Wizard button.
Procedure
Call Transaction SM36 or choose CCMS ® Jobs ® Definition .
Assign a job name. Decide on a name for the job you are defining and enter it in the Job Name field.
Set the jobs priority, or "Job Class":
High priority: Class A
Medium priority: Class B
Low priority: Class C
In the Target server field, indicate whether to use system load balancing.
For the system to use system load balancing to automatically select the most efficient application server to use at the moment, leave this field empty.
To use a particular application server to run the job, enter a specific target server.
If spool requests generated by this job are to be sent to someone as email, specify the email address. Choose the Spool list recipient button.
Define when the job is to start by choosing Start Condition and completing the appropriate selections. If the job is to repeat, or be periodic, check the box at the bottom of this screen.
Define the jobs steps by choosing Step, then specify the ABAP program, external command, or external program to be used for each step.
Save the fully defined job to submit it to the background processing system.
When you need to modify, reschedule, or otherwise manipulate a job after you've scheduled it the first time, you'll manage jobs from the Job Overview.
Note: Release the job so that it can run. No job, even those scheduled for immediate processing, can run without first being released.
For a simple job scheduling procedure, see the R/3 Getting Started Guide.

Reg. scheduling of program.......have ur points.

Hi all,
I want to schedule a program, so that i'll run automatically at 7 A.M in the morning for 1 year daily.
Pleas let me know step by step.
Thanx in advance& have ur points.
Regards,
pradeep phogat

HI,
Tcode SM36
Here we can Schedule the Executable Programs (Reports) in Background.
Create a Job.
Create the Steps - Assign the Report name and Variant.
Enter the Start Condition.
Save the Job.
Job status can be monitorEd in Tcode SM37
Or
Use the following function modules.
job_open
job_close
Thanks,
Reward If Helpful.

Scheduling ABAP Program to run Daily

Hi,
In my current scenario, i require my ABAP program(ZMYPROGAM) to run periodically (say every 1 hour/ 1 Day). Is it possible to schedule it so?
kindly let me know how to go about.
Regards,
Siva Maranani.

Hi Siva,
You can schedule your program as a background Job SM36. Please check the transaction. You will have to set your ABAP program as the job STEP (if you probably have selection screen for your report then you will also need a variant for your report).
Do schedule your program as a 'Periodic Job' and your personal period (daily/hourly).
After you save your settings you can also monitor your job (ABAP Program) from the same transaction or even from CCMS if you like.
hope this helps,
ZAM

Transaction Monitor u2013 Services Processes into BI

Hello Experts,
I'm wondering if it is possible to get tickets information from Solution Manager and have some statistics in BI.
For instance the average time of ticket resolution, number of tickets per team/person/month, etc, etc
At this moment Iu2019m using the ZNTP transaction in solution manager to access Transaction Monitor u2013 Services Processes.
I would like to install BI Content to have some summary queries about the activity within ZNTP.
I already connected BI to SMP and also replicated metadata, however the data sources available just by the name donu2019t help me much.
Perhaps I should check the content version within solution manager and BI but I don't know how to performe such verifications.
Thanks

using rsa3 in the source system I'm getting the follow error:
The extraction program does not support object 0CRM_SRV_PROCESS_H
Message no. R3009
Diagnosis
The application program for the extraction of data was called up using the invalid InfoSource/invalid InfoObject 0CRM_SRV_PROCESS_H.
System Response
The data extraction is terminated.
Procedure
Check the SAP Support Portal for the appropriate Notes and create a customer message if necessary.
Can you help me?

Write a program that would schedule another program in background multipul

Hi All,
I need to write a program that would schedule another program in background multipul times according to input we provided on different application servers.
For ex:- In table FKKVKP, i am having 2.1 millons contract accounts. I need to write a program(A) that should take 5000 contract accounts at a time as a input and schedule another program(B) in background with input of 5000 contract accounts. And program A ,again schedule program B on differrent application servers , till 2.1 millions contract accounts get used.
Thanx in advance.
Regards,
Dilip Kushwah

hi,
1) Carry out the recording for the required transaction using SHDB.
2) Use batch input method to create session this is done by three FMs
bdc: open_group, bdc_insert, close_group.
3) A session is created , which can be seen in transaction SM 35.
4) Use the standard program RSBDCSUB to configure the session created in SM35.
This way you can achieve your requirement.
Check this links too for more information
Batch input [http://help.sap.com/saphelp_nw04/helpdata/en/fa/097015543b11d1898e0000e8322d00/frameset.htm]
[http://help.sap.com/saphelp_nw04/helpdata/en/4c/4c0e8a725311d396a80004ac96334b/frameset.htm]
[http://help.sap.com/saphelp_nw04/helpdata/en/4c/4c0e8a725311d396a80004ac96334b/content.htm]
Thanks
Sharath

Splitter program for processing the IDOCS with status 64.

Hello folks,
I have a requirement explained below. I am looking for a sample code which could help me to jump start with initial development.
Requirement:
Develop a splitter program which retrieves all the IDCOS with status 64, create and schedule batch jobs with 200 IDOCS each. The splitter program with be scheduled every 15 min.
Restrictions
Only 150 batch jobs can be created at a time.
Estimation
There may be around 36000 IDOCS with status 64 every hour.

Srinivas,
Here is the explaination why we require a splitter program.
Using RBDAPP01 parellel processing
Let us assume there are 4 servers A, B, C and D in a server group. If we select RBDAPP01 parellel processing there will be too many RFC calls to the servergroup.
(there are some more RFC calls later on for Order creation for example). As there is a restriction with number of RFC calls, we need to clear other RFC's before making new one. This is making complex and messy.
Using RDBAPP01 serial processing
For the volume of orders getting created each day, it takes a week to process those Idoca created in one day. Hence it is not a good solution.
Splitter program
We spawn 150 batch jobs with 200 records each. This way if we think it takes 20 min for each job to complete...by the end of 20th min we must had processed 30000 idocs.
Hence it is the solution offered.
Srinivas...still this confuses me..but if I am wrong anywhere please correct me.
But your initial response has helped me a lot. Everything was blank to me, ur reply made it clear over the requirement.
Thanks a lot,
Naren

Toshiba PC Health Monitor Program

I have been using my Toshiba DX1215-D2101 for a week. What I have noticed is that the PC Health Monitor program uses 25% of the processing power. Is there a way to turn this on and off or is there a newer version that does not use so much processing power? Is this program necessary?

Further issues of interest...
What is my computer doing? will show no activity in its window even when the hard drive light is steady (I mean like lit...not blinking rapidly. Lit SOLID).
Clicking the start menu in windows is ok, but if I click all programs and attempt to open a folder, it loses its mind(little magnifying glass circling and circling and circling)...
Toshiba PC Health says the hdd health is good. I'm losing my faith that it is...

Calling r/3 program from process chain

Hi
I did the following steps to call r3 program from process chain :
Steps in R/3 system
1. Created a program which needs to be called from bw and at the end of the program i have calling 'RSPC_ABAP_FINISH' function module & passing the rfc id of bw system and bw process variant.
2. created a even in r/3 (please note that this is not there in bw)
3. Created a job and under start condiion selected 'after event' and mentioned the event created earlier. And under step1 enter the abap program to be executed
Steps in BW system :
1. Created a process chain
2. inserted a process type 'abap program' and selected the call mode 'Asynchronous' and under destination entered the RFC destination of R/3 system. Under scheduled program enterd the event name created in R/3
after executing above process chain, the abap program node is becoming green but the abap program is not getting executed in r/3 system. And also the process after the program node in process chain is not getting executed. The job which was created in r/3 still showing the released status
Kindly advice where i am wrong.
Thanks
RKS

any input

SCHEDULING & MONITORING PROGRAMS (TO PROCESS 100 TO 5000 FILES A DAY)

Similar Messages

Maybe you are looking for