Reduce/Eliminate the start up time for a batch job
I'm looking for ideas on how I might be able to modify either my jobs, workflow or data flow in an effort to reduce the "start up" time that I'm encountering.
Currently I have a batch job we execute several hundred times a day. One of the parameters we pass to this job is a file path. The data flow reads a flat text file in this file path, processes the data through several transforms, and writes a flat text file back into the file path.
The actual processing time varies with the number of records being processed -- as one would expect.
However, even for the smallest - single record - input file the job takes between 40 and 50 seconds to complete. If I watch the trace file during this time it appears that 95% of this time is spent getting ready to run the job. The actual processing is almost instantaneous.
Is there anything that I can look at changing in order to improve the overall performance of these jobs?
I should mention that I am aware of setting up real time jobs, and how they would not require this "start up" time for each job submitted. However this jobs we process could between 1 to 1,000,000 records. Passing that many records in a single web service call does not seem reasonable. I would deal with issue on the submission side and then the potential timeout issue when waiting on the response -- which could also be quite large.
Thanks Michael --
I must admit I'm a bit lost when it comes to the issue of codepages.
Is there any documentation you could refer me to with regard to setting the correct codepage for the data sources and targets?
Also, from the trace log I see the following statement:
Changed engine runtime locale <eng_us.cp1252> to <Unicode (UTF-16)> because data flow <DF_Suppression> contains a transform that requires processing with the Unicode (UTF-16) code page.
Other than the data source and data targets I do not see any option to set, or change, the codepage within any of transforms that I am using. (URAC, Name Cleanse, Query, Case, Match transforms are used)
Right now my source and targets are set to "default", which would seem to need to be changed. Where is the "default" set?
Sorry for so many questions about this.
Similar Messages
-
How do I change the start/stop time for multiple songs not just one song at a time?
How do I chnage the start/stop time for multiple songs not just one song at a time?
I don't think you can in iTunes.
Multiple editing in Get Info does not offer an option to change start/stop times. -
Separate Start/Stop times for a single song file
I am trying to create some playlists for different moods e.g. calm, happy, upbeat etc.
Now, my problem is this. I have some songs that have more than one mood within them, and I would like to set different start/stop times for the one song file.
I would like to be able to set different start/stop times in each playlist and not affect the start/stop times for the file in the music library.
Thanks in advance to anyone who can help!The only way to do this is to make a duplicate copy of the song for your playlist and adjust the times for the duplicate.
-
I try to install LR CC in Creativ Cloud (10 times) on Mac and as propose I logout from iCloud and login again. The Start Icon shows for short time an that is all.....is a deinstallation from the old LR necessary?? Or what is to do. Daniel Berne
Reboot the computer once.
Sign out from Creative Cloud and Sign in .
Important : Try to launch Lightroom CC apps from Apps tab menu. :
Still same issue ? let me know . -
PRVF-5636 : The DNS response time for an unreachable node exceeded "15000"
Friends,
OS: OEL 6.3 64 bit
DB: 11.2.0.3
Environment: VirtualBox 4.1.20
i have 3 nodes. one is for dns and other two for rac1 and rac2
i have done all the preliminary steps. then i started
[oracle@rac1 grid]$ ./runcluvfy.sh stage -pre crsinst -n rac1,rac2 -verboseand all are passed except the above subjected error.
Checking DNS response time for an unreachable node
Node Name Status
rac2 failed
rac1 failed
PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: rac2
File "/etc/resolv.conf" is not consistent across nodesok....now to solve this error...i tried many options...but not able to succeed.
Try 1:
[root@mydns ~]# time nslookup rac1
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: rac1.sathy.net
Address: 192.168.1.3
real 0m0.051s
user 0m0.003s
sys 0m0.011s
[root@mydns ~]# [oracle@rac1 grid]$ time nslookup mydns
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: mydns.sathy.net
Address: 192.168.1.2
real 0m0.019s
user 0m0.005s
sys 0m0.008s
[oracle@rac1 grid]$
[root@rac2 ~]# time nslookup rac1
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: rac1.sathy.net
Address: 192.168.1.3
real 0m0.019s
user 0m0.008s
sys 0m0.005s
[root@rac2 ~]# Try 2:
[root@mydns ~]# nslookup sathy-scan
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: sathy-scan.sathy.net
Address: 192.168.1.8
Name: sathy-scan.sathy.net
Address: 192.168.1.9
Name: sathy-scan.sathy.net
Address: 192.168.1.7
[root@mydns ~]# nslookup rac1
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: rac1.sathy.net
Address: 192.168.1.3
[root@mydns ~]# nslookup rac2
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: rac2.sathy.net
Address: 192.168.1.5
[root@mydns ~]#
[oracle@rac1 grid]$ nslookup sathy-scan
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: sathy-scan.sathy.net
Address: 192.168.1.9
Name: sathy-scan.sathy.net
Address: 192.168.1.7
Name: sathy-scan.sathy.net
Address: 192.168.1.8
[oracle@rac1 grid]$ nslookup rac2
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: rac2.sathy.net
Address: 192.168.1.5
[oracle@rac1 grid]$ nslookup rac1
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: rac1.sathy.net
Address: 192.168.1.3
[oracle@rac1 grid]$
[root@rac2 ~]# nslookup sathy-scan
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: sathy-scan.sathy.net
Address: 192.168.1.7
Name: sathy-scan.sathy.net
Address: 192.168.1.8
Name: sathy-scan.sathy.net
Address: 192.168.1.9
[root@rac2 ~]# nslookup rac1
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: rac1.sathy.net
Address: 192.168.1.3
[root@rac2 ~]# nslookup rac2
Server: 192.168.1.2
Address: 192.168.1.2#53
Name: rac2.sathy.net
Address: 192.168.1.5
[root@rac2 ~]# Try 3:
[root@mydns ~]# cat /etc/named.conf
// Default named.conf generated by install of bind-9.2.4-30.el4_7.2
options {
listen-on port 53 { any; };
directory "/var/named";
dump-file "/var/named/data/cache_dump.db";
statistics-file "/var/named/data/named_stats.txt";
forwarders { 192.168.1.1; };
include "/etc/rndc.key";
zone "sathy.net" IN {
type master;
file "sathy.net.zone";
allow-update { none; };
zone "1.168.192.in-addr.arpa." IN {
type master;
file "1.168.192.in-addr.arpa";
allow-update { none; };
zone "." in {
type hint;
file "/dev/null";
[root@mydns ~]# Try 4:
[root@mydns ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
#SELINUXTYPE=targeted
[root@mydns ~]#
[oracle@rac1 grid]$ cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
#SELINUXTYPE=targeted
[oracle@rac1 grid]$
[root@rac2 ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
#SELINUXTYPE=targeted
[root@rac2 ~]# what else i have to check to solve this error?
for the past 2 days..i am trying to solve this error. may be somewhere a small mistake is happened from my side, but i am not able to find out.
hope somebody will find out and help me.
thanksok...here it is....
[root@mydns ~]# cat /etc/resolv.conf
# Generated by NetworkManager
search sathy.net
nameserver 192.168.1.2
[root@mydns ~]#
[root@rac1 ~]# cat /etc/resolv.conf
# Generated by NetworkManager
search sathy.net
nameserver 192.168.1.2
[root@rac1 ~]#
[root@rac2 ~]# cat /etc/resolv.conf
# Generated by NetworkManager
search sathy.net
nameserver 192.168.1.2
[root@rac2 ~]#
[root@mydns ~]# cat /etc/hosts
#127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
#::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
# Public Network – (eth0)
192.168.1.3 rac1.sathy.net rac1
192.168.1.5 rac2.sathy.net rac2
# Public Virtual IP (VIP) addresses – (eth0:1)
192.168.1.4 rac1-vip.sathy.net rac1-vip
192.168.1.6 rac2-vip.sathy.net rac2-vip
# Private Interconnect – (eth1)
192.168.2.3 rac1-priv.sathy.net rac1-priv
192.168.2.5 rac2-priv.sathy.net rac2-priv
#Compute Node Client Access SCAN Interface details
#192.168.1.7 sathy-scan.sathy.net sathy-scan
#192.168.1.8 sathy-scan.sathy.net sathy-scan
#192.168.1.9 sathy-scan.sathy.net sathy-scan
[root@mydns ~]#
[root@rac1 ~]# cat /etc/hosts
#127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
#::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
# Public Network – (eth0)
192.168.1.3 rac1.sathy.net rac1
192.168.1.5 rac2.sathy.net rac2
# Public Virtual IP (VIP) addresses – (eth0:1)
192.168.1.4 rac1-vip.sathy.net rac1-vip
192.168.1.6 rac2-vip.sathy.net rac2-vip
# Private Interconnect – (eth1)
192.168.2.3 rac1-priv.sathy.net rac1-priv
192.168.2.5 rac2-priv.sathy.net rac2-priv
#Compute Node Client Access SCAN Interface details
#192.168.1.7 sathy-scan.sathy.net sathy-scan
#192.168.1.8 sathy-scan.sathy.net sathy-scan
#192.168.1.9 sathy-scan.sathy.net sathy-scan
[root@rac1 ~]#
[root@rac2 ~]# cat /etc/hosts
#127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
#::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
# Public Network – (eth0)
192.168.1.3 rac1.sathy.net rac1
192.168.1.5 rac2.sathy.net rac2
# Public Virtual IP (VIP) addresses – (eth0:1)
192.168.1.4 rac1-vip.sathy.net rac1-vip
192.168.1.6 rac2-vip.sathy.net rac2-vip
# Private Interconnect – (eth1)
192.168.2.3 rac1-priv.sathy.net rac1-priv
192.168.2.5 rac2-priv.sathy.net rac2-priv
#Compute Node Client Access SCAN Interface details
#192.168.1.7 sathy-scan.sathy.net sathy-scan
#192.168.1.8 sathy-scan.sathy.net sathy-scan
#192.168.1.9 sathy-scan.sathy.net sathy-scan
[root@rac2 ~]#
[oracle@rac1 ~]$ ssh rac2 date
Sun Sep 9 22:04:45 IST 2012
[oracle@rac1 ~]$ su - grid
Password:
[grid@rac1 ~]$ ssh rac2 date
Sun Sep 9 22:04:54 IST 2012
[grid@rac1 ~]$
[oracle@rac2 ~]$ ssh rac1 date
Sun Sep 9 22:04:18 IST 2012
[oracle@rac2 ~]$ su - grid
Password:
[grid@rac2 ~]$ ssh rac1 date
Sun Sep 9 22:04:31 IST 2012
[grid@rac2 ~]$ Now the error is ...
PRVF-5637 : DNS response time could not be checked on following nodes: rac2,rac1what else?
i wonder how it was successful in my previous installation which in oel 5.7....
Edited by: OraDB on Sep 9, 2012 9:44 AM -
Specific Start/End Times for Playlist Music?
I'm someone that dabbles in music production and i like to work with samples. I know that in iTunes you can set the Start Times and End Times of songs so that you may skip a rediculously long intro or just euthanize a song whose ending drags on for minutes at a time. (I'm looking at you, The Mars Volta.)
Anyway, when it comes to producing samples, sometimes when i listen to music i'll hear this one five-second piece that would make a great loop, and i write down the song name and the exact location of that interval. In iTunes, I attempted to create a "Samples" playlist with these songs in it, along with their Stop/Start Times sliced down exactly to that one sample. it's a great way to keep a log of what i plan to use in the future and keep good song ideas. Unfortunately, the Start/Stop Times set in iTunes are global, and affect the song no matter what Playlist is it on. Is this avoidable, or am i stuck? (Apple, please add this in your next iTunes release)
-DaveDave,
You're stuck.
Unless you want to duplicate the song and then make your adjustments.
If it were me, I'd put the track I'm interested into GarageBand and cutting it down to just the bit I want and the send that version back to iTunes.
Matt -
How to determine the creation date/time for a file?
The important operating systems maintain both a creation date/time and last modified date/time for files. But in the File class there is only a lastModified() method. How does one determine the creation date/time for a file?
As far as i know, there is no way to know creation time, since it is a OS dependant information.
-
I am having issues with adjusting the duration of time for each still frame and transition in imovie 11. Each time that I adjust these times, the application does not accept the change, and instead automatically enters its own time. Is there a work around this? Or am I doing something wrong? Please Help!
Thank you,
lagrlHave you tried turning off automatic transitions ? Choose file - project properties and follow the dialogue box to change duration with a slider. iMovie also doesn't allow a transition to be more than 50% duration of the clip its attached to. In other words if the clip is 4 seconds the transition cannot be more than 2 seconds and remember that relates to the first clip as the following clip (right side) is pulled back to overlay it.
Does this help. Perhaps you already know this ? -
CVU failing with error msg(PRVF-5636 : The DNS response time for an unreach
I am installing 4Node 10g RAC on OEL5.3, i downloaded CU from OTN and getting below warning messages.
Checking the file "/etc/resolv.conf"; to make sure only one of domain and search entries is defined
File "/etc/resolv.conf"; does not have both domain and search entries defined
Checking if domain entry in file "/etc/resolv.conf"; is consistent across the nodes...
domain entry in file "/etc/resolv.conf"; is consistent across nodes
Checking if search entry in file "/etc/resolv.conf"; is consistent across the nodes...
search entry in file "/etc/resolv.conf"; is consistent across nodes
Checking file "/etc/resolv.conf"; to make sure that only one search entry is defined
All nodes have one search entry defined in file "/etc/resolv.conf";
Checking all nodes to make sure that search entry is "soft1" as found on node "racnode4"
All nodes of the cluster have same value for 'search'
Checking DNS response time for an unreachable node
Node Name Status
racnode4 failed
racnode3 failed
racnode2 failed
racnode1 failed
PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: racnode4,racnode3,racnode2,racnode1
File "/etc/resolv.conf"; is not consistent across nodes
Check: Time zone consistency
Result: Time zone consistency check passed
Pre-check for cluster services setup was unsuccessful on all the nodes.
[oracle@racnode1 bin]$PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: racnode4,racnode3,racnode2,racnode1 have you checked this note? Runcluvfy - Dns Response Time Check - Fails with PRVF-5636 The DNS response time for an unreachable node exceeded "15000" ms [ID 1326997.1]
Also have you set Domain name on all servers?
[root@oracle-ha ~]# domainname
(none)
[root@oracle-ha ~]# domainname ckpt.com
[root@oracle-ha ~]# domainname
ckpt.com
[root@oracle-ha ~]#You can check DNS configuration of your setup using this link http://computernetworkingnotes.com/network-administrations/dns-server.html -
PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes
HI
I received this error message when i was trying to install 10g software on oracle linux 5.632 bit machine . how to resolve this issues
Checking consistency of file "/etc/resolv.conf" across nodes
Checking the file "/etc/resolv.conf" to make sure only one of domain and search entries is defined
File "/etc/resolv.conf" does not have both domain and search entries defined
Checking if domain entry in file "/etc/resolv.conf" is consistent across the nodes...
domain entry in file "/etc/resolv.conf" is consistent across nodes
Checking if search entry in file "/etc/resolv.conf" is consistent across the nodes...
search entry in file "/etc/resolv.conf" is consistent across nodes
Checking DNS response time for an unreachable node
Node Name Status
rac2 failed
rac1 failed
PRVF-5636 : The DNS response time for an unreachable node exceeded "15000" ms on following nodes: rac2,rac1
File "/etc/resolv.conf" is not consistent across nodes
Thanks[oracle@bdk-dev-db2 ~]$ time nslookup bdk-dev-db2
Server: 10.244.56.102
Address: 10.244.56.102#53
Name: bdk-dev-db2.dev.com
Address: 10.244.56.104
real 0m1.811s
user 0m0.003s
sys 0m0.004s
[oracle@bdk-dev-db2 ~]$ time nslookup bdk-dev-db1
Server: 10.244.56.102
Address: 10.244.56.102#53
Name: bdk-dev-db1.dev.com
Address: 10.244.56.103
real 0m0.007s
user 0m0.003s
sys 0m0.002s -
Can I change the moments and time for photos?
Can I change the moments and time for photos?
By editing the EXIF data. You'd need an EXIF editor for this...if using Windows, you can use Windows Exploroer. To do so on your phone, you need an app, like this:
https://itunes.apple.com/us/app/exif-fi-photo-gps-exif-viewer/id574247475?mt=8 -
I have to push the button several times for my phone to come on has this happened to anyone else?
I have to push the button several times for the phone to come on - has anyone else had this problem?
I have to push the button several times for the phone to come on - has anyone else had this problem?
-
I have a problem sending mail via smtp. I use a satellite system and the average return time for a ping is 675ms. Is this a problem with mail? If so can I change Mail to accept it. The problem also exists with Lion and on both my MacPro and my wife's Imac. I also see my mailboxes randomly disconnecting and reconnecting. Any other ideas of a possible cause?
I solved it myself, after the "note" which came back from FF/Mozilla just as I finished my message, commenting on what it was that my system had , I wnnt back to check my plug-ins etc. I downloaded the latest Java, BOTH 32bit AND 64 bit versions and latest Firefox.
Now all is working.
Thanks,
B. -
Manually specifying start/stop times for video
When we insert a video into a Keynote presentation, we can adjust the start/stop times of the video (inspector -> Quicktime icon). The problem here is that it's very difficult to accurately adjust the sliders due to the small slider. Is there a way to extend the slider so that it's longer, or simply a way to manually specify the start/stop times?
Unfortunately, there is not. Your best bet it to use Quicktime or iMovie to trim the movie down to just what you want to show and then you don't have to worry about setting it in Keynote. This will also save some file size as you don't have the video that you don't want to show taking up space in the Keynote file.
-
What is the boot up time for Mavericks using a 2.4 Ghz 13" MacBook Pro
What is the boot up time for Mavericks using a 2.4 Ghz 13" MacBook Pro
This will vary depending on what HDD/SSD option any system has been specked to, whether from apple or upgraded afterwards (depending on upgradable model)
I have a late 2011 2.4 i5 13" upgraded to 16Gb ram and a Samsung 840 Pro SSD and my install of Mavericks boots in around 15 seconds. This is actually about the same as my 2008 core2duo macbook booted snow leopard with a Samsung 840 (non pro)
You may find a system with less ram actually boots marginally quicker, but this would be marginal.
Maybe you are looking for
-
MSI DK790 GX (non platinum) - Flashing bios
Hello there! First things first, please accept my apologies if my language seems wrong. I'm not a native english speaker, and do not hesite to ask to repeat. Second thing, i've read some similar topic but i'm still confused about how i have to flash
-
Oracle forms 10g,multiple insert and update problem
Hi, I have tabular form(4 records displayed) with one datablock(based on a view). After querying the form could not update all the records but only first record value can select from LOV. I called a procedure in in-insert and on-update The query is l
-
Hi, I am installed AIA FP 2.2.1 so BSR is automatically installed with it. Afterwards i installed OSR 10.1.3.1. As per the AIA docs i have changed bsrConfig.properties to point to OSR registry. registry.url=http://myAPPServer/registry I was to publis
-
Midi stops working in Logic 9.1
When using Logic 9.1 I have been having midi crashes that require me to restart Logic. All Audio files will still play but I cannot play any 3rd party or Logic softsynths. I am getting a midi signal from my keyboard to my MOTU Ultralite interface but
-
my iphones buttons are not working and it shows a black screen whith apple logo on it what should i do now? and i have to post sth importent on instagram tonight so its ergent please help....