Is it possible to loop through files in a folder, but only load those with a specific filename?
Hello all. I have a folder with about 100 text files total, but it's really only about 25 different reports, each with multiple quarters. So, for example, it looks like this...
Report A 2014-12-31
Report A 2014-09-30
Report A 2014-06-30
Report A 2014-03-31
Report B 2014-12-31
Report B 2014-09-30
Report B 2014-06-30
Report B 2014-03-31
Report C 2014-12-31
Report C 2014-09-30
Report C 2014-06-30
Report C 2014-03-31
...and so on
I need to load only report B to a table in SQL Sever 2008 R2, but I need to load all quarters for report B. I'm trying to accomplish this using a foreach loop container, but I haven't figured
out hot to tell it to only look at report B and ignore the others. In testing, the loop always starts with report A.
In the process, I also need to capture the date section of the filename in a variable so I can create a derived column with the date to insert into the table.
Any ideas how to accomplish this? I'm trying to avoid putting each report in its own folder because it adds a manual step, but if there's a way to do that using a file system task before running the loop, I'm open to it.
Thanks for any advice you have!
WeeLass
Have you tried specified Report B*.txt for Files : under the Collections tab ?
Narsimha
Similar Messages
-
Looping through files in a folder, How to
Looking for plsql code to read through files in a folder, and pass the name of the file found to a parm.
The parm will then be used to open the table, process data to end of file, and then close the file. The file is then moved to an archive, and the next file in the folder is read, name of file passed to parm, file is opened, data is processed, and file is moved to archive....repeat until no files in folder.
I only need some help with reading through the folder to find the table. I have code on how to open, process and move a folder...it's the file in folder read loop I am trying to develop.
In summary, I am looking for some code showing how to read all tables in a folder and pass the name of the table found, until all files read.emss wrote:
Yes, you understand....thank you. That's the primary objective of this thread....determine the name of the .csv file.
The putline and so forth process the file. I need to find the file, it's name, and then process the .csv file.PL/SQL itself has no mechanism built in for determining the list of files in a directory.
The Java solution is one of the easiest to implement as PL/SQL can interface fairly simply with Java functions which you can create on the database.
There are other ways, but no way is going to be completely simple.
Remember, from PL/SQL you will only really be able to get to files that the database server itself can see, so either files on the database server itself, or files on machines that the server itself has network mappings to that it can treat as if they are local files. It certainly won't be able to hack across the network to any client machines hard drive to read a list of files. -
Is possible search all the files in one folder and get list all with the same extension.??
Hi,
I would like to get, if it is possible, do searching in one determinate folder and get all the files with the same extension, For example, Give the *.pou and get all the files on one list of the files in this folder and sub-folders...
If it is possible i would like to see any example.
Thanks a lot, Fonsi.Hi Dennis,
I got your advise, I download the openG (i had problems, and downloading directly and install one to one).
Ok, i got but i have one problem, which i can't solve. I entry the promt to search and put the directory to save, later i push 'Do it' and it search the files, show the paths and number, then save in the folder and finally show the window , all ok!.
The problem is with the target path, when i want use the browse, it doesnt run properly. I want select one carpet, and it demands one file, i dont understand why??, i changed the options browse but it didnt work. Also i would like when i put one path if folder doesnt exist, directly create it and save the files, if exist copy it.
Thanks for all!. I attached the file in lv 7.1
Attachments:
buscar3.vi 50 KB
capture.GIF 48 KB -
Looping through files with Regular expressions ?
Hi,
My Question is:
if i have the following Regular Expression,
String regrex = "tree\\s\\w{1,4}.+\\s=\\s(.*;)";
The file in which i am looking for the string has multiple entries, is it
possible to do another regular expression on the captured group (.*;)
which is in the original Regular expression ?
The text that is captured by the RE is of the type "(1,(2,((3,5),4)));" for
each entry, and different entries in the file have slightly different syntax
is it possible to loop through the file and first of all check for the presence
of the original RE in each entry of the file
and then secondly, check for the presence of another RE on the captured group?
[ e.g. to check for something like, if the captured group has a 1 followed by a 3
followed by a 5 followed by a and so on ].
Thanks Very much for any help, i've been struggling with this for a while!!
Much appreciated
The code that i have so far is as follows:
import java.util.*;
import java.util.regex.*;
import java.io.*;
import java.lang.*;
import javax.swing.*;
public class ExpressReg {
public String Edit;
public ExpressReg(String editorEx){
Edit = editorEx; // Edit = JTextArea
String regrex = "tree\\s\\w{1,4}.+\\s=\\s(.*;)";
//String regrex1 = "(.*;)";
Pattern p = Pattern.compile(regrex);
Matcher m = p.matcher(editorEx); // matcher can have more than one argument!
boolean result = m.find();
if(result){
JOptionPane.showMessageDialog(null, "String Present in Editor");
else if(!result){
JOptionPane.showMessageDialog(null, "String Not Present In Editor");if i have the following Regular Expression,
String regrex = "tree\\s\\w{1,4}.+\\s=\\s(.*;)";
The file in which i am looking for the string has multiple entries, is it
possible to do another regular expression on the captured group (.*;)
which is in the original Regular expression ?Yes, the capturing group is $1 (the only one) referenced in source code as m.group(1).
m.group() will return entire matching.
simply use this way:
String result = m.group(1);
// your stuff: could be another validation
The text that is captured by the RE is of the type "(1,(2,((3,5),4)));" for
each entry, and different entries in the file have slightly different syntax
is it possible to loop through the file and first of all check for the presence
of the original RE in each entry of the file
and then secondly, check for the presence of another RE on the captured group?Again "Yes", no limits!
Don't need to create another Matcher, just use m.reset(anotherSourceString)..loop using the same pattern.
Note: Take care with ".*" because regex nature is "greedy", be more specific, eg.: "\\d" just matches digits (0-9).
Can you give us some sample of "slight difference" ? -
SSIS - Loop through files from a file path based on the value in the variable
Experts,
I have a requirement where I'll be loading multiple files in to a SQL server table and archive the files when loaded. However, the challenge is , the file path should be dynamic based on the value of a variable (say, @ProductName).
For example: If I am running the package for variable @ProductName="Product", the file path would be "\\....\Src\Product", in that case the ForEachLoop will loop through all the files in that folder, load them to the table and Archive
the files to the "\\....\Src\Product\Archive" folder.
Similarly, if the @ProductName="Product_NCP", the foreachloop container should loop through files in the "\\....\Src\Product_NCP" folder, load them to the table and archive them to the ""\\....\Src\Product_NCP\Archive"
folder.
Any suggestions? I should be able to run the package manually just by passing the "@Product" value, create Archive folder if it doesn't exist, load the data and archive the files.Yes
1. Have a variable inside SSIS to get folder path. Set path based on your rule using an expression
like
(@[User::ProductName] == "Product" ? "\\....\Src\Product" : (@[User::ProductName] == "Product_NCP" ? \\....\Src\Product_NCP:..))
similary archive
(@[User::ProductName] == "Product" ? "\\....\Src\Product\Archive" : (@[User::ProductName] == "Product_NCP" ? "\\....\Src\Product_NCP\Archive" :..))
Please Mark This As Answer if it solved your issue
Please Vote This As Helpful if it helps to solve your issue
Visakh
My Wiki User Page
My MSDN Page
My Personal Blog
My Facebook Page -
I'm trying to view my movies through itunes and apple TV but only a few appear in itunes when i add to library, they include .avi and mp4 files. Whys have only a few copied and rest wont add to my library, can someone help? Do I need to convert them?
What are the file extensions of the movies that won't import to iTunes
What video content works in iTunes
What video content works with iTunes and mobile devices?
iTunes:
Video content purchased from the iTunes Store.
QuickTime and MPEG-4 movie files that end in ".mov", ".m4v", or ".mp4" and are playable in QuickTime Player.
Video podcasts.
iTunes Digital Copies.
iTunes Store Movie Rentals (Requires iTunes 9 or later). -
What would the Smart Collection criteria be to isolate files in a folder but NOT in a collection?
What would the Smart Collection criteria be to isolate files in a folder but NOT in a collection?
or else FAR more simply,
though in practice, the difficulty may lie with how fuzzily or precisely the search text used, applies.
QUESTION
I'd be very curious to know, does anyone know how to search on a section of a full folder path rather than on the individual parts of that?
My folders are structured by LR at import, as (e.g.) 2014 / 01 / 27 for 27th Jan 2014.
So if I wanted to use a Smart Collection to search for (among other things) images in July 2013, I can search multiply on includes all of "2013" and "07" but that also throws up in my case, images which were taken on the 7th day of other months during 2013. I could search on a date range, though that's a little more laborious, or else do what I otherwise prefer, which is by keywords and other attributes plus (possibly) just year.
If I could search on folder path includes the string "2013/07" this would work fine. But I haven't found a syntax that does that (yet, assuming it is possible at all).
RP -
Is it possible to have Thunderbird send an e-mail automatically when a message with a specific subject is received?
I basically want to create a rule that will send an automatically crafted form e-mail to the sender when I receive an e-mail with the subject line "New Swimmer". Is this possible?Thank you Todd, I was able to get it to work but I do have a few more questions...
1) When I tested this, at the top of the message, before any of the text I created, this showed up: This is a multi-part message in MIME format. --------------040406040801080102080500 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit
2) After the signature line, this showed up: --------------040406040801080102080500 Content-Type: image/jpeg Content-Transfer-Encoding: base64 Content-ID: /9j/4AAQSkZJRgABAQEAYABgAAD/4QAiRXhpZgAATU0AKgAAAAgAAQESAAMAAAABAAEAAAAA AAD/2wBDAAIBAQIBAQICAgICAgICAwUDAwMDAwYEBAMFBwYHBwcGBwcICQsJCAgKCAcHCg0K CgsMDAwMBwkODw0MDgsMDAz/2wBDAQICAgMDAwYDAwYMCAcIDAwMDAwMDAwMDAwMDAwMDAwM DAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAz/wAARCAC5ApUDASIAAhEBAxEB/8QA HwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQID AAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6 Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWm p6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/
(It was actually much longer than that)
Is there anyway that I can get that to go away? I obviously would prefer a nice, cleanly formatted e-mail to be sent just like I would send if I were creating it on the spot.
Thanks,
Evan -
i have an iPhone 4 that is stuck in a recovery loop. i try to restore it but it comes up with an error code (-1) towards the end of the restore. can someone please help me?
i just baught it off someone.. so im unsure if it was jailbroken or not.. and i dont think its under warranty now.
is there a software or something along those lines that would indicate what peice of hardware is faulty?
i baught it in this recovery loop thinking it was going to be a cheap way of getting a new phone without having to do much.. but now im starting to think its a brick -
I have a package that will be run from an Oracle Job and I want to process files in a directory and load each of the files into the database. I can load one specifically, but I want to loop through all the files in the directory and load each one at a time. Any help would be greatly appreciated.
LisaI followed the instructions but when I try to create the java, I get the following error (removed the numbers in front of each line):
and compile java source named "DirList"
ERROR at line 2:
ORA-29536: badly formed source: Encountered "<EOF>" at line 1, column 17.
Was expecting:
unknown command beginning "import jav..." - rest of line ignored.
unknown command beginning "public cla..." - rest of line ignored.
unknown command "{" - rest of line ignored.
unknown command beginning "public sta..." - rest of line ignored.
For a list of known commands enter HELP
and to leave enter EXIT.
unknown command beginning "throws SQL..." - rest of line ignored.
unknown command "{" - rest of line ignored.
unknown command beginning "File path ..." - rest of line ignored.
unknown command beginning "String[] l..." - rest of line ignored.
For a list of known commands enter HELP
and to leave enter EXIT.
unknown command beginning "String ele..." - rest of line ignored.
unknown command beginning "for(int i ..." - rest of line ignored.
unknown command "{" - rest of line ignored.
unknown command beginning "element = ..." - rest of line ignored.
For a list of known commands enter HELP
and to leave enter EXIT.
unknown command beginning "#sql { INS..." - rest of line ignored.
unknown command beginning "VALUES (:e..." - rest of line ignored.
unknown command "}" - rest of line ignored.
unknown command "}" - rest of line ignored.
For a list of known commands enter HELP
and to leave enter EXIT.
unknown command "}" - rest of line ignored.
and compile java source named "DirList"
ERROR at line 2:
ORA-29536: badly formed source: Encountered "<EOF>" at line 1, column 17.
Was expecting:
I researched it on metalink with no luck. I'm running it on my local database (Oracle8i Release 8.1.7.0.0 - Production JServer Release 8.1.7.0.0 - Production) so this may be the problem.
Help!
Lisa -
Sorry if this is a simple thing, I just can't seem to get it working. Ugh.
I'm following the example here:
http://www.sqlis.com/sqlis/post/Looping-over-files-with-the-Foreach-Loop.aspx
I got stuck on this section:
The important part of this dialog is highlighted and that is "Expressions". Click on the ellipses and we will be taken through to the next screen where we can start to create the expression. In the screen that follows, from the Property column
drop the list down and choose ConnectionString
In the 'Property Expressions Editor' window, i can't choose Property=ConnectionString. I just don't have it in my list. I'm using Visual Studio Ultimate 2010. The article is discussing SQL Server 2005 SSIS. Maybe something changed
b/w the two versions, or maybe I'm just not doing something right.
This is a big problem, b/c when I try to set the OLEDB, I get this.
Nothing is being feed into the Loop tool, and thus nothing can be mapped to the destination table in my DB.
I thought I could just make up my own, intuitive Expression, as such.
That doesn't do anything at all.
Can someone help me out?
Thanks!!
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it.So, I cancelled my other project and started over. I’m at this section in the original link that I posted:
The important part of this dialog is highlighted and that is "Expressions". Click on the ellipses and we will be taken through to the next screen where we can start to create the expression. In the screen that follows, from the Property column
drop the list down and choose ConnectionString
The whole problem here is that I don’t have the Property = ConnectionString in the Property Expression Editor window.
This is what I have.
There is no ConnectionString anywhere; it simply doesn't exist.
I did set the flat file source; that's fine now. But I have an exclamation mark in the OLEDB object (that doesn't seem right). No matter what I do, I can't get rid of it.
Knowledge is the only thing that I can give you, and still retain, and we are both better off for it. -
Loop through files locating specific data sets
Hi there,
I have been given the wonderful task of developing a setup which needs to do the following using JCAPS:
- open and read through various flat files in a folder. These flat files have random data sets in them, where a line starting with 01, marks the start of a data set and 99, marks the end of a data set. 03, marks the line where the account number can be found.
- retrieve the account number from the data sets and verify that against an external Oracle database table.
- if account number is found, then read the data set from the flat file and write the data set to an output file.
- if the account number is not found in the Oracle database table, just discard the dataset and continue with the next data set.
If you have any ideas on how to set this up, that would be much appreciated. No need to spell things out, but if anyone has worked on a scenario like this or similar using JCAPS to do the trick please let me know.
A scenario variant is that, instead of looking up the account number against the oracle database, we can use an if-statement like "if the currency is GBP, EUR, SEK, then take the data set from the input file and write the data set to the output file.
I can create an input OTD to read in the data sets, and an output OTD which creates the output file. There is a blank line between data sets. The structure of a data set looks like this:
01,CITIDIRECT,2978169,080922,0401,04010900,999,,2/
02,2978169,400515,1,080919,0000,USD,2/
03,57787314,USD,010,2191397371,,,015,2191397371,,,030,2191397371,,,040,2191397371,,,045,2191397371,,,060,2191397371,,,072,000,,,074,000,,,100,000,0,,400,000,0,,470,000,0,/
49,13148384226,2/
03,57787322,USD,010,1494730626,,,015,1494730626,,,030,1494730626,,,040,1494730626,,,045,1494730626,,,060,1494730626,,,072,000,,,074,000,,,100,000,0,,400,000,0,,470,000,0,/
49,8968383756,2/
03,57787349,USD,010,423382915,,,015,423382915,,,030,423382915,,,040,423382915,,,045,423382915,,,060,423382915,,,072,000,,,074,000,,,100,000,0,,400,000,0,,470,000,0,/
49,2540297490,2/
98,24657065472,3,8/
99,24657065472,1,10/
Thanks & Kind Regards,
RichardYes
1. Have a variable inside SSIS to get folder path. Set path based on your rule using an expression
like
(@[User::ProductName] == "Product" ? "\\....\Src\Product" : (@[User::ProductName] == "Product_NCP" ? \\....\Src\Product_NCP:..))
similary archive
(@[User::ProductName] == "Product" ? "\\....\Src\Product\Archive" : (@[User::ProductName] == "Product_NCP" ? "\\....\Src\Product_NCP\Archive" :..))
Please Mark This As Answer if it solved your issue
Please Vote This As Helpful if it helps to solve your issue
Visakh
My Wiki User Page
My MSDN Page
My Personal Blog
My Facebook Page -
Open Folder using Coldfusion and loop the files in the folder to the files names
Hello,
I have a folder with lot of .xls files in it.
I want to access a folder using some coldfusion code and then
"loop" through each .xls file in the folder, and while looping is
it possible to obtain the name of each .xls file?
Anyone who knows anything about this, your help would be
highly aprreciated.
ThxYup, I know all about it.
You can to if you read up on the <cfdirectory ...> tag
in the
documentation. A very hand tag for reading a directory, even
includes
the ability to filter to a certain file types. And produces a
record
set of the files with names, types and more. -
AIR application request a xml file from a server (but only legal users)
Hello guys
I am conceptualizing a project at the moment and I am thinking of using Adobe AIR to develop the application related to that project.
But it depends on the ability of AIR to fulfill my project requirements.
I want your ideas, please...
Imagine, I finished developing my application and compiled the distributed version using Adobe AIR.
And I am going to distribute the installation files of the intended application by burning on to a CD.
Imagine, the user who got that CD, installed it it on his computer and launched the application.
When the application is running, it needs to load a XML file which is saved in our company server.
Since it is an important data file I don't want to dispatch it with the installation CD.
So I am thinking of using URLRequest and URLLoader classes to load this XML file from server dynamically.
Lets imagine I did codes like this....
var request:URLRequest = new URLRequest("http://www.mycompany.com/sereverfolder/blabla.xml");
request.method = URLRequestMethod.POST;
var loader:URLLoader = new URLLoader();
loader.load(request);
But my question is, how safe hardcoding the path for the xml file in the class file as shown above.
Can anyone get access to xml file path by looking into installation files dispatched via CD?
Or is there any other safer way to do this?
Thank you for taking time to read my question and please add your input if you would like to share your knowledge.
cheerswrong forum.
-
Import Server detects file in Ready folder but does not process
I have a problem where MDIS detects a file in the ready folder, but it does not launch the import:
2772 2008/10/22 23:37:52.240 1.xml
Source file retrieval: Delta: 0.001451 seconds.
2772 2008/10/22 23:38:23.428 1.xml
Source file retrieval: Delta: 0.001151 seconds.
2772 2008/10/22 23:38:54.678 1.xml
Source file retrieval: Delta: 0.006335 seconds.
2772 2008/10/22 23:39:25.897 1.xml
Source file retrieval: Delta: 0.011595 seconds.
2772 2008/10/22 23:39:57.272 1.xml
Source file retrieval: Delta: 0.003268 seconds.
2772 2008/10/22 23:40:28.507 1.xml
Source file retrieval: Delta: 0.006276 seconds.
The log file shows the above. You can see it detects the file is there, but since it does not launch the import it does not process the file or even move it to the exception or archive folders. I have confirmed settings in the MDIS.ini file, e.g. password, etc. I can import the same file using Import Manager without any errors.
Please help!Hi Jitesh,
Your suggestion to try a different file type yielded positive reults. I created a simple tab delimited text file and import map and changed the existing Port from using XML based on the MATMAS05 schema to using the text file. The log shows:
6056 2008/10/23 17:32:34.066 [MDS=ROBLB2K1H2J Repos=JAN Master ClientSystem=U2K2_MDM Port=Materials_from_U2K2_MDM]: ImportTask: Task started. Chunk size[50000], No. parallel chunks[5]
4572 2008/10/23 17:32:34.144 [MDS=ROBLB2K1H2J Repos=JAN Master ClientSystem=U2K2_MDM Port=Materials_from_U2K2_MDM]: xStructuralTransformer: Thread started.
1160 2008/10/23 17:32:34.144 [MDS=ROBLB2K1H2J Repos=JAN Master ClientSystem=U2K2_MDM Port=Materials_from_U2K2_MDM]: xValueTransformer: Thread started.
4192 2008/10/23 17:32:34.144 [MDS=ROBLB2K1H2J Repos=JAN Master ClientSystem=U2K2_MDM Port=Materials_from_U2K2_MDM]: xImporter: Thread started.
4572 2008/10/23 17:32:34.144 [MDS=ROBLB2K1H2J Repos=JAN Master ClientSystem=U2K2_MDM Port=Materials_from_U2K2_MDM]: xStructuralTransformer: Thread finished; Start -> End: 0.000000000 seconds.
1160 2008/10/23 17:32:34.160 [MDS=ROBLB2K1H2J Repos=JAN Master ClientSystem=U2K2_MDM Port=Materials_from_U2K2_MDM]: xValueTransformer: Thread finished; Start -> End: 0.000000000 seconds.
4192 2008/10/23 17:32:34.660 Repository Load Successful. [JAN Master]: Delta: 0.085377 seconds.
4192 2008/10/23 17:32:35.958 [MDS=ROBLB2K1H2J Repos=JAN Master ClientSystem=U2K2_MDM Port=Materials_from_U2K2_MDM]: xImporter: Thread finished; Start -> End: 1.000000000 seconds.
6056 2008/10/23 17:32:36.458 [MDS=ROBLB2K1H2J Repos=JAN Master ClientSystem=U2K2_MDM Port=Materials_from_U2K2_MDM]: ImportTask: Task finished. Chunk size[50000], No. parallel chunks[5]
4192 2008/10/23 17:34:11.133 22355-3-1-10-23-08-06-32-57_081023-063303_999.xml
Source file retrieval: Delta: 0.006444 seconds.
4192 2008/10/23 17:34:11.242 Repository Load Successful. [JAN Master]: Delta: 0.028803 seconds.
4572 2008/10/23 17:34:43.825 22355-3-1-10-23-08-06-32-57_081023-063303_999.xml
Source file retrieval: Delta: 0.003005 seconds.
4572 2008/10/23 17:35:16.018 22355-3-1-10-23-08-06-32-57_081023-063303_999.xml
Source file retrieval: Delta: 0.010113 seconds.
4572 2008/10/23 17:35:47.742 22355-3-1-10-23-08-06-32-57_081023-063303_999.xml
Source file retrieval: Delta: 0.001994 seconds.
4572 2008/10/23 17:36:19.419 22355-3-1-10-23-08-06-32-57_081023-063303_999.xml
Source file retrieval: Delta: 0.012413 seconds.
The text file was immediately picked up and imported. This thus proves that (1) MDIS is functioning, (2) user / password is correct. Unfortunately you will also see from the log above that when I changed the port back to run on XML the problem has not been resolved.
Any ideas?
Maybe you are looking for
-
My ipone wont activate what can i do
Does anyone know why an iphone wont activate
-
How to check Navigational attribute in Planing book?
we have 5 characteristics and 1 navigational attribute in our MPOS. after loading data into PA and checking planning book, I do not see navigational attribute in the planninng book selection window. on the left side object selection and drill down my
-
An error occurred during the installation of assembly component
I am receiving this message when I try to install adobe reader (tried both reader X and reader 9) Same message came out on screen every time I try to install adobe reader program.... An error occurred during the installation of assembly component {B7
-
The Best kind of Laptob is suitable for InDesign CS4???
Hello I wanna to buy the best notebook for work with ID3 or higer (I prefer to work with ID4) but I,ve confused, beacause there are a lot of notebook from many companies and there are models. I need to help and some good advices to buy a great notebo
-
I have the folllowing types of taxcodes- 1.SRVTAX10 2.SRVVAT12 3.SRVVAT4 I am updating the pld of A/R credit memo.I want to uniquely identify all these taxcodes as SRV.