Sort records in a csv file in the descending order of time created attribute
I have an excel (.csv) file with the following column headers :-
Client,
SourceNetworkAddress
,TimeCreated,
LogonType
,User,
Message
Values like :- ABC, 10.5.22.27, 11/23/2014 9:02:21 PM, 10, testuser
The file is a combination of a report generated everyday using multiple scripts. The data is appended each day therefore, I would like to sort the final output file in descending order of time created (a combination of date and time) fetched from events
i.e. the latest record with the latest date and time should be at the top of the list.
I tried using the following command however, I get a sorted list according to the date but not the time. The command does not consider the AM/PM mentioned in the time instead simply sorts them as per numbers
Import-Csv "C:\Users\a\Desktop\report.csv" | sort Timecreated -Descending | Export-csv "C:\Users\a\Desktop\report_sorted.csv" -force -NoTypeInformation
So if I have a record with 9:02:21 PM(latest) and a record with time 10.44.10 AM on the same date, the command will sort the list with record 10:44:10 AM first and then record with 9:02:21 PM however it should be the opposite as per descending order.
Kindly help !!
Hi jrv,
Thanks for your response. However, I get errors while I run this command in Powershell :-
Import-Csv <file> | Select Client,SourceNetworkAddress,LogonType,User,Message,@{N='TimeCreated';E={[datetime]($_.TimeCreated)} | Sort TimeCreated -Descending | Export-csv <file> -force -NoTypeInformation
Missing expression after ','.
At line:1 char:150
Unexpected token 'LogonType' in expression or statement.
At line:1 char:151
Unexpected token ',' in expression or statement.
At line:1 char:160
Unexpected token 'User' in expression or statement.
At line:1 char:161
Unexpected token ',' in expression or statement.
At line:1 char:165
Unexpected token 'Message' in expression or statement.
At line:1 char:166
The hash literal was incomplete.
At line:1 char:174
Please help!
You are missing a second curly brace -
Import-Csv <file> | Select Client,SourceNetworkAddress,LogonType,User,Message,@{N='TimeCreated';E={[datetime]($_.TimeCreated)}} | Sort TimeCreated -Descending | Export-csv <file> -force -NoTypeInformation
Similar Messages
-
How to import an .csv file into the database?
and can we code the program in JSP to import the.csv file into the database.
It is better to use Java class to read the CSV file and store the contents in the database.
You can use JSP to upload the CSV file to the server if you want, but don't use it to perform database operations.
JSPs are good for displaying information on the front-end, and for displaying HTML forms, there are other technologies more suitable for the middle layer, back end and the database layer.
So break you application into
1) Front end - JSPs to display input html forms and to display data retrieved from the database.
2) Middle layer - Servlets and JavaBeans to interact with JSPs. The code that reads the CSV file to parse it's contents should be a Java Class in the middle layer. It makes use of Java File I/O
3) Database layer - Connects to the database using JDBC (Java Database Connectivity), and then writes to the database with SQL insert statements.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Keeping the above concepts in mind, first build a simple JSP and get it to work,
then research on Google , for Java File I/O , discover how to read a file,
Then search on how to readh a CSV file using Java.
After researching you should be able to read the CSV file line by line and store each line inside a Collection.
Then research on Google, on how to write to the database using JDBC
Write a simple program that inserts something to a dummy table in the database.
Then, read the data stored in the Collection, and write insert statements for each records in the collection. -
How to upload multiple .CSV files in the same time.
Hi Legends,
Can anyone please help me to resolve my issue?
This is very urgent and critical.
Description:
We have two users.1)edw_user_dump
2)prd_udm.
We need to upload the xx.csv file at the same time for these two users in oracle forms.
we have differentiated .csv file name based on the user names.
the main problem is in the sql loader command the xx.csv file name is created but the data is not captured in the server (kentucky) .
Below is my code for the upload.
-- To delete the Part_Mast.log file from the client
DECLARE
pid WEBUTIL_HOST.PROCESS_ID;
v_result PLS_INTEGER;
v_username varchar2(30) := GET_APPLICATION_PROPERTY(USERNAME);
BEGIN
v_result := WEBUTIL_HOST.Get_return_Code(pid ) ;
host('cat /dev/null > /tmp/'||v_username||'_'||'EDW_CF_IO_UPLOAD.log');
host('cat /dev/null > /tmp/'||v_username||'_'||'"CF615 IO Upload.csv"');
END;
DECLARE
l_success boolean:=FALSE;
l_bare_filename varchar2(100):=NULL;
v_username varchar2(30) := GET_APPLICATION_PROPERTY(USERNAME);
BEGIN
-- Delete the content of the log and bad file
host('cat /dev/null > /tmp/'||v_username||'_'||'EDW_CF_IO_UPLOAD.log');
--host('cat /dev/null > /tmp/Part_Mast.bad');
-- Upload the data file to Application Server
l_bare_filename := v_username||'_'||substr(:FIC_SOURCE,instr(:FIC_SOURCE,'\',-1)+1);
l_success := webutil_file_transfer.Client_To_AS_with_progress
(clientFile => :FIC_SOURCE
,serverFile => '/tmp/'||l_bare_filename
,progressTitle => 'Upload to Application Server in progress'
,progressSubTitle => 'Please wait'
,asynchronous => false
,callbackTrigger => null
IF l_success THEN
NULL;
ELSE
null;
END IF;
EXCEPTION
WHEN OTHERS THEN
RAISE Form_Trigger_Failure;
END;
DECLARE
v_username varchar2(30) := GET_APPLICATION_PROPERTY(USERNAME);
begin
host('cat /dev/null > /tmp/'||v_username||'_'||'EDW_CF_IO_UPLOAD.log');
host('cat /dev/null > /tmp/EDW_CF_IO_UPLOAD.bad');
end;
BEGIN
DECLARE
v_username varchar2(30) := GET_APPLICATION_PROPERTY(USERNAME);
v_password varchar2(30) := GET_APPLICATION_PROPERTY(PASSWORD);
v_connect_string varchar2(30) := GET_APPLICATION_PROPERTY(CONNECT_STRING);
a_host varchar2(500);
BEGIN
a_host :='/tmp/'||v_username||'_'||'"CF615 IO Upload.csv"';
host('sqlldr '||v_username||'/'||v_password||'@'||v_connect_string||' '|| 'control=/home/edw_bis/ctl/GLB_CF_IO_UPLOAD.CTL'||' '|| 'DATA=a_host'||' '|| 'LOG=/tmp/'||v_username||'_'||'EDW_CF_IO_UPLOAD.log SKIP=1 errors=200000 DIRECT=FALSE');
dbms_output.put_line(a_host);
END;
DECLARE
v_username varchar2(30) := GET_APPLICATION_PROPERTY(USERNAME);
begin
host('cat /dev/null > /tmp/'||v_username||'_'||'"CF615 IO Upload.csv"');
EXCEPTION
WHEN OTHERS THEN
RAISE Form_Trigger_Failure;
END;
end;
DECLARE
al_id3 ALERT;
al_button Number;
BEGIN
edw_user_dump.SANM_PRC_MERGE_CF_IO_UPLOAD(:global.v_plsql_res,:global.v_ins_rec,:global.v_upd_rec);
IF NVL(:global.v_plsql_res,0) = 0 and (:global.v_ins_rec !=0 OR :global.v_upd_rec != 0 ) then
al_id3 :=FIND_ALERT('ROWINS');
SET_ALERT_PROPERTY(al_id3,alert_message_text,' Process Completed Successfully!'||CHR(10)||' Rows Inserted : '||:global.v_ins_rec ||CHR(10)||' Rows Updated : '||:global.v_upd_rec);
al_button := SHOW_ALERT( al_id3 );
ELSIF (:global.v_plsql_res IN(-1,0) or :global.v_plsql_res > 0) and (:global.v_ins_rec =0 and :global.v_upd_rec = 0 and :global.v_del_rec =0 ) then
al_id3 :=FIND_ALERT('ROWINS');
SET_ALERT_PROPERTY(al_id3,alert_message_text,' Process Failed. Please Download the Log File '||CHR(10)||' Rows Failed : '||:global.v_plsql_res||CHR(10)||' Rows Inserted : '||:global.v_ins_rec ||CHR(10)||' Rows Updated : '||:global.v_upd_rec);
al_button := SHOW_ALERT( al_id3 );
ELSE
al_id3 :=FIND_ALERT('ROWINS');
SET_ALERT_PROPERTY(al_id3,alert_message_text,'Please Download the Log File '||CHR(10)||' Rows Failed : '||:global.v_plsql_res||CHR(10)||' Rows Inserted : '||:global.v_ins_rec ||CHR(10)||' Rows Updated : '||:global.v_upd_rec );
al_button := SHOW_ALERT( al_id3 );
END IF;
EXCEPTION WHEN OTHERS THEN
RAISE Form_Trigger_Failure;
END;
DECLARE
v_username varchar2(30) := GET_APPLICATION_PROPERTY(USERNAME);
begin
host('cat /dev/null > /tmp/'||v_username||'_'||'"CF615 IO Upload.csv"');
host('rm -rf /tmp/'||v_username||'_'||'"CF615 IO Upload.csv"');
end;
Thanks in advance!
Thanks,
MadhusudhananMadhusudhanan,
A couple of observations. First; always list your exact Forms version (eg; 10.1.2.0.2 not 10g R2). In most cases, the solution is different depending on the Forms version. Second; why must you use Forms to kick off a SQL Loader process? This is a server-side process and should be initiated by a server side process. If you absolutely must use Forms to kick off the process, again we need your Forms version in order to offer any solutions. Based on your code sample, I can asusme you are at least using Forms 9i becuase you are using WebUtil.
Is your Database and Application Server the same physical computer? If they are not, this would explain why your HOST command isn't working because HOST runs against the Application Server not the Database server.
Third; have you considered using and External Table (if your RDBMS version supports them) for each of the files you are attempting to upload? In this instance, it would be helpful to know your RDBMS version as well. External Tables can be a little frustrating to set up the first time, but as with any new construct you use - it gets easier the more you use it.
Fourth; are you getting any errors in your log file(s)? If so, what are the errors? Please list the full error message if you have one.
Finally, with respects to your statement:
Posted: Mar 18, 2011 2:30 PM - Madhu This is very urgent and critical.>
You have to understand that forum contributers are all volunteers - this is not our full-time job. If your issue is truely urgent I suggest you open a Service Request (SR) with Oracle Support! ;-)
Craig B-)
If someone's response is helpful or correct, please mark it accordingly. -
I also have a .csv file with the name of a jpeg in one column and a text description of each jpeg in a second column. Is there a way to automatically insert one jpeg (photo) and its corresponding text, each pair on one page, into a Indesign document?
I would also recommend to write the description into the meta data. This would allow to place a text frame above the image and it is possible to add meta information and file name automatically together with the image, when you place it or even in a prepared template.
Meta data information can be written easily in Bridge in the Meta File Workspace. -
Hi Everyone,
I want to create a powershell script file
1) Check a directory and upload all termset csv files into the SharePoint local taxonomy.
2) Input paramaters - directory that containss termset csv files, Local Termstore to import to,
3) Prior to updating get a backup of the existing termstore (for rollback/recovery purposes)
4) Parameters should be passed in via XML file.
Please let me know how to do it.
Regards,
SrinivasHi,
Please check this link
http://termsetimporter.codeplex.com/
Please remember to click 'Mark as Answer' on the answer if it helps you -
No luck syncing .cvs or vcs.csv files on the new iCal
This is so annoying, I can no longer sync my University timetable with the new iCal. Does anyone know why I can no longer add .cvs or .vcs.csv files to the new iCal? Thanks
Presumably you've tried trashing your preferences . . ?
Have you tried trashing the App and reinstalling it?
Andy -
SQL* Loader Loading specific column from CSV file to the table
Dear All,
Iam Loading specific column from .CSV file to the oracle table.
Could pls help how i can load only that cols into the table
Eg: CSV file having id, Frst_name,Last_name, Address,Phone,Insurance etc
out of this I want to load only Frst_name,Last_name columns to oracle table say fname and lname.
Thanks in Adv.
JunuLily,
I made some changes to your table def but you will get the idea
-- Table EMPLOYEE
CREATE TABLE EMPLOYEE
EMPID NUMBER NOT NULL,
EMPNICKNAME VARCHAR2(10 BYTE) NULL,
FNAME VARCHAR2(20 BYTE) NOT NULL,
MI VARCHAR2(20 BYTE) NULL,
LNAME VARCHAR2(20 BYTE) NOT NULL,
FULLNAME VARCHAR2(20 BYTE) NOT NULL,
HIREDATE DATE DEFAULT SYSDATE NOT NULL
-- data file employee.dat
1,amy,b,amy b
2,cindy,d,cindy d
3,eric,f,eric f
4,gary,h,gary
-- Control file : Employee.ctl ( you can use truncate, replace or append , see sqlldr for more options)
load data
Truncate into table employee
fields terminated by ","
optionally enclosed by '"'
TRAILING NULLCOLS
empId INTEGER EXTERNAL,
FName char(20),
LName char(20),
FullName char(30)
now to load use following or you can speicify infile in control fle
sqlldr username/passowrd control=employee.ctl data=employee.dat log=employee.log
{code}
Hope this help.
Regards -
IPhoto is changing the sequence of the photos I have in my file for the book I want to create. Help?
iPhoto is changing the sequence of the photos I have in my file for the book I want to create. Help?
The problem has been solved. As it turns out, I was highlighting the photos in the album I wished to create. And then generating the request for a book. This caused the photo images when loaded into the book format to revert to the sequence used when I originally loaded them into the album. Not the edited sequence. Turns out that I did not need to highlight the photos. By eliminating this step, they remained in the edited sequence.
Thank you for responding. Warmly, Janice huseby -
When restoring Time Machine after a failure, does it also restore the files to the same order, for example - within iPhoto I have pics/ videos in folders and sub folders organised as family/events etc - will it go back to this exact order after a restore?
Hi Stavros0203,
When restoring your entire system from a Time Machine backup, it is restored to the state it was when that backup was made. See this article for reference -
OS X Yosemite: Recover your entire system
Thanks for using Apple Support Communities.
Best,
Brett L -
Why we cannot view the move orders which are created by the other user?
Hi,
Can any one explain me the below point.
Why we cannot view the move orders which are created by the other user.
Ex: From ABC user I have created a Move Order: 123456 in M1 Org If I login to the other user BCD I cannot view the same move order in the same ORG i..e M1 ORG.
But we can perform the transact move orders for move orders which is created by the other user.
What could be the reason? Why the system is restricting this? Is this the standard functionality of oracle?
Can any one explain?
Regards,
Kv.This is due to a security function which allows to see the move orders created by the user only.
Please refer to the meta link document ID 280131.1 for detailed description.
I have tried these steps in one of my test instances long back and was able to see move orders created by other users.
Thanks
Karthik. -
Split records into Multiple csv files using a Threshold percentage
Hi Gurus,
I have a requirement to split the data into two csv file from a table using a threshold value(in Percentage) .
Assume that If my source select query of interface fetches 2000 records , I will provide a threshold value like 20%.
I need to generate a csv1 with 400 records(20% of 2000) and the rest of the records into another csv2.
For implementing this I am trying to use the following process.
1) Create a procedure with the select query to get the count of records.
Total Records count: select count(1) from source_table <Joins> <Lookups> <Conditions>;
2) Calculate the Record count to first CSV using the threshold_value.
CSV1_Count=Total records count /threshold_value
3) Create a view that fetches the CSV1_Count(400) records for CSV1 as follows.
Create view CSV1_view as select Col1,Col2,Col3 from source_table <Joins> <Lookups> <Conditions>
Where rownum<=CSV1_Count;
4) Generate CSV1 file using View 'CSV1_View'
5) Generate CSV2 File using the Interface with same select statement (with columns ) to generate a CSV.
select Col1,Col2,Col3 from source_table ST <Joins> <Lookups> <Conditions>
Left outer join (Select Col1 from CSV1_View ) CS on CS.Col1=ST.Col1 where CS.Col1 is null;
Which gives the Total records minus the CS1_View records.
The above process seems a bit complex and very simple . If any changes in my Interface I also need to change the procedure (counts the no:of records).
Please provide your comments and feedback about this and looking for your inputs for any new simple approach or fine tune the above approach.
Thanks,
ArjunArjun,
This are my thoughts and Lets do it in 3 Steps
Step 1. ODI Procedure
Drop table Temp_20 ;
Create table Temp_20 as select * from table where rownum < ( SELECT TRUNC( COUNT(1) /5) FROM TABLE ) .
[ ** This way iam fetching approx 20% of the table data and loading into Temp table . 1/5 th is 20% so i am dividing count by 5
I don't believe View will help you especially with RowNum as if you run the same query with rownum < N the rows order might differ . so Temp table is great ]
Step 2 . Use OdiSqlUnload with select columns from temp_20
Step 3 . Use again OdiSqlUnload with select columns from table where ( uk keys ) not in ( selecy uk_keys from temp_20)
[** this way you can pick the remaining 80% ** and the data will be not repeat itself across 20% and 80% , as might happen with view ]
what do you think ? -
Search and Delete a specific record from a CSV file
Hi All,
I am new to java . I want to search for the records from CSV file and delete the row form the file.
Below is my Sample .csv
100||a100||1b100
200||b200||dc300
200||bg430||ef850
400||f344||ce888
Now I need some help in below requirements.
1.How to delete a record having value 200 and b200?
2.If record already exists how to update the existing record with new values?
Please share your ideas or give me some code snippet..
Thanks in AdvanceIn that case Do i need to write the entire contents of my file to a hash table(sumthng like this) and modify the Second row in my case with the new values..
is it possible??I would have done like this (though there maybe better methods)
1- create a class representing the record.
class Record{
String field1;
String field2;
String field3;
// and so on....
//setters
public void setFeild1(String str){
field1=str;
// and so on....
//getters
public String getFeild1(){
field1=str;
// and so on....
public String toString(){
return(field1+"||"+field2+"||"+field3);
}//end class2- then create an ArrayList meant to have objects of this class (Generics).
3- read from the file , create a new Record Object and add that to the ArrayList
4- perform operations on the ArrayList (you can add new records, and delete record, update......)
5- write the record back to file using 'toString()' method.
is there ne sample code available for thisdon't know, but you rarely get full code on forums.....outline given can be followed
Thanks!
Edit: It appears that 'r035198x' and me have the same point. This shows that this methodology is almost a standard way( if we ignore the Random access files.....)
Edited by: T.B.M on Jan 13, 2009 2:39 PM -
Processing Several Records in a CSV File
Hello Experts!
I'm currently using XI to process an incoming CSV file containing Accounts Payable information. The data from the CSV file is used to call a BAPI in the ECC (BAPI_ACC_DOCUMENT_POST). Return messages are written to text file. I'm also using BPM. So far, I've been able to get everything to work - financial documents are successfully created in the ECC I also receive the success message in my return text file.
I am, however, having one small problem... No matter how many records are in the CSV file, XI only processes the very first record. So my question is this: Why isn't XI processing all the records? Do I need a loop in my BPM? Are there occurrence settings that I'm missing? I kinda figured XI would simply process each record in the file.
Also, are there some good examples out there that show me how this is done?
Thanks a lot, I more than appreciate any help!Matthew,
First let me explain the BPM Steps,
Recv--->Transformation1->For-Each Block->Transformation2->Synch Call->Container(To append the response from BAPI)->Transformation3--->Send
Transformation3 and Send must be outside Block.
Transformation1
Here, the source and target must be same. I think you must be know to split the messages, if not see the below example
Source
<MT_Input>
<Records>
<Field1>Value1</Field1>
<Field2>Value1</Field2>
</Records>
<Records>
<Field1>Value2</Field1>
<Field2>Value2</Field2>
</Records>
<Records>
<Field1>Value3</Field1>
<Field3>Value3</Field3>
</Records>
</MT_Input>
Now , I need to split the messages for each Records, so what I can do?
In Message Mapping, choose the source and target as same and in the Messages tab, choose the target occurrence as 0..Unbounded.
Now,if you come to Mapping tab, you can see Messages tag added to your structure, and also you can see <MT_Input> occurrence will be changed to 0..unbounded.
Here is the logic now
Map Records to MT_INPUT
Constant(empty) to Records
Map rest of the fields directly. Now your o/p looks like
<Messages>
<Message1>
<MT_Input>
<Records>
<Field1>Value1</Field1>
<Field2>Value1</Field2>
</Records>
</MT_Input>
<MT_Input>
<Records>
<Field1>Value2</Field1>
<Field2>Value2</Field2>
</Records>
</MT_Input>
<MT_Input>
<Records>
<Field1>Value3</Field1>
<Field3>Value3</Field3>
</Records>
</MT_Input>
</Message1>
</Messages>
raj. -
Read and sort a semicolon seperated csv file
Hi all,
I need to sort the given csv file columnwise (alphabatically)... which means the corresponding row/rows also have to be sorted. need help.
the csv file looks like:
Wear;9;;;
;;;Image;1
;;;Area;2
;;;ner;3
;;;Content;1
;;;BContainer;1
;;;View;1
VIEW;10;;;
;;;get;8
;;;ner;1
;;;View;1
ACTIVE;21;;;
;;;Image;1
;;;get;7
;;;Area;4
;;;ner;1
;;;de.vw.mqbkombi.widgets.TransitionContainer;3
;;;Aget2;2
;;;ner2;1
;;;Sget;1
;;;iView;1
The idea is to first sort Wear,VIEW, ACTIVE alphabatically (columnwise), making sure all the rows that they have are also moved.
after that, alphabatically sort all the in-between rows.
would appreciate the help from any expert on csv, excel table.doremifasollatido wrote:
If you know that your columns never have embedded semicolons, you could call String.split to split each row at semicolons. Although you could leave the results in the arrays that String.split gives, you should probably create a Java class to represent each row. It will make your code easier to understand, and you could also more easily represent numeric values as numbers such as primitive int values (so that they sort correctly as numbers instead of as Strings). Then create a List of instances of that Java class, and sort the list using Collections.sort and a custom java.util.Comparator.Or indeed make it Comparable; but I'd agree this is definitely the way to go.
Also, assuming that all your "secondary" rows, such as:
;;;Image;1
;;;Area;2
refer to the preceding "primary" (Wear;9;;;), I'd suggest you fill in the unsupplied values for them. That will allow you to sort them all in one go. It might also be worth including a type in your "Row" class (maybe as an enum); alternatively you could have two different subclasses of a common Row class.
Winston -
Cannot find the age in years csv file in the application server
Hi,
I want to upload the age in years hierarchy from the CSV file AGE_E.CSV that should be available in the application server.
When i try to preview file in the info package, a message is thrown which says that the file does not exist, which means that the perticular file is not present in the application server.
Can any one of you please tell me where else can i find this file.
Regards,
Chetana.dear Chetana,
transaction AL11 can be used to check the file location.
check DIR_TEMP or DIR_TRANS. or ask basis where the file is put.
hope this helps.
Maybe you are looking for
-
How can I see my Itunes store credit history
How can I see my store credit history? I know I can look at purchases, but I would like to know how I can see my beginning store credit balance and each purchase made there after...
-
FireFox crashes after a few seconds
Im Running windows 7 Ultimate 32 Bit. Fire fox works for about 10 seconds and then just doesnt do anything. FireFox stays open, but i can open new tabs but nothing works when i click it, the bookmarks tabs do nothing, refresh does nothing, google sea
-
I have got Real Player Plus and one of the features of it is being able to download any video. Now to do that you mouse on thevideo and a download icon appears on the top right of the video. Recently it has stopped working. I have all my Real Player
-
Slideshow in iphoto, once shared send to idvd, becomes lower quality
Hello, I made a slideshow in iphoto 8.1.2 that looks great, using the "snapshot" theme. I need to be able to play this in a regular dvd player attached to a tv. I hit share > send to idvd. In idvd 7.0.4, I set up my menu page (used "shelves" theme),
-
HR - Function module to change trip status
Hi, I'm trying to change trip approval status to '5' and settlement status to '3' using standard BAPI available BAPI_TRIP_CHANGE_STATUS and getting success message that trip has been changed. But when i go to trip approval status shown as '2' and set