CSV File Creation
I have 2 queries related to CSV file creation using ODI.
1) I need to generate CSV file out of data contained in Oracle DB Table. When the data gets extracted into CSV file, it needs to be sorted on one particular column ( e.g Account_Number). Is this possible?
2) When the data gets extracted in CSV format, the DATE is shown as ' 2008-06-06 00:00:00.0' . How can I get the date to be stored in just 'DDDD-MM-YY' format?
Thanks
KS
check out the OdiUnloadSql component. I have used it to create a csv file, but have not gone into date format and sorting detail. you might have to look into that.
Similar Messages
-
ODSI csv file creation from a data service that read data from many tables.
Hi,
We have few dataservices running in our enviroment. Now we have a requirement to save the data as CSV files on demand. My question, can we do this using ODSI 10gR3?
I know we can read the csv file and create a physical data service. We have a compleded a use case that data comes in CSV file and saved to Oracle table when required.
Now how to do the reverse is my question. I am not findind any help in documents for this. I have opend a case with Oracle support for this, looks like they are not understading and taking it as a issue in ODSI 10gR3.
Please help.
Thanks in advance.
Rahulwe have a requirement to save the data as CSV files on demand. My question, can we do this using ODSI 10gR3? Yes. But you have to write the java code to do it. Create a class with a static method that takes two arguments - an array of XmlObject (or of the specific type that extends XmlObject that is created by compiling the schema) and a filename - and writes the values of the children of the array elements into your CSV file.
Put that class in a jar and put the jar in DSP-INF/lib, then create a physical datasource -> Java from that method, make sure you specify that resulting datasource procedure is a libraryProcedure.
So you'll have to do a little homework on ODSI Java Functions with XmlObject arguments.
- Mike -
Web.show_document on XP SP2 does not open a .csv file
XP SP2
XE db
Forms 10g (9.0.4)
form90.conf:
AliasMatch /query_exports/(..*) "C:/mydir/query_exports/$1"
.csv file creation statement:
lv_ext_dir := 'query_export/';
lv_file_id := text_io.fopen(lv_ext_dir||lv_file_name,''w');
so far so good. the file is created and written to.
text_io.fclose(lv_file_id);
now, the open of the file created:
lv_temp_dir := 'query_exports/' -- note same value as above
web.show_documents(lv_temp_dir||lv_file_name);
at this point a new browser window is opened with HTTP 404 Not Found error. The URL is:
http:/local_machine:8888/forms90/query_exports/"lv_file_name"
if I go the following:
lv_temp_dir := 'file:///c:/mydir/query_exports/'
web.show_documents(lv_temp_dir||lv_file_name);
The new browser does not open and I do not get an error. The file is still written to.
If i use:
lv_temp_dir := 'query_exports/' -- note same value as above
web.show_documents(lv_temp_dir||lv_file_name);
and the new browser open with the HTTP 404 Not found message and I manually type in:
file:///c:/mydir/query_exports/"lv_file_name",
The .csv file opens correctly.
Why is this behavior happening?
TIA for any help you can provide,
Ed.Whats the machine name ?
Add the machine name/networkdomain name as a trusted site under
Internet Explorer ->Securitysettings
Rajesh ALex -
Creation of CSV file on client machine with data from forms
Hi,
My requirement is to generate a CSV file(or .XLS) on the client machine ie local drive with the details shown in a form.
Oracle version -
Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
PL/SQL Release 11.1.0.7.0 - Production
CORE 11.1.0.7.0 Production
TNS for Solaris: Version 11.1.0.7.0 - Production
NLSRTL Version 11.1.0.7.0 - Production.
I have searched the web for last couple of days and got to know that TEXT_IO is to be used to generate files on client machine. However, when I tried TEXT_IO, it was not able to generate the file on client rather it was able to generate on database server. After further browsing on this, there was a link which said that we need to use CLIENT_TEXT_IO to generate file on client side. For this, i was required to subclass the webutil.pll which i did and corrected the code to use CLIENT_TEXT_IO. The form was unable to compile and was not able to find "webutil_core" package.
I am very confused with the disparity in the information available on the web as in what to use to generate a file on client side. If anyone has use it in the past, can he/she please detail what to use to get things sorted.
Thanks,
ROracle version - Oracle Database 11g Enterprise Edition Release 11.1.0.7.0 - 64bit Production
PL/SQL Release 11.1.0.7.0 - Production
CORE 11.1.0.7.0 Production
TNS for Solaris: Version 11.1.0.7.0 - Production
NLSRTL Version 11.1.0.7.0 - Production. >
So, what is your FORMS version. This is more important than your database version.
My requirement is to generate a CSV file(or .XLS) on the client machine ie local drive with the details shown in a form. Depending on your Forms version, you would use TEXT_IO (if Forms 6i running in Client/Server mode) or WebUtil (if Forms 9i or higher).
I'm going to guess that you are at least using Oracle Forms 9i since you stated that your attempt at using TEXT_IO produced a file on the DB server.
There is more to using WebUtil than just attaching the WEBUTIL.PLL. If you had performed a simple search of the Forms Help System would have found numerous WebUtil topics to include: Introduction to WebUtil, Configuring WebUtil, Using WebUtil in Your Applications and the WebUtil User's Guide. If you have Oracle Forms release 10g or higher, WebUtil is included when you installed Forms, however, you do need to configure your installation to use WebUtil and you must download the Java COM Bridge (jacob.jar) from Source Forge. Take a look at the Configuring WebUtil Forms Help topic to find out which version of the Java COM Bridge you will need to download.
After you have successfully configured WebUtil, take a look at the Using WebUtil in Your Applications topic to find out how to implement WebUtil in a form.
Searching the Internet for answers is great, but don't forget the look at the Forms Help System because the majority of your questions can be answered there. :)
Lastly, configuration of WebUtil is primarily done on your Application Server (AS). However, if you plan to perform preliminary runtime testing by running your Form from the Forms Builder, then you will configure your local runtime to support WebUtil as well as configure your AS. The steps are exactly the same. A common mis-step is to skip a step during the configuration because you don't think the step applies. Take a look at the Forms Help Runtime Setup Checklist topic for a list of step you need to complete in order to enable WebUtil.
Hope this helps,
Craig B-)
If someone's response is helpful or correct, please mark it accordingly. -
HI,
I need a help for creating SSIS package which has flat file source (specifically , delimited csv files) which are placed in a shared folder
having file names as File_1_2015-04-30 08:54:13.900.csv, File_2_2015-04-30 07:54:13.900.csv.
So, I wanted to use foreach loop to find out the oldest file amongst that(through script task) and need to transform that file in Oracle Destination. After that, I want to archive that file and need to continue this action uptil the most recent file gets
loaded in destination.
So, Could you guys please guide me?
Thanks in advanceI'd say you need two loops, one to cycle through all the files and collect their names. Then sort them based on your criterion
Perhaps you can start by seeing http://www.sqlis.com/sqlis/post/Looping-over-files-with-the-Foreach-Loop.aspx
Collect the file names into a Recodset Destination, then shred it http://www.sqlis.com/sqlis/post/Shredding-a-Recordset.aspx
perhaps you need an outer loop to iterate over each file to compare the files for dates.
Do each piece and post here where you needed to stop.
Arthur
MyBlog
Twitter -
Csv file uploading for database table creation
Hi there,
I'm in the process of making an application that will be able to upload a csv file and create a table based on the same file. As of now, I have managed to make my application upload a csv file into the database. My problem now is to transfer the data in the csv into a table. If there is a function that can do this, please let me know. But as of now, I have tried all that I can but in vain. I would appreciate any assistance rendered as to how I can go about this.
Kind regards,
Lusunthahai Lusuntha ,
Go to search forum and type "upload within html db".here u will find the required information ,as well as the code.go for each topic in the search result. -
PowerShell - Bulk-New-ADUser Creation via a csv file
This script creates bulk AD users via a csv file. The script creates & configures the user accounts correctly even though the following error message appears. How can I correct this?
Set-Locaiton : Cannot find path 'AD:Mydomain,OU=MyDomain,DC=My,DC=Domain,DC=org' because it does not exist.
Import-Module ActiveDirectory
$csv = Import-CSV -Path "C:\Temp\CreateUsers.csv"
cd AD:
set-location -path "OU=MyDomain,DC=My,DC=Domain,DC=org" -PassThru
foreach($Item in $csv){
$newUserID=@{
Name=$item.userID
Description=$item.description
GivenName=$item.UserID
surName=$item.UserID
DisplayName=$item.UserID
UserPrincipalName=$item.UserID + "@MyDomain.org"
EmployeeID=$item.Owner
ScriptPath="login.cmd"
Try{
New-ADUser @newUserID -ErrorAction Stop -AccountPassword (ConvertTo-SecureString $Item.Password -AsPlainText -Force) -PassThru
Enable-ADAccount -Identity $item.userID
Set-ADUser -Identity $item.userID -ChangePasswordAtLogon $true
Write-Host "UserID $($item.UserID) created!" -ForegroundColor green
Catch{
Write-Host "There was a problem creating UserID $($item.UserID). The account was not created!" -ForegroundColor Red
set-location -path "c:\temp"
mamadukesSlight variation, populated from the AdventureWorks SQL database rather than a CSV
Import-Module "SQLPS" -DisableNameChecking
New-PSDrive -Name AWDB -PSProvider SQLServer -ROOT SQLSERVER:\sql\localhost\default\databases\adventureworks2012
Set-Location AWDB:\Tables
$SQLText = "SELECT e.BusinessEntityID, p.Title, p.FirstName, p.MiddleName, p.LastName, p.Suffix, "+
"e.JobTitle, d.Name AS Department, d.GroupName, edh.StartDate, e.LoginID"+
" FROM HumanResources.Employee AS e"+
" INNER JOIN Person.Person AS p ON p.BusinessEntityID = e.BusinessEntityID"+
" INNER JOIN HumanResources.EmployeeDepartmentHistory AS edh ON e.BusinessEntityID = edh.BusinessEntityID"+
" INNER JOIN HumanResources.Department AS d ON edh.DepartmentID = d.DepartmentID"+
" WHERE (edh.EndDate IS NULL)"+
" AND (p.FirstName ='Brian')"
$Query = Invoke-SQLCmd -Query $SQLText
$Password = "P@assword1"
foreach($Item in $Query)
$LoginID=$Item.LoginID
$LoginID="CORP\"+($LoginID).Substring(16)
$newUserID=@{
Name=$item.FirstName+$Item.LastName
Description="This is a test of a bulk user add"
GivenName=$item.FirstName
Surname=$item.LastName
DisplayName=$item.FirstName+" "+$Item.LastName
UserPrincipalName="$($item.FirstName+"."+$Item.LastName)@corp.contoso.com"
EmployeeID=$item.BusinessEntityID
ScriptPath='login.cmd'
Company="Contoso"
Department=$Item.Department
EmailAddress="$($item.FirstName+"."+$Item.LastName)@corp.contoso.com"
Title=$Item.JobTitle
$TargetOU="OU="+$item.Department+",DC=corp,DC=contoso,DC=com"
Try{
$newUserID
New-ADUser @newUserID -Path $TargetOU -ErrorAction Stop -AccountPassword (ConvertTo-SecureString $Password -AsPlainText -Force) -Passthru
Enable-ADAccount -Identity $newUserID.Name
Set-ADUser -Identity $newUserID.Name -ChangePasswordAtLogon $true
Write-Host "UserID $($newUserID.Name) created!" -ForegroundColor green
Catch{
Write-Host "There was a problem creating UserID $($item.UserID). The account was not created!" -ForegroundColor Red
No need to be on the PSDrive AD:, the OU can be specified in the Path parameter in the New-ADUser.
This was done on a member server with SQL Server (not a domain controller).
Thanks for your help -
Memory problem with loading a csv file and displaying 2 xy graphs
Hi there, i'm having some memory issues with this little program.
What i'm trying to do is reading a .csv file of 215 mb (6 million lines more or less), extracting the x-y values as 1d array and displaying them in 2 xy graphs (vi attacked).
I've noticed that this process eats from 1.6 to 2 gb of ram and the 2 x-y graphs, as soon as they are loaded (2 minutes more or less) are really realy slow to move with the scrollbar.
My question is: Is there a way for use less memory resources and make the graphs move smoother ?
Thanks in advance,
Ierman Gert
Attachments:
read from file test.vi 106 KBHi lerman,
how many datapoints do you need to handle? How many do you display on the graphs?
Some notes:
- Each graph has its own data buffer. So all data wired to the graph will be buffered again in memory. When wiring a (big) 1d array to the graph a copy will be made in memory. And you mentioned 2 graphs...
- load the array in parts: read a number of lines, parse them to arrays as before (maybe using "spreadsheet string to array"?), finally append the parts to build the big array (may lead to memory problems too).
- avoid datacopies when handling big arrays. You can show buffer creation using menu->tools->advanced->show buffer allocation
- use SGL instead of DBL when possible...
Message Edited by GerdW on 05-12-2009 10:02 PM
Best regards,
GerdW
CLAD, using 2009SP1 + LV2011SP1 + LV2014SP1 on WinXP+Win7+cRIO
Kudos are welcome -
Automatic Deprovisioning of AD resource Accounts from CSV file attribute
My scenario is somewhat like this,
I have a CSV flatfile Active Sync which contains the following columns:
accountId,firstname,lastname,department,location,region
ausmani,Arsalan,Bhagwan,Uphone,Milpitas,US
aahmed,Aftab,Singh,Telenor,Cairo,EMEA
hkhan,Hello,Khan,Lahore,Dublin,EMEA
I have created a role and has assigned AD resource to it. I have hardcoded this role in the waveset.roles field name in my creation form.
When I start FlatFileActiveSync, these above mentioned 3 accounts are created in IDM and they are also assigned AD role, and hence they are automatically provisioned to AD as, due to the fact that I am assigning resource on role base.
I am using Update User workflow in my poll-workflow configuration in my Flatfile synchronization policy.
Currently I am able to automatically provision a account from CSV file towards IDM and towards AD. All this process is automatic.
My problem is that how can I automatically disable and deprovision accounts via a CSV. What I should include in CSV that IDM will know that this account has to be disable and deprovision from resource??? Moreover, which workflow I have to use?
Thanks,
Farhan.Even I am struck at the same place. Please let me know if you find out.
Thanks you in advance
Prabhu -
Hi Guys
I have two csv files with the following headers and I need to import both files into the script to check whether the StaffCode is present in the Creation/Renewal of Contract csv in a DisableAccount Script so I can stop any action to disable the account as
the staff has renewed the contract with the company so the account should not be disabled.
However my accounts are still being disabled. I am not sure now to construct the query so that it detects that the account is to be left alone if the staffcode is present in both files
I does recognize that the $staffcodeN in the renewal file matches the $staffcode in the termination file
but still proceeds to disable or set an expiry date to the account anyway based on the termination file.
How do I stop it from doing that?
1)In the Creation/Renewal of contract file the following headers are present
- TranCode,StaffCode,LastName,FirstName,SocialSecurityNo,DateJoin,Grade,Dept,LastUpdateDate,EffectiveDate
2)In the Disable of contract file the following headers are present
- TranCode,StaffCode,LastName,FirstName,SocialSecurityno,LastDateWorked,Grade,Dept,LastUpdateDate,
My data is not very clean , I have a-lot of special characters such as = , ' ,/ and \ characters to remove first before i can compare the data
Thanks for the help in advance.
Yours Sincrely
Vicki
The following is a short snippet of the code
$opencsv = import-csv "D:\scripts\Termination.csv"
$opencsv2 = import-csv "D:\scripts\RenewContractandNewStaff.csv"
foreach ($usertoaction in $opencsv)
$Trancode = $usertoactionTranCode
$StaffCode = $usertoaction.StaffCode.replace("=","").replace('"','')
$LastName = [string]$usertoaction.LastName.Replace("/","\/").Replace(",","\,")
$FirstName = [string]$usertoaction.FirstName.Replace("/","\/").Replace(",","\,")
$socialsecurityno = $usertoaction.SocialSecurityNo.replace("=","").replace('"','')
$DateJoin = $usertoaction.DateJoin.replace("=","").replace('"','')
$LastDateWorked = $usertoaction.LastDateWorked.replace("=","").replace('"','')
$Grade = [string]$usertoaction.Grade
$Dept = [string]$usertoaction.Dept
$LastUpdateDate = $usertoaction.LastUpdateDate.replace("=","").replace('"','')
$AccountExpiry = [datetime]::Now.ToString($LastDateWorked)
foreach ($usertoaction2 in $opencsv2)
$TrancodeN = $usertoaction2.TranCode
$StaffCodeN = $usertoaction2.StaffCode.replace("=","").replace('"','')
$socialsecurityNoN= $usertoaction2.SocialSecurityNo.replace("=","").replace('"','')
$DateJoinN = $usertoaction2.DateJoin.replace("=","").replace('"','')
$GradeN = [string]$usertoaction2.Grade
$DeptN = $usertoaction2.Dept
$LastUpdateDate = $usertoaction.LastUpdateDate.replace("=","").replace('"','')
$EffectiveDate = $usertoaction.EffectiveDate.replace("=","").replace('"','')
$LastName2 = [string]$usertoaction2.LastName.Replace(",", "").Replace("/","").trim()
$FirstName2 = [string]$usertoaction2.FirstName.Replace("/","").trim()
# Use DirectorySearcher to find the DN of the user from the sAMAccountName.
$Domain = [System.DirectoryServices.ActiveDirectory.Domain]::GetCurrentDomain()
$Root = $Domain.GetDirectoryEntry()
$Searcher = [System.DirectoryServices.DirectorySearcher]$Root
$Searcher.Filter = "(sAMAccountName=$samaccountname)"
$doesuserexist1 = $Searcher.Findall()
if ($doesuserexist1 -eq $Null)
{Write-Host $samaccountname "account does not exist"}
elseif ($StaffCodeN -match $staffcode)
write-host "user has renewed the contract, no action taken"
else
if(($lastupdatedate -ne $null)-or($LastDateWorked -ne $null))
write-host "Setting Account Expiry to"$accountexpirydate
#$ChangeUser.AccountExpires = $accountexpirydate
#$Changeuser.setinfo()
if ($UserMailforwarding -ne $null)
#Set Account expiry date to Last Date Worked
# $ChangeUser.AccountExpires = $accountexpirydate
# $Changeuser.setinfo()
write-host "staff" $displayname "with staff employee no" $samaccountname "has
mailforwarding"
Write-host "Please disable the account manually via Active Directory Users & Computers and
Elseif ($accountexpirydate -lt $todaysdate)
#disable the accountHi Vicki,
This Forum has an insert-codeblock function. Using it will make your script far more readable
Your script is missing some parts, it is impossible to follow the problem.
You are performing the same string cleaning action on $opencsv2 for each element in $opencsv, when doing it once should suffice. Why not start it all by cleaning the values and storing the cleaned values in new arrays?
The Compare-Object function is great, why not take it out for a stroll on these lists, it might just safe you lots of unnecessarily complicated code ...
You are creating a new $Domain, $Root and $Searcher object each iteration, when doing it once should suffice. Probably not much of a time-saver, but every little thing contributes.
Try pinpointing the problem by doing extensive logging, not only by writing which action was taken, but writing the inidividual information (variables, mostly) before evaluation occurs. Your if/elseif/else looks sound, so if it's still not doing what you
want, the ingoing data must be different from what you think should be there.
Cheers,
Fred
There's no place like 127.0.0.1 -
Excel file creation on application server
Gurus,
I have seen a lot of threads on excel file creation in application server, but failed to see if anyone had a solution.
Problem: In background execution I need to create an excel sheet on application server, the data would be contained in internal table, The FM like WS_DOWNLOAD etc..cannot support these as they have GUI component.
All the documentation that I have been through states to use dataset which saves the file in.csv or txt format seperated by delimiter, is it possible to save as a true excel file.
same as this thread:
Re: Excel download in Application Server
let me know,
Thanks in advance .* Write column headers, if required
IF column_headers = 'X'.
* ... override field names with LABELS_IN, if specified
IF NOT labels_in[] IS INITIAL.
DESCRIBE TABLE labels_in LINES num_lines.
IF num_lines > num_fields.
RAISE too_many_labels.
ELSEIF num_lines < num_fields.
RAISE too_few_labels.
ENDIF.
ASSIGN COMPONENT 1 OF STRUCTURE labels_in TO <f>.
LOOP AT fields_in.
READ TABLE labels_in INDEX sy-tabix.
fields_in-name = labels_in-name.
MODIFY fields_in.
ENDLOOP.
ENDIF.
* ...write column headers to file...
position = 0.
LOOP AT fields_in.
IF fixed_length NE space.
*** INSERT: DAR was here for Issue 17817 --------------- dar012009 ---v
field_len = STRLEN( fields_in-name ).
*** INSERT: DAR was here for Issue 17817 --------------- dar012009 ---^
DO fields_in-len TIMES.
offset = sy-index - 1.
* IF sy-index > 15. dar012009
IF sy-index > field_len. "dar012009
line_out+position(1) = space.
ELSEIF fields_in-name+offset(1) NE space.
line_out+position(1) = fields_in-name+offset(1).
ELSE.
line_out+position(1) = space.
ENDIF.
position = position + 1.
ENDDO.
ELSE.
CONCATENATE line_out fields_in-name INTO line_out
SEPARATED BY delimiter.
ENDIF.
ENDLOOP.
IF fixed_length EQ space.
SHIFT line_out LEFT DELETING LEADING delimiter.
ENDIF.
IF no_write IS INITIAL. "dar041505
TRANSFER line_out TO file_out.
ENDIF. "dar041505
data_out = line_out. APPEND data_out. "dar041505
ENDIF.
* Write lines to file...
LOOP AT data_in.
CLEAR line_out.
position = 0.
DO num_fields TIMES.
ASSIGN COMPONENT sy-index OF STRUCTURE data_in TO <f>. -
Problem in handling multiple source of Complex CSV files in ODI
Hi,
I have a requirement where I have multiple CSV files.
In Details:
when I have one CSV file I use the following syntax in JDBC URL in creation of Data Server under Complex file of Topology navigator.
jdbc:snps:complexfile?f=C:\pqr\pqr.csv&d=C:\pqr\pqr.xsd&re=Root
When there are multiple CSV files what would be the syntax as? will it be like
jdbc:snps:complexfile?f=C:\pqr\*.csv&d=C:\pqr\pqr.xsd&re=Root (I am getting error as unexpected token * when I am using * and clicking on Test Connection Button).
Please help..
Thanks.Hi,
first of all according to documentation use backslash
http://docs.oracle.com/cd/E21764_01/integrate.1111/e12644/appendix_complex_driver.htm#CCHDJGHF
in second place under my opinion you are not able to use mask to identify more file. A workaround is to use a variable to store the file name. BUT i've never used Complex Files, so i'm speculating about it. -
Web.Show_document to open .csv file in Excel How??
We have a utility built in our menu (mmb)
It reads the screen output and creates a .CSV file on the server using text_io and then using Web.show_document displays the file on the client.
The file opens in MS-EXCEL. It gives a standard Excel prompt for Open , Save or run.
Now our application has moved to FMW 11gAS.
The file creation is happening but the file is opening in HTML format.
The excel prompt is not coming.
Where is setting to be done to open the .CSV files in EXCELIn addition to what I already mentioned, your issue may be partially caused by these two bugs: 10186820 & 10191154
Both have been fixed in 11.1.1.6, which was released last week. Information about the 11.1.1.6 patch set can be found in MyOracleSupport note 1364511.1
The fix should also already be in 11.1.2.0 -
Csv file processing using external table
Dear All,
Our database is oracle 10g r2 and OS is solaris
We would receive csv files to a particular directory on server each day.
File Format look like:
H00,SOURCE_NAME,FILE_CREATED_DATE
RECORD_TYPE,EMP_ID,EMP_NAME,EMP_DOB(DDMMYYYY),EMP_HIRE_DATE(DDMMYYYY),EMP_LOCATION
T00,RECORD_COUNT
EMPLOYEE TABLE STRUCTURE
EMP_ID NOT NULL NUMBER ,
EMP_NAME NOT NULL VARCHAR2(10) ,
EMP_DOB DATE,
EMP_HIRE_DATE NOT NULL DATE,
EMP_LOCATION VARCHAR2(80)
Sample File:
H00,ABC,21092013
"R01",1,"EMP1","14021986","06072010","LOC1"
"R01",20000000000,"EMP2","14021-987","06072011",""
,***,"EMPPPPPPPPPPP3","14021988","060**012","LOC2"
"R01",4,4,"14021989","06072013",
T00,4
we need to validate each record excluding header and trailer for:
DATATYPE, LENGTH,OPTIONALITY, and other date validations such as EMP_HIRE_DATE can not be less than EMP_DOB
In case of any data errors we need to send a response file for corresponding source file.
we have predefined error codes to be sent in the response file.
ERR001 EMP_ID can not be null
ERR002 EMP_ID exceeds 10 digits
ERR003 EMP_ID is not a number
ERR004 EMP_NAME has to be text
ERR005 EMP_NAME length can not exceed 10
ERR006 EMP_NAME can not be null
ERR007 EMP_DOB is not a date
ERR008 EMP_DOB is not in ddmmyyyy format
ERR009 EMP_HIRE_DATE is not a date
ERR010 EMP_HIRE_DATE is not in ddmmyyyy format
ERR011 EMP_HIRE_DATE can not be null
ERR012 EMP_LOCATION has to be text
ERR013 EMP_LOCATION length can not exceed 80
ERR014 EMP_HIRE_DATE can not be less than EMP_DOB
ERR015 Field missing in the record
ERR016 More number of fields than allowed
1.Do I need to create external table before processing each file.(EMP1.txt,EMP2.txt)?
2.How to generate these error codes in case of respective failure scenarios and to log into an exception table?
3.response file needs to have entire record and a concatination of all the error codes in the next line.
4.what would be a better approach among
creating an external table with all char(2000) fields and writing a select statement
such as select * from ext_table where (emp id is not null and length(emp_id)<=10 and....to select only proper data);
or creating the external table to be same as employee table and creating a bad file? if this is the preferred how can I generate the custom error codes?
Could you please help me in achieving this!
Warm Regards,
Shankar.You can do a one-time creation of an external table. After that, you can either overwrite the existing text file or alter the location. In the example below I have split your original sample file into two files, in order to demonstrate altering the location. You can make the external table all varchar2 fields as large as they need to be to accommodate all possible data. If you then create a staging table and rejects table of the same structure, you can create a trigger on the staging table such that, when you insert from the external table to the staging table, it inserts into either the employee table or rejects table with the concatenated error list. If you want this in a file, then you can just spool a select from the rejects table or use some other method such as utl_file. The following is a partial example. Another alternative would be to use SQL*Loader to load directly into the staging table without an external table.
SCOTT@orcl12c> HOST TYPE emp1.txt
H00,ABC,21092013
"R01",1,"EMP1","14021986","06072010","LOC1"
"R01",20000000000,"EMP2","14021-987","06072011",""
T00,4
SCOTT@orcl12c> HOST TYPE emp2.txt
H00,ABC,21092013
,***,"EMPPPPPPPPPPP3","14021988","060**012","LOC2"
"R01",4,4,"14021989","06072013",
T00,4
SCOTT@orcl12c> CREATE OR REPLACE DIRECTORY my_dir AS 'c:\my_oracle_files'
2 /
Directory created.
SCOTT@orcl12c> CREATE TABLE external_table
2 (record_type VARCHAR2(10),
3 emp_id VARCHAR2(11),
4 emp_name VARCHAR2(14),
5 emp_dob VARCHAR2(10),
6 emp_hire_date VARCHAR2(10),
7 emp_location VARCHAR2(80))
8 ORGANIZATION external
9 (TYPE oracle_loader
10 DEFAULT DIRECTORY my_dir
11 ACCESS PARAMETERS
12 (RECORDS DELIMITED BY NEWLINE
13 LOAD WHEN ((1: 3) != "H00" AND (1:3) != 'T00')
14 LOGFILE 'test.log'
15 FIELDS TERMINATED BY "," OPTIONALLY ENCLOSED BY '"' LDRTRIM
16 MISSING FIELD VALUES ARE NULL
17 REJECT ROWS WITH ALL NULL FIELDS
18 (record_type, emp_id, emp_name, emp_dob, emp_hire_date, emp_location))
19 LOCATION ('emp1.txt'))
20 /
Table created.
SCOTT@orcl12c> CREATE TABLE staging
2 (record_type VARCHAR2(10),
3 emp_id VARCHAR2(11),
4 emp_name VARCHAR2(14),
5 emp_dob VARCHAR2(10),
6 emp_hire_date VARCHAR2(10),
7 emp_location VARCHAR2(80))
8 /
Table created.
SCOTT@orcl12c> CREATE TABLE employee
2 (emp_id NUMBER NOT NULL,
3 emp_name VARCHAR2(10) NOT NULL,
4 emp_dob DATE,
5 emp_hire_date DATE,
6 emp_location VARCHAR2(80))
7 /
Table created.
SCOTT@orcl12c> CREATE TABLE rejects
2 (record_type VARCHAR2(10),
3 emp_id VARCHAR2(11),
4 emp_name VARCHAR2(14),
5 emp_dob VARCHAR2(10),
6 emp_hire_date VARCHAR2(10),
7 emp_location VARCHAR2(80),
8 error_codes VARCHAR2(4000))
9 /
Table created.
SCOTT@orcl12c> CREATE OR REPLACE TRIGGER staging_air
2 AFTER INSERT ON staging
3 FOR EACH ROW
4 DECLARE
5 v_rejects NUMBER := 0;
6 v_error_codes VARCHAR2(4000);
7 v_num NUMBER;
8 v_dob DATE;
9 v_hire DATE;
10 BEGIN
11 IF :NEW.emp_id IS NULL THEN
12 v_rejects := v_rejects + 1;
13 v_error_codes := v_error_codes || ',' || 'ERR001';
14 ELSIF LENGTH (:NEW.emp_id) > 10 THEN
15 v_rejects := v_rejects + 1;
16 v_error_codes := v_error_codes || ',' || 'ERR002';
17 END IF;
18 BEGIN
19 v_num := TO_NUMBER (:NEW.emp_id);
20 EXCEPTION
21 WHEN value_error THEN
22 v_rejects := v_rejects + 1;
23 v_error_codes := v_error_codes || ',' || 'ERR003';
24 END;
25 IF :NEW.emp_name IS NULL THEN
26 v_rejects := v_rejects + 1;
27 v_error_codes := v_error_codes || ',' || 'ERR006';
28 ELSIF LENGTH (:NEW.emp_name) > 10 THEN
29 v_rejects := v_rejects + 1;
30 v_error_codes := v_error_codes || ',' || 'ERR005';
31 END IF;
32 BEGIN
33 v_dob := TO_DATE (:NEW.emp_dob, 'ddmmyyyy');
34 EXCEPTION
35 WHEN OTHERS THEN
36 v_rejects := v_rejects + 1;
37 v_error_codes := v_error_codes || ',' || 'ERR008';
38 END;
39 BEGIN
40 IF :NEW.emp_hire_date IS NULL THEN
41 v_rejects := v_rejects + 1;
42 v_error_codes := v_error_codes || ',' || 'ERR011';
43 ELSE
44 v_hire := TO_DATE (:NEW.emp_hire_date, 'ddmmyyyy');
45 END IF;
46 EXCEPTION
47 WHEN OTHERS THEN
48 v_rejects := v_rejects + 1;
49 v_error_codes := v_error_codes || ',' || 'ERR010';
50 END;
51 IF LENGTH (:NEW.emp_location) > 80 THEN
52 v_rejects := v_rejects + 1;
53 v_error_codes := v_error_codes || ',' || 'ERR013';
54 END IF;
55 IF v_hire IS NOT NULL AND v_dob IS NOT NULL AND v_hire < v_dob THEN
56 v_rejects := v_rejects + 1;
57 v_error_codes := v_error_codes || ',' || 'ERR014';
58 END IF;
59 IF :NEW.emp_id IS NULL OR :NEW.emp_name IS NULL OR :NEW.emp_dob IS NULL
60 OR :NEW.emp_hire_date IS NULL OR :NEW.emp_location IS NULL THEN
61 v_rejects := v_rejects + 1;
62 v_error_codes := v_error_codes || ',' || 'ERR015';
63 END IF;
64 IF v_rejects = 0 THEN
65 INSERT INTO employee (emp_id, emp_name, emp_dob, emp_hire_date, emp_location)
66 VALUES (:NEW.emp_id, :NEW.emp_name,
67 TO_DATE (:NEW.emp_dob, 'ddmmyyyy'), TO_DATE (:NEW.emp_hire_date, 'ddmmyyyy'),
68 :NEW.emp_location);
69 ELSE
70 v_error_codes := LTRIM (v_error_codes, ',');
71 INSERT INTO rejects
72 (record_type,
73 emp_id, emp_name, emp_dob, emp_hire_date, emp_location,
74 error_codes)
75 VALUES
76 (:NEW.record_type,
77 :NEW.emp_id, :NEW.emp_name, :NEW.emp_dob, :NEW.emp_hire_date, :NEW.emp_location,
78 v_error_codes);
79 END IF;
80 END staging_air;
81 /
Trigger created.
SCOTT@orcl12c> SHOW ERRORS
No errors.
SCOTT@orcl12c> INSERT INTO staging SELECT * FROM external_table
2 /
2 rows created.
SCOTT@orcl12c> ALTER TABLE external_table LOCATION ('emp2.txt')
2 /
Table altered.
SCOTT@orcl12c> INSERT INTO staging SELECT * FROM external_table
2 /
2 rows created.
SCOTT@orcl12c> SELECT * FROM employee
2 /
EMP_ID EMP_NAME EMP_DOB EMP_HIRE_DATE EMP_LOCATION
1 EMP1 Fri 14-Feb-1986 Tue 06-Jul-2010 LOC1
1 row selected.
SCOTT@orcl12c> COLUMN error_codes NEWLINE
SCOTT@orcl12c> SELECT * FROM rejects
2 /
RECORD_TYP EMP_ID EMP_NAME EMP_DOB EMP_HIRE_D EMP_LOCATION
ERROR_CODES
R01 20000000000 EMP2 14021-987 06072011
ERR002,ERR008,ERR015
*** EMPPPPPPPPPPP3 14021988 060**012 LOC2
ERR003,ERR005,ERR010
R01 4 4 14021989 06072013
ERR015
3 rows selected. -
How to create a datasource for 0COSTCENTER to load data in csv file in BI
how to create a datasource for 0COSTCENTER to load data in csv file in BI 7.0 system
can you emil me the picture of the step about how to loaded individual values of the hierarchy using CSV file
thank you very much
my emil is <Removed>
allenStep 1: Load Required Master Data for 0CostCenter in to BI system
Step 2: Enable Characteristics to support Hierarchy for this 0Cost Center and specify the External Characteristic(In the Lowest Node or Last Node) while creation of this Characteristic InfoObject
Step 3: On Last Node of Hierarchy Structure in the InfoObject, Right Click and then Create Hierarchy MANUALLY by Inserting the Master Data Value as BI dosent Support the Hierarchy load directly you need to do it manually....
Step 4: Mapping
Create Text Node thats the first node (Root Node)
Insert Characteristic Nodes
Insert the Last Node of the Hierarchy
Then you need to create a Open hub Destination for extracting data into the .csv file...
Step1 : Create the Open Hub Destination give the Master Data table name and enter all the fields required....and create the transformations for this Open Hub connecting to the External file or excel file source...then give the location on to your local disk or path of the server in the first tab and request for the data...It should work alright let me know if you need anything else...
Thanks,
Sandhya
Maybe you are looking for
-
How to use multiple instances of the same dll ?
Hi, I'd like to use multiple instances of a jni dll . I have created different threads, in each thread, I have called System.loadLibrary(..), and I would like each thread to access a different instance of this library. Unfortunately, the loadLibrary
-
Phpvirtualbox gives me a grey screen
I followed the instruccions on the wiki page PhpVirtualBox. However, at the end, all I am getting is a grey image. # uname -r 3.16.2-1-ck # pacman -Qs virtualbox local/phpvirtualbox 4.3_1-1 PHP/AJAX web interface for VirtualBox 4.* local/vdfuse 82a-8
-
Bluetooth devices state they are disabled since upgrade to 10.2.1.537
Hi We have had (including myself) several users in my company that have upgraded their devices to this latest software. Since the upgrade we can no longer use Bluetooth devices. We are getting the message that Bluetooth has been disabled by your adm
-
All my purchases on itune are billed to a credit card that is not the credit card that is detailed on my billing profile? How is this possible? I have checked and reinserted my credit card details several times in my billing section and still charges
-
SCCM 2012 R2 not show Threat item in Malware Detected Report.
I had no "Malware detected" items fount. There are no report abount virus that Endpoint Protection client finds. I try to test be creating file with this test content: "X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*", my Endpoin