SQL Loader: handling difference datatypes in data file and table column

Hi,
I am not sure if my question is valid but I am having this doubt.
I am trying to load data from my data file into a table with just a single column of FLOAT datatype using SQL Loader. But very few insertions take place, leaving a large number of record rejected for the same reason-
Record 7: Rejected - Error on table T1, column MSISDN.
ORA-01722: invalid number
The data in my datafile goes like this: (with a single space before every field)
233207332711<EOFD><EORD> 233208660745<EOFD><EORD> 233200767380<EOFD><EORD>
Here I want to know if there is any way to type cast the data read from the data file to suit my table column's datatype.
How do I handle this? I want to load all the data from my datafile into my table.

Pl continue the discussion in your original post - Pls help: SQL Loader loads only one record

Similar Messages

SQL*Loader-510: Physical record in data file (clob_table.ldr) is long

If I generate loader / Insert script from Raptor, it's not working for Clob columns.
I am getting error:
SQL*Loader-510: Physical record in data file (clob_table.ldr) is long
er than the maximum(1048576)
What's the solution?
Regards,

Hi,
Has the file been somehow changed by copying it between windows and unix? Ora file transfer done as binary or ASCII? The most common cause of your problem. Is if the end of line carriage return characters have been changed so they are no longer /n/r could this have happened? Can you open the file in a good editor or do an od command in unix to see what is actually present?
Regards,
Harry
http://dbaharrison.blogspot.co.uk/

Data File and Table I/O

Good day.
We're currently experiencing very high I/O on a datafile.
We would like to remove the table that is causing the high I/O but are having difficulties in identifying it.
We've upped the StatsPack level to 7 to give us I/O for tables, but unfortunately, the top 5 tables in the StatsPack report are not located within the datafile (based on DBA_EXTENTS)
Any help would be greatly appreciated.
Thank you,
Charles.
Ottawa, Canada

You can use v$filestat to find out the datafile which has more load on it, like, physical read/writes, and other stuff.
SELECT name, phyrds, phywrts
FROM v$datafile df, v$filestat fs
WHERE df.file# = fs.file#;
Make sure that you dont have any temp segments/objects in the datafile which is having more i/o.
Mabe you can list the objects present in the particular datafile and if they are heavily accessed, you can move objects into another tablespace.
Jaffar

ERROR LOADING bcp -w option generated data files

Hi,
I have to migrate data from SQL Server to Oracle.
The unload_script is this:
bcp "CLEPSIDRA.dbo.InversionesAceptadas" out "[CLEPSIDRA].[dbo].[InversionesAceptadas].dat" -q -c -t "<EOFD>" -r "<EORD>" -Usa -Pas -STIEPO
When I execute the load script, the data is loaded ok.
But I have a field that contains character like á,é,ñ,...so I changed my unload script to:
bcp "CLEPSIDRA.dbo.InversionesAceptadas" out "[CLEPSIDRA].[dbo].[InversionesAceptadas].dat" -q -w -t "<EOFD>" -r "<EORD>" -Usa -Pas -STIEPO
If I execute the new unload_script, the data file generated is ok, with the characters like á,é,ñ...
But when I trying to execute the load script, I have errors like 'field is not a valid number..'...
But if I open the first (-c option generated) data file and replace it's content with the content of the second (-w option generated) data file and save it, the load script works ok!
How can I solve this?
Thanks

Hi user616069,
There is a preference in SQLDeveloper for encoding.
Tools->Preferences->Environment->Encoding
Do you have a reproducible test case you can give us a URL to, including for example a single fictional record?
-Turloch

[svn:osmf:] 17548: Remove left-over event handler, which could trigger an unnecessary event if a player loaded plugins through the config XML file and manually (e.g.

Revision: 17548
Revision: 17548
Author:   [email protected]
Date:     2010-09-01 14:09:14 -0700 (Wed, 01 Sep 2010)
Log Message:
Remove left-over event handler, which could trigger an unnecessary event if a player loaded plugins through the config XML file and manually (e.g. for static plugins).
Modified Paths:
    osmf/trunk/libs/samples/ChromeLibrary/org/osmf/chrome/configuration/PluginsParser.as

Remember that Arch Arm is a different distribution, but we try to bend the rules and provide limited support for them. This may or may not be unique to Arch Arm, so you might try asking on their forums as well.

System Center 2012 R2 install: SQL server Data file and log file

This might be a dumb question, but I can't find the answer anywhere.
I'm installing a new instance of System Center 2012 R2 on a new server, I'm stuck on the SQL Server data file section. Everytime I put in a path, it says that tne path does not exist. I'm I supposed to be creating some sort of SQL Server
data file and log file before this installation, I didn't get this prompt when installing System Center 2012 SP1 or hen I upgraded from System Center 2012 SP1 to System Center 2012 R2
My SQL is on a different server
Thank you in advanced

Have you reviewed the setup.log?
On a side note, why would you put the database file on the same drive as the OS? That defeats the whole purpose of having a remote SQL Server. Why use a remote SQL Server in the first place.
Jason | http://blog.configmgrftw.com

Sql server data file and log file

hello experts
what is the best way to save data file and log file in a two node cluster environment. i have an active\passive cluster with windows server 2008r2 enterprise and sql server 2008r2. i am new to the environment and i noticed that all system and user databases
including their data and log files are stored in one drive, just curious, what is the best practise in such kinds of scenario, thank you as always for your help.

Make sure you have valid/tested backup strategy for both system and user databases.
Best Regards,Uri Dimant SQL Server MVP,
http://sqlblog.com/blogs/uri_dimant/
MS SQL optimization: MS SQL Development and Optimization
MS SQL Consulting:
Large scale of database and data cleansing
Remote DBA Services:
Improves MS SQL Database Performance
SQL Server Integration Services:
Business Intelligence

How to design SQL server data file and log file growth

how to design SQL DB data file and log file growth- SQL server 2012
if my data file is having 10 GB sizze and log file is having 5 GB size
what should be the size in MB (not in %) of autogrowth. based on what we have to determine the ideal size of file auto growth.

It's very difficult to give a definitive answer on this. Best principal is to size your database correctly in advance so that you never have to autogrow, of course in reality that isn't always practical.
The setting you use is really dictated by the expected growth in your files. Given that the size is relatively small why not set it to 1gb on the datafile(s) and 512mb on the log file? The important thing is to monitor it on an on-going basis to see if that's
the appropriate amount.
One thing you should do is enable instant file initialization by granting the service account Perform Volume Maintenance tasks in group policy. This will allow the data files to grow quickly when required, details here:
https://technet.microsoft.com/en-us/library/ms175935%28v=sql.105%29.aspx?f=255&MSPPError=-2147217396
Also, it possible to query the default trace to find autogrowth events, if you wanted you could write an alert/sql job based on this
SELECT
[DatabaseName],
[FileName],
[SPID],
[Duration],
[StartTime],
[EndTime],
CASE [EventClass]
WHEN 92 THEN 'Data'
WHEN 93 THEN 'Log' END
FROM sys.fn_trace_gettable('c:\path\to\trace.trc', DEFAULT)
WHERE
EventClass IN (92,93)
hope that helps

How to read the data file and write into the same file without a temp table

Hi,
I have a requirement as below:
We are running lockbox process for several business, but for a few businesses we have requirement where in we receive a flat file in different format other than how the transmission format is defined.
This is a 10.7 to 11.10 migration. In 10.7 the users are using a custom table into which they are first loading the raw data and writing a pl/sql validation on that and loading it into a new flat file and then running the lockbox process.
But in 11.10 we want to restrict using temp table how can we achieve this.
Can we read the file first and then do validations accordingly and then write to the same file and process the lockbox.
Any inputs are highly appreciated.
Thanks & Regards,
Lakshmi Kalyan Vara Prasad.

Hello Gurus,
Let me tell you about my requirement clearly with an example.
Problem:
i am receiving a dat file from bank in below format
105A371273020563007 07030415509174REF3178503 001367423860020015E129045
in this detail 1 record starting from 38th character to next 15 characters is merchant reference number
REF3178503 --- REF denotes it as Sales Order
ACC denotes it as Customer No
INV denotes it as Transaction Number
based on this 15 characters......my validation comes.
If i see REF i need to pick that complete record and then fill that record with the SO details as per my system and then submit the file for lockbox processing.
In 10.7 they created a temporary table into which they are loading the data using a control file....once the data is loaded into the temporary table then they are doing a validation and updating the record exactly as required and then creating one another file and then submitting the file for lockbox processing.
Where as in 11.10 they want to bypass these temporary tables and writing it into a different file.
Can this be handled by writing a pl/sql procedure ??
My findings:
May be i am wrong.......but i think .......if we first get the data into ar_payments_interface_all table and then do the validations and then complete the lockbox process may help.
Any suggestions from Oracle GURUS is highly appreciated.
Thanks & Regards,
Lakshmi Kalyan Vara Prasad.

How to avoid duplicate data while inserting from sample.dat file to table

Hi Guys,
We have issue with duplicate data in flat file while loading data from sample.dat file to table. How to avoid duplicate data in control file.
Can any one help me on this.
Thanks in advance!
Regards,
LKR

No, a control file will not remove duplicate data.
You would be better to use an external table and then remove duplicate data using SQL as you query the data to insert it to your destination table.

Timeouts increased after we moved USR, SAP data files and TLogs to new SAN

We are having issues with timeouts after we moved our USR, SAP SQL Datafiles and SAP Transaction Logs from our old SAN to a new SAN.
Timeouts for SAPGUI users are set to 10 minutes.
We are running Windows Server 2003 with SQL Server 2005.
The SAP database has 8 datafiles with a total size of about 350GB.
Procedure we used to move SAP to new SAN:
1. Attached 3 new SAN Volumes
     -a. USR
     -b. Data Files
     -c. Transaction Logs
2. Shutdown SAP and SQL services
3. Alligned the new volumes with a 1024kb offset and gave the Data files and Transaction log volumes a 64kb allocation
    size. (The alignment and 64kb allocation size were not setup for these volumes on the old SAN)
4. Copied the 3 volumes from old to new.
5. Changed the new volumes drive letter to the drive letters of the old volumes.
     -a. I had to restart in order to change the USR volume.
     -b. Because of this I had to resetup the sapmnt and saploc shares.
6. Started SQL services and then SAP services and everything came up just fine.
The week before we had anywhere from 1 to 9 timeouts per day.
This week: Monday had 20 and Tuesday had 26.
On Monday we saw that MD07 was the only transaction that was timing out, but Tuesday had others as well.
The amount of users in the system is about the same. The amount of orders going in are about the same. No big transports went in right before we switched.
Performance counters that I know about for disk look a lot better on the Data Files.
- PAGEIOLATCH_SH ms/request is about 50% better
- Under I/O Performance in DBACOCKPIT:
- MS/OP is now anywhere from 5 to 30 - Old SAN: 50 to 300
- The Hit Ratio is over 99% - same as the old SAN
Looking at Wiley Introscope graphs:
- The "SAP Host: Average queue length" is about 30% to 40% lower then the old SAN.
- the "SAP Host: Disk utilization in %" is about the same.
Questions:
1. Did we do anything wrong or miss anything with our move procedure?
     a. Do we have to do anything in SQL since we changed volumes even though we kept the drive letters the same?
2. What other logs or performance counters should I be looking at?
Thank you,
Neil

Our new SAN Vendor is Compellent. They have been fantastic. I would highly recommend checking them out.
The reasons for the timeouts had nothing to do with the SAN...Well kind of anyway.
I decided to check t-code SM20 to see what users were doing when these timeouts were happening. What I found was the program R_BAPI_NETWORK_MAINTAIN was being called thousands of times in a matter of 10 to 15 minutes at random times through out the day. It would take up about 50 to 80 percent of the amount of programs being executed during these times.
So, I sent this information to our developers and they found out that R_BAPI_NETWORK_MAINTAIN was being called from another program that was looping thousands of times. The trigger to stop the loop wasn't happening fast enough. They made a change and we haven't seen the timeouts since.
I think that the performance increases allowed the loop to run faster which caused the slow downs and timeouts to happen more often.
Thank you to everyone for their help!
Neil

How to create new .dat file and its contents?

Hi There
Can anybody let me know the procedure of how to create new .ctl or .dat file to load data to tables.
i working on 2day dba chapter 8. it shows me how to creat table and use .dat file to load data. but doesnot giv any clue how new .dat file and its contents can be created. please help.
thanks in advance.

Thanks for ur help
I ve created txt file in notepad and saved it as dat file. tried to load that data thru Enterprise manager. used load data from User files everything went well and job was showing suceeded but dont know why that data is not showing in the table. i ve tried it now for three four times. used right table and pathe but dont know. has anybody got idea why that ll be?
Thanks

SQL ENTERPRISE: The edition of Reporting Services that you are using requires that you use local SQL Server relational databases for report data sources and the report server database

The error below makes absolutely no sense! I'm using Enterprise Core...yet I'm being told I can't use remote data sources:
w3wp!library!8!03/05/2015-19:08:48:: i INFO: Catalog SQL Server Edition = EnterpriseCore
w3wp!library!8!03/05/2015-19:08:48:: e ERROR: Throwing Microsoft.ReportingServices.Diagnostics.Utilities.OperationNotSupportedException: , Microsoft.ReportingServices.Diagnostics.Utilities.OperationNotSupportedException: The feature: "The edition of Reporting
Services that you are using requires that you use local SQL Server relational databases for report data sources and the report server database." is not supported in this edition of Reporting Services.;
Really? This totally contradicts the documentation found here:
https://msdn.microsoft.com/en-us/library/ms157285(v=sql.110).aspx
That article says remote connections are completely supported.
ARGH! Why does this have to be so difficult to setup?!?

Hi jeffoliver1000,
According to your description, you are using Enterprise Core edition and you are prompted that you can’t use remote data sources.
In your scenario, we neither ignore your point nor be doubt with what you say. But actually we have met the case before that even though the SQL Server engine is Enterprise but the reporting services is still standard. So I would recommend you to find the
actual edition of reporting services you are using. You can find Reporting Services starting SKU in the Reporting Service logs ( default location: C:\Program Files\Microsoft SQL Server\<instance name>\Reporting Services\LogFiles). For more information,
please refer to the similar thread below:
https://social.technet.microsoft.com/Forums/en-US/f98c2f3e-1a30-4993-ab41-acbc5014f92e/data-driven-subscription-button-not-displayed?forum=sqlreportingservices
By the way, have you installed the other SQL Server edition before?
Best regards,
Qiuyun Yu
Qiuyun Yu
TechNet Community Support

Took backup of data files and controlfile but not archive file db is 11g xe

took backup of data files and controlfile but not redo file db is 11g xe r2..was trying complete shut backup,..missed redo..
anyway to restore the db back..
I had failed to backup redo log file..
db was shut...properly
any info highly appreciated.
cheers
Edited by: zycoz100 on Feb 27, 2013 4:42 AM

A controlfile create is a fairly trivial task, for either a cold backup or hot backup scenario. Well, its an easy task after you've done the process a few times.
If you have a healthy oracle instance running any place else, preferably at 11gR2, or even a 10g instance can work as an example, get a system connection and do an `alter database backup controlfile to trace;` and the instance will create a new trace file with all SQL commands needed to rebuild the controlfile in the instance trace directory.
Have a look at the trace file, a `show parameter diag` reveals the diagnostic_dest parameter, look for the "trace" directory under that folder, for the latest *trc file. In 10g its `show parameter dump` and the trace file goes straight to the user_dump_dest directory, no digging required.

Error: SQL*Loader-926: OCI error while OCIStmtExecute(trigger_hp) for table

HEllo, i've a problem when i try to load a file, the message is:
SQL*Loader-926: OCI error while OCIStmtExecute(trigger_hp) for table <table>
ORA-00904: invalid column name
what is the solution for this?
thanks

The root cause is in the error ora-1426, which you can look up in the online error documentation at http://tahiti.oracle.com . No one knows every error message by heart. This means it is expected you look up the error prior to posting, and you don't expect any volunteer in this forum to look up the error on your behalf.
Also this is a typical candidate for being a known problem, and known problems can be found on My Oracle Support.
Sybrand Bakker
Senior Oracle DBA

SQL Loader: handling difference datatypes in data file and table column

Similar Messages

Maybe you are looking for