Length semantics

Hi,
There are odbc apis which have character i/o parameters. These arguments have length parameters associated with them. These lengths can be number of bytes or number of wide characters/code points. I want to know about these length semantics. I could not find any relevant documentation. It would be fine if somebody points me to such a reference.
Thanks and regards,
Vivek.

Hi,
What does codepoint mean
In simple terms Code point Assinging a distinct numeric value for each character.
As coming to length Semantics, that totally depends upon the Language setting like UTF-8. Since in Language Setting in order to maintain/store a Asian character it takes(Single Character).. it takes three code units some thing like that...
Let me put the nice link for you from where you can get some understanding about the length semantics...
Here we go finally got it.. for you... But nice link..
http://www.oracle.com/technology/oramag/oracle/03-nov/o63tech_glob.html
- Pavan Kumar N

Similar Messages

SQL Loader Multibyte character error, LENGTH SEMANTICS CHARACTER

Hi,
startet SQL Loader Multibyte character error
{thread:id=2340726}
some mod locked the thread, why?
the solution for others:
add LENGTH SEMANTICS CHARACTER to the controlfile
LOAD DATA characterset UTF8 LENGTH SEMANTICS CHARACTER
TRUNCATE
INTO TABLE utf8file_to_we8mswin1252
ID CHAR(1)
, TEXT CHAR(40)
)Regards
Michael

Hi Werner,
on my linux desktop:
$ file test.dat
test.dat: UTF-8 Unicode text, with very long lines
my colleague is working on a windows system.
On both systems exact the same error from SQL Loader.
Btw, try with different number of special characters (german umlaute and euro) and there is no chance to load without the error
when to many (?) special characters or data is long as column length and special characters included.
Regards
Michael

Querying CHAR columns with character length semantics unreliable

Hi again,
It appears that there is a bug in the JDBC drivers whereby it is highly unlikely that the values of CHAR columns that use character length semantics can be accurately queried using ResultSet.getString(). Instead, the drivers return the value padded with space (0x#20) characters out to a number of bytes equal to the number of characters multiplied by 4. The number of bytes varies depending on the number and size of any non-ascii characters stored in the column.
For instance, if I have a CHAR(1) column, a value of 'a' will return 'a ' (4 characters/bytes are returned), a value of '\u00E0' will return '\u00E0 ' (3 characters / 4 bytes), and a value of '\uE000' will return '\uE000 ' (2 characters / 4 bytes).
I'm currently using version 9.2.0.3 of the standalone drivers (ojdbc.jar) with JDK 1.4.1_04 on Redhat Linux 9, connecting to Oracle 9.2.0.2.0 running on Solaris.
The following sample code can be used to demonstrate the problem (where the DDL at the top of the file must be executed first):
import java.sql.*;
import java.util.*;
This sample generates another bug in the Oracle JDBC drivers where it is not
possible to query the values of CHAR columns that use character length semantics
and are NOT full of non-ascii characters. The inclusion of the VARCHAR2 column
is just a control.
CREATE TABLE TMP2
TMP_ID NUMBER(10) NOT NULL PRIMARY KEY,
TMP_CHAR CHAR(10 CHAR),
TMP_VCHAR VARCHAR2(10 CHAR)
public class ClsCharSelection
private static String createString(char character, int length)
char characters[] = new char[length];
Arrays.fill(characters, character);
return new String(characters);
} // private static String createString(char, int)
private static void insertRow(PreparedStatement ps,
int key, char character)
throws SQLException
ps.setInt(1, key);
ps.setString(2, createString(character, 10));
ps.setString(3, createString(character, 10));
ps.executeUpdate();
} // private static String insertRow(PreparedStatement, int, char)
private static void analyseResults(PreparedStatement ps, int key)
throws SQLException
ps.setInt(1, key);
ResultSet results = ps.executeQuery();
results.next();
String tmpChar = results.getString(1);
String tmpVChar = results.getString(2);
System.out.println(key + ", " + tmpChar.length() + ", '" + tmpChar + "'");
System.out.println(key + ", " + tmpVChar.length() + ", '" + tmpVChar + "'");
results.close();
} // private static void analyseResults(PreparedStatement, int)
public static void main(String argv[])
throws Exception
Driver driver = (Driver)Class.forName(
"oracle.jdbc.driver.OracleDriver").newInstance();
DriverManager.registerDriver(driver);
Connection connection = DriverManager.getConnection(
argv[0], argv[1], argv[2]);
PreparedStatement ps = null;
try
ps = connection.prepareStatement(
"DELETE FROM tmp2");
ps.executeUpdate();
ps.close();
ps = connection.prepareStatement(
"INSERT INTO tmp2 ( tmp_id, tmp_char, tmp_vchar " +
") VALUES ( ?, ?, ? )");
insertRow(ps, 1, 'a');
insertRow(ps, 2, '\u00E0');
insertRow(ps, 3, '\uE000');
ps.close();
ps = connection.prepareStatement(
"SELECT tmp_char, tmp_vchar FROM tmp2 WHERE tmp_id = ?");
analyseResults(ps, 1);
analyseResults(ps, 2);
analyseResults(ps, 3);
ps.close();
connection.commit();
catch (SQLException e)
e.printStackTrace();
connection.close();
} // public static void main(String[])
} // public class ClsColumnInsertion

FYI, this has been mentioned as early as November last year:
String with length 1 became 4 when nls_lang_semantics=CHAR
and was also brought up in Feburary:
JDBC thin driver pads CHAR col to byte size when NLS_LENGTH_SEMANTICS=CHAR

How to determine column length semantics through ANSI Dynamic SQL ?

I am looking for a way to determine the length semantics used for a column through ANSI Dynamic SQL.
I have a database with NLS_CHARACTERSET=AL32UTF8.
In this database I have the following table:
T1(C1 varchar2(10 char), C2 varchar2(40 byte))
When I describe this table in SQL*Plus, I get:
C1 VARCHAR2(10 CHAR)
C2 VARCHAR2(40)
In my Pro*C program (mode=ansi), I get the select statement on input, use PREPARE method to prepare it and then use the GET DESCRIPTOR method to obtain colum information for output:
GET DESCRIPTOR 'output_descriptor' VALUE :col_num
:name = NAME, :type = TYPE,
:length = LENGTH, :octet_length = OCTET_LENGTH
For both C1 and C2 I get the following:
:type=12
:length=40
:octet_length=40
So, even if I know that my database is AL32UTF8, there doesn't seem to be a way for me to determine whether char or byte length semantics were used in C1 and C2 column definitions.
Does anybody know how I can obtain this information through ANSI Dynamic SQL?
Note: the use of system views such as ALL_TAB_COLUMNS is not an option, since we wish to obtain this information even for columns in a complex select statements which may involve multiple tables.
Note: I believe OCI provides the information that we need through OCI_ATTR_DATA_SIZE (which is in bytes) and OCI_ATTR_CHAR_SIZE (which is in chars). However, switching to OCI is something we would like to avoid at this point.

Yes, I was wondering which forum would be the best for my question. I see similar questions in various forums, Call Interface, SQL and PL/SQL and Database - General. Unfortunately there is no Pro*C or Dynamic SQL forum which would be my first choice for posting this question.
Anyway I now posted the same question (same subject) in the Call Interface forum, so hopefully I'll get some answers there.
Thank you for the suggestion.

Oracle length semantics migration documention

Due to the oracle documention to migrate length semantics:
To convert an existing schema and its associated data from byte semantics and a single-byte character set to character semantics and a multibyte character set, such as UTF-8, you need only follow these steps: [The following steps have been corrected since the magazine was printed.]
1. Export the schema.
2. Issue an ALTER SYSTEM SET NLS_LENGTH_SEMANTICS=CHAR SCOPE=BOTH command on the target database.
3. Stop and restart the instance so that the parameter change takes effect.
4. Drop the original schema.
5. Recreate the original schema and its tables (you can use import's show=Y option to get the CREATE TABLE statements). Columns in the recreated tables will use character semantics, because that's now the default.
6. Import the schema into the target database using the IGNORE=Y import option.
What is the meaning of the terms target and original?
Suppose there is a (source) database with length semantics byte. If a (target) database
is to be created as a clone of the (source) database, except for the length semantic char, does one have to migrate the (source) database first?
Or rather, why is it not possible to
1. Export the data from the source database with length semantic byte,
2. Create a target database with length semantics char,
3. Import the data from the source database?)

This documentation is, unfortunately, poorly written.
If you want to migrate data from one database to another, with both databases having different character sets, you can avoid some data expansion issues, if you migrate to character set semantics at the same time.
You cannot just export data, and import it into a character semantics database, because export/import preserves original semantics of the exported tables.
Note: there is actually no such thing as a character semantics database. Character semantics is a column/variable property. The "character semantics database" is a confusing, though commonly used, term that actually means an instance which has NLS_LENGTH_SEMANTICS set to CHAR in its initialization file. The only significant meaning of this parameter is to be the default for session-level NLS_LENGTH_SEMANTICS. It is used for sessions that do not set this parameter explictly (through environment variable or ALTER SESSION). The session-level parameter is significant for CREATE TABLE/PROCEDURE/FUNCTION/PACKAGE [BODY] statements and tells the default for column and variable declarations that do not specify BYTE or CHAR explicitly.
To migrate semantics of an original schema you need to create a script that will contain all CREATE statements needed to recreate this schema (at least CREATE {TYPE | TABLE | MATERIALIZED VIEW | PROCEDURE | FUNCTION | PACKAGE [BODY]}). Then, you can just add the ALTER SESSION SET NLS_LENGTH_SEMANTICS=CHAR after any CONNECT command in this script. You can than run the script in the target database. How you create the script is irrelevant. You can use any reverse-engineering tool available (e.g. SHOW=Y option of import, DBMS_METADATA package, etc.)
After you pre-create the schema with the new semantics, you can import the data from the original (source) database with IGNORE=Y. The original semantics saved in the export file will be ignored for pre-created objects.
Note: PL/SQL may need manual corrections to migrate to character semantics. For example, SUBSTRB used to trim values before assignment may need to be replaced with SUBSTR.
-- Sergiusz

Convert all VARCHAR2 data types to character length semantics

Hi,
I am wondering if there is an easy way to convert all columns in the database of data type VARCHAR2(x BYTE) to VARCHAR2(x CHAR)?
Regards
Håkan

The DMU does not allow character length semantics migration for the following type of objects:
- Columns already in character length semantics
- Data dictionary columns
- Columns under Oracle-supplied application schemas
- CHAR attribute columns of ADT
- Columns in clusters
- Columns on which partition keys are defined
Please check if the disabled nodes you observed in the wizard fall under one of these categories.

Export with data length semantics

Hello,
I've following problem.
I have a table abcd which contains 2 VARCHAR2 columns with different data length semantics (one with BYTE, one with CHAR). Charset is Single Byte; let's say WE8MSWIN1252, so data length semantics should not be a problem. should not. details later.
So this would be:
create table abcd (a_char VARCHAR2(2 CHAR), a_byte VARCHAR2(2 BYTE));after that I export the table via exp. I'm not setting NLS_LENGTH_SEMANTICS environment variable, so BYTE is used.
In the dump file the data length semantics for the byte col is omitted, as I exported it with BYTE:
create table abcd (a_char VARCHAR2(2 CHAR), a_byte VARCHAR2(2));after that, I "accidently" import it with data length semantics set to CHAR, and the table looks like this now
abcd
a_char VARCHAR2(2 CHAR)
a_byte VARCHAR2(2 CHAR)Same happens vice versa when using CHAR for export and BYTE for import...
In single byte charsets this might not be so much of a problem, as one CHAR is equal to one BYTE, but...
If I compile plsql against the original table, and run against the outcoming table after export, I get an ORA-4062, and I have to recompile...
Would not be a problem if the plsql I compile would be on the database...Big problem is that the ORA-4062 occurs in forms, where it's difficult for me to recompile (I would have to transfer all the sources to customer and compile there).
Is there any possibility to export data length semantics regardless which environment variable is set?
database version would be 9.2.0.6; but if there exists a solution in higher versions I would also be happy to hear them...
many thanks,
regards

I can't reproduce your problem:
SQL> show parameter nls_length_semantics
NAME                                 TYPE        VALUE
nls_length_semantics                 string      BYTE
SQL> create table scott.demo( col1 varchar2(10 byte), col2 varchar2(10 char) );
SQL> describe scott.demo
Name                                      Null?    Type
COL1                                               VARCHAR2(10)
COL2                                               VARCHAR2(10 CHAR)
$ export NLS_LENGTH_SEMANTICS=BYTE
$ exp scott/tiger file=scott.dmp tables=demo
SQL> drop table scott.demo;
$ export NLS_LENGTH_SEMANTICS=CHAR
$ imp scott/tiger file=scott.dmp
SQL> describe scott.demo
Name                                      Null?    Type
COL1                                               VARCHAR2(10 BYTE)
COL2                                               VARCHAR2(10)
SQL> alter session set nls_length_semantics=byte;
SQL> describe scott.demo
Name                                      Null?    Type
COL1                                               VARCHAR2(10)
COL2                                               VARCHAR2(10 CHAR)Can you post a test like mine?
Enrique
PS If you have access to Metalink, read Note:144808.1 Examples and limits of BYTE and CHAR semantics usage. From 9i and up, imp doesn't read nls_length_semantics from the environment.
Edited by: Enrique Orbegozo on Dec 16, 2008 12:50 PM
Edited by: Enrique Orbegozo on Dec 16, 2008 12:53 PM

Data Length Semantics?

Dear memebers,
what is Data Length Semantics property of text box?
what's its behavior?
thanks
Muhammad Nadeem
[email protected]

Values CHAR, BYTE, null
Refer to Built-in
GET_ITEM_PROPERTY
Usage Notes
- If a null value is specified, then byte semantics will be used unless the environment variable NLS_LENGTH_SEMANTICS is set to CHAR when the form is compiled.
- When the Synchronize with Item property is set, DATA_LENGTH_SEMANTICS will be ignored in a subordinate mirror item. The DATA_LENGTH_SEMANTICS property is always taken from the master mirror item. A compiler (generator) warning will be issued if a non-null value is specified in a subordinate mirror item.
- A compiler (generator) warning will also be issued if a non-null value is specified in an item whose datatype is neither CHAR, ALPHA, nor LONG.

Character length semantics

Hi,
in 10g R2 how to enable Character length semantics ?
Thank you.

You cannot just enable character length semantics.
The following link would be helpful.
You need to export schema and import schema after setting the parameter
NLS_LENGTH_SEMANTICS=CHAR
[Character semantics |http://www.oracle.com/technology/oramag/oracle/03-mar/o23sql.html]

Issues caused by changing length semantics

Hi All,
Our database was formerly using BYTE for columns of tables and for stored procedures. We recently changed the Length Semantics to CHAR but caused some issues. An ORA-06502 pl/sql numeric or value error character string buffer too small appears when we access the database's stored procedures via Java. What could be the possible cause of this? Could you give me some paths to take in troubleshooting this issue?
Thanks to all!
Edited by: 1002671 on 25.4.2013 23:55

1002671 wrote:
Thanks for answering Sir! Are you kidding!!! No 'Sir' please... Common I don't know anything yet.
Correct me if I'm wrong but doesn't CHAR already handle multi-byte characters passed to or used in stored procedures? I'm really not that knowledgeable when it comes to the effects of changing the Length Semantics. We already changed the columns from VARCHAR2(BYTE) to VARCHAR2(CHAR). The problem lies within the stored procedures.I'm not clear on your doubt, but please check this -
Link: http://docs.oracle.com/cd/E11882_01/appdev.112/e10472/datatypes.htm (Section 'Declaring Variables for Multibyte Characters')
>
When declaring a CHAR or VARCHAR2 variable, to ensure that it can always hold n characters in any multibyte character set, declare its length in characters—that is, CHAR(n CHAR) or VARCHAR2(n CHAR), where n does not exceed FLOOR(32767/4) = 8191.
>
What i feel is you getting confused with the SQL data-type 'VARCHAR2' (i.e. used in specifying column type) and PL/SQL data-type 'VARCHAR2' (i.e. used while declaring variables)
Then check this : difference between BYTE & CHAR
Read and thoroughly research each comment given by the Experts there.

Column Length Semantics and Table Columns - NLS_LENGTH_SEMANTICS

Is NLS_LENGTH_SEMANTICS defined in oracle 10g?
if not what is its equivalent.
Please let me know about this.

$ sqlplus / as sysdba
SQL*Plus: Release 10.2.0.1.0 - Production on Mon May 22 23:30:14 2006
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to:
Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
SYS@db102 SQL> select * from nls_database_parameters
2 where parameter = 'NLS_LENGTH_SEMANTICS';
PARAMETER                      VALUE
NLS_LENGTH_SEMANTICS           BYTE
SYS@db102 SQL>

Field length misreported in application with AL32UTF8 database

Hi all,
I have the following problem with two different applications running against an Oracle 10.2 database with AL32UTF8 character set. The tables in the database are created using NLS_LENGTH_SEMANTICS=CHAR. The problem is that in the applications, the length of the fields are misreported as being 3 or 4 times longer, so that the users can input too much data which can then not be stored. For example with a field created as VARCHAR2(20 CHAR), it is reported in the application as either 60 or 80 characters depending on where it is run from.
I suspect this is because both applications don't implement proper support for length semantics. One is a Win32 application written in Delphi and using the SqlDirect interface to OCI. The other is a Microsoft .NET framework 1.1 application using the Microsoft System.Data.OracleClient interface to OCI. The application can read and write data including international characters but the lengths are incorrect.
I don't know much about the low-level OCI details but when scanning the documentation I found some information about OCI_ATTR_CHAR_SIZE which is what I suppose the application should use to get the character length of a field.
I made a quick test program using the Oracle Data Provider for .NET and with that it seems to report the column lengths correctly, but changing to that for the .NET application is not something that can be done quickly.
I would be very interested to know if anyone else has encountered similar issues, and if there was a solution that did not involve changing the application, i.e. is there some parameter or something that could be changed so that the application reports the size in characters?
If this is indeed an application/driver bug as I suspect, then I will take the question to the corresponding supplier.
Thank you very much in advance for any input you can give on this issue
Robert

The third-part drivers, like those you mentioned, are usually built on OCI and, if they are not aware of character semantics, they always use OCI_ATTR_DATA_SIZE attribute to retrieve the column length. This attribute is always in bytes and there is no option to change this.
-- Sergiusz

How to load unicode data files with fixed records lengths?

Hi!
To load unicode data files with fixed records lengths (in terms of charachters and not of bytes!) using SQL*Loader manually, I found two ways:
Alternative 1: one record per row
SQL*Loader control file example (without POSITION, since POSITION always refers to bytes!)<br>
LOAD DATA
CHARACTERSET UTF8
LENGTH SEMANTICS CHAR
INFILE unicode.dat
INTO TABLE STG_UNICODE
TRUNCATE
A CHAR(2) ,
B CHAR(6) ,
C CHAR(2) ,
D CHAR(1) ,
E CHAR(4)
) Datafile:
001111112234444
01NormalDExZWEI
02ÄÜÖßêÊûÛxöööö
03ÄÜÖßêÊûÛxöööö
04üüüüüüÖÄxµôÔµ Alternative2: variable length records
LOAD DATA
CHARACTERSET UTF8
LENGTH SEMANTICS CHAR
INFILE unicode_var.dat "VAR 4"
INTO TABLE STG_UNICODE
TRUNCATE
A CHAR(2) ,
B CHAR(6) ,
C CHAR(2) ,
D CHAR(1) ,
E CHAR(4)
) Datafile:
001501NormalDExZWEI002702ÄÜÖßêÊûÛxöööö002604üuüüüüÖÄxµôÔµ Problems
Implementing these two alternatives in OWB, I encounter the following problems:
* How to specify LENGTH SEMANTICS CHAR?
* How to suppress the POSITION definition?
* How to define a flat file with variable length and how to specify the number of bytes containing the length definition?
Or is there another way that can be implemented using OWB?
Any help is appreciated!
Thanks,
Carsten.

Hi Carsten
If you need to support the LENGTH SEMANTICS CHAR clause in an external table then one option is to use the unbound external table and capture the access parameters manually. To create an unbound external table you can skip the selection of a base file in the external table wizard. Then when the external table is edited you will get an Access Parameters tab where you can define the parameters. In 11gR2 the File to Oracle external table can also add this clause via an option.
Cheers
David

Loading "fixed length" text files in UTF8 with SQL*Loader

Hi!
We have a lot of files, we load with SQL*Loader into our database. All Datafiles have fixed length columns, so we use POSITION(pos1, pos2) in the ctl-file. Till now the files were in WE8ISO8859P1 and everything was fine.
Now the source-system generating the files changes to unicode and the files are in UTF8!
The SQL-Loader docu says "The start and end arguments to the POSITION parameter are interpreted in bytes, even if character-length semantics are in use in a datafile....."
As I see this now, there is no way to say "column A starts at "CHARACTER Position pos1" and ends at "Character Position pos2".
I tested with
load data
CHARACTERSET AL32UTF8
LENGTH SEMANTICS CHARACTER
replace ...
in the .ctl file, but when the first character with more than one byte encoding (for example ü ) is in the file, all positions of that record are mixed up.
Is there a way to load these files in UTF8 without changing the file-definition to a column-seperator?
Thanks for any hints - charly

I have not tested this but you should be able to achieve what you want by using LENGTH SEMANTICS CHARACTER and by specifying field lengths (e.g. CHAR(5)) instead of only their positions. You could still use the POSITION(*+n) syntax to skip any separator columns that contain only spaces or tabs.
If the above does not work, an alternative would be to convert all UTF8 files to UTF16 before loading so that they become fixed-width.
-- Sergiusz

Oracle 11g special character(á) length count

Hi Experts,
I'm facing an issue while inserting the special character (á) in Oracle 11g. The database table length for the column 'descr' declared as varchar2(30). I'm inserting a string ('Aabbccee/ddffggh Ffrán y Plaos') in column descr which is having the length 30. Please note the character 'a' in Ffrán is not letter 'a' but special character *'á'*. It gives the below error,
java.sql.SQLException: ORA-12899: value too large for column ""DESCR" (actual: 31, maximum: 30)
The strange is the same query works/insert in Oracle 9i without any issue. I suspect the special character *'á'* will take/count as 2 character in Oracle 11g but single character in Oracle 9i.
Please provide the insight views and solution for this issue with out altering the column length.
I hope the solution at the earliest, since this is an urgent requirement.
Thanks.

Your 9i database has character set WE8ISO8859P1 which is a single byte characterset.
But on 11g, you're using AL32UTF8 - a multi-byte unicode characterset.
Obviously with a multi-byte characterset, characters which took up a single byte might now need multiple bytes.
So the default meaning of VARCHAR2(30) which is VARCHAR2(30 BYTE) may not be sufficient to hold 30 characters.
Assuming that you want to use the different characterset - (and if not then you might as well rebuild your 11g database with the 9i characterset), then you need to do is to change your LENGTH SEMANTICS to CHAR not BYTE.
This means two things.
1. Changing NLS_LENGTH_SEMANTICS to CHAR via ALTER SYSTEM. I believe this needs a restart to take effect. This will affect newly created column definitions so that VARCHAR2(30) means VARCHAR2(30 CHAR) not VARCHAR2(30 BYTE) - so for changes going forward.
2. Changing existing column definisition via ALTER TABLE xyz MODIFY colname VARCHAR2(30 CHAR) etc.

Length semantics

Similar Messages

Maybe you are looking for