Unusual characters in database

If an XSQL query returns certain characters, the resulting XML is
invalid (note the character below is hex 1F):
<?xml version="1.0" encoding="UTF-8" ?>
<xsql:query connection="cons" xmlns:xsql="urn:oracle-xsql">
<![CDATA[
SELECT '' AS "BAD",
'&' AS "OK1",
'<' AS "OK2"
FROM dual
]]>
</xsql:query>
if you run XSQLCommandLine on the file and save the results to a
file, the ampersand and angle bracket are properly encoded in the
resulting XML file, but the first character is not. Any attempts
to parse it fail, saying the document is not well formed.
I've tried changing the encoding value, but nothing seems to
work.
I've also tried using XSQLRequest in my own program. The query
completes and returns a seemingly correct XMLDocument object, but
when printed, the character is not escaped properly.
If anyone has a tip, I'd appreciate it.

Hi altenbach,
I see what you mean - I will add this to my list of bugs.
Thanks!
Laura

Similar Messages

Entry of non-English characters into database

Hi
We are facing a problem in inserting non-English characters into the database.For example, we have a company name field which can accept German characters. This field has been defined as of varchar2 type of size 50 in the db. When we enter 49 English characters and then one German character, the database is throwing the error that the inserted value is too large for the column.Is it that the German character is taken as equivalent to two English characters ? Or is there any database level setting that can be done for this ? For the time being we have identified certain critical fields and have doubled the size of their fields in the db. But I guess there has to be another solution to this....
Please help.
TIA
Vinoj

Indeed, your German character is using two bytes to store itself. Consult the Oracle JDBC Developer's Guide.
null

Formatting non-English Characters in Database Extracts

Hi
I am trying to create data flat file with Oracle SQL extracts. The data file is position based i.e. Position 1-30 for First Name, 31-50 for Last name, etc. I encountered problems when the data fields contain non-English characters. The position shift right by 1 with every non-English characters. For example, if there is one Spanish char in First Name, Last Name will start from 32, instead of 31. If there are two Spanish chars, Last Name will start from 33 (shift 2). Is there any way, in a database session, to restrict the formats of these text fields such that non-English characters will not affect the data field position in the flat files?
Thanks in advance
Jason

An alternative might be to tab or comma delimit your data.
Eric

How to get 10 characters if there are 18 characters in database field

IF there are four fields in a internal table and each field have 18 character length. i should less the filed length of only 3 fields to 10 characters and should get this fields in Application layer and flat file ( out bound interface),
so can u guide me the logic how to populate the data in application layer and flat file with new field length of 3 fields and old field length of fourth field.

Hi Azeez,
First declare your internal by referring database fields and fetch data from data base into this internal table.
Ex:
types: begin of stru,
                             f1 type <t1>-f1,
                             f2 type <t1>-f2,
                             f2 type <t1>-f2,
                             f2 type <t1>-f2,
                          end of stru.
Lets consider all the above fields have length 18 in database.
data: itab type standard table of stru.
data: wa_itab type stru.
data: lv_char(48) type c.
select f1 f2 f3 f4 from t1 into table itab.
loop at itab into wa_itab.
concatenate wa_itab-f1+0(10) wa_itab-f2+0(10) wa_itab-f3+0(10) wa_itab-f4
              into lv_char.
<b>now transfer this variable (lv_char) to application server.</b>
endloop.
hope this helps.
Regards,
Phani.

Need to find out extended ASCII characters in database

Hi All,
I am looking for a query that can fetch list of all tables and columns where there is a extended ASCII character (from 128 to 256). Can any one help me?
Regards
Yadala

yadala wrote:
Hi All,
I am looking for a query that can fetch list of all tables and columns where there is a extended ASCII character (from 128 to 256). Can any one help me?
Regards
YadalaThis should match your requirement:
select t.TABLE_NAME, t.COLUMN_NAME from ALL_TAB_COLUMNS t
where length(asciistr(t.TABLE_NAME))!=length(t.TABLE_NAME)
or length(asciistr(t.COLUMN_NAME))!=length(t.COLUMN_NAME);The ASCIISTR function returns an ASCII version of the string in the database character set.
Non-ASCII characters are converted to the form \xxxx, where xxxx represents a UTF-16 code unit.
The CHR function is the opposite of the ASCII function. It returns the character based on the NUMBER code.
ASCII code 174
SQL> select CHR(174) from dual;
CHR(174)
Ž
SQL> select ASCII(CHR(174)) from dual;
ASCII(CHR(174))
174
SQL> select ASCIISTR(CHR(174)) from dual;
ASCIISTR(CHR(174))
\017DASCII code 74
SQL> select CHR(74) from dual;
CHR(74)
J
SQL> select ASCII(CHR(74)) from dual;
ASCII(CHR(74))
74
SQL> select ASCIISTR(CHR(74)) from dual;
ASCIISTR(CHR(74))
J

Unable to insert Chinese characters in Database

My problem is that I am not able to insert chinese
(to traditional chinese) characters into my tables in the
database.
I have changed the character set to UTF8 while creating the
database and also tried the alter session command in SQL to
alter the NLS_LANGUAGE and NLS_TERRITORY (to say traditional chinese).
But this did not solve my problem.
Also tried all possibilites like getting Chinese characters
in my notepad by copy - paste from a Chinese web site
but while giving the insert into command in my database
it takes some junk values.
Someone PLEASE HELP!!!URGENT!!!
Thanks in advance.
RKP
null

You mentioned in your first note that you have set your database character set to UTF-8? If so, then you are able to store and retrieve multilingual data, including Chinese and Japanese characters. Your issue is not the database. Your client OS must be able to support these languages as well. It is likely that your version of OS supports only Latin and Western European characters. By the way changing your NT regional setting only effects sorting, date formats etc. It doesn't help you change the languages that your keyboard will support.
1.To determine your Win32 operating system's current ANSI CodePage (ACP). This can be found by bringing up the registry editor (Start --> Run..., type "regedit.exe", and click "OK") and looking at the
registry entry HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Nls\CodePage\ACP (there are many registry entries with very similar names, so please make sure that you are looking at the right place in the registry).
2.To find the character set in the table below based on the ACP you got above.
ANSI CodePage (ACP) Client character set (3rd part of NLS_LANG) (*1)
1250 EE8MSWIN1250
1251 CL8MSWIN1251
1252 WE8MSWIN1252
1253 EL8MSWIN1253
1254 TR8MSWIN1254
1255 IW8MSWIN1255
1256 AR8MSWIN1256
1257 BLT8MSWIN1257
1258 VN8MSWIN1258
874 TH8TISASCII
932 JA16SJIS
936 ZHS16GBK
949 KO16MSWIN949
950 ZHT16MSWIN950
others UTF8 (*2)
(*1) The character sets listed here are compatible with Win32's non-Unicode graphical user interface (GUI). Since Win32's MSDOS Box (Command Prompt) uses different character sets, NLS_LANG needs to be manually set in the MSDOS Box (or set in a batch script) in order to handle the difference
between Win32's GUI and MSDOS Box. (Please see "NLS_LANG Settings in MS-DOS Mode and Batch Mode" in the Oracle8i Installation Guide Release 2 (8.1.6) for Windows NT, part# A73010-01.)
(*2) If you use UTF8 for the 3rd part of NLS_LANG on Win32, client programs that you can use on this operating system would be limited to the ones that explicitly support this configuration. Recent versions of Oracle Forms' Client/Server mode (Fat-Client) on NT4.0 would be an example of such client
programs. This is because the user interface of Win32 is not UTF8, therefore the client programs have to perform explicit conversions between UTF8 (used in Oracle side) and UTF16 (used in Win32 side).

Attach file to mail with "unusual" characters

Hi
I have made a procedure that can send mail with attached file/files. But when I attach a text file that contains characters like (Å, Ä or Ö) these characters gets like (A, A and ?) when I recieve the mail to my mail client. When I look at the file on the disk before I attach it the characters is OK but when I attach the file and mails it the characters get replaced. So why does the charaters get messed up when I attach the file to a mail??
My procedure looks like this:
procedure mail_files(from_name varchar2,
to_name varchar2,
subject varchar2,
message varchar2,
max_size number default 9999999999,
filename1 varchar2 default null,
filename2 varchar2 default null,
filename3 varchar2 default null,
debug number default 0) is
mottagare VARCHAR2(2048) := to_name;
v_smtp_server varchar2(32) := 'mail.telia.se';
v_smtp_server_port number := 25;
v_directory_name varchar2(100);
v_file_name varchar2(100);
v_line varchar2(1000);
crlf varchar2(2) := chr(13) || chr(10);
mesg varchar2(32767);
conn UTL_SMTP.CONNECTION;
type varchar2_table is table of varchar2(200) index by binary_integer;
file_array varchar2_table;
i binary_integer;
v_file_handle utl_file.file_type;
v_slash_pos number;
mesg_len number;
mesg_too_long exception;
invalid_path exception;
mesg_length_exceeded boolean := false;
begin
-- first load the three filenames into an array for easier handling later ...
file_array(1) := filename1;
file_array(2) := filename2;
file_array(3) := filename3;
-- Open the SMTP connection ...
conn := utl_smtp.open_connection(v_smtp_server, v_smtp_server_port);
-- Initial handshaking ...
utl_smtp.helo(conn, v_smtp_server);
utl_smtp.mail(conn, from_name);
WHILE (mottagare IS NOT NULL) LOOP
utl_smtp.rcpt(conn, get_element(mottagare));
END LOOP;
--utl_smtp.rcpt( conn, to_name );
utl_smtp.open_data(conn);
-- build the start of the mail message ...
mesg := 'Date: ' || TO_CHAR(SYSDATE, 'dd Mon yy hh24:mi:ss') || crlf ||
'From: ' || from_name || crlf || 'Subject: ' || subject || crlf ||
'To: ' || to_name || crlf || 'Mime-Version: 1.0' || crlf ||
'Content-Type: multipart/mixed; boundary="DMW.Boundary.605592468"' || crlf || '' || crlf ||
'This is a Mime message, which your current mail reader may not' || crlf ||
'understand. Parts of the message will appear as text. If the remainder' || crlf ||
'appears as random characters in the message body, instead of as' || crlf ||
'attachments, then you''ll have to extract these parts and decode them' || crlf ||
'manually.' || crlf || '' || crlf ||
'--DMW.Boundary.605592468' || crlf ||
'Content-Type: text/html; charset=8859-1' || crlf ||
'Content-Disposition: inline;' || crlf ||
'Content-Transfer-Encoding: 8bit' || crlf || '' || crlf ||
message || crlf;
mesg_len := length(mesg);
if mesg_len > max_size then
mesg_length_exceeded := true;
end if;
utl_smtp.write_data(conn, mesg);
--utl_smtp.write_raw_data(conn, utl_raw.cast_to_raw(mesg));
-- Append the files ...
for i in 1 .. 3 loop
-- Exit if message length already exceeded ...
exit when mesg_length_exceeded;
-- If the filename has been supplied ...
if file_array(i) is not null then
begin
-- locate the final '/' or '\' in the pathname ...
v_slash_pos := instr(file_array(i), '/', -1);
if v_slash_pos = 0 then
v_slash_pos := instr(file_array(i), '\', -1);
end if;
-- separate the filename from the directory name ...
v_directory_name := substr(file_array(i), 1, v_slash_pos - 1);
v_file_name := substr(file_array(i), v_slash_pos + 1);
-- open the file ...
v_file_handle := utl_file.fopen(v_directory_name, v_file_name, 'r');
-- generate the MIME boundary line ...
mesg := crlf || '--DMW.Boundary.605592468' || crlf ||
'Content-Type: application/octet-stream; name="' ||
v_file_name || '"' || crlf ||
'Content-Disposition: attachment; filename="' ||
v_file_name || '"' || crlf ||
'Content-Transfer-Encoding: 8bit' || crlf || crlf;
mesg_len := mesg_len + length(mesg);
utl_smtp.write_data(conn, mesg);
-- and append the file contents to the end of the message ...
loop
utl_file.get_line(v_file_handle, v_line);
if mesg_len + length(v_line) > max_size then
mesg := '*** truncated ***' || crlf;
utl_smtp.write_data(conn, mesg);
mesg_length_exceeded := true;
raise mesg_too_long;
end if;
mesg := v_line || crlf;
utl_smtp.write_data(conn, mesg);
mesg_len := mesg_len + length(mesg);
end loop;
exception
when utl_file.invalid_path then
-- All other exceptions are ignored ....
when others then
null;
end;
mesg := crlf;
utl_smtp.write_data(conn, mesg);
-- close the file ...
utl_file.fclose(v_file_handle);
end if;
end loop;
-- append the final boundary line ...
mesg := crlf || '--DMW.Boundary.605592468--' || crlf;
utl_smtp.write_data(conn, mesg);
-- and close the SMTP connection ...
utl_smtp.close_data(conn);
utl_smtp.quit(conn);
end;
Any help is highly appreciated.
Regards
Nils

Hi Madhu,
Using the default modules PayloadSwapBean and MessageTransformBean you can able to send multiple attachments.
swap.keyName :payload-name or payload-description or content-type, content-description. swap.keyValue: If you have a multiple attachments then give the multiple key value.
Like you have payload name
Attachment1, Attachment2 then
swap.keyName payload-name
swap.keyValue Attachment1, Attachment2
Using the MessageTransformBean you can able to change the file name.
PayloadSwapBean:
[http://help.sap.com/saphelp_nw70/helpdata/EN/2e/bf37423cf7ab04e10000000a1550b0/content.htm]
MessageTransformBean:
[http://help.sap.com/saphelp_nw70/helpdata/EN/57/0b2c4142aef623e10000000a155106/content.htm]
Regards,
Prakasu

Trouble in writing Chinese Characters into database

I'm developing a JSP web site using ACCESS2002, Tomcat4.0.4 and J2EE 1.4.1. I found my data(chinese chatacters) in mess when I wrote it in to database through sun's own JDBC-ODBC driver though it is displayed correctly in web-browser. I tried in JSP, Servlet and JavaBean, and the situation seems no better.
Is it the problem of sun's JDBC-ODBC driver or is there a solution to this problem?
Eager to hear a reply, thank millions!!

Have you tried something like the following?
first, set this page quality:
<%@ page contentType="text/html;charset=Big5"%>
then, change the Chinese string to this:
String zhtounicode=new String(yourzhstring.getBytes("8859_1"));
then, do with the converted string.

Chinese and Japanese characters in database

I have Customer table in which Name column can contain Chinese characters. I have given it's datatype as NVARCHAR. I am writing simple SELECT statement. But it is showing question marks instead of Chinese characters. What should I do to get correct data?
In Below Image, you can see that after 2nd statement, @NAME variable value automatically gets those question marks which is not returning correct answer.

Hello,
You have to add the Unicode literal N in front of the string, like
SET @NAME = N'here comes the text';
Olaf Helper
[ Blog] [ Xing] [ MVP]

Insert chinese big5 characters into database

database: mysql
default charset of web page: utf-8 (specified in apache, all
pages are in utf-8)
if I enter english, database insert is ok but if I enter
chinese, error = "You have an error in your SQL syntax; check the
manual that corresponds to your MySQL server version for the right
syntax to use near '?, ?)' at line 1"
display inputted fields are OK.
*** form.cfm ***
<form action="insert.cfm" method="post">
name: <input type="text" name="name"
value=""><BR>
Password: <input type="text" name="password"
value=""><BR>
<input type="submit" name="login_user"
value="insert"><BR>
</form>
*** insert.cfm ***
<cfoutput>
#Form.name#<Br>
#form.password#<br>
</cfoutput>
<cfquery name="insert" datasource="test">
insert into test (name,password) values ('#Form.name#',
'#Form.password#')
</cfquery>

dynason wrote:
> database: mysql
> default charset of web page: utf-8 (specified in apache,
all pages are in
> utf-8)
what version of cf5? what version of mysql? has to be at
least 4.1. is the db,
etc. setup to handle that encoding? do you really want to use
BIG5 instead of
unicode? what db driver? it can't be ODBC.

UTF-8 characters in database

Hi there,
I'm having problems with UTF-8 characters displaying incorrectly. The problem seems to be that the Content-Type HTTP headers have the character set as "Windows 1252" when it should be UTF-8. There is a demonstration of the problem here:
http://teasel.homeunix.net/~rah/screenshots-unicode-db/apex-unicode-display.html
I though I'd solved this previously, because I changed the nls_lang variable in wdbsvr.app to use single-quotes instead of double-quotes. After I did this, the server started sending Content-Type HTTP headers with UTF-8 in them and the characters in the example above displayed fine. For some reason, it seems to have reverted to sending "Windows 1252" and I don't know why.
I noticed that the logs contain the following lines:
[Wed Nov 1 09:50:45 2006] [alert] mod_plsql: Wrong language for NLS_LANG 'ENGLISH_UNITED KINGDOM.AL32UTF8' for apex DAD
[Wed Nov 1 09:50:45 2006] [alert] mod_plsql: Wrong charset for NLS_LANG 'ENGLISH_UNITED KINGDOM.AL32UTF8' for apex DAD
Any help getting it to send UTF-8 would be greatly appreciated.
Thanks,
Robert

However I believe that it is correct to use double
quotes rather than single quotes where the setting in
the nls_lang contains space.Well, this is odd; using double quotes gives the same error, but with the double quotes instead of the single in the error message:
[Thu Nov 2 09:43:42 2006] [alert] mod_plsql: Wrong language for NLS_LANG "ENGLISH_UNITED KINGDOM.AL32UTF8" for apex DAD
[Thu Nov 2 09:43:42 2006] [alert] mod_plsql: Wrong charset for NLS_LANG "ENGLISH_UNITED KINGDOM.AL32UTF8" for apex DAD
This implies that the parser is pulling the string out verbatim and including the quotes when it shouldn't. Lo and behold, removing any quotes:
nls_lang = ENGLISH_UNITED KINGDOM.AL32UTF8
causes the error to go away, and causes the HTTP headers to declare UTF-8; problem solved.
I'm loathe to step ahead again and say that the docs need updating, but it certainly looks that way.
Robert

Unusual characters displayed... missing a locale or..? [solved]

What is the cause of these usual characters when I "man patch"? Am I missing a locale or something?
Last edited by graysky (2011-10-19 17:29:53)

@SF - I think I am:
$ grep utf8 /etc/rc.conf
LOCALE="en_US.utf8"
$ grep en_US /etc/locale.gen
en_US.UTF-8 UTF-8
en_US ISO-8859-1
The screenshot came from terminal 0.4.8 using monospace/10. HOWEVER, if I install gnome-terminal and look at the same manpage, I do indeed see a bullet point! Perhaps this is a limitation of terminal?
EDIT: Yes! This is the problem. I followed the advice on this wiki page adding the following to /etc/environment:
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
Rebooted and now there are no funny characters. Thanks for pointing me in the right direction.
Last edited by graysky (2011-10-19 17:30:06)

Update invalid characters in database using regexp_replace

I have a table in which a column has invalid characters like ` Is it possible to update this using regexp_replace with correct character '?
Thanks

Thank You Volder this works.
I also need to replace if testcol has CHR(10) or CHR(11) with CHR(20) can this be done with one single regexp_replace? I don't want to use multiple regexp_replace statements one after the other.
like
update test set testcol = REGEXP_REPLACE(testcol, CHR(10), CHR(11) , CHR(20));
Thank You very much!

Store big5 characters in database using JDBC

The key point is the character set of database.
Try character set 'WE8ISO8859P1'
Good Luck.
null

Thanks for ur help:
another Q:
in my GUI i have a button called Insert. but can't update my database table unless i create textField(e.g Firstname, Surname..ect..), so that when I click Insert button in actionEvent it will bring up popup which have field name in which i am going to insert new records.
so what iam trying to say is, how iam going to tell the actionEvent to trigger this.
the following is my code
public void actionPerformed(ActionEvent event)
ExamResults exam= new ExamResults ();
String studentId = studentTextField.getText();
if (event.getSource() == insert) //if insert is clicked
Thanks

Help ! getting hexadecimal characters from database

Hello !
I am writing a servlet using jdbc 2.0 ( bidirectional cursor) ... exactly, oracle 8.1.7r2 under Solaris 2.8, JDBC drivers from Oracle 9.0.1i -2.0.
I am getting hexadecimal characters from a query, somethings like :
Title > 0x436F6D65646961
0x4170726F6261646F20656E204361737469646164
It wonderful !
I get the following data from oracle configuration:
--> select * from v$nls_parameters:
PARAMETER(VARCHAR2) VALUE(VARCHAR2)
NLS_LANGUAGE AMERICAN
NLS_TERRITORY AMERICA
NLS_CURRENCY $
NLS_ISO_CURRENCY AMERICA
NLS_NUMERIC_CHARACTERS .,
NLS_CALENDAR GREGORIAN
NLS_DATE_FORMAT DD-MON-RR
NLS_DATE_LANGUAGE AMERICAN
NLS_CHARACTERSET WE8ISO8859P15
NLS_SORT BINARY
NLS_TIME_FORMAT HH.MI.SSXFF AM
NLS_TIMESTAMP_FORMAT DD-MON-RR HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT HH.MI.SSXFF AM TZH:TZM
NLS_TIMESTAMP_TZ_FORMAT DD-MON-RR HH.MI.SSXFF AM TZH:TZM
NLS_DUAL_CURRENCY $
NLS_NCHAR_CHARACTERSET WE8ISO8859P15
NLS_COMP BINARY
But, i wonder, why is different result from this query:
-->select * from v$parameter
60 nls_language 2 SPANISH FALSE TRUE FALSE FALSE FALSE NLS language name
61 nls_territory 2 SPAIN FALSE TRUE FALSE FALSE FALSE NLS territory name
62 nls_sort 2 null TRUE TRUE FALSE FALSE FALSE NLS linguistic definition name
63 nls_date_language 2 null TRUE TRUE FALSE FALSE FALSE NLS date language name
64 nls_date_format 2 null TRUE TRUE FALSE FALSE FALSE NLS Oracle date format
65 nls_currency 2 null TRUE TRUE FALSE FALSE FALSE NLS local currency symbol
66 nls_numeric_characters 2 null TRUE TRUE FALSE FALSE FALSE NLS numeric characters
67 nls_iso_currency 2 null TRUE TRUE FALSE FALSE FALSE NLS ISO currency territory name
68 nls_calendar 2 null TRUE TRUE FALSE FALSE FALSE NLS calendar system name
69 nls_time_format 2 null TRUE TRUE FALSE FALSE FALSE time format
70 nls_timestamp_format 2 null TRUE TRUE FALSE FALSE FALSE time stamp format
71 nls_time_tz_format 2 null TRUE TRUE FALSE FALSE FALSE time with timezone format
72 nls_timestamp_tz_format 2 null TRUE TRUE FALSE FALSE FALSE timestampe with timezone format
73 nls_dual_currency 2 null TRUE TRUE FALSE FALSE FALSE Dual currency symbol
74 nls_comp 2 null
Why is differente language ?
Sometimes i get an exception:
java.sql.SQLException: Character Set Not Supported !!: DBConversion
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:180)
at oracle.jdbc.dbaccess.DBError.throwSqlException(DBError.java:222)
at oracle.jdbc.dbaccess.DBError.check_error(DBError.java:916)
What is wrong ?
Anyone happend something similar ?
Help please !
[email protected]

Charset in the machine you develop must be the same with the Charset in DB (WE8ISO8859P15). Or try request.setCharacterEncoding("ISO_8859_15"); on top of servlet.

Unusual characters in database

Similar Messages

Maybe you are looking for