Smart quotes stored in DB are displaying as ? after jdbc select

I originally posted this to the general forum, but it was suggested that I post it here where Sergiusz Wolicki from Oracle might be able to help.
I have an Oracle 10 DB set up with the following character set
NLS_LANGUAGE AMERICAN
NLS_CHARACTERSET WE8MSWIN1252
NLS_NCHAR_CHARACTERSET AL16UTF16
I am connecting to the database using JDBC thin client. I have a web page that is encoded using
<%@ page language="java" pageEncoding="Windows-1252" contentType="text/html;charset=Windows-1252" %>
The scenarion I am having problems with is the following. In an input field the user copies a so-called smart quote from Word and submits. I then update the corresponding database field and then return to the page and display the result.
If I display the string that I wrote to the database back in the page, without selecting it from the database, it displays correctly. However, if I select it from the database and then display it in the page I get a question mark ('?') - note that this is NOT the upside down variety.
I believe it is inserting it into the database correctly. I can go into sql plus and do a select on the field and it appears to be correct. I can also use the ascii function and it shows the character as character 145. Here is what I did:
select middle_name from employee where employee_id=100;
MIDDLE_NAME
select ascii(middle_name) from employee where employee_id=100;
ASCII(MIDDLE_NAME)
145
So my problem is getting it out of the database and into a java string properly.
I know I can solve this issue by changing the database character set to UTF-8, but I am not in control of this and it is not an option.
Thanks in advance for any help on this.

I suspect this may be the result of bug #4659157. Upgrade your JDBC driver to 10.2.0.4 or to any 11g driver and retest.
-- Sergiusz

Similar Messages

Re: Smart Quotes crashes reports running on Reports Server 11g/Weblogic 10.

Anyone experience smart quotes stored in oracle tables that causing report to crash. Is there a parameter option in Reports 11g/Weblogic or maybe a characterset change. We are looking at modifying the call package but would prefer a parameter change. Thanks.

Ricardo,
I've done a lot of VM installations of WebLogic 11g w/ Forms but never used these VM "templates" that you use. I've always installed weblogic and Forms on a freshly installed operating system (Solaris,OEL,Redhat,Windows,etc..).
Please refer to forms installation documentation provided by Oracle: http://download.oracle.com/docs/cd/E14571_01/install.1111/e10421/toc.htm
Secondly, the statement "Oracle weblogic 10gR3" concerns me - what exact version of oracle weblogic server is installed?
You need 11g Forms with 11g Oracle WebLogic Server. The versions are a little confusing but you need one of the following setups:
1.) WebLogic Server 11g (v10.3.2) with Forms 11g (11.1.1.2)
1.) WebLogic Server 11g (v10.3.3) with Forms 11g (11.1.1.3)
2.) WebLogic Server 11g (v10.3.4) with Forms 11g (11.1.1.4) (Just recently released)
Note: Forms 11.1.1.3 and Forms 11.1.1.4 are patchsets.
Thirdly, for system specs. All minimum requirements are noted in the installation guide link i mentioned above. It depends on what environment this is and how many apps, users, etc. etc. However, for development environments i've done: usually 25GB+ HDD space, 4GB RAM, 1 CPU (1.5 Ghz+) has been sufficient.
Hope this helps.
Thanks,
Gavin

CharConversionException when displaying smart quotes

My servlets are throwing a CharConversionException when I attempt to display "smart quotes." The quotes and other symbols (i.e. long hyphen) are entered into an online form when a user composes something in Word 2000 then copies/pastes it into a text field. The servlet is able to get the parameter and update the MS SQL 7 database but not display the value back in the confirmation page. I am running this application on iPlanet web server v.6 SP 1. It seems to be related to my servlet engine as it doesn't have this problem when using the servlet runner (srun)--that will display the characters as a question mark. This is a set of �smart quotes� and a long�hyphen

problem solved:
1.- I changed the response content type of my Servlet to:
response.setContentType("text/html;charset=\"windows-1252\"");
2.- In my DB I changed the type of the field to the "nText" type which support more characters like Unicode, etc

Where are Smart Slices stored (server / client)

Hi,
I wanted to know where are the Smart Slices stored ? Are they stored up in the client machine (machine dependent) OR are they stored up in the server (and hence accessible to the user from any machine) ?
We have big business group who are enthusiastic about Smart Slices and the wonders it could do in their needs.
Thanks in advance for your interest in this thread !!!

They are not stored on the client as they are available to everyone. I'm not sure if they are stored in APS (which I think) or in Essbase.

HT204291 I cannot stream TV Shows stored on iTunes from Mac to Apple Tv - Checkered Boxes are displaying

I Cannot stream my TV shows to be played from MAC BOOK Air to Apple TV using Mirror Airplay, Checkered Boxes are displaying instead.

Welcome to the Apple Community.
Which application are you using to watch the TV shows.

Disabling Smart Quotes in FrameMaker 7.2

Hi,
My team uses FrameMaker 7.2 to create and edit DITA XML files. We recently noticed that the smart quote feature inserts non-printable characters in our XML content files. Our help system cannot display the characters properly, and so we've started to search for and replace the smart quote characters with straight quotes (outside of FrameMaker).
The smart quote characters seem to be defined in the maker.ini file.
; Smart Quote Characters
; SmartQuotes \xd4\xd5\xd2\xd3 ) English curved quotes
; SmartQuotes \xe2\xd4\xe3\xd2 ) German-style quotes with base quotes
; SmartQuotes \xd5\xd5\xc7\xc8 ) French-style quotes using guillemets
; SmartQuotes \xd5\xd5\xd3\xd3 ) Swedish- and Finnish-style quotes
; SmartQuotes \xd4\xd5\xd2\xd3 ) Italian curved quotes
; English curved quotes:
SmartQuotes=\xd4\xd5\xd2\xd3
I've been searching unsuccessfully online for hex code characters to replace the smart quote characters with straight quotes. Does anyone here know which character codes to use?
Incidentally, I've fiddled with the feature for disabling smart quotes per document (Format > Document > Text Options), but don't want my team mates to have to perform this task every time they open a file.
Thanks,
Mark

Mark,
the Text Options are stored with the FrameMAker file and since you are opening XML files the setting must be made once in the template used when opening XML files.
As far as I know FrameMaker writes curly quotes as perfectly valid codepoints to XML, so it must be a weakness of your help system not being able to display those characters. There is nothing wrong with those characters. Have you found the problem there?
- Michael

Microsoft Word "Smart Quotes"

I hope this will save other developers some time.
This may be obvious to others, but I just spent several hours Googling and testing to determine what actually happens when a user copies text containing "Smart Quotes" from Microsoft Word into a Java JTextComponent. For those not familiar with Smart Quotes, by default, MS Word changes double-quoted strings from using the US-ASCII character for quote (0x22) into left- and right- curly quotes (UTF-16: 0x201c and 0x201d). Word also does this with serveral other characters. This plays havoc with the display and Java Strings later encoded with java.beans.XMLEncoder, unless treated carefully. Here is what I discovered (obviously, this applies to MS Windows):
All values are in hexadecimal.
- Word is storing the character for double quote as UTF-16 internally (201C).
- When the character is copied to the clipboard, it is copied as UTF-8 (E2 80 9C).
- When the clipboard is pasted into Java, Java is assuming the it was originally Windows-1252 encoded, because that is the default for the US-EN locale in Windows XP (probably also Vista, but I only tested in XP).
- Java translates this into a-circumflex, euro-sign, o-e-ligature, the characters corresponding to E2, 80, and 9C respectively in Windows-1252 and represents it internally in UTF-16 as 00E2 20AC 0153.
-When the String is XML-encoded using java.beans.XMLEncoder, it is written in UTF-8 as C3A2 E282AC C593, which equates to UTF-16 00E2 20AC 0153 -- the characters a-circumflex, euro-sign, o-e-ligature.
I am not sure how to fix this, but maybe another reader does. I am experimenting with the Clipboard (java.awt.datatransfer) to see if I can programmatically find out the original character encoding (in this case, UTF-16).

Doesn't the DataFlavor contain the character encoding? What is the content of the InputStream returned by
            InputStream is = (InputStream)contents.getTransferData(DataFlavor.getTextPlainUnicodeFlavor());
If I use
                DataFlavor df = DataFlavor.getTextPlainUnicodeFlavor();
                String mimeType = df.getMimeType();
                String encoding = mimeType.replaceAll(".*?charset=(.*?)\\s*$", "$1");
                InputStream is = (InputStream) contents.getTransferData(df);
                ByteArrayOutputStream baos = new ByteArrayOutputStream();
                byte[] buffer = new byte[1024];
                for (int count = 0; (count = is.read(buffer)) != -1;)
                    baos.write(buffer, 0, count);
                baos.close();
                result = baos.toString(encoding);to transfer
Hello "World"
which Word changes the quotes to the smart 'smart quotes' version I get as a result
Hello World
which is what I expect.
Am I missing something?
Edited by: sabre150 on Sep 4, 2009 1:27 PM

Characters such as apostrophes and smart quotes turning into boxes or question marks

We recently upgraded from CF5 to CF7 and are having a problem
with previously saved text that no longer displays correctly. Some
characters (apparently, non-ASCII characters such as curly
apostrophes and smart quotes) are rendering as boxes or question
marks. We recently upgraded to Oracle 10g from Oracle 8i, but this
problem appears to be independent of the database that the text is
stored in. Here is sample code that will illustrate the problem:
<CFSET string1="Department’s">
<CFSET string2="hey—there">
<CFOUTPUT>
string1 is #string1#
<BR>
string2 is #string2#
</CFOUTPUT>
output looks like this:
string1 is Department?s
string2 is hey?there
These are rendered as boxes when viewed in Internet Explorer.
(They show up as question marks when I copy and paste them here.)
The Demoronize UDF helps *some* of the time, but this is
still happening with a lot of text, especially text that gets
pasted from a website into a form, then saved to a database. Does
anybody have a solution for this? This is breaking my applications
and is incredibly annoying. I'd like to either replace the
problematic characters at the time they are displayed, or replace
them when they are input in the database in the first place (and go
back and update all the previously saved data to replace the
problematic characters with plain text equivalents).
Any suggestions appreciated.

I finally isolated the problematic characters so I edited the
DeMoronize UDF (available at cflib.org) by adding the following
text replacements at the bottom:
text = Replace(text, chr(8208), "-", "ALL");
text = Replace(text, chr(8209), "-", "ALL");
text = Replace(text, chr(8210), "–", "ALL");
text = Replace(text, chr(8211), "–", "ALL");
text = Replace(text, chr(8212), "—", "ALL");
text = Replace(text, chr(8213), "—", "ALL");
text = Replace(text, chr(8214), "||", "ALL");
text = Replace(text, chr(8215), "_", "ALL");
text = Replace(text, chr(8216), "‘", "ALL");
text = Replace(text, chr(8217), "’", "ALL");
text = Replace(text, chr(8218), ",", "ALL");
text = Replace(text, chr(8219), "'", "ALL");
text = Replace(text, chr(8220), "“", "ALL");
text = Replace(text, chr(8221), "”", "ALL");
text = Replace(text, chr(8222), """", "ALL");
text = Replace(text, chr(8223), """", "ALL");
text = Replace(text, chr(8226), "·", "ALL");
text = Replace(text, chr(8227), ">", "ALL");
text = Replace(text, chr(8228), ".", "ALL");
text = Replace(text, chr(8229), "..", "ALL");
text = Replace(text, chr(8230), "...", "ALL");
text = Replace(text, chr(8231), "·",
"ALL");

Convert smart quotes and other high ascii characters to HTML

I'd like to set up Dreamweaver CS4 Mac to automatically convert smart quotes and other high ASCII characters (m-dashes, accent marks, etc.) pasted from MS Word into HTML code. Dreamweaver 8 used to do this by default, but I can't find a way to set up a similar auto-conversion in CS 4. Is this possible? If not, it really should be a preference option. I code a lot of HTML emails and it is very time consuming to convert every curly quote and dash.
Thanks,
Robert
Digital Arts

I too am having a related problem with Dreamweaver CS5 (running under Windows XP), having just upgraded from CS4 (which works fine for me) this week.
In my case, I like to convert to typographic quotes etc. in my text editor, where I can use macros I've written to speed the conversion process. So my preferred method is to key in typographic letters & symbols by hand (using ALT + ASCII key codes typed in on the numeric keypad) in my text editor, and then I copy and paste my *plain* ASCII text (no formatting other than line feeds & carriage returns) into DW's DESIGN view. DW displays my high-ASCII characters just fine in DESIGN view, and writes the proper HTML code for the character into the source code (which is where I mostly work in DW).
I've been doing it this way for years (first with GoLive, and then with DW CS4) and never encountered any problems until this week, when I upgraded to DW CS5.
But the problem I'm having may be somewhat different than what others have complained of here.
In my case, some high-ASCII (above 128) characters convert to HTML just fine, while others do not.
E.g., en and em dashes in my cut-and-paste text show as such in DESIGN mode, and the right entries
    –
    —
turn up in the source code. Same is true for the ampersand
    &
and the copyright symbol
    ©
and for such foreign letters as the e with acute accent (ALT+0233)
    é
What does NOT display or code correctly are the typographic quotes. E.g., when I paste in (or special paste; it doesn't seem to make any difference which I use for this) text with typographic double quotes (ALT+0147 for open quote mark and ALT+0148 for close quote mark), which should appear in source code as
    “[...]”
DW strips out the ASCII encoding, displaying the inch marks in DESIGN mode, and putting this
    "[...]"
in my source code.
The typographic apostrophe (ALT+0146) is treated differently still. The text I copy & paste into DW should appear as
    [...]’[...]
in the source code, but instead I get the foot mark (both in DESIGN and CODE views):
I've tried adjusting the various DW settings for "encoding"
    MODIFY > PAGE PROPERTIES > TITLE/ENCODING > Encoding:
and for fonts
    EDIT > PREFERENCES > FONTS
but switching from "Unicode (UTF-8)" to "Western European" hasn't solved the problem (probably because in my case many of the higher ASCII characters convert just fine). So I don't think it's the encoding scheme I use that's the problem.
Whatever the problem is, it's caused me enough headaches and time lost troubleshooting that I'm planning to revert to CS4 as soon as I post this.
Deborah

Smart quotes in languages other than English

Hi to all,
As a technical writer, I deal with texts in several languages. One of the problems I have with FrameMaker, is that French and German "smart" (or "typographic") quotes are not entered automatically. Quotation marks remain in the English format. The solutions offered in Adobe's Knowledge Base are less than optimal:
- Replacing the way FrameMaker writes smart quotes in the Maker.ini file is made on a general base, and not per language.
- Manually entering all quotation marks cannot be done easily when you receive a text from others.
Is there a viable solution to this problem?
Paolo

Paolo,
One way would be to create a small custom config file for each specific language that modifies the definition of the standard double-quote characters used for the Smart quotes. The definition for the standard characters in cmds.cfg (in the \fminit\configui folder) is:
<Command CharLeftDblQuote
    <Label Left Double Quote>
    <Definition \xd2>>
<Command CharRightDblQuote
    <Label Right Double Quote>
    <Definition \xd3>>
where the Definition parameter specifes the hex value (i.e. character in the FrameRoman encoding) to use for the character that you wish to display in FM.
For the French, version create a file with those two entries (call it French_smartquotes.cfg) and change the hex values to the appropriate language-specific quote, i.e. guillemotleft is \xc7 and guillemotright is \xc8. Likewise for the German.
Then, when you need to work in any of theose languages, use View > Menus > Modify... and select the appropriate .cfg file. This will then temporarily change the mapping for the double-quote characters such that when you enter them from the keyboard, the appropraite language specific quote will be used. To switch languages, simply select another .cfg file using the View > Menus > Modify... The setting only persists for the current FM session.
The alternative is to edit the maker.ini file to enable the language specific smart quotes, listed in the [Spelling] section. The entry is:
; Smart Quote Characters
; SmartQuotes \xd4\xd5\xd2\xd3 ) English curved quotes
; SmartQuotes \xe2\xd4\xe3\xd2 ) German-style quotes with base quotes
; SmartQuotes \xd5\xd5\xc7\xc8 ) French-style quotes using guillemets
; SmartQuotes \xd5\xd5\xd3\xd3 ) Swedish- and Finnish-style quotes
; SmartQuotes \xd4\xd5\xd2\xd3 ) Italian curved quotes
; English curved quotes:
SmartQuotes=‘’“”
Delete the ";" at the front to un-comment the appropriate entry and add a ";" to the english one.
However, this route requires you to close FM every time yo uwish to make a change.

Re: Smart quotes won't work

My situation is a bit different to the main thrust of this thread but is still 'smart quotes don't work'.
I'm on a 2009 iMac, using Yosemite 10.10.1, Pages 5.5.2
I've followed the instructions on Apple's page (http://help.apple.com/pages/mac/5.0/#/tanad45f9cce) to make all quotes smart quotes in my document and it didn't work.
When I opened Substitutions in Pages the box for smart quotes was ticked, unticking and reticking and doing 'replace all' made no difference.
Prior to this I'd just changed the font style setting for the document from Courier New to Times New Roman.
I've also checked in System Prefs/Keyboard/Text where Use smart quotes is ticked.
I'm formatting a book, this is holding up production of my book as all quotes are non-curly, straight quotes and need to be changed to smart quotes.
I'm wondering whether I shouldn't go back to the previous version of Pages where everything seemed to work, including auto capitalisation.

Pages '08 and '09 are unaware of the Operating System on this point.
You get the described characters because the app is running in German.
At this time, you
may just disable the quotes substitution feature.
Uncheck the box which is checked here.
I will search in the app resources if I may change the set of substitution quotes.
Yvan KOENIG (VALLAURIS, France) dimanche 30 janvier 2011 17:52:29
I got it.
The resource is stored in :
Macintosh HD:Library:Application Support:iWork '09:Frameworks:SFWordProcessing.framework:Versions:A:Resources:German.lproj:Aut oCorrect.plist
To change the behavior, you must edit the pointed set of characters.

Smart quotes won't work

Hello there dear community,
for some reason, Pages will not put quotation marks as I want them to be. As I am from Germany and therefore gernerally write in German, quotation marks are not “ ” but „ ” . I have not figured out how to get Pages to do that. I have already changed spelling language to German, enabled and disabled Smart Quotes, and set the System Preferences to the format I'd like the marks to be, but nothing happens.
What am I doing wrong?
Thank you.

Pages '08 and '09 are unaware of the Operating System on this point.
You get the described characters because the app is running in German.
At this time, you
may just disable the quotes substitution feature.
Uncheck the box which is checked here.
I will search in the app resources if I may change the set of substitution quotes.
Yvan KOENIG (VALLAURIS, France) dimanche 30 janvier 2011 17:52:29
I got it.
The resource is stored in :
Macintosh HD:Library:Application Support:iWork '09:Frameworks:SFWordProcessing.framework:Versions:A:Resources:German.lproj:Aut oCorrect.plist
To change the behavior, you must edit the pointed set of characters.

Find open and close smart quotes

Dear scripter,
Here I am trying find whether all the open double smart quotes(") are closed with close double smart quotes("). Here is my small work to find how many open and close quotes.
myopenqu=app.activeDocument.search("^{", false, false, "^{");
myclosequ=app.activeDocument.search("^}", false, false, "^}");
myopenqulength = myopenqu.length;
myclosequlength = myclosequ.length;
if(myopenqulength!=myclosequlength)
alert("Quotes not matched\n"+ myopenqulength +" Open quotes found\n" + myclosequlength +" Close quotes found")
It works well, the script just alert if I have 25 open quotes and 23 close quotes
Quotes not matched
25 Open quotes found
23 close quotes found
Now I want to find the quote set where the 24,25th set close quotes are missing?
Is there any way to find the misisng quotes
Thanks in advance
regards
a r u l
vpublish.net

You'd have to look for unmatched quotes, in this case two cases of two open quotes without an intervening closed quote. This is difficult to script in CS2 because it doesn't have GREP natively and because of footnote problems. In CS3/4 it should be possible. But even then it will work only if you don't have quotes within quotes.
Peter

Smart quotes and em-dashes in epub file

When I create an epub file from my InDesign book file, I find that sometimes (not always, thought!) my smart quotes and em-dashes revert to gibberish.
For example, when I open the epub file, the em-dash is fine, but in chapter 2, it changes to â€”.
Does anyone know how I might correct this?
Thanks so much!

It sounds like somethings wrong with the chapter's character encoding. The character encoding is declared in the header of the ePub HTML. From ID it's:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
This test has UTF-8 declared in the first line of its source code:
http://www.zenodesign.com/forum/encodetest/UTF8.html
And this has no encoding
http://www.zenodesign.com/forum/encodetest/None.html
Most browsers let you override or choose the character set, so in Safari if you set the character set to View>Text Encoding>default, you'll see a difference between the above links, but if you set it to Unicode (UTF-8) they both display correctly. In FireFox it's View>Character Encoding.
You can check and edit the contents of the .epub file by changing the extension from .epub to .zip, and then extracting the contents.

Problem with javascript/PHP/ oracle 10g smart quotes

I have a problem with my php form that passes the text field to a javascript object. When I copy text from MS Word that includes smart quotes, the form inputs that into the database as ? (upside down) marks. The charset of the DB is WE8MSWIN1252. How do I store these smart quotes as regular quotes? And also if I can do any conversions on the front end (js or php). I tried doing some conversion but to no avail. Any help would be appreciated thank you. “double”

Decide if you want your HTML pages in Windows code page 1252 or Unicode UTF-8. Then, make sure the pages are properly tagged as either "text/html;charset=windows-1252" or "text/html;charset=utf-8". Use HTTP header Content-type or the corresponding <META HTTP-EQUIV=...> tag. Then, set NLS_LANG environment variable for your PHP engine to either AMERICAN_AMERICA.WE8MSWIN1252 or to AMERICAN_AMERICA.AL32UTF8, depending on which encoding you selected for your HTML.
-- Sergiusz

Smart quotes stored in DB are displaying as ? after jdbc select

Similar Messages

Maybe you are looking for