JTidy with extended characters?

Hi all,
I've been using JTidy to tidy up html to valid xml, and it works fine, except when I have extended characters in the HTML.
Has anybody else come across this problem? Is there any solution? JTidy doesn't look like its had any releases since 2001. Is there any other active projects that do the same thing?
thanks,
Justin

Just looking at the input, and all extended characters are entities.
So the input is:
<html>
<head>
</head>
<body>
ýÿăĈ
</body>
</html>and the output is:
<html>
<head>
<meta name="generator" content="HTML Tidy, see www.w3.org" />
<title></title>
</head>
<body>
��[][]
</body>
</html>where the "[]" characters represent a box. I.e. a character System.out.println can't render.
If I look at the integer value of them, they are 3 & 8.
So I guess I'm trying to turn off the entity replacement in the JTidy. Is this possible?
thanks,
Justin

Similar Messages

Error with extended characters

hi i have a problem with extended characters and it only happens when i have an attachment with the mail
MimeBodyPart bp = new MimeBodyPart();
String content = "� � � � � � � � � � � � �� ";
bp.setContent( content, "text/plain; charset=UTF-8" );
mp.addBodyPart( bp );
//code for adding a attachment is added to the mp using MimeBodyPart
msg.setContent( mp ); // add Multipart
msg.saveChanges(); // generate appropriate headers
ByteArrayOutputStream baos = new ByteArrayOutputStream();
msg.writeTo( baos );msg is a MimeMessage.
Now this msg is stored in a cache and later retrieved using
String s = "";
            java.lang.Object o = msg.getContent();
            if ( o instanceof String )
                s = (String) o;
            else if ( o instanceof Multipart )
                try
                    Multipart mp = (Multipart) o;
                    MimeBodyPart bp = (MimeBodyPart) mp.getBodyPart( 0 );
                    s = (String) bp.getContent();
                catch ( Exception e )
                    cLog.error(e.getMessage(),e);
            }now s contains "� ��  � � � � � � � � � � �� " which is wrong.
This has been bugging me for a couple of days now.
could anyone please give a solution.
Many Thanks in advance

Using these literal non-ASCII characters in the string constant might
look great in your editor but might not be giving the correct Unicode
values. Try replacing the special characters with Unicode escapes
(\u####).
If that doesn't work, step through the original string and print out
each character as an integer. Then do the same for the string you
fetch from the message. Compare the integer values. If they're
not the same, post the details. If they are the same, the problem
is most likely in how you're displaying the characters you read
from the message.

Working with Extended Characters

I have CS3 on XP SP2
In InDesign, both upper and lower case Pi show up amongst the other letters of the Greek Alphabet in fonts with extended character sets (Times or Arial). However, in Illustrator, the lower case pi is missing... it goes straight from omnicron to rho, skipping pi. I searched and finally found pi clustered with a select few "math" Greek symbols like sigma and mu. InDesign has glyph sorting options, but apparently not Illustrator. Is it possible to sort the glyphs by unicode in Illustrator to make it easier to locate certain characters?
Also, the lower case pi I found in Illustrator had no unicode and it is not rendered in many of the fonts that do in fact contain a lower case pi, under Unicode U+03C0.
Another related annoyance is that when I create a line of text (say in Times New Roman), I sometimes like to highlight the text and rapidly step through the fonts with the up and down arrows until I find one I like. This works fine in In-design and will show pink squares wherever a glyph is not supported by a font, but in Illustrator if the selected text contains certain extended characters, when I step to the next font in the list, the results can be unpredictable.

In Illustrator CS3, I found that both Arial and Times New Roman have TWO lowercase pi characters in the glyphs panel. The first one, in the math chars near the top of the glyphs panel, says "No Unicode". The second one is lower down in the glyphs panel, after the Greek, after the Cyrillic, and before the Hebrew. It has the expected Unicode value U+03C0. I'm guessing this second pi with a Unicode value probably works better when changing to other fonts.

Extended characters

I previously used Dreamweaver 4 without CSS and then moved onto Dreamweaver 8, again without CSS. I am trialing Dreaweaver CS5 and have updated my site content to <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> from just <HTML>
Extended characters such as e acute which used to appear correctly in earlier versions of Dreamweaver, although they appear correctly in Dreamweaver Live View now appear rather differently in Internet Explorer 8 e.g an e acute appears as A with a tilda followed by a copyright symbol. How can I correct this please?

Thanks John,
Since reading your message I discovered that the error occured only where I had copied text with extended characters from another website and pasted it into mine.
Best wishes,
Brian

Internal Server Error while sending mail with extended property

I am using EWS Managed API. I just started fiddling with Extended properties. So I wrote simple code to send a simple mail with extended property attached to it.
Forming the mail part I simply copy pasted from this
MSDN page. For testing purpose I suppressed certificate validations.
This is my complete code:
1 ServicePointManager.ServerCertificateValidationCallback = delegate(object s, X509Certificate certificate, X509Chain chain, SslPolicyErrors sslPolicyErrors) { return true; };
2 service = new ExchangeService(ExchangeVersion.Exchange2007_SP1);
3 service.TraceListener = new TraceListener();
4 service.TraceEnabled = false;
5
6 service.Credentials = new WebCredentials("[email protected]", "password@123");
7 service.Url = new Uri("https://exchng.domain.com/EWS/Exchange.asmx");
8
9 Guid MyPropertySetId = new Guid("{C11FF724-AA03-4555-9952-8FA248A11C3E}");
10
11 // Create a definition for the extended property.
12 ExtendedPropertyDefinition extendedPropertyDefinition = new ExtendedPropertyDefinition(MyPropertySetId, "Expiration Date", MapiPropertyType.String);
13
14 // Create an e-mail message that you will add the extended property to.
15 EmailMessage message = new EmailMessage(service);
16 message.Subject = "Saved with extendedPropertyDefinition of two days";
17 message.Body = "The expiration date is contained within the extended property.";
18 message.ToRecipients.Add("[email protected]");
19
20 // Add the extended property to an e-mail message object named "message".
21 message.SetExtendedProperty(extendedPropertyDefinition, DateTime.Now.AddDays(2).ToString());
22
23 // Save the e-mail message.
24 message.SendAndSaveCopy();
Please find the rest of the question in first reply, as I got "Body must be 4 to 60000 characters long".

Rest of the question:
I am getting below exception (with no nested inner exceptions) on line 24:
An internal server error occurred. The operation failed.
at Microsoft.Exchange.WebServices.Data.ServiceResponse.InternalThrowIfNecessary()
at Microsoft.Exchange.WebServices.Data.ServiceResponse.ThrowIfNecessary()
at Microsoft.Exchange.WebServices.Data.MultiResponseServiceRequest`1.Execute()
at Microsoft.Exchange.WebServices.Data.ExchangeService.InternalCreateItems(IEnumerable`1 items, FolderId parentFolderId, Nullable`1 messageDisposition, Nullable`1 sendInvitationsMode, ServiceErrorHandling errorHandling)
at Microsoft.Exchange.WebServices.Data.ExchangeService.CreateItem(Item item, FolderId parentFolderId, Nullable`1 messageDisposition, Nullable`1 sendInvitationsMode)
at Microsoft.Exchange.WebServices.Data.Item.InternalCreate(FolderId parentFolderId, Nullable`1 messageDisposition, Nullable`1 sendInvitationsMode)
at Microsoft.Exchange.WebServices.Data.EmailMessage.InternalSend(FolderId parentFolderId, MessageDisposition messageDisposition)
at Microsoft.Exchange.WebServices.Data.EmailMessage.SendAndSaveCopy()
If I comment line number 21, the code works fine and sends the message. So why it fails with extended properties?

How to use a select statement with chinese characters?

I am currently developing a java servlet<using tomcat 4.x> which allows me to use select statement to retrieve results from the Microsoft SQL Server 2000 database. I am using a simple form to get the parameter for querying. The main problem i'm facing is that there are chinese information in the SQL database, but i can't retrieve it through the sql statement with the chinese characters input<thru the form with the help of NJ STAR>in the WHERE condition. When i execute the statement, it returns me no results even though the rows are present in the database.
Does anyone have the solution to using chinese words in the WHERE clause of the select statement to retrieve results with columns which contains chinese characters? Please help me. Thanks everyone. :)
PS: when i cut and paste those characters in the sql database and paste onto java.. it is ??? in questionmarks.. but when i paste them into excel 2000.. its shown as chinese chars again..
please heelppp~~

Greetings,
PS: when i cut and paste those characters in thesql
database and paste onto java.. it is ??? in
questionmarks.. but when i paste them into excelThis is why the SELECT is not returning any results.
You need to set the character encoding set on your
statement and parameters for the characters to be
properly translated. Refer to the charsetName
parameter in the String class constructor in your API
docs and also to
$JDK_DOCS/guide/intl/encoding.doc.html in your JDK
documentation.
2000.. its shown as chinese chars again..Because Office programs are performing the same kind
of character translation with the appropriate MS APIs.
please heelppp~~Regards,
Tony "Vee Schade" Cookis it possible for you to show me some coding examples? i don't really understand what is to be done in order to set the char set and what does it really do.. tried reading up but still dun understand.. :(
pardon my shallow knowledge of java..
ok..
The thing is when i used an insert statement with chinese characters of GBK format hardcoded into the java servlet and then i use the insert statement to insert the chars into the database, it cannot be seen as a chinese word when i off the NJStar. and then it can be searched out with my current form of servlet.. below is my coding of the servlet..
note: i've set my html file to charset = GBK
//prototype of Search engine...
//workable for GBK input and output...
import java.io.*;
import java.io.OutputStream;
import java.io.IOException;
import javax.servlet.http.*;
import javax.servlet.ServletException;
import java.util.*;
import java.sql.*;
import java.nio.charset.Charset;
public class SearchBeta extends HttpServlet {
     private Vector musicDetails = new Vector();
     private String query = "";
     public void service (HttpServletRequest req, HttpServletResponse res) throws IOException, ServletException, UnsupportedEncodingException {
          query = req.getParameter ("T1");
          System.out.println("before:"+query);
          String type = req.getParameter ("D1");//type
          query = req.getParameter ("T1");
          //query = "��";
          System.out.println("after:"+query);
          getResults(type,query);
          System.out.println("locale = :"+req.getLocale());
          res.setContentType ("text/html;charset=GBK");
          PrintWriter out = res.getWriter();
          out.println("<html>");
          out.println("<head>");
          out.println("<body bgcolor = \"black\">");
          out.println("<font face = \"comic sans ms\" color=\"Cornsilk\">");
          if (query.length()==0)
               out.println ("Please key in your search query.");
          else if (musicDetails.size()==0)
               out.println ("Sorry, no results matching your search can be found.");
          else {
               out.println("<center>");
               out.println("<table cellspacing = \"50\">");
               int i = 0;
               //Display the details of the music
               while (i<musicDetails.size()) {
                    Results details = (Results)musicDetails.get(i);
                    String dbArtist = "";
                    String dbAlbum = "";
                    String dbTitle = "";
                    String dbCompany = "";
                    dbAlbum = details.getAlbum();
                    dbTitle = details.getTitle();
                    dbCompany = details.getCompany();
                    dbArtist = details.getArtist();
                    try{
                         dbAlbum = new String(dbAlbum.getBytes("ISO-8859-1"),"GBK");
                         dbTitle = new String(dbTitle.getBytes("ISO-8859-1"),"GBK");
                         dbCompany = new String(dbCompany.getBytes("ISO-8859-1"),"GBK");
                         dbArtist = new String(dbArtist.getBytes("ISO-8859-1"),"GBK");//correct translation.
                    catch(UnsupportedEncodingException e){
                         System.out.print(e);
                         e.printStackTrace();
                    String dbImage_loc = details.getImage();
                    out.println("<tr>");
                         out.println("<td><table>");
                              out.println("<img src=C:\\Program Files\\Apache Group\\Tomcat 4.1\\webapps\\examples\\ThumbNails\\"+dbImage_loc+">");
                         out.println("<tr>");
                              out.println("<th><font color=\"violet\"> Artist: </font></th>");
                              out.println("<td><font color=\"Cornsilk\">"+dbArtist+"</font></td>");
                         out.println("</tr>");
                         out.println("<tr>");
                              out.println("<th><font color=\"violet\"> Title: </font></th>");
                              out.println("<td><font color=\"Cornsilk\">"+dbTitle+"</font></td>");
                         out.println("</tr>");
                         out.println("<tr>");
                              out.println("<th><font color=\"violet\"> Company: </font></th>");
                              out.println("<td><font color=\"Cornsilk\">"+dbCompany+"</font></td>");
                         out.println("</tr>");
                         System.out.println("album: "+ dbAlbum);
                         out.println("<tr>");
                              out.println("<th><font color=\"violet\"> Album: </font></th>");
                              out.println("<td><font color=\"Cornsilk\">"+dbAlbum+"</font></td>");
                         out.println("</tr>");
                         System.out.println("company: "+ dbCompany);
                         out.println("</table></td>");
                    out.println("</tr>");
                    i++;
               out.println("</table>");
               out.println("</center>");
          out.println("</font>");
          out.println("</body>");
          out.println("</head>");
          out.println("</html>");
          out.close();
          //to remove all the elements from the Vector
          musicDetails.removeAllElements();
     //get Searched Music Details and store in Results object which is stored in musicDetails vector
     public void getResults (String type, String searchQuery) {
          try {
               Class.forName ("com.microsoft.jdbc.sqlserver.SQLServerDriver");
               Connection con = DriverManager.getConnection("jdbc:microsoft:sqlserver://localhost:1433;DatabaseName=music","sa","kokkeng");
               Statement stmt = con.createStatement();
               String query = "SELECT * FROM MusicDetails WHERE "+type+" = '"+searchQuery+"'";
               ResultSet rs = stmt.executeQuery(query);
               while (rs.next()) {
                    String artist = rs.getString("Artist");
                    String title = rs.getString("Song");
                    String company = rs.getString("Company");
                    String album = rs.getString("Album");
                    String image_loc = rs.getString("Image");
                    Results details = new Results (artist,title,company,album,image_loc);
                    musicDetails.add(details);
               stmt.close();
               con.close();
          catch (Exception e) {
               System.out.println(e.getMessage());
               e.printStackTrace();
with the above servlet i created, i can search out the data in the database which i've inserted through the insert statement. I still can't search for things i've keyed into the database directly using NJStar..
thank you so much for helping.. really hope any one else who knows the answer to this will reply too... thank you all so much...
-KK

Vibe 3.2 problems with special characters?

I'm not sure where the issue lies here, the client or Vibe, so I'll throw
this out there to see if anyone has run into this problem. I have a user
with a + sign in their password. If he tries to log in from either his
Android phone or his Android tablet, he gets a bad password error. Using a
different browser on the tablet, he was able to get logged in. Using my
Android phone, he was also able to get logged in.
Are there any known issues with Vibe not supporting certain browsers for
special characters in passwords? I don't think he had the problem with Vibe
3.1.
thanks,
Todd Bowman
Senior Network Analyst
University of Minnesota Physicians
612-884-0744
[email protected]

Yes, you're right. Same issue here with a '' character. Found this:
'6.2 Character Restrictions in Usernames and Passwords
Do not use extended characters or double-byte characters in Novell Vibe usernames and passwords.
This includes usernames and passwords that are synchronized from an LDAP directory into Vibe.
This limitation is caused by the open source Spring Security (http://static.springframework.org/
spring-security/site) that Vibe uses for authentication, in combination with the various authentication
methodsboth basic authentication (http://en.wikipedia.org/wiki/Basic_a...authentication) and
form-based authentication (http://en.wikipedia.org/wiki/Form_ba...tication)used by
single sign-on products such as Novell Authentication Manager, by Web services, and by WebDAV.
Only ASCII characters are properly interpreted in all combinations.'
source: https://www.novell.com/documentation...dme_novell.pdf
Extremely frustrating for anyone using strong passwords...

Filenames with strange characters

(With apologies if this is a previously answered question.)
One of my Java programs copies users' home areas around our network. I'm getting filenotfoundexceptions with some of the users' files, specifically those which use extended character sets (we have many international students who use special programs to create files with e.g. Chinese characters). I'm at a loss to know how to diagnose this problem and would appreciate any advice. The filenames are retrieved as result of a listFiles() and the copy method uses BufferedInput\OutputStreams.
David R.

Your problem must be with the names of the files, rather than with the content. I did a few experiments and found that Java (I'm using SDK 1.3 on Windows 2000) can't open files with Cyrillic characters in the file name, either for input or for output. Throws a FileNotFoundException, as you said. There shouldn't be any problem with Chinese content if you're using input and output streams.
So, the problem is diagnosed. There are several bug reports that sound like this. Don't know if it has been fixed in SDK 1.4.

How to create users with i18n characters in SunONE directory server?

Was trying to create users and groups with i18n characters in SunONE directory server
1. Started LDAP console using -l option
2. Chaged the Locale to Japanese
3. Entered few japanese character as username (meaning internationalization user name)
4. However, I could not able to type the password using the "soft keyboard" that comes with Japanese Locale
5. to overcome with #4, for now, I typed english chars as the password
6. Click OK to save the above username/pwd
7. It says "netscape.ldap.LDAPException: error result (19); value of attribute "uid" contains extended (8-bit) characters"
Has anyone ever created i18n user names in SunONE Directory Provider? Please help...

Hi LostLad,
Soryy for my ignorance...Could you please be elaborate on how to remove "uid attribute from 7-bit ASCII plugin?
Thanks in advance..

Java app is writing question marks instead of extended characters

People,
since I've installed an Sun E3000 with Solaris9 I'm having problems with accents (extended characters) in my java application: - it writes question marks instead of accents.
This machine used to be Solaris 2.6 and we formatted/installed Solaris9 and got the problem since then.
My application uses a thin driver to connect to Oracle9i database, and via unix I can call sqlplus and insert chr(199) and SELECT it (�) with no problems, but application writes ? in nohup.out.
If you can point me in any direction, please let me know.
Regards, Orlando.

I'm pretty sure that the default locale "C" corresponds to the 7-bit US-ASCII character set. Again, this is all the methaphorical shot in the dark, but it's the only time that I've seen this happen. Sun's own documentation says to use something other than C or US ASCII if international or special characters will be needed.
Here is my /etc/default/init for ISO 8859-15.
TZ=US/Eastern
CMASK=022
LC_COLLATE=en_US.ISO8859-15
LC_CTYPE=en_US.ISO8859-1
LC_MESSAGES=en_US.ISO8859-1
LC_MONETARY=en_US.ISO8859-15
LC_NUMERIC=en_US.ISO8859-15
LC_TIME=en_US.ISO8859-15
If the system that you're using can be rebooted, try changing the /etc/default/init file to something like the above, but with the specific for your locale. Obviously, en_US is not your area, but I have no idea what Brazil's would be.

Entering extended characters?

As a recentish switcher I'm completely stumped by how I can enter extended characters. I need to enter the character code 0239 to get a special character in a font I'm using. On a PC I'd simply type 0239 on the numeric pad while holding down the Alt key. But that doesn't work on the Mac.
Any help much appreciated. GeoffT.

Most apps have an item in the "Edit" menu called "Special Characters..." which brings up a palette with just about every conceivable character set you could want. That should do the trick, unless the app is so old that it doesn't enable this system-level item. Assuming it works, once you've found the character you want, just click the "Insert" button in the lower-right of the palette to insert it into the active text field.
Here's Apple's description of the same feature:
http://docs.info.apple.com/article.html?path=Mac/10.5/en/8164.html
There are also many longstanding tricks to enter the more common special characters such as curly quotes, emdashes, and accented characters. Here's a very old but still valid cheat sheet:
http://home.earthlink.net/~awinkelried/keyboard_shortcuts.html

Extended characters not getting through VM properly in JDK1.4 versions

This is on WIndows platform with regular US keyboard.
When you type "ALT+0252" (it is �) from your regular (English) keyboard using it's numeric keypad, somewhere in the VM there is translation happening that is converting the numeric value of 252 to 242 and there by displaying the character mapped to ALT+0242 (which is �).
Similarly, every other extended character is getting translated somewhere thereby displaying the wrong character.
The way to look at Character Map on Windows platform is Start|Programs|Accessories|System Tools|Character Map.
Best way to compare what is getting through VM is by running
regular Notepad (Start | Programs | Accessories | Notepad) and typing
above character and then running the Notepad that comes with J2SDK install (java -JAR JDKInstallDir\demo\jfc\Notepad\Notepad.jar) and typing the same character.
You will notice that the regular Notepad displays � when you type ALT+0252 but the java Notepad displays � when you type the same ALT+0252 character.

Here is the sample code. You can first run it in jdk1.3 environment and enter the extended characters in the textfield. You will notice that they get rendered correctly (such as ALT+0252 gets rendered as �)
Then try running the same applet in jdk1.4 environment and enter the same extended characters in the textfield and you will notice the difference. ALT+0252 will now get rendered as � which is not correct.
JComponent class in JDK1.4 seems to be translating the string some how before rendering it.
Please let me know if you have any ideas or suggestions on this weird behavior in JDK1.4. Thanks.
import javax.swing.*;
import java.applet.*;
import java.awt.event.*;
import java.awt.*;
public class DemoApplet extends JApplet
public JTextField jtf = new JTextField ("Testing");
public String text = "";
public void init()
jtf.addActionListener (new ActionListener() {
public void actionPerformed(ActionEvent e)
text = ((JTextField)e.getSource()).getText();
try {
System.out.println ("numeric value of text is : " + (int)(text.charAt (0)));
catch (Exception ex)
ex.printStackTrace ();
JPanel pan = new JPanel ();
pan.add (jtf);
pan.validate();
getContentPane().add (pan);

Tabular model column rename with special characters - ), *, /, +, %, [ etc

I am working in tabular model 2012. I have requirement to rename the column names with special characters like ), *, /, +, %, [ etc. When I try to rename in my model, it gives me below error:
I also tried renaming the columns after creating a new test model with no luck. I may have lived with this error however we have another tabular model which contains special charaters in the name. This makes me believe that there must be some setting that
I am missing.
I tried changing the column name in other model having some special characters and it takes it.
I have gone through this link#
https://connect.microsoft.com/SQLServer/feedback/details/802009/ssdt-ssas-tabular-column-name-with-special-characters
and
http://msdn.microsoft.com/en-us/library/gg492144(v=sql.110).aspx which says that special characters are not allowed in column names however I am able to do it in one of our
models then why not others.
Any help is appreciated! Thank you for you time.
If this post answers your query, please click "Mark As Answer" or "Vote as Helpful".

Well Mr. Google helped me.. I got the solution:
Databases of a specific modality and compatibility level can have object names that include reserved characters. Dimension
attribute, hierarchy, level, measure and KPI object names can include reserved characters, for tabular databases (1103 or higher) that allow the use of extended characters.
How to change the compatibility, just right click on the .bim file from SSDT and go to properties and change the compatibility to 1103.
I hope this will help someone. :-)
Useful MSDN Links:
http://msdn.microsoft.com/en-us/library/bb522632.aspx
http://msdn.microsoft.com/en-us/library/jj674204.aspx
If this post answers your query, please click "Mark As Answer" or "Vote as Helpful".

Problem with some characters in complex objects

Hi all,
I've built a webservice which returns a complex object with several fields inside. All fields are public and accessable via getter and setter methods.
The problem is, that some of these fields contains numbers or underscores in their names.
For example:
public int field_a;
or
public String house3of4;
When I try to import these webservice as a model in a Web Dynpro project, it doesn't work until I remove these characters.
Is this a known problem or is there any solution for it?
Thanks
Thomas

NLS_LANG in registry is "ARABIC_UNITED ARAB EMIRATES.AR8MSWIN1256"
I use oracle form 10g for developer
oracle form 9i for database
when I build a form in client side and make a text with farsi characters, when I run the form,all characters shows me correct in farsi except four characters(گ چ ژ پ)

CRVS2010 Beta - Cannot export report to PDF with unicode characters

My report has some unicode data (Chinese), it can be previewed properly in the windows form report viewer. However, if I export the report document to PDF file, the unicode characters in exported file are all displayed as a square.
In the version of Crystal Report 2008 R2, it can export the Chinese characters to PDF when I select a Chinese font in report. But VS2010 beta cannot export the Chinese characters even a Chinese font is selected.

Barry, what is the specific font you are using?
The below is a reformatted response from Program Management:
Using non-Chinese font with Unicode characters (Chinese) the issue is reproducible when using Arial font in Unicode characters field. After changing the Unicode character to Simsun (A Chinese font named 宋体 in report), the problem is solved in Cortez and CR both.
Ludek

JTidy with extended characters?

Similar Messages

Maybe you are looking for