[SOLVED] File name encoding issue

Hi all,
I have a large series of files with accented characters, they were all displayed nicely, but at some point, when I copied them to another computer, the characters were replaced by codes, for instance: "ó" --> "Ã³".
+Renaming ie. "PasÃ³" (bad encoding of "Pasó") --> Pasó, while writing it, it shows the correct character, but when pressing enter the name remains ("PasÃ³")
+If I rename the file to something else and then to the correct name, it will accept it: PasÃ³ --> Pas --> Pasó will display correctly.
I don't know if it's a system wide encoding issue because new files are displayed correctly, but I would like to know if I have to change file names manually to make them right.
PS. When copying bad encoded files to another FS (like a USB drive), nautilus and bash refuse to copy them.
Last edited by Wasser (2012-09-17 21:10:52)

My fstab:
# /etc/fstab: static file system information
# <file system> <dir> <type> <options> <dump> <pass>
tmpfs /tmp tmpfs nodev,nosuid 0 0
# /dev/sda2 LABEL=ROOT
UUID=d2243d9c-b8e7-442a-8446-5a43a4d9221b / ext4 rw,relatime,data=ordered 0 1
# /dev/sda5 LABEL=HOME
UUID=e67f5cfa-3ec3-4c06-9c2c-62c4cc188ffe /home ext4 rw,relatime,data=ordered 0 2
# /dev/sda3 LABEL=VAR
UUID=caac4924-2a13-4c97-9926-668ac0595ba3 /var reiserfs rw,relatime 0 2
# /dev/sda1 LABEL=UEFI
UUID=1E70-6485 /boot/efi vfat rw,relatime,fmask=0022,dmask=0022,iocharset=iso8859-1,shortname=mixed,errors=remount-ro 0 2
# /dev/sda4
UUID=14993c2e-4bc4-42e4-b2e5-9dbc286abb4c none swap defaults 0 0
Files in question are in /dev/sda5 (HOME)
Last edited by Wasser (2012-09-16 08:37:52)

Similar Messages

RIDC check in the file name encoding issue

Hi,
When I use the RIDC to upload the chinese file name, the native file field become ??.pdf.
TransferFile tf = new TransferFile(new File("c:/中文.pdf"), "application/pdf");
dataBinder.addFile("primaryFile", tf);
Thanks!

This is rather a Java than RIDC question.
Check these links, maybe you will find something useful:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4185525
http://stackoverflow.com/questions/3610013/file-listfiles-mangles-unicode-names-with-jdk-6-unicode-normalization-issues

File Name Encoding resets to Roman from Cyrillic when importing into iTunes

I have added a large number of Russian songs to my library. Many of the file names were not imported using the correct file name encoding. I downloaded the file name encoding repair app and fixed all the file names in my iTunes library, but whenever I add the songs into iTunes, the names are reset to Roman encoding from Cyrillic. Is there any way to set and lock the encoding so iTunes doesn't reset it?

Fixing the filenames probably doesn't do what you need. iTunes uses the ID3 tags. See this note for info on fixing those:
http://homepage.mac.com/thgewecke/mlingos9.html#itunes

File name Encoding

We have a requirement from our customer for some simple content management. Basically they will get a directory on our server to place pdf files. We then read this folder at runtime and dynamically create a JavaScript tree so that users can view and select their required pdf.
In terms of display we have a simple policy. The display text is the name of the file with underscores replaced by spaces (e.g. My_Document.pdf --> My Document). We require this functionality (at first) in 6 languages (English, French, Spanish, Dutch, German and Italian).
The problem we have is concerning the file names with "non-english" chars such e grave, e acute etc used in French, German and Dutch. We have copied the pdfs from a windows server to a Unix server. The code works fine on windows but once ported to Unix we get question marks replacing the chars mentioned. Basically what we do is given a root (e.g. /usr/local/documents/fr) we then call listFiles() on the File object.
Do you think this is something to do with the encoding of the actual file names? Or perhaps to do with a missing language pack on the Solaris machine? Unfortunately I know very little about Unix and we only have one Unix admin guy who has never encountered this problem before.
Has anybody else encountered this problem, or know how I can correct this?
Thanks in advance, Laurence.

This is the output from the locale command whatever
this means:
LC_CTYPE="C"This means you are using the default locale (called "C" probably because of the programming language). In the default locale the encoding that Java picks up is 7bit ASCII which doesn't contain characters like e with acute accent. There are a number of things you can do to change this: you can change the locale through environment variables, or change only the encoding that Java picks up.
Locale is changed by modifying the LANG environment variable. You can list all available locales with the command "locale -a". For instance to change to US English (which uses the iso-8859-1 encoding with é etc) you would use these commands before running your program (choose one for the shell you use):tcsh% setenv LANG en_US
or
bash$ export LANG=en_USYou can change the encoding that Java picks up from the system with "-Dfile.encoding", for example:java -Dfile.encoding=ISO-8859-1 YourClass> Unfortunatley I cannot get a local login to the
server. It is a production server in a rack and only
has remote access. My ssh client (Putty) displays the
characters correctly.
Then hopefully this time the file names have been transferred ok and you are able to run the program after either of the modificatios above :)
Yet again they don't display correctly....Yep, and old bug that the admins haven't arsed to fix, kind of sad considering the i18n support Java has...

Dynamic file name configuration issue

Hi,
My scenario is Idoc to multiple file scenario..
I need to drop 2 files at the target with the file names to be configured dynamically, at present i have used dynamic UDF to drop the files with the dynamic name. but i am getting the output files with the same name i mean the name which i used in the second structure.
eg: Input -- IDOC
ouput: i need to have abc.idocnumber.sysdate.xml & xyz.idocnumber.sysdate.xml
But now we are getting as xyz.idocnumber.sysdate.msgid.xml & xyz.idocnumber.sysdate.xml
so correct me if i am wrong .. i should not use 2 dynamic UDF in a single scenario..
else can i use variable substitution for one structure and Dynamic UDF for another structure..
please suggest some solution
Thanks,
--Kishore

Hi Kishore,
you can append the system date to the filename of the receiver.
Try the below code:
SimpleDateFormat simpledatFormat = new SimpleDateFormat("yyyyMMdd");
Date date = new Date();
String datewithYear = simpledatFormat.format(date);
SimpleDateFormat simpledatFormat1 = new SimpleDateFormat("HHmmss");
Date date1 = new Date();
String datewithMs = simpledatFormat1.format(date1);
DynamicConfiguration conf = ((DynamicConfiguration) container.getTransformationParameters().get(StreamTransformationConstants.DYNAMIC_CONFIGURATION));
DynamicConfigurationKey key = DynamicConfigurationKey.create("http:/""/""sap.com""/""xi""/""XI""/""System""/""File" , "FileName");
String oldFileName=conf.get(key);
String valueNew;
if( oldFileName == null){
valueNew = "HRXML"datewithYeardatewithMs+".xml";
else{
valueNew =oldFileNamedatewithYeardatewithMs+".xml";
//set the new filename
conf.put(key, valueNew);
Are you using multi-mapping? If so Dynamic configuration does not work.

Receiver File Adapter - Encoding issue.

Hi Everybody,
The file format (encoding) is different to the format generally we used to get.
Currently we are get the flat files in DOS format.The current file when we are downloading it we are getting it in the UNIX or other format.
For eg: 20 has been changed to 0D in the file.
Can somebody help me on the same.
Thanks,
Zabiulla

Hi,
Check on this for file adapters
Text
Under File Encoding, specify a code page.
The default setting is to use the system code page that is specific to the configuration of the installed operating system. The file content is converted to the UTF-8 code page before it is sent.
Permitted values for the code page are the existing Charsets of the Java runtime. According to the SUN specification for the Java runtime, at least the following standard character sets must be supported:
■       US-ASCII
Seven-bit ASCII, also known as ISO646-US, or Basic Latin block of the Unicode character set
■       ISO-8859-1
ISO character set for Western European languages (Latin Alphabet No. 1), also known as ISO-LATIN-1
■       UTF-8
8-bit Unicode character format
■       UTF-16BE
16-bit Unicode character format, big-endian byte order
■       UTF-16LE
16-bit Unicode character format, little-endian byte order
■       UTF-16
16-bit Unicode character format, byte order
Regards
Vijaya

Automount by hal+ default file name encoding

It seems default file name is iso8859-1 now (correct me), why isn't it UTF8? Which makes names of all my windows files becoming messed up.

Hi,
you can use a user Exit after assigning the file. The file name is existing as variable in the function so that you can set this value as document description.
regards
Thomas

Error in file name encoding on OS X

This little java program fails when creating a new file with the Danish letter "�" (U+00C5) and then trying to find the same file. It cannot find it - the name has been messed up in the encoding/decoding.
The program only fails on OS X 10.4. The charset is MacRoman. If I look for the file in OS X it looks fine.
It runs fine on Windows XP.
Anyone who can explain this?
public class Test
    public static void main(String []args)
        try
            String arr[] = {
                    "\u0026\u0416\u4E2D\u10346"    //http://www.tbray.org/ongoing/When/200x/2003/04/26/UTF
                    ,"\u00C5" //AA
                    ,"\u00d8" //OE
                    ,"\u00E6" //AE
            for (int i = 0; i<arr.length; i++)
                String filename = arr;
String folder = "/Users/.../shared/1/";
java.io.File f = new java.io.File(folder + filename);
f.createNewFile();
f = new java.io.File(folder);
boolean found = false;
String[] files = f.list();
for (int j=0; j<files.length; j++)
if (files[j].equals(filename))
found = true;
System.out.println(arr[i] + " : " + found);
catch (Exception ex)
System.out.println(""+ex);

Here is a even more clear example that something is wrong. CompareTo returns != 0 for all files in a folder, even when compared to the file object, from which a file was just created.
Again this works fine with Windows XP.
public class Test
     public static void main(String []args)
          try
               String filename = "\u00C5"; //Danish �
               String folderPath = "/Users/.../shared/";
               //Create file.
               java.io.File file = new java.io.File(folderPath + filename);
               file.createNewFile();
               //Loop folder
               java.io.File folder = new java.io.File(folderPath);
               java.io.File[] files = folder.listFiles();
               for (int j=0; j<files.length; j++)
                    java.io.File folderFile = files[j];
                    System.out.println(folderFile + " compareTo: " + file.compareTo(folderFile));
          catch (Exception ex)
               System.out.println(""+ex);
}Message was edited by:
stigcAtSun2

How do i solve fonts/display/encoding issue?

This problem only appears on Firefox and no other browsers. When i go into some website the letters will not appear correctly. You may see an example on the link below which contains a screenshot
http://img291.imageshack.us/img291/3134/20152698.jpg

This issue can be caused by the Helvetica or Geneva bitmap font or another (bitmap) font that can't be displayed by Firefox in that font size.
Firefox can't display that font in the specified size and displays gibberish instead.
You can test that by zooming out (View > Zoom > Zoom Out, Ctrl -) to make the text smaller.
Uninstall (remove) all variants of that not working font to make Firefox use another font or find a True type version of that font that works better.

Character encoding issue

I'm using the below give code to send mail in Trukish language.
MimeMessage msg = new MimeMessage(session);
msg.setText(message, "utf-8", "html");
msg.setFrom(new InternetAddress(from));
Transport.send(msg);
But my customer says that he gets sometime unreadable characters in mail. I'm not able to understand how to solve this character encoding issue.
Should i ask him to change his mail client's character encoding settings?
If yes which one he should set.

Send the same characters using a different mailer (e.g., Thunderbird or Outlook).
If they're received correctly, come the message from that mailer with the message
from JavaMail. Most likely other mailers are using a Turkish-specific charset instead
of UTF-8.

Encoding failed in "file name" with error: -43

Hi!
I've tried everything but still, after about 20 hours of encoding DVD Studio Pro just stoppes, saying "encoding failed in "file name" with error: -43". The movie is in DV PAL format and made in Final Cut Pro version 5. I'm using DVD Studio Pro 4.0. The movie is about 1 hour and 30 minutes long and I'm trying to put it on a double layer DVD. I've tried to shorten it (it was nearly 2 hours before), but that didn't help. I also tried to shorten it even more and put it on a single layer DVD, but still the same message.
So if somebody has any idea of how I can solve this problem, please let me know as soon as possible!
Thank you!

Error -43 means "File not found." If you are compressing from a reference movie, make sure that all referenced movie clips are present on the disk and that permission settings are not preventing DVD Studio Pro from accessing them.
Error -1309 means "Out of bounds." This means that the source file is pretending to be bigger than it actually is.
So it sounds like there are issues with your source movie.
P.S. Compressor has a much better MPEG-2 encoder and I would recommend using that instead.

Encoding issue for file manager

I am using the ditto command to duplicate a file. This file has unicode filename and as per http://developer.apple.com/qa/qa2001/qa1173.html I am first normalizing the name to kCFStringNormalizationFormD and then converting it to utf-8 before calling ditto on it. This all works smoothly but when I try to get the FSRef using the original unicode name I get fnfErr. Dosn't the API CFURLGetFSRef convert the string to kCFStringNormalizationFormD? Or is there any alternate for ditto on Tiger.

no encoding issues if i use xml (xlf or xliff) bundle as xml supports utf-8 encoding.

When syncing iTunes reports "The iPod "name" cannot be synced. A duplicate file name was specified. ". the music is updated but all the sleeve imagary is not including deleting the existing images. How do I solve please?

When syncing iTunes reports "The iPod “name” cannot be synced. A duplicate file name was specified. ". The music is updated but all the sleeve imagery is not including deleting the existing images. How do I solve please?

Hello WGT52,
It sounds like this error message is preventing you from syncing your content to the iPod. I would recommend the troubleshooting steps in the following article:
iTunes: Troubleshooting issues with third-party iTunes plug-ins
Thank you for using Apple Support Communities.
Take care,
Sterling

[SOLVED] Find and replace quotes with underscores in file names

I was looking to replace all of the quotes in file names in a directory with underscores. I seem to be having a problem doing so, though. Running the find command:
find . -name '*[\`\'\"]*' -exec sh -c "mv -i '$0' '${0//[\`\'\"]/_}'" {} \;
I get a > inviting me to continue the command. What am I doing wrong?
Last edited by gogi-goji (2010-01-03 02:19:13)

@lolilolicon: yeah the -exec part does give another level of quotes, so you can run into trouble with scripts inside -exec.
I didn't know you could give {} as an argument after sh -c and $0, that solves the problem of not knowing which quotes to use around it in the -exec part.
$ find -mindepth 1 -exec sh -c 'mv -v --backup=t "$0" "$(echo "$0" | sed s/[\\x27\"\`]/___/g)"' {} \;
mv: `./a b' and `./a b' are the same file
`./\'' -> `./___'
`./\'"\'' -> `./_________'
mv: `./$0' and `./$0' are the same file
mv: `./a' and `./a' are the same file
mv: `./$(echo crash)' and `./$(echo crash)' are the same file
`./"' -> `./___' (backup: `./___.~1~')
`./\'\'\'' -> `./_________' (backup: `./_________.~1~')
Or with the other substitution method, it doesn't interpret \x27 (you could use $(echo -e \\x27) or you have to use ...'\\\''...
$ find -mindepth 1 -exec sh -c 'echo mv -v --backup=t "$0" "${0//['\\\''\"\`]/___}"' {} \;
$ find -mindepth 1 -exec sh -c 'echo mv -v --backup=t "$0" "${0//[$(echo -e \\x27)\"\`]/___}"' {} \;
mv -v --backup=t ./a b ./a b
mv -v --backup=t ./' ./___
mv -v --backup=t ./___ ./___
mv -v --backup=t ./'"' ./_________
mv -v --backup=t ./$0 ./$0
mv -v --backup=t ./___.~1~ ./___.~1~
mv -v --backup=t ./a ./a
mv -v --backup=t ./$(echo crash) ./$(echo crash)
mv -v --backup=t ./_________ ./_________
mv -v --backup=t ./" ./___
mv -v --backup=t ./''' ./_________
mv -v --backup=t ./_________.~1~ ./_________.~1~

I am using itunes 10 and trying to consolidate my files. I keep getting the error "Copying files failed. The file name was invalid or too long". How can I indentify what file is causing this problem or resolve this issue?

I am using itunes 10 and trying to consolidate my files. I keep getting the error "Copying files failed. The file name was invalid or too long". How can I indentify what file is causing this problem or resolve this issue?

BUMP
Yes, I just get that message. I don't see how I could investigate this problem.
I didn't mention that this happened when I was consolidating my library, not copying files to another computer.
In other words, I'm using a "normal" itunes procedure, itunes won't complete it, and won't tell me exactly why or how to figure out how to fix it...
Is there at least some easy way to tell which files were successfully copied to my itunes music folder so I can work on moving the uncopied files?
Can anybody help me?

[SOLVED] File name encoding issue

Similar Messages

Maybe you are looking for