Cp and tar files with non printable characters

Hi all,
Maybe it's a silly question but just got stuck with this.
We have an XSan with diverse material from varios departaments. Besides having a backup on tape I was trying to just do a plain copy from a terminal of all the files to another disk just using cp or tar.
But whenever cp or tar encounters a file with a nonprintable char they don't copy it.
Let's say in the client Finder the named the file "opción.txt"
The file shows up in terminal with an ? but cp or tar won't get the file.
any clues?
thanks!

Hi
Have a look at csv upload -- suggestion needed with non-English character in csv file it might help you.
Thanks,
Manish

Similar Messages

Robohelp 9 .properties file inserting non-printable characters @ export

I have a mapped help file that I am generating for integration to an online application. When we export the .properties file from the Project Set-up pod, the mapped files appear to be fine, if viewed in Notepad (see below).
However, when this is viewed in a different text editor, you can see that RoboHelp added additional non-printable characters to the .properties file (see below).
We've tried generating this from different computers, exporting it to different locations, retyping the initial entry, and haven't found a solution to this issue.
Does anyone know if there is a fix available? Are we doing something wrong?
Thanks!!
Kelly

Ask your developers if they think these characters could be what are known as BOM (byte order marks).
That is something can be seen in some files using the default encoding. There it can be changed by changing the encoding in the SSL dialog.
Maybe that explains it and if that is the cause, I don't know how you would prevent it here in Rh. I think you will have to live with your own solution.
See www.grainge.org for RoboHelp and Authoring tips
@petergrainge

Upload text files with non-english characters

I use an Apex page to upload text files. Then i retrieve the contents of files from wwv_flow_files.blob_content and convert them to varchar2 with utl_raw.cast_to_varchar2, but characters like ò, à, ù become garbage.
What could be the problem? Are characters lost when files are stored in wwv_flow_files or when i do the conversion?
Some other info:
* I see wwv_flow_files.DAD_CHARSET is set to "ascii", wwv_flow_files.FILE_CHARSET is null.
* Trying utl_raw.cast_to_varchar2( utl_raw.cast_to_raw('àòèù') ) returns 'àòèù' correctly;
* NLS_CHARACTERSET parameter is AL32UTF8 (not just english ASCII)

Hi
Have a look at csv upload -- suggestion needed with non-English character in csv file it might help you.
Thanks,
Manish

Cannot rename file with non-ASCII characters when using the

My application moves files from one directory to another by calling File[] srcFiles = srcDir.listFiles() to get a list of files in the source directory, and then calling srcFiles.renameTo(destFile) to rename each file.
This does not work (renameTo returns false and the file is not moved) under the following circumstances:
- the file's leaf name contains non-ASCII characters, for example "�"
- the OS is Solaris 9
- the LANG and LC_* environment variables are unset, i.e. the C locale is being used
If I set the LANG environment variable to, for example, en_GB.UTF-8 then the rename succeeds.
I have tried calling srcFiles[index].getName().getBytes("UTF-8") and the non-ASCII characters are being replaced with ? (0x3f) characters when LANG is unset.
Is this a bug in the JRE? I would argue that since my code does not actually manipulate the filename (I just use the File object that File.listFiles() gives me) then the rename should succeed. Of course I would not expect the file name to be displayed correctly if I printed it out.
I have reproduced this behaviour with JDK 1.4.2_05 and 1.5.0_04 on Solaris 9.
Francis

Thanks for the info Alan.
I considered setting the locale in the environment (this sounds like the "correct" fix to me and we might implement it later), but this application shares a WebLogic server with many other applications so we would have to do a huge amount of testing to make sure that the locale change wouldn't break the other apps. In the end I worked around the problem by making the code that generates the filenames in the first place strip out any non-ASCII characters (the names of the files are not critically important).
Looking forward to JSR-203, in the meantime perhaps a note about this behaviour in the java.io.File javadoc would be useful.

Naming files with non English characters.

I'm using filemaker to creat PDF's through Acrobat 10.1.12. I need to use Polish, Hungarian, Czech and Slovakian characters in the file name but the characters are not recognised and so the file name will not create. This is for Windows, the problem does not occur on a mac.

Hi
Have a look at csv upload -- suggestion needed with non-English character in csv file it might help you.
Thanks,
Manish

Cannot create file with Non-latin characters- I/O

I'm trying to create a file w/ Greek (or any other non-latin) characters ... for use in a RegEx demo.
I can't seem to create the characters. I'm thinking I'm doing something wrong w/ IO.
The code follows. Any insight would be appreciated. - Thanks
import java.util.regex.*;
import java.io.*;
public class GreekChars{
     public static void main(String [ ] args ) throws Exception{
          int c;
          createInputFile();
//          String input = new BufferedReader(new FileReader("GreekChars.txt")).readLine();
//          System.out.println(input);
          FileReader fr = new FileReader("GreekChars.txt");
          while( (c = fr.read()) != -1)
               System.out.println( (char)c );
     public static void createInputFile() throws Exception {
          PrintStream ps = new PrintStream(new FileOutputStream("GreekChars.txt"));
          ps.println("\u03A9\u0398\u03A0\u03A3"); // omega,theta,pi,sigma
          System.out.println("\u03A9\u0398\u03A0\u03A3"); // omega,theta,pi,sigma
          ps.flush();
          ps.close();
          FileWriter fw = new FileWriter("GreekChars.txt");
          fw.write("\u03A9\u0398\u03A0\u03A3",0,4);
          fw.flush();
          fw.close();
// using a printstream to create file ... and BufferedReader to read
C:> java GreekChars
// using a Filewriter to create files .. and FileReader to read
C:> java GreekChars
*/

Construct your file writer using a unicode format. If
you don't then the file is written using the platform
"default" format -probably ascii.
example:
FileWriter fw = new FileWriter("GreekChars.txt",
"UTF-8");I don't know what version of FileWriter you are using, but not that I know of take two string parameters. You should try checking the API before trying to help someone, instead of just making things up.
To the OP:
The proper way to produce a file in UTF-8 format would be this:
OutputStreamWriter writer = new OutputStreamWriter(new FileOutputStream("filename"), "UTF-8");Then to read the file, you would use:
InputStreamReader reader = new InputStreamReader(new FileInputStream("filename"), "UTF-8");

[Solved]Amarok don't play files with non latin1 characters in filename

I know that this problem is not in amarok, i patched phonon from aur http://aur.archlinux.org/packages.php?ID=27465
/src/import/gstreamer/mediaobject.cpp
at line 365 in bool MediaObject::createPipefromURL(const QUrl &url)
QByteArray encoded_cstr_url = url.toEncoded();
m_datasource = gst_element_make_from_uri(GST_URI_SRC, encoded_cstr_url.constData(), (const char*)NULL);
//for utf8 locale replace toLocal8Bit to toAscii
//toLocalFile fails for files with "?" in filenames. toLocalFile("a?c.s") => "a"
+ if ( url.toString().indexOf("file://") == 0 )
+ g_object_set(m_datasource, "location", url.toString().replace(0,7,"").toLocal8Bit().constData(), NULL);
if (!m_datasource)
return false;
and now amarok plays ALL files
Last edited by klama (2009-08-03 12:23:16)

which phonon backend are you using? try with a different one

Inserting strings of printable and non printable characters

I would very much appreciate some help with the following
To handle an interface with a legacy system I need to create strings containing both printable and non-printabel ascii characters. And with non printable characters I mean in particular those in the range of ASCII 128 to 159.
It seems it is not possible to insert a string containting both printable and not printable characters from the afore mentioned range into a VARCHAR2 table column as the following demonstrates:
insert into test values(chr(156)); -- this inserts the 'œ' symbol.
SQL> select test, ascii(test), length(test), substr(test,1,1), ascii(substr(test,1,1))from test;
TEST       ASCII(TEST) LENGTH(TEST) SUBSTR(TEST,1,1) ASCII(SUBSTR(TEST,1,1))
┐                  156            1That the the character mapped is shown as '┐' and not 'œ' is not really issue for my application, what is important is that the ASCII value is shown as 156, which is the ASCII code of the character I inserted.
What is however strange (actually probably not strange but has to do with the lack of understanding of the issue at hand) is that substr returns an empty string...
Now I try to insert a concatenated string, first the "non printable" character then a printable character
insert into test values(chr(156)||chr(65));
SQL> select test, ascii(test), length(test), substr(test,1,1), ascii(substr(test,1,1))from test;
TEST       ASCII(TEST) LENGTH(TEST) SUBSTR(TEST,1,1) ASCII(SUBSTR(TEST,1,1))
A                   65            1 A                                     65For some reason the not printable character (chr(156)) is now not inserted or at least does not appear when I selected the data from the table, this effect seems to apply to all characters in the range of ASCII 128 to 159 (tried some but not all) However for instance CHR(13) can be inserted as part of a string as shown above .
For our application I really don't care much what character is shown or not show, what is important is that I can retrieve the ASCII value and that this value matches the one I inserted which for some reason does not seem to work.
This seems to be, at least to some extent a character set issue. I have also tested this on a database with character sets set as follows
NLS_CHARACTERSET
WE8MSWIN1252
NLS_NCHAR_CHARACTERSET
AL16UTF16
With WE8MSWIN1252 the described issue does NOT occur, however unfortunately I must use NLS_CHARACTERSET AL32UTF8 which produces the results as described above!
As said any insights would be much appreciated as I am slowly but surely starting to despair.
For completions sake, character sets are set as follows (changing it is NOT an option):
NLS_CHARACTERSET
AL32UTF8
NLS_NCHAR_CHARACTERSET
AL16UTF16
The test table is created as follows
CREATE TABLE TEST
TEST VARCHAR2(1000 BYTE)
Database Version 11.2.0.3.0
Edited by: helios.taraba on Dec 2, 2012 10:18 AM --Added database version
Edited by: helios.taraba on Dec 2, 2012 10:24 AM Added description of test results using NLS_CHARACTERSET WE8MSWIN1252

Hello Orafad,
Thanks for your reply, at least I understand the effects I'm seeing i.e.
+"For multibyte character sets, n must resolve to one entire code point. Invalid code points are not validated, and the result of specifying invalid code points is indeterminate."+
http://docs.oracle.com/cd/E11882_01/server.112/e26088/functions026.htm
You are absolutely right I could use chr(50579) to get the ligature symbol. However as what we are trying to achieve is to implement a legacy interface to a 20+ years old subsystem we are actually not so much interested in the symbol itself but rather in the ascii value of that symbol (156 as you so rightly point out in the win-1252 characterset), this particular field represents the lenght of the message being sent to the subsystem and can vary from decimal 68 to 164 and is also considered in a checksum calculation which is part of the message.
As changing the nls_characterset of the database is not an option I guess I only have one reasonable avenue to resolve this namely to push the functionality to added the "encoded" length of the message (and the calculation of the checksum) to the java driver which is responsible for sending the message (tcp/ip) to the subsystem. Here we should not have any issues adding a byte with the value 156 (or any other for that matter) to the datastream.
Thankfully all other fields have characters with ascii values below 128 and above 31.
I'm going to leave my question as un-answered for a bit longer in the hopes of someone coming up with a golden bullet, although not getting my hopes up.
Thanks, Helios

Need help in removing non printable characters

hi
I am having an issue with non printable characters in webservice. This webservice dishes out xml in B2B communication to my clients programs. Due to data corruption in oracle (dont know who is creating bad data ) I am having non printable characters in the xml file which is generated from database. I am dishing out this to our customers. since the data in updated every day it is imposible to fix the data every time. I need to write a very very effficient method to strip non printable characters from strings from the xml. Can some one Please help on this one. I want to make sure this method is very efficient because this method could be potentially be called lots of times. I am using JDK 1.3.1 and oracle 8i
Any help will be appreciated
Thanks
Ashok Pappu

At some point you existing program is probably converting from String data to the XML bytes through a CharsetEncoder, probably inside a java.io.Writer.
Perhaps your best approach might be to write your own java.nio.charset.CharsetEncoder which deals with the bad characters as you see fit.
You can register a new java.nio.charset.CharSet as a private character set type. Because this should result in simply replacing a standard CharsetEncoder with a non-standard one hopefully the overheads would be low.

[AS] Problem with non English characters in file path

I wrote a script that exports a pdf file from ID, rasterizes it in PS, applies an action, saves it as another pdf file, and finally creates a Mail message, and attaches the file to it (the last part is written in AppleScript).
The problem is that it doesn't work when the path to this file contains non English characters.
This works:
make new attachment with properties {file name:"/Volumes/Macintosh HD/BackUp Tetard/Test.pdf"}
but this doesn't:
make new attachment with properties {file name:"/Volumes/Macintosh HD/BackUp Têtard /Test.pdf"}
I remember vaguely that I read somewhere that AppleScript can work with Unicode — in other words with such characters — starting from some version, don't remember which exactly, but it seems to me — Leopard.
I am on Mac OS X 10.4.11 right now. Will updating solve this problem? Does anybody know any solution to this problem: a scripting addition, some hidden setting, etc.
I made a little test: used a Russian character — ё and it works, but when I use — ê (Dutch) it doesn't. May it have something to do with the Region setting in International panel?
Thanks in advance,
Kasyan

Kasyan, as of Leopard AppleScript treats all text as Unicode pre this you can specify 'as Unicode text'. Try a test with these.
-- Leopard
set x to POSIX path of (path to desktop)
-- Pre Leopard
set x to POSIX path of (path to desktop as Unicode text)
-- Leopard
set x to POSIX path of (choose file without invisibles)
-- Pre Leopard
set x to POSIX path of ((choose file without invisibles) as Unicode text)

Non printable characters in a text file..

hi,
How to get blank lines and non-printable characters
and remove those characters from the text file being uploaded from application server .
thanks,
Anil.

Take a look at the constants in cl_abap_char_utilities. A simpler solution would be to ask for a file without such characters...

Removing non printable characters from an excel file using powershell

Hello,
anyone know how to remove non printable characters from an excel file using powershell?
thanks,
jose.

To add - Excel is a binary file. It cannot be managed via external methods easily. You can write a macro that can do this. Post in the Excel forum and explain what you are seeing and get the MVPs there to show you how to use the macro facility
to edit cells. Outside of cell text "unprintable" characters are a normal part of Excel.
¯\_(ツ)_/¯

Detecting non printables characters in a text file

Hi,
I need to remove some non printable characters like tabs, carriage returns, line feeds,.... and so!
i want to do something like
aString.replaceAll(<the non-printable char>, "");

str = str.replaceAll("\\P{Print}+", "");From http://java.sun.com/j2se/1.5.0/docs/api/java/util/regex/Pattern.html, \p{print} is printable characters and \P is the negative of \p.

Error when import file with non-english character

Hi, I have images file with non-english character (unicode), for example ABC<X>.png where <X> is non-english character such as japanese, chinese, etc. Whenever I want to import the file to After Effects (right click -> import -> file), I always encounter error: Finding file/dir info for the file "C:\...\ABC?.png" -- file not found (-43) (3::30) Can't import file "ABC?.png": unsupported filetype or extension. (0::1) My PC is Windows XP Professional 2002 SP2 English. How to solve this problem? Thanks

Adjust your system language settings. Proper file name conventions require a consistent Unicode environment, so install the respective foreign language support files or switch the language system-wide. Mixing different zones/ code ranges is always a bad idea. If your system is not in Japanese, AE will always misinterpret the characters and refuse to import. If that's not feasible, simply rename the files.
Mylenium

Problems with non-ASCII characters on Linux Unit Test Import

I found a problem with non-ASCII characters in the Unit Test Import for Linux. This problem does not appear in the Unit Test Import for Windows.
I have attached a Unit Test export called PROC1.XML It tests a procedure that is included in another attachment called PROC1.txt. The unit test includes 2 implementations. Both implementations pass non-ASCII characters to the procedure and return them unchanged.
In Linux, the unit test import will change the non-ASCII characters in the XML file to xFFFD. If I copy/paste the the non-ASCII characters into the Unit Test after the import, they will be stored and executed correctly.
Amazon Ubuntu 3.13.0-45-generic / lubuntu-core
Oracle 11g Express Edition - AL32UTF8
SQL*Developer 4.0.3.16 Build MAIN-16.84
Java(TM) SE Runtime Environment (build 1.7.0_76-b13)
Java HotSpot(TM) 64-Bit Server VM (build 24.76-b04, mixed mode)
In Windows, the unit test will import the non-ASCII characters unchanged from the XML file.
Windows 7 Home Premium, Service Pack 1
Oracle 11g Express Edition - AL32UTF8
SQL*Developer 4.0.3.16 Build MAIN-16.84
Java(TM) SE Runtime Environment (build 1.8.0_31-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.31-b07, mixed mode)
If SQL*Developer is coded the same between Windows and Linux, The JVM must be causing the problem.

Set the System property "mail.mime.decodeparameters" to "true" to enable the RFC 2231 support.
See the javadocs for the javax.mail.internet package for the list of properties.
Yes, the FAQ entry should contain those details as well.

Cp and tar files with non printable characters

Similar Messages

Maybe you are looking for