File Content Conversion: Remove reserved characters from Content
Hi PI Gurus,
we're facing a little problem with FCC in File Receiver Adapter. Sometimes the defined fieldSeparator ("|") comes within the field content.
e.g.
<?xml version="1.0" encoding="UTF-8"?>
<ns0:Positions xmlns:ns0="http://test.com">
<Position>
<Value1>001</Value1>
<Value2>Test|Content</Value2>
</Position>
</ns0:Positions>
So, my first thought was to replace all "|" with a litte java mapping beind the actual mapping.
But, is there any possibility to achieve this via configuration of FCC (something like fieldSeperatorSubstitution)?
Help is as always highly appreciated!
Cheers,
Matthias Kral
Hello Hareenkumar,
yes you're totally right. I ran in several OutOfMemoryError while testing the code above. The errors started with messages over 15 MB.
Here's the code, that is productive now. It replaces all Pipes "|" by Slashes "|" and really works fine and performant.
public void execute(InputStream in, OutputStream out)
throws StreamTransformationException {
try {
int totalCount = 0;
byte[] buf = new byte[40960];
int count = 0;
while (count >= 0) {
count = in.read(buf);
totalCount += count;
if (count > 0) {
for (int i=0; i<count; i++){
if (buf<i>==124){
buf<i> = 47;
out.write(buf, 0, count);
} catch (IOException e) {
throw new StreamTransformationException(e.getMessage());
Thank you all again!!!
Cheers Matthias
Similar Messages
-
Remove LF characters from file names
I have a folder full of files with filenames that contain LF character (ASCII code 10). I want to use Automator's "Replace Text" funcion to remove these non printing characters from file names. Is there a way to do it?
If automator is not able to do this task, I will take a bash script or applescript solution as well...Take a look at: http://stackoverflow.com/questions/4417588/sed-command-to-fix-filenames-in-a-dir ectory
(I changed tr -d "\r\n" to tr -d "\n", but try both)
for f in ~/Desktop/*
do
new="$(printf %s "$f" | tr -d "\n")"
if [ "$f" != "$new" ]; then
mv "$f" "$new"
fi
done -
Removing offending characters from string...
Hello. I have some code that receives a csv file that is generated by an AS/400 - info about store orders, etc. However occasionally there is a character that causes an exception when I try to send the data into XML (prep for a web service call).
So, I added some code to my process to check the individual bytes. It's a byte value of 28 that causes me problems. I sometimes find it where I am expecting an apostrophe. So for testing purposes I added a routine that replaces byte=28 with an apostrophe, and all is well.
My question revolves around the fact that I don't have a great understanding of character encoding, etc. Essentially, what should I be doing? Should I be using different character encoding when reading the file (currently using ISO8859-1)? I don't have much control over the actual XML generation, so my hands are tied on that end. It just seems that the solution that I implemented is less-than-ideal. (Although searching through this forum, I did find it as a suggestion...)
Any help is greatly appreciated.
ChrisSo any thoughts on the question of how to get rid of it?
Just iterate over all bytes as I am now? Or is there a better solution?I assume you are using something like a buffered reader to read in the CSV file contents. Does the ISO 8859-1 encoding have problems with the FS control code byte or does it simply read it in and decode it into \u0028 ? if the latter, my own inclination would be to handle it on the char level, (by replacement or removal) before you generate your XML. What you replace it with depends on its original function within the CSV file, which I don't know.
Joe -
Sql to remove special characters from my search
Hi everyone. I'm very new to sql and have hit another road block. I am doing a query on my database in oracle sql developer. I want to search manufacturer numbers but sometimes they were entered with dashes ( 999-99-9999) and other times not (999999999) is it possible to apply a function to overlook the dash in both my query numbers and in the database mfr_nbr column?
any help would be appreciated.
Kellyok, I have built a nesting string of replaces to remove all of my special characters and it worked perfectly but now I am not sure where to place the nest later in the string to remove it from my search of mfr numbers. The reason I need to do it again is because I want to remove the characters so I am searching in terms of " apples to apples" so to speak. here is my string so far. I still need to add the part where I put in my search for the manuf_item_nbr. my question is where do I need to place the nested replace's to remove it from my numbers I'm going to search?
SELECT MAX(item_nbr) ,REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(REPLACE( manuf_item_nbr, ' '), ','), '<'), '.'), '>'), '?'), ''), '"'), ';'), ':'), '\'), '|'), ']'), '}'), '['), '{'), '='), '+'), '_'), '-'), ')'), '('), '*'), '&'), '^'), '^'), '%'), '$'), '#'), '@'), '!'), '~'), '`'),
manuf_item_nbr ,MAX(description), MAX(description2), MAX(GHX_FULL_ITEM_DESCR), MAX(Cntrct_nbr_txt), MAX(uom_cd) ,
MAX(item_qty), MAX(tier_descr), MAX(tier_prc_amt),MAX(list_prc_amt), MAX(vndr_nm), MAX(vndr_id), MAX(unspsc_nbr),MAX(iss_account) FROM
( SELECT '' AS item_nbr, manuf_item_nbr,'' AS description, '' AS description2,'' AS GHX_FULL_ITEM_DESCR, Cntrct_nbr_txt, uom_cd, CAST(item_qty AS VARCHAR (255)) AS item_qty,tier_descr, CAST ( tier_prc_amt AS VARCHAR (255)) AS tier_prc_amt, CAST (list_prc_amt AS VARCHAR (255)) AS list_prc_amt,
vndr_nm, '' AS vndr_id, '' AS unspsc_nbr,'' AS iss_account FROM ROI.CNTRCT_PRC_LIST
WHERE ACTN_CD <> 'D'
AND ROW_UPDT_TSP IS NULL
UNION ALL
SELECT item_nbr, manuf_item_nbr,'', '', GHX_FULL_ITEM_DESCR,'', purch_uom_txt AS uom_cd,
purch_qoe_txt AS item_qty, '', '' AS tier_prc_amt,'' AS list_prc_amt,
vndr_nm, vndr_id, unspsc_nbr,
gl_cd AS iss_account
FROM ROI.ROI_ITEM_ENRCHD_NUVIA
UNION ALL
SELECT trim(item) AS item_nbr,
trim(manuf_nbr) AS manuf_item_nbr,
trim(description),
trim(description2), '' AS GHX_FULL_ITEM_DESCR, '',
trim(stock_uom ) AS uom_cd,
'' AS item_qty,'', '','' AS tier_prc_amt, '' AS list_prc_amt,'' AS vndr_id, '' AS unspsc_nbr,
CAST( trim(iss_account) AS VARCHAR(255))
FROM ITEMMAST_LAW
)GROUP BY manuf_item_nbr
ORDER BY manuf_item_nbr -
Problem with macro expansion removing ? characters from macros
Hi,
When compiling the following code (CC ex.cpp), we always receive the error line 28: Error: Unexpected ":" found.
ex.cpp
#include <stdio.h>
#include <stdarg.h>
#include <string.h>
#define ONE 1
#define MCheck(n) (n == ONE ? "One" : "???")
#define F1(buff, size, ...) func(buff, size, __VA_ARGS__)
#define F(...) F1(acBuff, sizeof(acBuff), __VA_ARGS__)
void func(char* buff, size_t len, const char* fmt, ...) {
va_list args;
va_start(args, fmt);
size_t nWrote = vsnprintf(buff, len, fmt, args);
va_end(args);
int main(void) {
char acBuff[10];
// Parsed OK
F1(acBuff, sizeof(acBuff), "%s\n", MCheck(1));
printf("%s", acBuff);
// Causes error
F("%s\n", MCheck(1));
printf("%s", acBuff);
return 0;
}This appears to be because the pre-processed code (found using CC -E ex.cpp) has removed the '?' characters from the parsed macro 'F' expansion as shown:
(Code inserted from headers)
#13 "ex.cpp"
void func ( char * buff , size_t len , const char * fmt , ... ) {
va_list args ;
( void ) ( args = ( __va_list ) & __builtin_va_alist ) ;
size_t nWrote = vsnprintf ( buff , len , fmt , args ) ;
( void ) 0 ;
int main ( void ) {
char acBuff [ 10 ] ;
func ( acBuff , sizeof ( acBuff ) , "%s\n" , ( 1 == 1 ? "One" : "???" ) ) ;
printf ( "%s" , acBuff ) ;
func ( acBuff , sizeof ( acBuff ) , "%s\n" , ( 1 == 1 "One" : "???" ) ) ;
printf ( "%s" , acBuff ) ;
return 0 ;
}Is this an error with the compiler, or is there some flag we need to set to ensure the macros are expanded to the correct level?Bwadly wrote:
Hi Maxim,
Thanks for the feedback.
We are using Solaris 8, so I'm guessing we should use Sun Studio 11? Correct, the latest release that supports Solaris 8 is Sun Studio 11. Be sure to patch both Solaris and Sun Studio once you install it.
BTW, I was able to reproduce the problem using Studio 9 compiler (CC 5.6). Apparently, it got fixed in later versions of compiler - patched version of CC 5.6 does not have it.
Edited by: MaximKartashev on 15.08.2008 12:55 -
Remove special characters from incoming data
Hi Varun, You could use either of below.. REG_REPLACE(YOUR_INPUT_STR,^[A-Za-z0-9 ],NULL) -- Replaces all non-alphanumeric with null
REG_EXTRACT(YOUR_INPUT_STR,[A-Za-z0-9 ]) -- Extracts only alphanumeric data -Rajanii have special character coming in the source data and i want to remove it before loading into target, currently i am getting one special character , it may come as some other type of special character other than alpha numeric. so how to remove those special characters from data and load the alphanumeric data into target.
-
Excel - Remove specific characters from entire worksheet if they are the first 2 in the cell
Hello,
I have a very large worksheet that is used to populate a SQL database.
I need to have the data clean before I pass it on to the owner of that database.
The data is riddled with "na" where the supplier of the data sometimes leaves it blank, like they should, but instead sometimes puts in "na" as either "not applicable" or "information not available".
I just need the all "na" removed. If I highlight the entire sheet and replace "na" with nothing then where it says "international" it will say "inertional" and I can't have that.
I need a routine, with an on-click for a button, probably, that removes all of the "na" that comes first in the cell.
I've seen where you can do it with individual cells, but not looping through the whole spreadsheet.
Thank you in advance for any help I can get!
Gee>>then where it says "international" it will say "inertional"
Selecting the "match entire cell contents" option may give you what you want. -
Removing unwanted characters from imported string
Hello,
I have a tab-delimited .txt file which I have to import into Indesign for further processing.
The file is composed by a 3 columns header row at the beginning (Code, Description, price) followed by a series of 3 columns data rows.
The problem is that sometimes, depending on the way the txt/csv file has been created, may include unwanted characters (such as spaces, double spaces, etc.).
Is there a way to "clean" the imported strings from these unwanted characters?
This is my starting code:
function processImportedTxt(){
//Open .csv file
var csvFile = File.openDialog("Open file .csv","tab-delimited(*.csv):*.csv;");
datafile = new File(csvFile);
if (datafile.exists){
datafile.open('r');
var csvData = new Array();
while(!datafile.eof){//read every row till the end of file
csvData.push(datafile.readln());
datafile.close();
for(a=1; a<csvData.length; a++){
var myRowData = csvData[a];//row of data
var mySplitData = myRowData.toString().split("\t");//divide columns
var myRowCode = mySplitData[0];
var myRowDesc = mySplitData[1];
var myRowPrice = mySplitData[2];
// Here goes code for cleaning strings from unwanted characters
processImportedTxt();
Any help would be much appreciated
Thanks in advanceHi,
If you want to safe 1-space occurences just a small correction:
i.e.:
var myRowCode = mySplitData[0].replace(/\s\s+/g,'');
Jarek -
[Forum FAQ] How to remove div characters from multiline textbox field in SharePoint 2013
Scenario:
Need to avoid the div tags and get data alone from multiline textbox field using JavaScript Client Object Model in SharePoint 2013.
Solution:
We can use a regular expression to achieve it.
The steps in detail as follows:
1. Insert a Script Editor Web Part into the page.
2. This is the complete code, add it into the Script Editor Web Part and save.
<script type="text/javascript">
ExecuteOrDelayUntilScriptLoaded(retrieveListItems, "sp.js");
function retrieveListItems() {
// Create an instance of the current context to return context information
var clientContext = new SP.ClientContext.get_current();
//Returns the list with the specified title from the collection
var oList = clientContext.get_web().get_lists().getByTitle('CustomListName');
//use CAML to query the top 10 items
var camlQuery = new SP.CamlQuery();
//Sets value that specifies the XML schema that defines the list view
camlQuery.set_viewXml('<View><RowLimit>10</RowLimit></View>');
//Returns a collection of items from the list based on the specified query
this.collListItem = oList.getItems(camlQuery);
clientContext.load(this.collListItem, 'Include(Title,MultipleText)');
clientContext.executeQueryAsync(Function.createDelegate(this, this.onQuerySucceeded), Function.createDelegate(this, this.onQueryFailed));
function onQuerySucceeded() {
//Returns an enumerator to iterate through the collection
var listItemEnumerator = this.collListItem.getEnumerator();
//Remove div tag use a regular expression
var reg1 = new RegExp("<div class=\"ExternalClass[0-9A-F]+\">[^<]*", "");
var reg2 = new RegExp("</div>$", "");
//Advances the enumerator to the next element of the collection
while (listItemEnumerator.moveNext()) {
//Gets the current element in the collection
var oListItem = listItemEnumerator.get_current();
alert(oListItem.get_item('MultipleText').replace(reg1, "").replace(reg2, ""));
function onQueryFailed(sender, args) {
alert('Request failed. ' + args.get_message() + '\n' + args.get_stackTrace());
</script>
Result:<o:p></o:p>
References:
http://www.w3schools.com/jsref/jsref_obj_regexp.asp
Please click to vote if the post helps you. This can be beneficial to other community members reading the thread.Nice article :)
If this helped you resolve your issue, please mark it Answered -
How to remove the characters from first and last position of a string?
Hi all,
I am creating an application where i am using vb script to export the data to excel.
In the excel sheet most of the values are entered with double " quotes.
so the entry looks like ""http://www.sap.com"".
Notice that it is starting with "" and not " . so the vb takes it as a line feed and shows an error .
It will be great if anybody can tell me how to remove these set of quotes ,so that the value should come as "http://www.sap.com" . A sample code wil be of great help.
Thanks in advance,
mahima.Hi,
To modify you excel you have 2 options, 1) you can filter them in excel itself by using filter ...etc in excel itself. or 2) you can do it by using ABAP code.
Below code may help you.
Get the data from Excel to Internal table using FM :
call function 'ALSM_EXCEL_TO_INTERNAL_TABLE'
exporting
filename = 'c:\temp\test.xls'
i_begin_col = 1
i_begin_row = 1
i_end_col = w_values
i_end_row = 10
tables
intern = t_alsmex_tabline
exceptions
inconsistent_parameters = 1
upload_ole = 2
others = 3.
Now replace all unwanted "" in internal table with space.
replace all occurrences of '""' in itab-field with ' '.
Again you can export to Excel using
call function 'GUI_DOWNLOAD'
exporting
filename = lv_file
filetype = 'DAT'
append = ' '
write_field_separator = 'X'
tables
data_tab = itab. -
How can I remove special characters from spreadsheet filename?
Hi
Having a spreadsheet filename coded (on the Report Attributes page) as &VARIABLE..csv works fine unless the VARIABLE contains special characters such as space, lt, gt and so on.
In the case of space, it is replaced by %20 in the generated filename. Is it possible convert the %20s to underscores (or remove them)?
thanks
TrevorTrevor,
I would use a computation on that page to alter the value of variable, perhaps a SQL Expression such as:
replace (:VARIABLE, ' ', '_')Alternatively, look at the process that puts the value into VARIABLE in the first place and ensure no spaces >, etc are assigned.
Sergio -
Removing special characters from a string
Hi All,
I have this in one my column in my table
ABC-DEF
ABC/DEF
ABC,DEF
ABC DEF
I want to remove -, /, comma and space so i want the result like this
ABCDEF
ABCDEF
ABCDEF
ABCDEF
How can I do this in PL/sql.
Thanks.Use REGEXP_REPLACE:
with sample_table as (
select 'ABC-DEF' str from dual union all
select 'ABC/DEF' from dual union all
select 'ABC,DEF' from dual union all
select 'ABC DEF' from dual
-- end of on-the-fly sample table
select str,
regexp_replace(str,'[-/, ]') new_str
from sample_table
STR NEW_STR
ABC-DEF ABCDEF
ABC/DEF ABCDEF
ABC,DEF ABCDEF
ABC DEF ABCDEF
SQL> SY. -
Re: remove special characters from incoming data
Dear Varun Kumar, Please use this logic. REG_REPLACE(INPUT FIELD,'[^A-Za-z0-9]' ,'')Only Characters and Numbers it will come. I Tried Same Logic it working. Please let me Know. Thanks & RegardsKasireddy+966545281845
Hi Varun, You could use either of below.. REG_REPLACE(YOUR_INPUT_STR,^[A-Za-z0-9 ],NULL) -- Replaces all non-alphanumeric with null
REG_EXTRACT(YOUR_INPUT_STR,[A-Za-z0-9 ]) -- Extracts only alphanumeric data -Rajani -
How do you removing special characters from a string
hi,
i'm a novice to java, if you couldn't tell. i need to write a program to recognise palindromes (eg. madam etc.) but i can't work out how to strip or clean the input string of white space or punctuation. any help appreciated. by the way, any other ideas on how to go about it?? (i don't expect it to be done, just after guidance)
my thanks in advanceOK Here you are:
protected static String removeJunk(String string)
int i, len = string.length();
StringBuffer dest = new StringBuffer(len);
char c;
for (i = (len - 1); i >= 0; i--)
c = string.charAt(i);
if (Character.isLetterOrDigit(c))
dest.append(c);
return dest.toString();
Klint -
Removing unwanted characters..
Hey guys,
I'm back for help again. Unfortunately my brain isn't creative enough, so please help! :-)
Ok, I need to remove unwanted characters from a file...the problem is that the characters look like this:
in any text editor. I'm using JEdit, and it's ISO-8859-1 encoding. The text was initally from a html file, and i think that most of the html is displayed well as text in JEdit. But these squares, which are bits of info that I don't need, are making it a little trick to do my extraction.
Ex: the word I want to extract is "trouble". But in the file, it looks like this:
troble....
Anyone know how to get rid of all that stuff???
Thanks in advance.
...DJVege...you could try to set a filter on the characters you accept. Process each character and only accept those that fall into some ASCII boundary. If you accept ASCII characters that have values between 33-255, most blocks should be eliminated.
Something like this should help:
import java.io.*;
public class Example {
protected static final int MIN_ASCII = 33;
protected static final int MAX_ASCII = 255;
public Example(String file) throws IOException {
BufferedReader b = new BufferedReader(new InputStreamReader(new FileInputStream(file)));
PrintStream p = new PrintStream(new FileOutputStream(file+"_fix.txt"));
String s = "";
int j;
while ((s = b.readLine()) != null) {
for (j = 0;j < s.length();j++) {
if (valid(s.charAt(j))) {
p.print(s.charAt(j));
p.println();
b.close();
p.close();
protected boolean valid(char c) {
int asc = (int)c;
// allow for tabs and spaces
if (asc == 9 || asc == 32) {
return true;
return (asc >= MIN_ASCII && asc <= MAX_ASCII);
public static void main(String args[]) {
if (args.length > 0) {
try {
new Example(args[0]);
catch (IOException e) {
e.printStackTrace();
}useage: java Example <file>
of course, something like this will probably only work on "english" files as I dont have an understanding on how foreign characters are encoded..
if this doesnt solve your problem, you might want to adjust the range to 33-127, which will eliminate all "block" characters and all "special formatted" characters (i.e. accented characters, currency signs, etc.)
see http://asciitable.com/ for more information on ASCII characters
Maybe you are looking for
-
I was syncing my iphone with itune and there was a update which system started downloading then something happened and i had to restart my computer. when i restarted my computer, itune started backing up my iphone eventhought there was nothing left o
-
HT204053 How to set up E-Mail for ICloud on IPad
Hoe do I change Password for Email set up for ICloud ?
-
Show URL in browser without toolbar and others
Hello, I am new in calling URL. In a CRM transaction we have an action that must show a variable URL (loads clients data). I am using a modified version of the Reuse Library RSDEMO_HTML_VIEWER. It uses classes cl_gui_html_viewer and cl_gui_custom_con
-
Get a report of cubes/measures information?
Hi, I don't know what are the reporting capabilities of OWB. I just finished building all my stars and would like to get a report of the information on all the measures that are defined in my cubes. Is this possible in OWB? Do you know of any other m
-
Converting Number into words while Display
Hello Sir, I am working on Project System. I am getting Field MAT_PSPNR(Valuated Sales Order Stock WBS Element) in MSEG Table. In table the Corresponding field stored in Numbers while Display its converted into Some other Content Please Explain Me