Hex Output - Byte Order

Hi Community,
I'm a little bit confused about a Hex output ordered one time byte by byte and a second time by data words:
Here is the Output by byte:
# od -t x1 /home/mbr.dmp | tail -6
00660 00 00 00 00 00 00 00 00 03 7e 09 00 00 00 80 20
00700 21 00 83 93 29 2a 00 08 00 00 00 68 0a 00 00 b4
00720 09 2a 05 fe ff ff fe 77 0a 00 02 80 f5 00 00 00
00740 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00760 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa
Here is the output by data words:
# od -t x2 /home/mbr.dmp | tail -6
00660 0000 0000 0000 0000 7e03 0009 0000 2080
00700 0021 9383 2a29 0800 0000 6800 000a b400
00720 2a09 fe05 ffff 77fe 000a 8002 00f5 0000
00740 0000 0000 0000 0000 0000 0000 0000 0000
00760 0000 0000 0000 0000 0000 0000 0000 aa55
Here is the byte order reversed, for example the last data word is aa55 and in the byte by byte output above 55 aa...
Why?
Thx & Bye Tom

Here is the byte order reversed, for example the last data word is aa55 and in the byte by byte output above 55 aa...
The intel computer is called a "Little Endian" computer vs "Big Endian" (PowerPC was "Big Endian"). I know that does not mean much to you, but I'll try to explain.
In a Little Endian computer, the address of any numeric value is the "Least Significant Byte" (LSB) for that number. So if you have a 16 bit number (eg. 0xAA55), the address of that number (eg. 0x076E) points to the Least Significant Byte, which in this case would be 0x55. The 16 bit number's address +1 (eg. 0x076F) will point to the "Most Significant Byte" (MSB), which in this case is 0xAA.
If this were a 32 bit number, such as the value at 0x0720, then the LSB would be 0x09, the next significant byte would be 0x2F, the next would be 0x05, and finally the MSB would be 0xFE. Reading this as a 32 bit value it would look like 0xFE052F09.
All that this really means is that when the intel computer loads data from memory into a working register, it will place the LSB in the lowest 8 bits of the register (bits 0-7), then next byte in the register's 8-15 bits, etc... The computer can load values into the working registers in 1 byte units, 2 byte units, 4 byte units, and the 64-bit intel chips can also load 8 bytes at a time. And in all cases the address of the numeric value points to the Least Significant byte.
A "Big Endian" computer always points to the Most Significant Byte of any numeric value. From a hex dump output point of view this makes sense to your brain, but the computer hardware doesn't really care, and as a programmer, I find there are some benefits to Little Endian.
Digital Equipment Corporation was also a Little Endian computer manufacture, and they had a hex dump format where the address was on the right and you read the dump lines right to left. This allowed the individual bytes to also be grouped into 16, 32, and 64 bit words without any special reformatting.
For example:
hexdump -C tmp.tmp
00000000 54 68 65 20 71 75 69 63 6b 20 62 72 6f 77 6e 20 |The quick brown |
00000010 66 6f 78 20 6a 75 6d 70 73 20 6f 76 65 72 20 74 |fox jumps over t|
00000020 68 65 20 6c 61 7a 79 20 64 6f 67 2e 0a           |he lazy dog..|
0000002d
myhexdump tmp.tmp # my own version of DEC's hexdump utility:
20 6e 77 6f 72 62 20 6b 63 69 75 71 20 65 68 54      0 |The quick brown |
74 20 72 65 76 6f 20 73 70 6d 75 6a 20 78 6f 66     10 |fox jumps over t|
         0a 2e 67 6f 64 20 79 7a 61 6c 20 65 68     20 |he lazy dog..|
You read the later Right to Left. It takes a little getting used to, but when dealing with Little Endian computers it is extremely useful.

Similar Messages

How can I create files in unicode format without "byte order mark"?

Hello together,
I have to export files in UTF-8 format and have to send them to another partner system which works with linux as operating system.
I have tried the different possibities to create files with ABAP. But nothing works 100% how I want.
Some examples:
1.)
OPEN DATASET [filename] FOR OUTPUT IN TEXT MODE ENCODING UTF-8.
If I create a file in this way and download it from application server to local system the result for file format in a unicode text edior like NotePad is "ANSI AS UTF-8". This means I have no BYTE ORDER MARK inside.
But it is also possible that the file format is only ANSI if the file contains no "special characters", isn't it?
In my test cases I create 3 files. 2 of them has format "ANSI AS UTF-8", and one only "ANSII".
After transfer to Linux I get as result 2 times UTF8 and one time ASCII as file format.
2.)
OPEN DATASET [filename] FOR OUTPUT IN TEXT MODE ENCODING UTF-8 WITH BYTE ORDER MARK.
With this syntax the result in local editor looks like ok. I get as format really "UTF-8".
But I get problems with the system which receives the files.
All files has the file format UTF-8 on Linux but the interface / script can not read the file with BYTE ORDER MARK.
This is a very big problem for me.
Do anybody of you know if it possible to force creation in UTF-8 without a BYTE ORDER MARK?
This means more or less the first example but all files should have UTF-8 format!
Thanks in advance
Christian

This means it is not possible to create a pure unicode file without the byte order mark?
You wouldn't happen to know how a file with byte order mark should read on a Linux system?
Or if this possible or not?
Regards
Christian

FIle Creation in the Application Server With Unicode-8 and Byte-Order Mark

Hi Guys,
I've requirement of creating a file in the Application server with the Data.
The Data Format Should be in UTF-8 and Byte-Order Mark.
I need to supply this data from SAP to PRMS.
I'm able to create a file with Unicode, but any of the guys have worked on Umicode with Byte-Order Mark, please let me know.
Thanks,
Adi.

Hi Mathieu,
If you haven't found an aswer yet, you can check in transaction SE24 CL_ABAP_FILE_UTILITIES method CREATE_UTF8_FILE_WITH_BOM. You can check the code of the method (it's very short) so you can understand how it works. It's also a static method so you can call it directly in your program.
Ex:
CALL METHOD cl_abap_file_utilities=>create_utf8_file_with_bom(your_file_name).
I hope this helps.
Pax Vobiscum.
~ Eric

Byte Order Mark (BOM) not found in UTF-8 file download from XI

Hi Guys,
Facing difficulty in downloading file from XI in UTF-8 format with byte order mark.
Receiver File adapter has been configured to download the file in UTF-8 file format. But the byte order mark is missing. Same works well for UTF-16. Could see the byte order mark at the beginning of file "FEFF" for UTF-16BE - Unicode big endian.
As per SAP help, UTF-8 supposed to be the default encoding for TEXT file type.
Configuring the Receiver File/FTP Adapter in the SAP help link.
http://help.sap.com/saphelp_nw04/helpdata/en/d2/bab440c97f3716e10000000a155106/frameset.htm
Could you please advice on how to achieve BOM in UTF-8 file as it is very important for the outbound file to get loaded in our vendor system.
Thanks.
Best Regards
Thiru

Hi! 
 
Had the same problem. But here, we create a "CSV"-File which must have the BOM otherwise it will not be recogniced as UTF-8.
 
Therefore I've done the folowing:
Created a simple destination-structure which represents the CSV and done the mapping with the graphical-mapper. The destination-Structure looks like:
 
(?xml version="1.0" encoding="UTF-8"?) 
(ONLYLINES) 
 (LINE) 
 (ENTRY)Hello I'm line 1(/ENTRY) 
 (/LINE) 
 (LINE) 
 (ENTRY)and I'm line 2(/ENTRY) 
 (/LINE) 
(/ONLYLINES)
As you can see, the "ENTRY"-Element holds the data. 
 
Now I've created the folowing Java-Mapping and added that mapping within the Interface-Mapping as second step after the graphical mapping: 
 
---cut--- 
package sfs.biz.xi.global; 
 
import java.io.InputStream; 
import java.io.OutputStream; 
import java.util.Map; 
 
import javax.xml.parsers.DocumentBuilder; 
import javax.xml.parsers.DocumentBuilderFactory; 
 
import org.w3c.dom.Document; 
import org.w3c.dom.Element; 
import org.w3c.dom.NodeList; 
 
import com.sap.aii.mapping.api.StreamTransformation; 
import com.sap.aii.mapping.api.StreamTransformationException; 
 
public class OnlyLineConvertAddingBOM implements StreamTransformation { 
 
 public void execute(InputStream in, OutputStream out) throws StreamTransformationException { 
 try { 
 byte BOM[] = new byte[3]; 
 BOM[0]=(byte)0xEF; 
 BOM[1]=(byte)0xBB; 
 BOM[2]=(byte)0xBF; 
 String retString=new String(BOM,"UTF-8"); 
 Element ServerElement; 
 NodeList Server; 
 
 DocumentBuilderFactory docBuilderFactory = DocumentBuilderFactory.newInstance(); 
 DocumentBuilder docBuilder = docBuilderFactory.newDocumentBuilder(); 
 Document doc = docBuilder.parse(in); 
 doc.getDocumentElement().normalize(); 
 NodeList ConnectionList = doc.getElementsByTagName("ENTRY"); 
 int count=ConnectionList.getLength(); 
 for (int i=0;i<count;i++) { 
 ServerElement = (Element)ConnectionList.item(i); 
 Server = ServerElement.getChildNodes(); 
 retString += Server.item(0).getNodeValue().trim() + "\r\n"; 
 } 
 
 out.write(retString.getBytes("UTF-8")); 
 
 } catch (Throwable t) { 
 throw new StreamTransformationException(t.toString()); 
 } 
 } 
 
 public void setParameter(Map arg0) { 
 // TODO Auto-generated method stub 
 
 } 
 
/* 
 public static void main(String[] args) { 
 File testfile=new File("c:\\instance.xml"); 
 File testout=new File("C:\\testout.txt"); 
 FileInputStream fis = null; 
 FileOutputStream fos= null; 
 OnlyLineConvertAddingBOM myFI=new OnlyLineConvertAddingBOM(); 
 try { 
 fis = new FileInputStream(testfile); 
 fos = new FileOutputStream(testout); 
 myFI.setParameter(null); 
 myFI.execute(fis, fos); 
 } catch (Exception e) { 
 e.printStackTrace(); 
 } 
 
 
 } 
 */ 
 
} 
--cut---
 
This Mapping searches all "ENTRY"-Tags within the XML-Strucure and creates a big string which startes with the UTF-8-BOM and than combined each ENTRY-Element, separated by CR/LF. 
 
We use this as Payload for an Mail-Adapter (sending via SMTP) but it should also work on File-Adapter. 
 
Hope it helps. 
Rene 
 
Besides: could someone tell SAP that this editor is the WORSEST editor I've ever seen. Maybe this guys should copy somethink from wikipedia :-((
Edited by: Rene Pilz on Oct 8, 2009 5:06 PM

CS4 won't open TIFF with IBM PC Byte order

When our team tries to open images saved as TIFFs with a Byte order for IBM PC with Photoshop CS4, we get an error message that we do not have enough RAM to open the image.
We then opened the image on a machine running Photoshop CS2 with no problem, resaved the TIFF with a Macintosh Byte order and that seemed to fix the problem.
Is this a glitch that Adobe is aware of and will it be fixed in any upcoming updates?

I know of one bug like that, reading PackBits compressed images (the bug is very rare).
But if you could send me a copy of the file that won't open, I'll take a look and see if it might be a problem we don't know about.

ESB Routing Service Byte Order Mark error

Hi,
I have a esb routing service to accept soap messages from an external system. The external system sents messages with a Byte Order Mark for UTF8 at the start. If i look at the tcp messages i see the following:
POST /event/DefaultSystem/CaseVerhuizing/EsbStuf0204Service HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; MS Web Services Client
Protocol 2.0.50727.42)
Content-Type: text/xml; charset=utf-8
SOAPAction: "http://www.egem.nl/StUF"
Host: ux920:7777
Content-Length: 2382
Expect: 100-continue
Connection: Keep-Alive
HTTP/1.1 100 Continue
...<?xml version="1.0" encoding="utf-8"?><soap:Envelope
xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
The three dots signify the hexadecimal value EF BB BF which is for UTF8. This raises an error however in iAS:
HTTP/1.1 400 Bad Request
Date: Mon, 16 Jul 2007 14:41:32 GMT
Server: Oracle-Application-Server-10g/10.1.3.1.0 Oracle-HTTP-Server
Content-Length: 158
Connection: close
Content-Type: text/html; charset=ISO-8859-1
Bad Request
Error parsing envelope: (1, 1) Start of root element
expected
It seems the esb routing service is trying to parse the Byte Order Mark as xml and therfore can not find the soap envelope tag? Any help is appreciated!
Kind Regards,
Andre Jochems
Message was edited by:
ajochems

Hi Andre,
We got exactly the same error as you did. Your analysis is also correct; the first 3 bytes of the reply are not interpreted by BPEL/ESB as a BOM, but as 'real' characters.
The problem is due to the fact that the UTF-8 BOM is actually optional in the specifications, and obligatory for UTF-16. The UTF-8 BOM practically has no meaning, and is never used in most SOA applications. However the guys that made the webservice you need to consume are (wrongly) convinced that the UTF-8 BOM is supposed to be there according to the OASIS/W3C specs. Which is not true btw...
For more info on BOMs, check: http://en.wikipedia.org/wiki/Byte_Order_Mark
Unfortunately I don't have the code for our 'proxy' service anymore. I remember that we simply made a little servlet, that uses input- and outputstreams. The inputstream then filters the BOM. Remember to also proxy the WSDL to 'rewrite' the endpoint ;-)
HTH,
Bas

Will this work to determine the system byte order?

I need to be able to determine, preferably without relying on conditional disable symbols, whether the system on which a VI is run uses big-endian byte order. What I came up with was to flatten a U32 containing 0xDDCCBBAA to string using the unknown system byte order, and unflatten using a big-endian byte order. If the pre- and post-flattening U32's are equal, the system must be use big-endian byte order. Two questions:
1. Will this work? It recognizes, correctly, that my PC does not use big-endian byte order, but I don't have any big-endian targets to test it out on (see #3).
2. Is there a better way? Perhaps something already built into LabVIEW?
3. Would somebody with a big-endian target verify that my VI detects it as big-endian?
Thanks,
Mark Moss
Attachments:
Detect Big-Endian System.vi ‏8 KB

You can simplify your VI a little. Since you know what the result of the string flattening should be in big endian, just compare the string.
Attachments:
Detect Big-Endian System 2.vi ‏7 KB

UNICODE Byte Order Marker at beginning of text files

Hi,
I'm running in to problems when reading text from a number of text files, some of which are plain US-ASCII text and others which are also plain US-ASCII content but contain a UNICODE UTF-8 Byte Order Mark at the beginning of the file i.e. the bytes 0xEF 0xBB 0xBF.
I open each file using standard :
InputStream fis = new FileInputStream(fileName);
Reader fileReader = new InputStreamReader(fis);
However, in those cases where a BOM is present, the first 3 characters of my stream are the BOM above which I would have expected to have been automatically stripped. When I set the encoding in the InputStreamReader I still get a single garbage character, whereas when I perform above with UTF-16 I get only the files chars as expected.
Do I need to open this file as a byte stream and check the BOM myself and then derive the type of encoding I should be opening the file with? And if so, for UTF-8 I also then must discard the first 3 bytes?
Please help as I don't want to have to do this if possible and I hope someone can understand my problem.
My JVM environment is 1.4.2 on XP.
Many THanks,
Henry

Probably not. Some of the Unicode encoding types... the list is here:
http://java.sun.com/j2se/1.4.2/docs/guide/intl/encoding.doc.html
support the BOM, so you'd only need to know that it's UnicodeBig or UnicodeLittle or whatever it really is.
Of course, if you don't know what it is, that is a problem. You can probably assume the BOM bytes are actually that, but technically, you can't generally infer any particular encoding type by just reading the file. I mean, who's to say that a file is UTF-8 encoded or ISO8859-1? Yes, if it is UTF-8, and it includes chars that are of multi-byte sets (Chinese, for example), then many characters, if read as ISO8859-1, would look on screen like gibberish. But from the standpoint of reading a file at the character level, Java doesn't care and can't know.
So to really know, you would either have to know ahead of time what the encoding is, or do some analysis of the data to see if it's likely 1 or the other, which is probably hard to do cuz it would require some sort of natural language knowledge.

Weblogic and byte order mark in files

We have web application with static content - html files, js files, images, etc.
There are byte order mark at the beginning of all html files.
These files were genereted by some tool. So I cannot modify them.
We deploy this application on Weblogic.
When I try to access this web application via direct link to Weblogic, then I have a lot of javascript errors.
But in case when I try to access this page via Apache proxy - then all is ok.
But Apache forwards all request direct to Weblogic.
And I do not have such errors in case if application was deployed on JBoss.
In this case I can access application both via direct link to JBoss and via proxy.
Anybody have some assumptions - why I cannot access application via direct link to Weblogic?

I have found a solution for this problems.
I've just added following mime mapping to web.xml:
<mime-mapping>
<extension>xml</extension>
<mime-type>text/xml</mime-type>
</mime-mapping>
<mime-mapping>
<extension>js</extension>
<mime-type>text/javascript</mime-type>
</mime-mapping>

Byte order mark

I've seen this asked one other time but it wasn't actually answered. In dreamweaver when you are including php files in webpage, such as to create a temple, unicode automatically inserts these byte order marks. In the preferences you can deselect an option that will include the byte order marks, but when they are deselected, they still manage to show up, causing random spaces in your formatting. This is an issue because it overrides the CSS and the characters are hidden until you view your page in a browser or in Live view. If anyone can answer this question it would be greatly appreciated.

In dreamweaver when you are including php files in webpage, such as to create a temple, unicode automatically inserts these byte order marks.
That's not my experience. UTF-8 does not require a BOM.
Set title and encoding properties for a page
Nancy O.

ConvertToClob and byte order mark for UTF-8

We are converting a blob to a clob. The blob contains the utf-8 byte representation (including the 3-byte byte order mark) of an xml-document. The clob is then passed as parameter to xmlparser.parseClob. This works when the database character set is AL32UTF8, but on a database with character set WE8ISO8859P1 the clob contains an '¿' before the '<'AL32UTF8');
I would assume that the ConvertToClob function would understand the byte order mark for UTF-8 in the blob and not include any parts of it in the clob. The byte order mark for UTF-8 consists of the byte sequence EF BB BF. The last byte BF corresponds to the upside down question mark '¿' in ISO-8859-1. Too me, it seems as if ConvertToClob is not converting correctly.
Am I missing something?
code snippets:
l_lang_context number := 1;
dbms_lob.createtemporary(l_file_clob, TRUE);
dbms_lob.convertToClob(l_file_clob, l_file_blob,l_file_size, l_dest_offset,
l_src_offset, l_blob_csid, l_lang_context, l_warning);
procedure fetch_xmldoc(p_xmlclob in out nocopy clob,
o_xmldoc out xmldom.DOMDocument) is
parser xmlparser.Parser;
begin
parser := xmlparser.newParser;
xmlparser.parseClob(p => parser, doc => p_xmlclob);
o_xmldoc := xmlparser.getDocument(parser);
xmlparser.freeParser(parser);
end;The database version is 10.2.0.3 on Solaris 10 x86_64
Eyðun
Edited by: Eyðun E. Jacobsen on Apr 24, 2009 8:58 PM

can this be of some help? http://download.oracle.com/docs/cd/B19306_01/server.102/b14200/functions027.htm#SQLRF00620
Regards
Etbin

Executing output of order header not by RSNAST00

Hi to all.
I would like to execute an output in order header that created with dispatch time 1-Send with periodically scheduled job. Is there any way to execute it except using RSNAST00? I would like to execute it from an ABAP program after creating the sales doc. and commit work.
I have copied the RSNAST00 and used it in an ABAP program but it checks authorizations that my users doesn't currently have.
Thanks, Yuval Kaduri at Cellcom.

Hi Yuval,
You can use the transaction VF31 and select menu Program -> Execute in Background.
Kindly reward points if your question is answered.
Thanks
Vinod

Byte Order Mark and CSV download

referencing this thread:
Re: BUG?? UTF-8 non-Latin database chars in IR csv export file not export right
Is there any way in APEX to incorporate the BOM (Byte Order Mark) to be included within the excel download? Or must I write my own custom excel download to include the BOM?

referencing this thread:
Re: BUG?? UTF-8 non-Latin database chars in IR csv export file not export right
Is there any way in APEX to incorporate the BOM (Byte Order Mark) to be included within the excel download? Or must I write my own custom excel download to include the BOM?

StAX parser does not handle UTF-8 byte order mark

Hello,
i am playing around with the reference implementaion of the StAX API using the XMLStreamReader.
When i parse UTF-8 encoded xml files with the UTF-8 byte order mark i get the following exception when the method next() is called on the reader instance:
javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,7]
Message: processing instruction can not have PITarget with reserveld xml name
     at com.bea.xml.stream.MXParser.parsePI(MXParser.java:2734)
     at com.bea.xml.stream.MXParser.parseProlog(MXParser.java:1775)
     at com.bea.xml.stream.MXParser.nextImpl(MXParser.java:1717)
     at com.bea.xml.stream.MXParser.next(MXParser.java:1180)
The XMLStreamReader is created on a FileInputStream.
When parsing xml's without a byte order mark, parsing works without any problems.
Any idea how to solve this problem, or is this an internal problem of the StAX implementation.
Thanks for help.
Jörg Eichhorn

Issue related to handling the BOM were fixed as part of the 10g project which added NLS Support to the protocols. I just verified that an UTF8 file containing BOM is correctly processed via FTP in 10.1.0.2.0

Byte Order for Floating Point Types

I need to send an array of floating point values from an x86 C++ application to a Java application. I am using a socket to transfer the data. What, if anything, do I have to do about preserving the correct byte order?

Probably nothing.
If you do end up having a problem, look at the java.nio package. Specfically, java.nio.ByteOrder, and java.nio.Double/FloatBuffer.
To be completely safe, I think I'd probably send an enum in some kind of header that indicates what byte order your C++ is sending and use the *Buffer classes to automatically handle the conversion.

Hex Output - Byte Order

Similar Messages

Maybe you are looking for