How to detect encoding file in ANSI, UTF8 and UTF8 without BOM

Hi all,
I am having a problem with detecting a .txt/.csv file encoding. I need to detect a file in ANSI, UTF8 and UTF8 without BOM but the problem is the encoding of ANSI and UTF8 without BOM are the same. I checked the function below and saw that ANSI and UTF8
without BOM have the same encoding. so, How can I detect UTF8 without BOM encoding file? because I need to handle for this case in my code.
Thanks.
public Encoding GetFileEncoding(string srcFile)
            // *** Use Default of Encoding.Default (Ansi CodePage)
            Encoding enc = Encoding.Default;
            // *** Detect byte order mark if any - otherwise assume default
            byte[] buffer = new byte[10];
            FileStream file = new FileStream(srcFile, FileMode.Open);
            file.Read(buffer, 0, 10);
            file.Close();
            if (buffer[0] == 0xef && buffer[1] == 0xbb && buffer[2] == 0xbf)
                enc = Encoding.UTF8;
            else if (buffer[0] == 0xfe && buffer[1] == 0xff)
                enc = Encoding.Unicode;
            else if (buffer[0] == 0 && buffer[1] == 0 && buffer[2] == 0xfe && buffer[3] == 0xff)
                enc = Encoding.UTF32;
            else if (buffer[0] == 0x2b && buffer[1] == 0x2f && buffer[2] == 0x76)
                enc = Encoding.UTF7;
            else if (buffer[0] == 0xFE && buffer[1] == 0xFF)
                // 1201 unicodeFFFE Unicode (Big-Endian)
                enc = Encoding.GetEncoding(1201);
            else if (buffer[0] == 0xFF && buffer[1] == 0xFE)
                // 1200 utf-16 Unicode
                enc = Encoding.GetEncoding(1200);
            return enc;

what you want is to get the encoding utf-8 without bom which can only be detected if the file has special characters, so do the following:
public Encoding GetFileEncoding(string srcFile)
            // *** Use Default of Encoding.Default (Ansi CodePage)
            Encoding enc = Encoding.Default;
            // *** Detect byte order mark if any - otherwise assume default
            byte[] buffer = new byte[10];
            FileStream file = new FileStream(srcFile, FileMode.Open);
            file.Read(buffer, 0, 10);
            file.Close();
            if (buffer[0] == 0xef && buffer[1] == 0xbb && buffer[2]
== 0xbf)
                enc = Encoding.UTF8;
            else if (buffer[0] == 0xfe && buffer[1] == 0xff)
                enc = Encoding.Unicode;
            else if (buffer[0] == 0 && buffer[1] == 0 && buffer[2]
== 0xfe && buffer[3] == 0xff)
                enc = Encoding.UTF32;
            else if (buffer[0] == 0x2b && buffer[1] == 0x2f &&
buffer[2] == 0x76)
                enc = Encoding.UTF7;
            else if (buffer[0] == 0xFE && buffer[1] == 0xFF)
                // 1201 unicodeFFFE Unicode (Big-Endian)
                enc = Encoding.GetEncoding(1201);
            else if (buffer[0] == 0xFF && buffer[1] == 0xFE)
                // 1200 utf-16 Unicode
                enc = Encoding.GetEncoding(1200);
           else if (validatUtf8whitBOM(srcFile))
                enc = UTF8Encoding(false);
            return enc;
private bool validateUtf8whitBOM(string FileSource)
            bool bReturn = false;
            string TextUTF8 = "", TextANSI = "";
            //lread the file as utf8
           StreamReader srFileWhitBOM  = new StreamReader(FileSource);
           TextUTF8 = srFileWhitBOM .ReadToEnd();
           srFileWhitBOM .Close();
            //lread the file as  ANSI
           StreamReader srFileWhitBOM  = new StreamReader(FileSource,Encoding.Defaul,false);
           TextANSI  = srFileWhitBOM .ReadToEnd();
           srFileWhitBOM .Close();
           // if the file contains special characters is UTF8 text read ansi show signs
            if(TextANSI.Contains("Ã") || TextANSI.Contains("±")
                 bReturn = true;
            return bReturn;

Similar Messages

  • How to download a file from the net and save it into .txt format in a datab

    Can some one show me a tutorial on how to download a file from the net and save it into .txt format in a database?
    Thank you,

    http://java.sun.com/docs/books/tutorial/networking/urls/readingWriting.html

  • How do I transfer files from one computer to another without using cables??

    How do I transfer files from one computer to another without using cables?

    Wifi network
    Email
    Thumb Drive
    Online storage such as, iCloud Google Docs, SkyDrive, etc.

  • Hi, i purchased a 2 dvd digital set. i cant just download it straight to my ipad. they said i have to down load it to my i tunes, then can transfer to i pad. im not seeing how to click the files from my email, and get them into the i tunes acct... ugh.

    hi, i purchased a 2 dvd digital set. i cant just download it straight to my ipad. they said i have to down load it to my i tunes, then can transfer to i pad. im not seeing how to click the files from my email, and get them into the i tunes acct... ugh.
    i do not have a mac home pc. just a regular pc

    I had the same problem after I gave my old iPad to my parents and tried to install Netflix. This is what you have to do:  Open iTunes on your computer, the one you sync your iPad to. Then go to iTunes Store and search for and download Netflix app. After you download it, if your iPad is set to download new purchases it may start downloading on your iPad. If so, tap and hold to delete the app (because it is trying to install the new version on the iPad) Next step, go to the App Store on your iPad and find Netflix and it should say install since you already purchased it on the computer. Tap to install, and it will say the version is not compatible, tap to download a previous version. Click that and it will install the older version!    One more thing, if and when you sync to your computer again it will say something like " Unable to install Netflix on your iPad" Just click the box to never remind you again, because it's trying to sync the newer Netflix app to your iPad, but it doesn't work so it displays the message. The old app will remain on the ipad. Hope this helps, good luck

  • How to save .pdf file using office word and excel

    Can someone help me how to save .pdf files using office word and excel?  I reinstalled my adobe 7.0 pro in my new pc and before I was able to do it but with my old pc.

    For anything after Office 2003, you have to print to the Adobe PDF printer. If you installed AA 7 on OS newer than XP, you may have to do a workaround to get it to work. With later versions of WORD you can always use the MS plugin for creating PDFs.

  • Thanks.. can u please help me out in one more question. how can i transfer files like pdf, .docx and ppt from my laptop to iPhone 5 ? please its urgent.

    thanks.. can u please help me out in one more question. how can i transfer files like pdf, .docx and ppt from my laptop to iPhone 5 ? please its urgent.

    See your other post
    First, i want to know how can i pair my iPhone 5 with my lenovo laptop?

  • How to transfer mp3 files to 4s from a PC without itunes?

    How to transfer mp3 files to 4s from a PC without itunes?

    No can do. Closest you can do is to use iCloud and iTunes Match. Otherwise, for mp3 files you will have to import them to iTunes on the computer and then sync them to the phone.

  • How can detect that cdrom drive were opened and closed?

    How can detect that cdrom drive were opened and closed?

    I'm sure there are more elegant and complicated ways to accomplish this using Windows SDK function calls to kernel32.dll or something extravagant like that, but if you want the bare-bones easy way to check if the CD-ROM tray is currently opened, then simply use System Exec to query the CD-ROM drive from the command line. One example of this is shown below.
    Use the command line function cmd /c d:, where d: is your CD-ROM drive. If the Standard Error output from System Exec.vi is "The device is not ready." followed by a carraige return and line feed, then voila, your CD-ROM drive is open. If not, it's closed. Anyone have a better idea? I'm sure one exists...
    Message Edited by Jarrod S. on 02-16-2006 12:23 PM
    Jarrod S.
    National Instruments
    Attachments:
    CDROM_Check.JPG ‏22 KB

  • How to disconnect internet connection in Time Capsule and reconnect without restarting the whole device?

    Hi, I would be very grateful if someone could tell me how to disconnect internet connection in Time Capsule and reconnect without restarting the whole device?
    The thing is that as I have a dynamic IP address, everytime I restart the internet connection my IP address changes. I sometimes need it to change, so I need to restart the connection. I don't know another way to restart the connection than restarting the whole Time Capsule.
    Thanks!!
    PD: the computer I use is Windows based, using the Airport Utility for Windows, but I also have a MacBook Pro I could use and an iPhone too.

    The utility in windows is very like Mac so the screenshots are from Mac as I don't have windows available now.
    How this works depends on the setup of the TC.. which you did not tell us.. you cannot change anything if you do not have the TC as the main router in the network.
    So on the internet tab you need to share a public IP address.
    On the next tab where you set IP address.
    Click the renew IP address.
    It might work.. and it might not.. the problem is you are not disconnecting the network.. sometimes the hard disconnection is required before the address will renew..
    What about unplugging the WAN port.. then click renew the IP which will fail.. then plug the wan back in again.
    In the end it is easier and faster to power cycle the TC.

  • How do I take a small website photo and enlarge without pixelation in PS CC?

    How do I take a small website photo and enlarge without pixelation in PS CC?

    If the resolution is too low, you can't create the extra pixels.

  • When I try to open a pages document on my mac book pro it says "The file 'thesis.pages' couldn't be opened." But i can open the same file on my iPad and iPhone without a problem.

    When I try to open a pages document on my mac book pro it says "The file 'thesis.pages' couldn't be opened." But i can open the same file on my iPad and iPhone without a problem.

    Thanks very much for the instructions.  I encountered this problem this morning myself and the recommendation fixed the problem immediately.  When things like this happen with iCloud and Pages it makes me wonder if I might be better off using something else, having just recently had to fix the problem that the current Pages can't read documents I created with a much earlier version until they were "upgraded" by an interim version.

  • How do I delete email from my iPad and iPhone without deleting it from my laptop?

    how do I delete email from my iPad and iPhone without deleting it on my laptop. I'm trying to free up space.

    It would depend on the type of mail account you are using. If you have an IMAP or Exchange account deleting mail from one client is going to sync to the other clients. You cannot remove from one device and not the others. POP mail however, does allow one to delete mail from one client and it remain on the other clients. Because POP mail is downloaded to the individual clients, whereas mail in IMAP and Exchange resides on the server, and is just viewed on the individual machines.

  • How to download a file from AL11 inot Excel format without collapse the col

    Hi,
    Please suggest how to download a file from AL11 to Excel sheet.currenlty all field are merging into single column.I writing this file via using DATASET.
    Regards
    Ricky

    Hi,
       Try this code,
    ==============================================
    TYPES : BEGIN OF ty_emp,
              empno(2) TYPE c,
              empid(10) TYPE c,
              empname(3) TYPE c,
            END OF ty_emp.
    DATA : it_emp TYPE TABLE OF ty_emp,
            wa_emp TYPE ty_emp.
    DATA : pbk TYPE string.
    SELECTION-SCREEN BEGIN OF BLOCK b1 WITH FRAME TITLE text-001.
    PARAMETERS : p_file TYPE rlgrap-filename.
    PARAMETERS : p_asfile TYPE rlgrap-filename.
    SELECTION-SCREEN END OF BLOCK b1.
    pbk = p_file.
    OPEN DATASET p_asfile FOR INPUT IN TEXT MODE ENCODING DEFAULT.
      DO.
        READ DATASET p_asfile INTO wa_emp.
        IF sy-subrc <> 0.
          EXIT.
        ENDIF.
        APPEND wa_emp TO it_emp.
      ENDDO.
    CLOSE DATASET p_asfile.
    Filling the already created file with download PS radiobutton
      CALL FUNCTION 'GUI_DOWNLOAD'
        EXPORTING
      BIN_FILESIZE                  =
          filename                      = pbk
      FILETYPE                      = 'ASC'
         append                        = 'X'
         write_field_separator         = ' '
      HEADER                        = '00'
      TRUNC_TRAILING_BLANKS         = ' '
      WRITE_LF                      = 'X'
      COL_SELECT                    = ' '
      COL_SELECT_MASK               = ' '
      DAT_MODE                      = ' '
    IMPORTING
      FILELENGTH                    =
        TABLES
          data_tab                      = it_emp
    EXCEPTIONS
      FILE_WRITE_ERROR              = 1
      NO_BATCH                      = 2
      GUI_REFUSE_FILETRANSFER       = 3
      INVALID_TYPE                  = 4
      NO_AUTHORITY                  = 5
      UNKNOWN_ERROR                 = 6
      HEADER_NOT_ALLOWED            = 7
      SEPARATOR_NOT_ALLOWED         = 8
      FILESIZE_NOT_ALLOWED          = 9
      HEADER_TOO_LONG               = 10
      DP_ERROR_CREATE               = 11
      DP_ERROR_SEND                 = 12
      DP_ERROR_WRITE                = 13
      UNKNOWN_DP_ERROR              = 14
      ACCESS_DENIED                 = 15
      DP_OUT_OF_MEMORY              = 16
      DISK_FULL                     = 17
      DP_TIMEOUT                    = 18
      FILE_NOT_FOUND                = 19
      DATAPROVIDER_EXCEPTION        = 20
      CONTROL_FLUSH_ERROR           = 21
      OTHERS                        = 22
      IF sy-subrc <> 0.
    MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO
            WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.
      ENDIF.
    ==============================================
    Regards,
    Krrishna

  • How can user attach file to interactive form, and be read by abap program?

    Hello,
    I created an abap interactive form, sent to the user, then user fills out the form fields, then it is uploaded back to sap, then my program reads this data and process it.
    It works.
    Now I want users to be able to attach any file they want, and the abap program to extract this file out of the PDF, to put it into SAP (or anywhere).
    I succeeded in attaching a file, I can "see" it via function module get_annotations, but I get the file in compressed form by Adobe (filter="FlateDecode").
    Do you know how to unzip the file, or is there another way to attach files in adobe reader/extract them via abap?
    I use acrobat reader 8.1.2
    Thx a lot!
    sandra
    Below is the xml obtained by GET_ANNOTATIONS. We see that there are other text annotations, and that the attached file has {length="98005"}, though real file (before "attach file" to the PDF, and also after "detach file") has length 111 995 bytes. It's why I deduced that there is a compression (I also tried to extract it as-is and open it with adobe but it doesn't work.
    <?xml version="1.0" encoding="UTF-8"?>
    <xfdf xmlns="http://ns.adobe.com/xfdf/" xml:space="preserve">
      <annots>
        <highlight rect="212.347,624.257996,252.979004,636.539978" creationdate="D:20080604100938+02'00'"
    name="2114308b-275d-4eca-8bfc-c8288ef4a77c" color="#FFFF00" flags="print"
    date="D:20080604100938+02'00'" title="mtresori" subject="Texte surligne "
    coords="215.432999,636.179016,249.893005,636.179016,215.432999,624.619019,249.893005,624.619019"
    page="0">
          <popup rect="595.276001,516.179016,775.276001,636.179016" flags="print,nozoom,norotate"
    open="no" page="0"/>
        </highlight>
        <fileattachment rect="15.345001,802.841003,29.345001,822.841003"
    creationdate="D:20080604101011+02'00'" name="f6e12648-c9b0-4cd1-b08b-82876c8300d7"
    color="#4055FF" flags="print,nozoom,norotate" date="D:20080604101019+02'00'" title="mtresori"
    subject="Piece jointe" file="SAP_WIDGETS2.pdf" mimetype="application/pdf"
    creation="D:20080603112937+02'00'" modification="D:20080603112937+02'00'" size="111995"
    checksum="9F846412B510089F7C6DBCC6527C6339" page="0">
          <contents-richtext>
            <body xmlns="http://www.w3.org/1999/xhtml" xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/"
    xfa:APIVersion="Acrobat:8.0.0" xfa:spec="2.0.2">
              <p>SAP_WIDGETS2.pdf</p>
            </body>
          </contents-richtext>
          <data MODE="raw" encoding="hex" length="98005" filter="FlateDecode">
    4889CC570B5414D719F651A38EC468D5C6A2980104E528BBF3B8F3B2626405242920020A0AA4AEBB
    C3B2BAECEAEEA2A018030A628C8FFA4A0F568F58426A1EF5118FB5266A8D9AA2B16A55044541C5
    A62656EBA315935AE8BDB3BBEC2C0BD9D5B39EB3671966FEFBFCE6BF
    ETC. (all hexa code is the attached file, but with compression by adobe)
          </data>
        </fileattachment>
        <text rect="96.187408,680.099976,116.186996,698.099976" creationdate="D:20080613144537+02'00'"
    name="73360fcf-f5e5-4d5b-9315-fe4c42924300" color="#FFFF00" flags="print,nozoom,norotate"
    date="D:20080613144617+02'00'" icon="Comment" title="srossi" subject="Note" page="0">
          <contents-richtext>
            <body xmlns="http://www.w3.org/1999/xhtml" xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/"
    xfa:APIVersion="Acrobat:8.0.0" xfa:spec="2.0.2">
              <p dir="ltr">
                <span dir="ltr" style="font-size:10.0pt;text-align:left;color:#000000;
    font-weight:normal;font-style:normal">
    ffffffffffffffffffffff
                </span>
              </p>
            </body>
          </contents-richtext>
          <popup rect="595.276001,578.099976,775.276001,698.099976" flags="print,nozoom,norotate"
    open="yes" page="0"/>
        </text>
        <text rect="100,82,120,100" creationdate="D:20080613144627+02'00'" state="Marked"
    name="d524b7d5-1a85-4b62-80b3-5875dd152c51"
    color="#FFFF00" flags="hidden,print,nozoom,norotate" date="D:20080613144627+02'00'"
    icon="Comment" title="srossi" subject="Note"
    inreplyto="73360fcf-f5e5-4d5b-9315-fe4c42924300" statemodel="Marked" page="0">
          <contents-richtext>
            <body xmlns="http://www.w3.org/1999/xhtml" xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/"
    xfa:APIVersion="Acrobat:8.0.0" xfa:spec="2.0.2">
              <p>Marked definie par srossi</p>
            </body>
          </contents-richtext>
          <popuprect="595.276001,-20,775.276001,100" flags="print,nozoom,norotate" open="no" page="0"/>
        </text>
        <text rect="100,82,120,100" creationdate="D:20080613144633+02'00'" state="Accepted"
    name="88af2cb6-7516-4f21-81e6-97edc7a85184"
    color="#FFFF00" flags="hidden,print,nozoom,norotate" date="D:20080613144633+02'00'" icon="Comment"
    title="srossi" subject="Note" inreplyto="d524b7d5-1a85-4b62-80b3-5875dd152c51" statemodel="Review"
    page="0">
          <contents-richtext>
            <body xmlns="http://www.w3.org/1999/xhtml" xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/"
    xfa:APIVersion="Acrobat:8.0.0" xfa:spec="2.0.2">
              <p>Accepted definie par srossi</p>
            </body>
          </contents-richtext>
          <popup rect="595.276001,-20,775.276001,100" flags="print,nozoom,norotate" open="no" page="0"/>
        </text>
        <text rect="100,82,120,100" creationdate="D:20080613144642+02'00'" state="Marked"
    name="a8c6b485-c3f9-44e2-ad53-c61b2766d9d5" color="#FFFF00" flags="hidden,print,nozoom,norotate"
    date="D:20080613144642+02'00'" icon="Comment" title="srossi" subject="Note"
    inreplyto="f6e12648-c9b0-4cd1-b08b-82876c8300d7" statemodel="Marked" page="0">
          <contents-richtext>
            <body xmlns="http://www.w3.org/1999/xhtml" xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/"
    xfa:APIVersion="Acrobat:8.0.0" xfa:spec="2.0.2">
              <p>Marked definie par srossi</p>
            </body>
          </contents-richtext>
          <popup rect="595.276001,-20,775.276001,100" flags="print,nozoom,norotate" open="no" page="0"/>
        </text>
        <text rect="100,82,120,100" creationdate="D:20080613144647+02'00'" state="Accepted"
    name="0136b689-575c-46b8-a21a-86e3c8d73904" color="#FFFF00" flags="hidden,print,nozoom,norotate"
    date="D:20080613144647+02'00'" icon="Comment" title="srossi" subject="Note"
    inreplyto="a8c6b485-c3f9-44e2-ad53-c61b2766d9d5" statemodel="Review" page="0">
          <contents-richtext>
            <body xmlns="http://www.w3.org/1999/xhtml" xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/"
    xfa:APIVersion="Acrobat:8.0.0" xfa:spec="2.0.2">
              <p>Accepted definie par srossi</p>
            </body>
          </contents-richtext>
          <popup rect="595.276001,-20,775.276001,100" flags="print,nozoom,norotate" open="no" page="0"/>
        </text>
        <text rect="100,82,120,100" creationdate="D:20080613144657+02'00'" state="Accepted"
    name="a21fc3c5-cf6a-4c6f-bc1f-328a91050b4e" color="#FFFF00" flags="hidden,print,nozoom,norotate"
    date="D:20080613144657+02'00'" icon="Comment" title="srossi" subject="Note"
    inreplyto="2114308b-275d-4eca-8bfc-c8288ef4a77c" statemodel="Review" page="0">
          <contents-richtext>
            <body xmlns="http://www.w3.org/1999/xhtml" xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/"
    xfa:APIVersion="Acrobat:8.0.0" xfa:spec="2.0.2">
              <p>Accepted definie par srossi</p>
            </body>
          </contents-richtext>
          <popup rect="595.276001,-20,775.276001,100" flags="print,nozoom,norotate" open="no" page="0"/>
        </text>
      </annots>
      <f href="acTempF-00000028453-00000000089"/>
      <ids original="03473EFED11DB2110A00000000000000" modified="959796E85292D54DB7FA558CAAD4A346"/>
    </xfdf>

    Hello everybody,
    I am sure you have an answer to my question above. It was :
    how is it possible for a user to ATTACH files to a PDF interactive form, and then be able to READ them via a program ? (I guess the solution to read them via ABAP or JAVA is quite the same?)
    Thx a lot !
    sandra

  • How to search a file from a hierarchy and delete it

    hi ,
    i have to delete a file from a directory structure .i dno't know where is file stored in my folder hierarchy .i have to search that file and delete that file .
    for example ;
    i have a folder name A , which have folder B ,C ,D ,E . B folder have other 2 folder or 3 file (anything) , same C and D And E can have file or folder .
    means we dnot know that A folder have how many files or folder in it . in that we have to search a specific file and delete it .

    hi vineet ,
    this is only for u .
    here is my code
    class serDir
         public static void main(String[] args)
              String dirname="/java";
              File f1=new File(dirname);
              if(f1.isDirectory()){
                   System.out.println("directory name "+dirname);
                   String s[]=f1.list();
              for(int i=0;i<s.lenght();i++){
              File f=new(dirname +"/"+ s);
              if(f.isDirectory()){
    System.out.println("directory name "+s[i]);
                   } else {
                   System.out.println("file name "+s[i]);
              } else {
              System.out.println(dirname+"is not directory" );
    but not working properly , now plz tell me what changes i should make to full fill my requirement . question is still same ....................waiting

Maybe you are looking for

  • Multiple Materials on a single object

    I'm sure I'm missing something...but I'm finding it impossible to use multiple materials form the 3D toolkit on a single object unless I separate the material differences to separate objects....which I'm sure is crazy. But if I import an object with

  • PSA had delta records but while updating to DSO it shows zero records.

    Hi all, I have a delta info package which picks up the delta records. The data is successfully extracted from source system and updated in the PSA. But the no of records extracted to the DSO are zero. Couldnt understand what is the problem. Request y

  • "Your role in this website is no longer valid" error message.

    Hello all, Several of my clients have recently begun reporting that they are receiving this error message: "Your role in this website is no longer valid. Please obtain a new connection key." Issuing a new connection key doesn't solve the problem, the

  • After upgrading to Lion some messages for gmail account are messed

    hello ever since upgrading to Lion (installed GoldMaster first) the new mail app doesn't seem to work properly for some messages, i was hoping someone already experienced this problem too so it would be solved in the realse version, but this didn't s

  • Can't open Final Cut Express HD v.3.51

    really annoying when you pay a lot of money for an application and are trying to get some work done and you get this message:"This software requires certain hardware or software which is missing. • Quartz Extreme Support." What is this all about? Tha