Record based file IO suggestions

I'm working with some files that store variable length data fields. Each field is preceded by a four byte preface that holds the length value for the fields. This supports reading the variable length fields. We recently moved the system over to UTF-8. The old code was determining the field length by examing the length of the String that represents the field value inside of Java, like:
int fieldLength = fieldValue.length();
In the case of chars that are require more than 8 bits of representation, this causes a discrepancy between the true byte length of the value and my codes idea of the length. Thus, when trying to read these files; everything blows up.
I'm NOT looking for a fix to this; I can handle that. What I want to know is:
What is the best way to implement a file like I'm describing these day? Should I use character streams? That would solve the robustness issue in the face of such things as our recent change of encoding. Should I use a data object stream? This would make the need to keep track of field lengths go away. Is there something in the NIO that would help?
Recommendations?

ejp wrote:
What is the best way to implement a file like I'm describing these day? With a database. Seriously.A database has a range of tools for altering and mainating the quatlity of your data set. If you are not going to use a database I suggest using a text file as you can easily reuse tools like editors and text search facilities. If you use binary, you will have the problem that if your data is corrupted or need to be used in a new way, you will have to write support for this from scratch.
Should I use character streams?If you must use files for this, I would use DataInputStreams and DataOutputStreams. That gives you all the primitive datatypes plus writeUTF() and readUTF() for your variable-length strings.writeUTF and readUTF is a good choice for write text in a binary mode.
To do the same thing with text, you can write the length as text, followed by the text itself. The problem with using predefined lengths is you may have to changed this yourself which can be difficult to get right at times. If you really are using text, you should be able to determine a sensible record seperator character or character sequence. Then adding/removing characters to edit your database is trivial.

Similar Messages

  • How to create a record based on the name of a file in the file-system?

    Hi,
    With a lot of pictures I want to have a database to gather some information about these pictures.
    First question is how to generate a record based on a file in the file system?
    e.g. the pictures are "c:\fotos\2009\01\disc_001.jpg" to "c:\foto\2009\01\dis_98.jpg" .
    now i want to create records with as one of the attributes the name of the picture (not the picture itself). how to create these records (based on the information of the file-ssytem). i.e. the number of records should be the same as the number of pictures.
    any suggestions?
    any reaction will be appreciated.
    Leo

    Link to Create directory
    http://www.adp-gmbh.ch/ora/sql/create_directory.html
    You can create a list of files in the directory and read the list files from that directory.
    [UTL_FILE Documentation |http://download.oracle.com/docs/cd/B14117_01/appdev.101/b10802/u_file.htm#996728]
    [Solution using Java|http://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:439619916584]
    SS

  • Idoc to file -- Records based on the segment repitition. Urgent..Pls.!!

    Hi Experts,
    Iam doing a Idoc to file scenario.
    Based on the sales organization(VKORG),i need to create the multiple records in the file.
    For example :
    E1MARAM 0..Unbounded
    |
    |E1MARMM 0..Unbounded
    |
    E1MARMM
    |
    E1MARMM
    |
    E1MARMM
    |
    E1MVKEM 0..Unbounded
    |
    E1MVKEM
    When E1MVKEM-VKORG = Z200 then i need to create a file with 4 records (for 4 E1MARMM segments 4 records will be created)
    When one more E1MVKEM-VKORG = Z201 present then we need to create 4 more records for Z201 in the same file.
    So now i will be getting a single file with 8 records.
    Each E1MVKEM segment will be having only one VKORG value.
    I tried with <b>oneasmany</b> function,but its creating only for VKORG=Z200 with 4 values but its not providing the values for VKORG=Z201.
    Please suggest me what to do.. I need to write UDF's or can i achieve without using them.??
    PS: we are not using BPM in our scenario.
    Thank You.
    Regards
    Seema.

    Hi Prabhu,
    I need to display the records in the file for each sales organisation (MVKEM-VKORG) .
    MARAM is the parent node and MARMM,MVKEM are child nodes.
    MARAM                                                  
             MARMM                                              
             MARMM                                              
             MARMM                                              
             MVKEM                                                               
             MVKEM                    
    Consider that first MVKEM has VKORG = Z200 then 3 records has to display in file (for 3 MARMM segments 3 records has to display)
    Simillarly consider second MVKEM has VKORG = Z201 then 3 more records has to display in file. ( same as above for 3 MARMM segments = 3 records in file)
    So for the above structure i will be getting a total of 6 Records in my output file.
    Iam trying with different Node functions but it is taking only for first VKORG and not taking the second one.so iam getting only 3 records in my file.
    Please help me in cracking this.
    Regards
    Seema.

  • Sending XML file from SAP to Windows Based file server with FTP function

    Hi Gurus,
    We are using SAP BW 3.0B version.
    I need to convert data in ODS to XML format and send this XML file to remote server which  is not a SAP application server, it is just a Window Based file server with FTP function..
    By writing some ABAP code I have converted ODS data into XML format (which gets saved in my local system)
    (Is that I need to put this file in Application Server to send it to the other servers? )
    Now the thing is how I can send this file to that Windows Based file server.
    plz suggest me.... what can be done......
    Thanks in Advance
    Madhusudhan
    Edited by: Madhusudhan Raju on Dec 3, 2009 4:25 AM

    I dont think the above code support windows OS. Because I always execute this script via UNIX.
    I think you can try this option, go to command prompt, goto the destination path where you have an XML file using cd....
    ftp (destination servername), specify the username and password.
    afterthat, use the command put and filename.
    check whether the file had reached destination successfully or not.
    For automation purpose, you can use the following script like
    ftp: -s: test.txt  (servername)
    In test.txt,
    UserName
    Password
    bin
    cd /files
    put file.xml
    bye
    Also, you can check in SM69, there will be some SAP external commands to automate the file transfer.
    Thanks
    Sat
    http://support.microsoft.com/?kbid=96269

  • How to record .jar file using Java Protocol

    Hi,
    I Tried to record .jar using Java protocol by using HR Loadrunner 11.0 Version tool.
    Am unable to record the application.
    Can any one suggest me the process of recording .jar file using load runner?
    Thanks,
    Venkat

    Look at the manual page for jar:
    # man jar
    Also you can run them by doing:
    # java -jar Prog.jar

  • Simultaneous Recording in file and  playing incoming rtp stream

    I want to Simultaneous Recording in file and  playing incoming rtp stream.I have cloned the data source. After cloning I created two data sinks and two processors.One processor/datasink is meant for recording the stream in a file and other processor/data sink is used to play the rtp through speaker.
    But at a time I am only able to achieve one operation i.e either recording or playing.For recording into the wav file I have to put Thread.sleep(30) so that I can record the rtp into the file.During this time I am not able to play the audio through speaker.I also create separate thread but this does not help.
    Please give suggestion on how to resolve this issue.

    thanks for the reply, but I think I've already solved the problem.

  • Recording audio files in PCM_FLOAT format

    Hello!
    I'm trying to record voice sample and I've got a problem with my .wav format. With:
    AudioFormat format = new AudioFormat(AudioFormat.Encoding.PCM_SIGNED, 44100.0F, 16, 1, 2, 44100.0F, false);everything works fine. But what I need (because of further analysis of this audio file in Matlab) is PCM_FLOAT format:
    AudioFormat format = new AudioFormat(AudioFormat.Encoding.PCM_FLOAT, 8000.0F, 32, 1, 4, 8000.0F, true);which unfortunately returns an error:
    Exception in thread "AWT-EventQueue-0" java.lang.IllegalArgumentException:
    No line matching interface TargetDataLine supporting format PCM_FLOAT 8000.0 Hz, 32 bit, mono, 4 bytes/frame,  is supported.I've tried a lot of different combinations with PCM_FLOAT but every one of them returns same exception. I'm probably missing something but the question is: what? How can I find what audio formats could work with my sound card? If I can only use PCM_SIGNED or UNSIGNED how can I convert recorded .wav file to PCMFLOAT?

    I've finally been able to try what you suggested. As server received from the client an array of bytes I tried to convert it to an array of floats:
    IntBuffer sbuf = ByteBuffer.wrap(myAudioBytes).order(ByteOrder.LITTLE_ENDIAN).asIntBuffer();
    int[] audioInts = new int[sbuf.capacity()];
    sbuf.get(audioInts);
    float[][] audioFloats = new float[audioInts.length][1];
    for (int k = 0; k < audioInts.length; k++)
        audioFloats[k][0] = ((float)audioInts[k])/0x8000; // Two dimensions in audioFloats because Utils.floatsToWav needs to know whether it's mono or not.
    }and then save it to file using http://code.google.com/p/j-simple-wav-io/ :
    Utils.floatsToWAV(audioFloats,new File("output.wav"),8000);Unfortunately it didn't work. I checked the .wav files with AudioFileInfo from jsresources.org and it recognizes the format of output.wav as:
    Type: WAVE
    AudioFormat: PCM_SIGNED, 8000.0 Hz, 16 bit, mono, 2 bits/frame, little-endian
    Lenght of audio data: 22050 frames, 44100 bytes, 2,75625 seconds
    Total lenght of file (including headers): 44144 byteswhile valid audio file (I mean the one that actually works with my Matlab script) is:
    Type: WAVE
    AudioFormat: PCM_FLOAT, 8000.0 Hz, 32 bit, mono, 4 bits/frame
    Lenght of audio data: unknown
    Total lenght of file (including headers): unknownBefore that I tried to do it another way:
    - getTargetEncodings(originalFormat) --> returned PCM_FLOAT (amongst others);
    - isConversionSupported(targetFormat, originalFormat) --> returned true;
    (where originalFormat is "PCM_SIGNED, 8000.0F, 16, 1, 2, 8000.0F, false" and targetFormat is "PCM_FLOAT, 8000.0F, 32, 1, 4, 8000.0F, false" - as you can see above I don't really know if Matlab wants little or big endian).
    so I thought sth like this will do the trick:
    ByteArrayInputStream bais = new ByteArrayInputStream(myAudioBytes);
    AudioInputStream AIS = new AudioInputStream(bais, originalFormat, myAudioBytes.length / originalFormat.getFrameSize());
    if(AudioSystem.isConversionSupported(targetFormat, originalFormat))
        AIS = AudioSystem.getAudioInputStream(targetFormat, AIS);
    File file = new File(fileName);
    int nWrittenBytes = 0;
    try
        nWrittenBytes = AudioSystem.write(AIS, AudioFileFormat.Type.WAVE, file);
    }It didn't work either (but it seems that this bit can help switching from 16 to 32 bits and from 2 to 4 bits/frame).
    So - clearly I've done something wrong but again have no idea what. I'm sort of new to Java and an absolute novice to processing audio files so thanks for all the advices :-).

  • Recording mp3 files

    How do I record MP3 files using ITunes? It wants to record as audio files which will take more than disk. Is there some other easy program which will be less cumbersome than ITunes. Must be freeware. Welcome suggestions. Thanks

    The link does not work. Here is the detail:
    Choose iTunes > Preferences, then click the Burning button at the top of the window.
    Choose MP3 CD as the Disc Format, then click OK.
    Select the playlist containing the songs you want to burn to the CD, and make sure all the songs you want to include on the CD have a checkmark beside them.
    You can only burn a CD from the songs in a playlist, and only checked songs are included. For instructions on creating a playlist, click "Tell me more."
    If the playlist contains more songs than will fit on one CD, iTunes will burn as many songs as fit on the CD. You may want to move some of the songs to a new playlist, and then burn each playlist to a separate CD. (You can see the size of the selected playlist at the bottom of the iTunes window.)
    Click the Burn Disc button in the upper-right corner of the iTunes window.
    Insert a blank CD-R disc and click Burn Disc again.
    If you plan to only play the CD using your computer, you can also use a CD-RW disc. To play it on a stereo, use a CD-R disc.
    If your playlist includes songs in formats other than MP3, they will not be burned on the CD.
    Regards

  • Can not get access files from Windows 7 to Claims-based file authorization share

    We have AD level 2012R2, DCs running 2012R2 of course, and we have clustered File Server (3 FSNodes running 2012R2).
    We enabled 2 policies 
    KDC Support for claim
    Kerberos support for claim
    We created 1 claim type in ADAC (For example "Division" Source Property). Filled this property to all IT AD Accounts by our value "IT"
    On FS made a share folder ITDivision:
    - set permissions  Domain Users can Modify if User.Division equals "IT"
    so on Windows 8 IT Users can access files on this share and on Windows 7 they cant
    =\ . We know from many presentations about Dynamic Access Control that File Server must enroll user claims if client do not support this claims (Service-for-User-To-Self)

    Hi,
    >>so on Windows 8 IT Users can access files on this share and on Windows 7 they cant
    =\ . We know from many presentations about Dynamic Access Control that File Server must enroll user claims if client do not support this claims (Service-for-User-To-Self)
    How is it going? Was there any error message? As far as I know, Dynamic Access Control (DAC) should work for downlevel clients. It’s backwards compatible. As Florain explains in the following blog:
    For non-Windows 8 and non-Windows Server 2012 boxes accessing DAC-protected file shares, the users do not carry any claims. For them, the Server 2012-based file share will query Active Directory and proxy the claims request to figure out what claims
    the user and machine bring. The file server checks in the name of the user, whether they should have claims. With that information, the file server evaluates the access to the file share. So yeah – DAC works for downlevel clients, too. It’s backwards compatible.
    And totally transparent to Windows 7.
    Questions regarding Dynamic Access Control (FAQ)
    http://www.frickelsoft.net/blog/?p=293
    In addition, regarding dynamic access control, the following blog can also be referred to for more information.
    Dynamic Access Control in Windows Server 2012
    http://www.infoq.com/news/2012/10/Dynamic-Access-Control
    Please Note: Since the above two website are not hosted by Microsoft, the link may change without notice. Microsoft does not guarantee the accuracy of this information.
    Best regards,
    Frank Shen

  • I am programming a DVD-Rom in DVD Studio Pro 4 and need to display and open pdf files, any suggestions?

    I am programming a DVD-Rom in DVD Studio Pro 4 and need to display and open pdf files, any suggestions?

    http://documentation.apple.com/en/dvdstudiopro/usermanual/index.html#chapter=22% 26section=9%26tasks=true

  • HELP: Appending new records to File containing 1 Record

    Hi, im transferring records to a file, the first line of the file contains the headers (used 'NO END OF LINE'). The problem is that when I write the data records to the file, the first record continues from the end of the Headers line, hence i have a record missing in my statement. I want to append the records below the Headers line. I tried opening the dataset 'for appending' instead of 'for output' but I get the same result.
    lv_ds_name_ex  = Filename
    Code:
    Getting field descriptions to use as headers and tranferring to file
      open the dataset
      OPEN DATASET lv_ds_name_ex FOR OUTPUT IN TEXT MODE ENCODING DEFAULT.
    CLEAR: lt_dfies.
      " Get field descriptions from settlement table
      CALL FUNCTION 'DDIF_FIELDINFO_GET'
        EXPORTING
          tabname   = 'zrl_generic_record'
        TABLES
          dfies_tab = lt_dfies.
    write column headers to the file
      LOOP AT lt_dfies.
        IF lt_dfies-scrtext_m = 'Char15'.
          IF lv_counter < 1.
            lt_dfies-scrtext_m = 'Remuneration Amount'.
            lv_counter = lv_counter + 1.
          ELSE.
            lt_dfies-scrtext_m = 'Commission Amount'.
          ENDIF.
        ENDIF.
        TRANSFER lt_dfies-scrtext_m  TO lv_ds_name_ex NO END OF LINE.
        TRANSFER ';' TO lv_ds_name_ex NO END OF LINE.
      ENDLOOP.
    Close  DATASET lv_ds_name_ex.
    Transferring data records to file
    open the dataset
      OPEN DATASET lv_ds_name_ex FOR APPENDING IN TEXT MODE ENCODING DEFAULT.
    write records to the file
      LOOP AT lt_generic_record_csv INTO ls_generic_record_csv.
            TRANSFER ls_generic_record_csv TO lv_ds_name_ex.
      ENDLOOP.
      "close the dataset
      CLOSE DATASET lv_ds_name_ex.

    Solution: TRANSFER " " TO lv_ds_name_ex. (This simply adds an empty line/record with an end of line) see code before closing the dataset 
    open the dataset
    OPEN DATASET lv_ds_name_ex FOR OUTPUT IN TEXT MODE ENCODING DEFAULT.
    CLEAR: lt_dfies.
    " Get field descriptions from settlement table
    CALL FUNCTION 'DDIF_FIELDINFO_GET'
    EXPORTING
    tabname = 'zrl_generic_record'
    TABLES
    dfies_tab = lt_dfies.
    write column headers to the file
    LOOP AT lt_dfies.
    IF lt_dfies-scrtext_m = 'Char15'.
    IF lv_counter < 1.
    lt_dfies-scrtext_m = 'Remuneration Amount'.
    lv_counter = lv_counter + 1.
    ELSE.
    lt_dfies-scrtext_m = 'Commission Amount'.
    ENDIF.
    ENDIF.
    TRANSFER lt_dfies-scrtext_m TO lv_ds_name_ex NO END OF LINE.
    TRANSFER ';' TO lv_ds_name_ex NO END OF LINE.
    ENDLOOP.
    TRANSFER " " TO lv_ds_name_ex.
    Close DATASET lv_ds_name_ex.
    open the dataset
    OPEN DATASET lv_ds_name_ex FOR APPENDING IN TEXT MODE ENCODING DEFAULT.
    write records to the file
    LOOP AT lt_generic_record_csv INTO ls_generic_record_csv.
    TRANSFER ls_generic_record_csv TO lv_ds_name_ex.
    ENDLOOP.
    "close the dataset
    CLOSE DATASET lv_ds_name_ex.
    Edited by: Daniel Lebotse on May 15, 2008 5:42 PM

  • HT1338 Left hand side of monitor dims on 2011/intel based iMac-any suggestions?

    Left hand side of monitor dims on 2011 /intel based iMac-any suggestions? Called apple support. Restarted-didn't help. Unplugged my second monitor from the thunderbolt port-didn't help. Unplugged center cord in the middle on the back of the monitor-waited 15 seconds, restarted, held down option-command-p-r and this didn't help. Apple support thought that it was most likely a video card/hardware issue, but would like to hear of any other trouble shooting suggestions.

    Not a hardware issue, since it shows up in a screenshot. Please boot in safe mode, launch Safari, and test. Same problem?
    Some things won't work in safe mode, possibly including wireless networking.
    After testing, reboot as usual (not in safe mode.)
    Mac OS X: Starting up in Safe Mode

  • Mac Book Pro 13" LPTP-Photo Booth Recorded Video File won't find my video/movie that I recently recorded from Sun 9-1-13. Is there a way I can retrieve the files under the Time Machine or will I need to download something for iPhoto Booth?

    I have been having the most difficult times with the use of iPhoto Booth for recording movie files of church services. It seems if I make a rename on the files, for some odd reason, I cannot find it in any of my Files Folders in Finder. I need to know is there a more simple way to retrieve movie files thru iPhoto Booth or is there something I may can download to fix this issue? Also, is there a way the videos can be flipped around to not a mirror image, but to a reverse camera? Please anyone who recognizes this problem, contact me ASAP. Thank you for your support!

    You used the data.  Verizon can not see what it was sued for.  However your phone can see whats apps used the data.  go to settings-data usage- there will be a place that says data usage cycle.  line the dates up with your cycle.  then there will be a bar graph below that   extend bother white bars one all the way to the left and one all the way to the right.  after those are extended below that will be a list of apps,  there should be one that used over 2 gb and that will show you what app used that data in her purse

  • I have a large library that I ripped in AIFF using iTunes.  I converted to Mp3 to save drive space, keeping the originals. I have a new computer and want to revert to my original files. iTunes sees the library but is looking for Mp3 files.  Suggestions?

    I have a large library that I ripped in AIFF using iTunes.  I converted to Mp3 to save drive space, keeping the originals. I have a new computer and want to revert to my original files. iTunes sees the library but is looking for Mp3 files.  Suggestions?

    You're welcome.
    For tips on logical organization within iTunes see my article on Grouping Tracks Into Albums.
    tt2

  • UDF - Removing Header Record from File

    Hi Team,
    In my file(File Content Conversion) to ABAP server proxy scenario...
    The source File structure is like this...
    KUNNR|Matnr|Description----->(This is Hearder Record.Each File contains this header record.)
    1001|0077321|Special Materials
    1002|0077323|Raw Material
    File adapter will picks the file , In Message Mapping The first record is the Header Record from the above file structure.So I need to remove the Header Record.
    Can we handle  this requirement with any FCC parameter ;I think we can do it with the help of UDF in MessageMapping?
    Can anybody ghelp me the sample code for this.
    Thanks.
    Drumi

    Hi,
    You can use Document Offset in FCC to  specify the number of lines that are to be ignored at the beginning of the document.
    Regards,
    Priyanka

Maybe you are looking for