Non-blocking file access?

1. Did I miss something: is there a way in java 1.5 (or even java 1.6) to read files and query file metadata (size, last modified, that kind of thing) in a non-blocking manner?
2. Is there a way to figure out where physical harddrive boundaries lie without resorting to tricks like running 'df' and parsing the output, which is rather un-portable? I obviously don't want to create a thread per file read (one aim is that the maximum number of threads is constant - even with a gazillion requests the amount of threads will never exceed a (low) constant.) but just 1 thread to perform all required reads would possibly end up wasting time if there are multiple harddrives and the CPU is idling anyway. I'd guess at this point that 1 thread per physical harddrive handling all file access is the best way to go, but how do I figure out those boundaries?
3. Is NIO2 (JSR 203) going to add features to work with files in a non-blocking way?
4. Is there a full-stack (templating, database access, writing web services, serving up the data) web server out there someplace that works entirely on a non-blocking methodology? I don't think so, but I might have missed something. JETTY tries something, but the servlet specs fundamentally do not work with non-blocking I/O, so it doesn't count.
background:
I'm writing a non-blocking webserver. KISS is the attitude - so far the basic system and any services (my take on the servlet spec, as the servlet spec doesn't play nice with NIO principles) on it all run inside a single thread, no thread pool.
So far it all works, and stress testing it with 1000 connections, 200 at the same time shows a process time of about half a millisecond per request, on a simple macbook, which is nice, because that beats apache even at serving up 404s.
So far I have only written very simple test apps that for example do calculations and the like, nothing that you'd actually do with web apps, like, say, serving (template-powered) data from files on a harddrive.
Because of the absolute requirement for each service NEVER to block while serving up bytes, manually working with files would be an unbelievable pain on the part of the web service developer. Hence I'm building templating and DB access into the system.
Unfortunately, I realized, reading NIO specs, that there doesn't seem to be any way to perform file operations in a non-blocking manner (not just reading them, but getting file size and last modified timestamps as well). I'll obviously have to use a lot of caching tactics, but before I can cache, I need to read files somehow.
The aim of this webserver is basically to survive massive spikes in traffic, be it from digg/slashdot, or from a DDOS attack, as best as possible, and where that isn't possible, to gracefully fall apart. That is, service as many requests as possible at a certain minimum bandwidth per request, prioritizing repeat visitors and site moderators, and deny the rest instantly, instead of just falling apart in shambles as happends in your usual LAMP or servlet design, not serving even 1 request right most of the time.
Thanks a lot for everyone's time!

ejp wrote:
BufferedInputStream will improve the performance by a factor of several thousand because it cuts down on system calls.Well, that might be true if you're reading individual bytes with a call to read(), or making many small read( buffer ) requests. But if you call read( buffer ) with a reasonably sized buffer (BufferedInputStream uses 8192 bytes as a default buffer size), then you're going to get performance equal to or possibly better than a BufferedInputStream.

Similar Messages

  • Non-blocking file selector?

    Running W98/Labview 5.1.1, I have a DAQ application that collects data and
    streams it to file. For this I would like to be able to change output
    files without breaking the DAQ loop. For some reason, however the DAQ loop
    occurence will time out the moment I open the file dialog, even though
    the dialog runs in it's own while loop and even in it's own sub VI with
    low priority. Any suggestions how I could get a non-blocking dialog box?
    TIA,
    Rudolf

    Hi Rudolf,
    take a look
    http://zone.ni.com/devzone/devzoneweb.nsf/opendoc?openagent&75C636329327A87B862568690074800A&cat=8ABEC12D4C0AA4A3862567AC00583899
    I hope it help you in many cases.
    Mike
    Rudolf Potucek wrote:
    > Running W98/Labview 5.1.1, I have a DAQ application that collects data and
    > streams it to file. For this I would like to be able to change output
    > files without breaking the DAQ loop. For some reason, however the DAQ loop
    > occurence will time out the moment I open the file dialog, even though
    > the dialog runs in it's own while loop and even in it's own sub VI with
    > low priority. Any suggestions how I could get a non-blocking dialog box?
    >
    > TIA,
    >
    > Rudolf

  • Non-blocking file behaviour for Reader

    Hi!
    It will be really nice if Reader will not block files for writing. Assume someone is using TeX to create PDF and he need to
    a) examine the results;
    b) edit the source.
    It becomes a nightmare with Reader. Compile TeX, open Reader, examine results, close Reader. And then from the beginning.
    Auto reloading of a PDF file is also a nice feature!
    So far I can see only one solution: one should use another viewer to get non-blocking behaviour.
    Thanks!

    OK, let us see. I use Windows XP Professional SP3, Reader version is 9.3.0. I made a simple test: openned a file in Reader, then openned this file in my favourite text editor and tried to change "%PDF-1.4" to "%PDF-1.3" and save. I received a "File sharing violation error". I can also try to do it in Ubuntu.
    As for the open source viewers. That is what I am currently doing: I use additional viewer to view the files I create and I use Adobe Reader to view all other files. One can leave with such solution, but it is not the best.

  • FileInputStream.read()  and non-blocking file i/o

    According to the API, FileInputStream.read() is :
    public native int read() throws IOException
    Reads a byte of data from this input stream. This method blocks if no input is yet available.
    Returns:
    the next byte of data, or -1 if the end of the file is reached.In what instances does the read() blocks? Is there a danger that the call to read() would block forever?

    thanks martian!
    is the ff code right enough to prevent i/o blocking?
      FileInputStream fis = new FileInputStream(src);
      FileOutputStream fos = new FileOutputStream(sPersistorFolder+src.getName());
      if(fis.available() > 0) {
        int c = -1;
        while((c=fis.read()) != -1) {
          fos.write(c);
      fis.close();
      fos.close();

  • Non-Blocking Multicast Sockets in JDK 1.4?

    Hi,
    I've been trying to create non-blocking multicast sockets in JDK1.4, which essentially seems (at this stage) to boil down to the simpler problem of creating a DatagramChannel that uses MulticastSockets, or at least DatagramSockets that can join a Multicast group. Not having found any obvious way to do it, I created this extraordinary hack:
    package java.net; // Wicked, wicked!
    import java.io.*;
    public class MyDatagramSocket {
    public static void join(java.net.DatagramSocket socket, InetAddress addr)
    throws IOExceptio DatagramSocket ds = new DatagramSocket(port);
    ds.setReuseAddress(true);
    MyDatagramSocket.join(ds, InetAddress.getByName("224.0.0.104"));
    DatagramPacket dp = new DatagramPacket(array, 5000);
    ds.receive(dp);          /* READS FINE */
    n
    socket.impl.join(addr); // Uses knowledge of DatagramSocket culled from examining source to access DatagramSocketImpl
    Now I compile this, and drop the class file into my rt.jar files (in the JDK and the JRE), so that I can use MyDatagramSocket.join (DatagramSocket, InetAddress), which looks like it should work from code like this:
    try {
    int port = 58501;
    DatagramChannel dc = DatagramChannel.open();
    dc.socket().setReuseAddress(true);
    dc.socket().bind(new InetSocketAddress(port));
    MyDatagramSocket.join(dc.socket(), InetAddress.getByName("224.0.0.104"));
    byte [] array = new byte[5000];
    ByteBuffer bb = ByteBuffer.wrap(array);
    dc.receive(bb);
    System.out.println("Read from dc");
    } catch (Exception x) {
    x.printStackTrace();
    But it doesn't work - it just doesn't read. A simpler example is this:
    DatagramSocket ds = new DatagramSocket(port);
    ds.setReuseAddress(true);
    MyDatagramSocket.join(ds, InetAddress.getByName("224.0.0.104"));
    DatagramPacket dp = new DatagramPacket(array, 5000);
    ds.receive(dp);          /* READS FINE */
    So I know that my hack is working, but this fails:
    DatagramChannel dc = DatagramChannel.open();
    dc.socket().bind(new InetSocketAddress(port));
    dc.socket().setReuseAddress(true);
    MyDatagramSocket.join(dc.socket(), InetAddress.getByName("224.0.0.104"));
    DatagramPacket dp = new DatagramPacket(array, 5000);
    dc.socket().receive(dp);     /* NEVER READS */
    I've reduced the problem to one of the difference between a java.net.DatagramSocket - the standard DatagramSocket, and a sun.nio.ch.DatagramSocketAdaptor, which is what DatagramChannels seem to use.
    My questions are:
    a) Is there a proper way to do this, without my adding my own classes to java.net?
    b) If NO is the answer to a), any ideas what I'm doing wrong in my code?
    Many thanks for any assistance,
    Craig

    I've encountered the same problem in my code. The datagramChannel never receives incoming data. Doesn't matter the rate at which you send it or anything else. I don't see any way around this problem at the moment. If i find something i'll post it. Interesting enough, my friend who programs with C++ got non-blocking I/O with datagrams to work in windows. So this might just be java.

  • How do I replace someone else's Mac ID with my own? Bought iPhone second hand. Previous owner re-set the phone but his ICloud account stayed on now my phone. He uses his macID on his new iPhone now?? Any ideas, as this issue is blocking my access to iTune

    How do I replace someone else's Mac ID with my own?
    Bought iPhone second hand on TradeMe.
    Previous owner re-set the phone but his ICloud account stayed on now my phone. He uses his AppleID on his new iPhone now and, understandably, does not want to give me his password. Any ideas?? Please.
    This issue is blocking my access to iTunes and any other file from my home computer. It keeps on telling me that I have to autorise my computer to pass on files, yet, it seems, for all this to happen, I need to get logged in through the phone's Apple ID. This of course is different to my one on my computer - and I have no password for it.
    This phone is not stolen!! I payed still a fair bit for it. Am still in contact with the previous owner. He doesn't know how to fix the problem either.
    Would appreciate any suggestion ????
    Thanks
    SamSings

    Settings>general>resets>erase all content and settings.
    That will put it back to its out of the box state. Set it up with your own apple Id.

  • SQL Developer blocking vs non-blocking

    Hi,
    Is there a way in SQL Developer 3.0 to have a non-blocking SQL call? In Oracle Forms we can choose between blocking and non-blocking SQL Net connections to the database. Does SQL Developer have the same capability? If so where exactly are those options hidden in the preferences menus?
    Annoyed,
    ScottK
    Edited by: ScottK on Jun 1, 2011 11:20 AM

    Hi ScottK,
    SQLDeveloper locks access to individual connections when they are in use, if you want to simulate
    some aspects of non blocking behaviour you can use other connections.
    The following features might help:
    Use a different connection name back to the same schema in your database - it will be treated as an independent connection and 'blocking' on this connection will not affect your other connections.
    This can be achieved temporarily in the worksheet by:
    ctrl shift n
    which creates a worksheet with new connection independent of your existing worksheet connection
    though connected to the same schema and database.
    -Turloch
    SQLDeveloper team

  • Acrobat 9.3.4 no longer finds bookmarked non-PDF files (and then launches assoc app)

    I produce a documentation DVD consisting of a one-page PDF file of ~ 1400 bookmarks and a lot of content.  The bookmarks point to the content on the DVD which includes pdf files as well as text, DOC, XLS, and other misc. file types, and a large number of html, swf, flv, .jpg, etc. files which make up 4 complete web sites.  The web sites are entirely contained on the DVD (no extermal internet access is referenced or required)
    Content is accessed by clicking a bookmark which simply opens the file (regardless of type.)  Prior to Acrobat 9.3.4, appropriate warnings popped up for non-PDF files but if allowed by the user, the appropriate app was launched for the requested file.
    Since applying 9.3.4, a "Launch File" warning box pops up showing the file name (which lacks a device) and when the warning's "Open" button is pressed, a Windows error appears with the message "Windows cannot find 'file.txt'.  Make sure you typed the name correctly, and then try again.  To search for a file....  ", and finally an Adobe Acrobat information box pops up and says "Could not open the file 'path/file.txt'  ".
    I have read and re-read about launching external apps, restricted urls and attachments, trust manager preferences, etc. etc.  I've added the files to the privileged locations list under the enhanced security settings and on and on.  It no longer works, yet I tested this DVD on several computers just last week and it worked fine. So, as a shot in the dark, I decided to fire up my laptop and test it there again.
    Once on, the laptop immediately wanted to update about 12,000 software products but I politely said "No, wait until i test this out!"  The test was successful and the DVD's pdf file worked great again!  At that point I let the machine apply the Acrobat 9.3.4 update (and some uncountable number of Windows patches and fixes.)  It's running a current version of Win7.  After the reboot, i tried the DVD again (under Acrobat 9.3.4) and it failed as described above.  Because of Win7 I was able to revert to acrobat 9.3.2 using System Restore.  After doing that, the DVD worked again.
    Most of our user's machines are either Macs or WinXP (current versions) and i haven't had time to figure out if you can even recover from this update under those OS's.  (It says you cannot remove it from XP, however.)  Regardless, even if i can do it for my laptop and get it working again, i cannot do it for all of our users.
    So, my question is if i am not doing something wrong, how do i let someone at Adobe know this release is broken??  If i am doing something wrong on the other hand, what is it and how do i get this to work again??
    Thanks all.

    Thanks.  I did submit a report at the site.  I hope somebody reads it as this is a big problem for us.
    Thanks again.

  • Blocked file after downloading iTunes on Windows XP

    I have downloaded the iTunes onto my Windows XP from apple.com/ipod/start. However, each time I try to open file, I get the same message: "Windows found problem. Unknown publisher. iTunes set-up.exe, blocked file." I am so frustrated!!! I bought an iPod Nano for my 11-year-old for his birthday and haven't been able to access a thing! Please help. Since I am "new" to this whole iPod thing, I need explicit instructions on how to fix the problem. Thanks for your help!

    Hello, pamela2142. 
    Thank you for the question.  Try the steps in the article below as they may help you resolve the issue that you are experiencing with iTunes becoming unresponsive.
    iTunes for Windows Vista, Windows 7, or Windows 8: Fix unexpected quits or launch issues
    http://support.apple.com/kb/TS1717
    Cheers,
    Jason H.

  • Hyperlink to a non-pdf file

    I have a number of pdf files as pages in an ASP.NET website.<br />I want to include hyperlinks to other pages that are not pdf files.<br />I get the message "Could not open the file <file name>" which means that acrobat tries to open the file and not to transfer the control.<br />Is there a means to make Acrobat 9 pro Extended to return the control to the website?<br /><br />Thanks,<br />Dinu

    Thanks for the suggestion.<br />Indeed AA9Ext offers Edit | Preferences | Trust Manager where I ticked "Allow opening of non-PDF file attachments with external applications". It doesn't work, apparently my problem is not related to "file attachments".<br />I get "Security Block - Acrobat does not allow connection to: <url>"

  • Remote file access doesn't work with Mac and Q10

    I'm running the latest BlackBerry Link (version 1.20, build 7) and Mac OS X (version 10.9) software on my Mac Mini computer. 
    I setup BlackBerry Link for remote file access.
    On my Q10 in File Manager I see my Mac computer but for some reason I don't see any folders. It's just a black blank screen. My carrier is Telus. 
    In link I chose folders to access. It seems like everything is setup properly and I still see only a black blank screen on my Q10. 
    Any Mac users have this issue?

    Me too. None of the sync functions/remote file access/wifi sync using BB Link are working for me. I have a Mac (10.8.5) and a BB Z10 (10.2.0.429). I hope BB comes up with a solution soon. 

  • Reading variable block file

    Hi,
    I have a variable block file which was transmitted from mainframe side.
    When i browse the variable block file in mainframe i can see it as
    !Line 1
    !Line 2
    !Line 3
    but once we transmit the file in Unix box, the file contents appear as
    !Line1!Line2!Line3
    does java differentiate between variable block and fixed block files..
    does java has separate API to read Variable block file

    Ah. Then perhaps the file browser that you use in the mainframe is displaying the data in lines even though there are actually no new-line characters separating the records. For your convenience of course.
    And perhaps there's an option in the mainframe FTP client that can insert those new-line characters if you need them. Although it's quite possible that you don't need them in the Unix system. If the only problem is that you don't see any, then just write your program assuming that there are none.

  • "Block Files That Cannot Be Scanned" in TMG malware inspection feature didn't work

    hi friends.
    in my TMG test lab, i have enabled malware inspection both Globally & on my web access rule & in its settings i have selected
    "Block Files That Cannot Be Scanned".
     now in my test external website i have a text file which is archiced it via winrar & set a password on it. password is 123.
    now when i connect  to that website, i can download that file & unzip it.
    why TMG didn't block this pasword protected archive ? it must block according to this option : ( "Block Files That Cannot Be Scanned".
    any help ?
    thanks in advance
     

    Hi,
    Thank you for your post here.
    Before going further, we must make sure that we have enabled Malware inspection and we need to note that web protection need license.
    Please follow the steps below to verify if your malware inspection is enabled meanwhile update the latest definitions.
    http://technet.microsoft.com/en-us/library/dd441078.aspx
    Additionally, you could try to check both “Block encrypted files” and “Block Files That Cannot Be Scanned” options to see if it works.
    Best Regards
    Quan Gu 
    hi quan, thanks.
    malware inspection is enabled with evaluation license & is not expired.
    both chek marks are selected but it again doesn't block password-protected rar files. if possible please test that.
    many thanks in advance

  • AppleScript - Writing to non-existent directories, and non-existent files..

    All,
    AppleScript - Writing to non-existent directories, and non-existent files...
    Creating directories several levels deep on the fly.
    How do we write to a file that does not exist, buried deep down in a hierarchy of directories that don't exist either...
    In trying to do this I explored two options. One used AppleScript, assisted by UNIX, which was simplicity itself, the other one used only AppleScript and was considerably more complex, and slower.
    http://www.mac-specialist.com/r/asckdirs.html
    Hope these are useful,
    Best Regards,
    Bill Hernandez
    Plano, Texas

    Simplified code examples - lacking extensive error checking -
    UNIX example 001:
    set file_Name to "Sales Figures.txt" -- File to create.
    set file_Path to "2006 Sales:Forecast:Fruits:Flordia:Oranges:Large" -- Folder to create.
    set UNIXfilePath to (quoted form of (POSIX path of file_Path))
    try
    do shell script "mkdir -p " & UNIXfilePath -- Attempt to create a folde, and respective intermediary folder(s).
    end try
    try
    do shell script "touch " & UNIXfilePath & "/" & file_Name -- Attempt to create a blank file.
    end try
    UNIX example 002:
    set file_Name to "Sales Figures.txt" -- File to create.
    set file_Path to "2006 Sales:Forecast:Fruits:Flordia:Oranges:Large" -- Folder to create.
    try
    do shell script "mkdir -p " & (quoted form of (POSIX path of file_Path)) -- Attempt to create a folde, and respective intermediary folder(s).
    end try
    -- Create a file, and enter some text.
    set FREF to open for access file (file_Path & ":" & file_Name) with write permission
    write "Beispieltext" to FREF
    close access FREF
    AppleScript example:
    set file_Name to "Sales Figures.txt" -- File to create.
    set file_Path to "2006 Sales:Forecast:Fruits:Flordia:Oranges:Large" -- Folder to create.
    -- Obtain list of text items of 'file_Path'.
    set {oAStID, AppleScript's text item delimiters} to {AppleScript's text item delimiters, ":"}
    set filePathList to text items of file_Path
    set AppleScript's text item delimiters to oAStID
    tell application "Finder"
    set folder_Path to name of startup disk -- Obtain name of boot disk.
    repeat with i in filePathList -- Cycle through list.
    try
    make new folder at folder_Path with properties {name:i} -- create folder.
    set folder_Path to folder_Path & ":" & i -- Create new path for next new folder.
    end try
    end repeat
    end tell
    -- Create a file, and enter some text.
    set FREF to open for access file (folder_Path & ":" & file_Name) with write permission
    write "Beispieltext" to FREF
    close access FREF
      Mac OS X (10.4.4)  

  • Non-Blocking Writing and Strings

    Hello.
    My question is simple, is it possible to write strings using a non-blocking method?
    I was praying that there was a method in the NIO API that allowed me to, yet i can't find one.
    If the answer is blatantly obvious please forgive me, i'm tired and hungry :)
    Thank you for looking at this topic

    Strings are written to files or sockets using a certain encoding. I usually use UTF-8, but your application might be different.
    1. Get the SocketChannel from your non-blocking socket.
    SocketChannel ch = mySocket.getChannel(); // mySocket is java.net.Socket2. Make a CharBuffer out of the String you want to send.
    CharBuffer chars = CharBuffer.wrap(myString); // myString is your data3. Encode it so it becomes a ByteBuffer. I'll use the [UTF-8|http://www.joelonsoftware.com/articles/Unicode.html] Charset here.
    ByteBuffer bytes = Charset.forName("UTF-8").encode(chars);4. Use the write(ByteBuffer) method in the SocketChannel from 1.
    ch.write(bytes);Import declarations:
    import java.nio.channels.SocketChannel;
    import java.nio.CharBuffer;
    import java.nio.ByteBuffer;
    import java.nio.charset.Charset;s

Maybe you are looking for