Java.nio and databases

Please, need help.
I'm using java.nio to get large files contents (txt) and need to put these data into MSSQL SERVER 2000 database tables. NIO have capabilityies to get data and transfer directly to the tables? If so, how can i do it?
Thanks a lot
Aloisio

ChuckBing, i'm connected with database trough JDBC yet and i can retrieve data meaning my connection is OK.
I'm able to read data from a txt file (250mb large) trough MappedByteBuffer and write into another with the same size and content. It's working fine.
My question is if i can read the txt file and, instead write the data into another txt file, put these data into a table.
Thanks for your reply

Similar Messages

Java NIO and sockets in 1.4.1FCS

Hi,
I have built a client server system using the new 1.4 java.nio packages. In 1.4 everything worked fine, yet in 1.4.1 FCS it definitely does not. The process is as follows.
Everything work in non-blocking mode:
- Server receives connection and accepts it, the new SocketChannel is put in non-blocking mode and registered with the same selector as the server.
- New SocketChannel waits for data from the client so it waits for OP_READ from the selector. All it gets is OP_WRITE, OP_CONNECT. With 1.4 it received OP_READ, OP_WRITE.
I tried adding a call to finishConnect, but that didn't help.
Anyone have any thoughts about this?
Thanks,
Mike.

The code in the bug report is incorrect. The key will of course always be writable if you do not write anything to the socket! And since it is writable the readable part of the code will never be executed (its in an else-statement). When you are not interested in the OP_WRITE, change the ops on the selection key.

Java Nio and http proxy

Hello,
I would send a http request through a http proxy with a Nio Client. So I wrote by hand the
http request :
buffer.append("HTTP/1.1\n");
buffer.append("Content-type: text/xml\n");
Then I send this request with a Nio Client but the request doesn't pass.
Can you help me ?

I use tcp to send my request with a nio Client. The header of my http request :
StringBuffer buffer = new StringBuffer();
buffer.append("POST ");
String path = "/";
buffer.append(path + " ");
buffer.append("HTTP/1.1\r\n");
buffer.append("Content-type: text/xml\r\n");
buffer.append("Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2\r\n");
buffer.append("Host: 10.194.55.23:80\r\n");
buffer.append("Connection: keep-alive\r\n");
buffer.append("Content-Length: 0\r\n");
buffer.append("\r\n");

Help "ascending" to Java NIO

Hello there.
I've been using Java for quite some time, but the few really networked client/server applications I have used, have used java.io and java.net. Now, even I have heard the rumours about Java NIO, and I am curious. I have tried reading up and down the API on java.nio.* , but I must say I am still kind of confused.
Basically, I am looking for some example code, or a thorough explanation of how all the classes and components come together to form efficient asynchronous non-blocking I/O.
What I picture is a server over TCP/IP meant to handle 100+ clients, written with an emphasis on performance. Think MMORPG, I guess (no no, I am not ambitious enough to want to write one, but I can imagine that gives a good analogy of how I would have the communication set up). The server needs to quickly figure out which clients need to be handled at any given time, with as little overhead or reduntant iteration as possible.
Ok, I am asking alot perhaps.. I just wonder if anyone could offer links to example source-code, write something to show me, or basically just try to explain to me how this all works.
I would be very very grateful if anyone stepped up to the plate.
Thanks,
-Terje

Take a look at my thread Taming the NIO Circus. It starts with simple examples of a single thread echo server and a swing client.

Announcement: Java NIO Framework

Hi all,
After reading Ron Hitchen's book "Java NIO" and watching his presentation "How to Build a Scalable Multiplexed Server With NIO" at the 2006 JavaOne Conference, I implemented a framework based on his ideas.
Here is the website: http://nioframework.sourceforge.net/
I would be very glad if you could take a look and tell me what you think about it.
Regards and thanks for your input!
Ronny Standtke
PS: I will add dukes to this topic and spend it for contributors (found bugs, hints, ...).

Finally I found the time to write some benchmark programs for the NIO Framework and take a look at the MINA API.
If you download the latest NIO Framework release you can run the plaintext benchmark as follows:
Start the benchmark server (this is just the echo server without any session debugging output or logging) with the following command:
java -cp nio_framework-0.9.3_all.jar ch.unifr.nio.framework.examples.BenchmarkServer 8080
Then start the benchmark client with the follwing command:
java -cp nio_framework-0.9.3_all.jar ch.unifr.nio.framework.examples.BenchmarkClient localhost 8080
On my machine the resulting bandwidth is around 80 MB/s.
And now to MINA:
I used the very simple echo server example from the MINA project and just removed the logger from the chain. You may download this package from here (it includes all relevant source code):
http://nioframework.sourceforge.net/mina_echo_server.jar
You can start this server with the following command:
java -jar mina_echo_server.jar
If I run my BenchmarkClient against the MINA echo server the resulting bandwidth is only around 20 KB/s!? That is about 4000 times slower.
Is MINA really THAT slow? I seriously doubt that. Are there any stops that I have to pull to get a higher bandwidth with MINA?

Java NIO locking and NTFS network resources

Hi all - just ran into a really nasty situation and I was wondering if anyone else has hit it and might have some suggestions.
Platform: JRE 1.4_02 on a Win XP machine
The following test code locks a file, then copies it to another location using NIO.
When I run it with source path on my local drives (C), it works fine. If I run it with source path on a network shared resource, it fails with an IOException with description 'Error performing inpage operation'.
If I disable the lock immediately before the copy operation, it works fine.
My conclusion is that there is something about the NIO locking implementation that prevents it from working properly with NTFS volumes on other hosts. Can this be right? I've found the following bug report:
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4774175
but this seems like a huge problem that would prevent folks from using NIO in many, many applications. Maybe I'm wrong on something here...
Anyway, here's the test code:
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.nio.channels.FileChannel;
import java.nio.channels.FileLock;
* Created on May 28, 2004
* (c) 2004 Trumpet, Inc.
* @author kevin
public class test {
     private void createFile(File f) throws IOException{
          FileOutputStream os = new FileOutputStream(f);
          for(int i = 0; i < 10; i++){
               os.write(i);
          os.close();
     public test() {
          boolean testWithReleasingLockPriorToCopy = false;
          final File f1= new File("w:/temp/test2.lok");
          final File f2 = new File("w:/temp/test.lok");
          f1.delete();
          f2.delete();
          try {
               createFile(f1);
               RandomAccessFile raf1 = new RandomAccessFile(f1, "rw");
               RandomAccessFile raf2 = new RandomAccessFile(f1, "rw");
               FileChannel ch1 = raf1.getChannel();
               FileChannel ch2 = raf2.getChannel();
               FileLock flock1 = ch1.lock();
              if (!f2.getParentFile().exists() && !f2.getParentFile().mkdirs())
                   throw new IOException("Unable to create directories for destination file '" + f2 + "'");
              if (testWithReleasingLockPriorToCopy)
                   flock1.release();
               ch1.transferTo(0, raf1.length(), ch2);
               raf1.close();
               raf2.close();
          } catch (Exception e) {
               // TODO Auto-generated catch block
               e.printStackTrace();
     public static void main(String[] args) {
          test t = new test();
}Does anyone have any pointers here? I need to be able to exclusively lock a file on a network drive (preventing any other applications from opening it), then make a copy of it. I can't use regular stream operations, because the lock prevents them from working properly (it appears that, once you grab a file lock using NIO, the only way your application can use the file is via the NIO operations - using stream operations fails...).
Thanks in advance for any help!
- Kevin

i've run into the same problem recently, channels working fine for local file locking, but when you turn to the network, they fail to accurately handle locks.
i ended up writing a jni utility to ship with my java application that locks files using native windows calls.
my .c file ends up looking something like this:
JNIEXPORT jint JNICALL Java_Mapper_NativeUtils_LockFile
(JNIEnv *env, jobject obj, jstring filename)
const char* ntvFilename = (*env)->GetStringUTFChars(env, filename, 0);
int retVal = (int)CreateFile
ntvFilename
, GENERIC_WRITE
, FILE_SHARE_READ
, 0
, OPEN_EXISTING
, FILE_FLAG_SEQUENTIAL_SCAN
, 0
//add code to throw java exceptions based on retVal
if (retVal == (int)INVALID_HANDLE_VALUE)
return retVal;
(*env)->ReleaseStringUTFChars(env, filename, ntvFilename);
return retVal;
JNIEXPORT jboolean JNICALL Java_Mapper_NativeUtils_UnlockFile
(JNIEnv *env, jobject obj, jint handle)
     CloseHandle((void *)handle);
return 1;
it's a little shy on the error checking side, but it provides support for network file locking that java seems to lack.

J2ME and java.nio

I'm not sure which forum this should go in so applogies if I got it wrong.
I would like to use the java.nio package from JDK1.4 with the J2ME. Is this possible in any way?
cheers
Andrew

I'm not sure which forum this should go in so
applogies if I got it wrong.
I would like to use the java.nio package from JDK1.4
with the J2ME. Is this possible in any way?this topic should be post at
CLDC and MIDP
http://forum.java.sun.com/forum.jsp?forum=76
or
K Virtual Machine (KVM)
http://forum.java.sun.com/forum.jsp?forum=50
CLDC 1.0 and MIDP 1.0 doesn't include java.nio
please go to
http://java.sun.com/j2me/docs/
two documents will be useful
CLDC Specification, V1.0
MIDP 1.0 Specification, Final (JSR 37)

Java NIO, ByteBuffers and Linksys router

I have a client server app/game that uses NIO for communication sending ByteBuffers. On a LAN with 5-8 users it runs great. On the internet, through a Linksys router, with one user, it has a blip. I get all my data transmissions except for one buffer. Whenever I chat the buffer contains a size, an int typeID and the encoded string for chat. This particular buffer never makes it to the client on the outside of the router. I have a port forwarded and regular tcp/ip java io sockets stuff works fine. As does al lof the other NIO buffer traffic for locational data, login in and out, etc... ANy thoughts??

But not sure what would be the performance of those clients?? when compared to Java NIO performance....Telnet isn't a high-performance protocol anyway. Don't worry about it. Use existing code. Get it working. Measure. If you have a performance issue, then worry, while at least you have something you can deploy. It won't be a problem. The router is there to route, not to talk high-speed telnet.

Java io and Java nio, which is faster to binary io?

Anybody can advise me about java io and java nio ?
I want to write the faster code to read and write binary files.
I'm going to read/write
- individual elements (int, double, etc)
- arraylists
- objects
Also I'm going (or I'd want) to use seek functions.
Thanks

Anybody can advise me about java io and java nio ?
I want to write the faster code to read and write binary files.Which is "faster" depends on exactly how you're using them. For example, a MappedByteBuffer is usually faster for random access than a RandomAccessFile, unless your files are so large (and your accesses so random) than you're constantly loading from disk. And it's not at all faster for linear uni-directional access than a simple FileInputStream.
So, rather than expecting some random stranger telling you that one is faster than the other without knowing your project, perhaps you should tell us exactly how you plan to use IO, and why you think that one approach may be faster than the other.

Java and database

I want to develop an application software using java however i dont know which data base systems / software to use apart from mysql, and access. Is sql just a language or is it also a database software
please help

OK, that enough, but just to set the record straight. You said "SQL is a software too", no it is not. SQL is a language.
http://en.wikipedia.org/wiki/SQL
SQL (IPA: /ˌɛsˌkju??ˈɛl/ or /ˈsi??kwəl/), commonly expanded as Structured Query Language, is a computer language designed for the retrieval and management of data in relational database management systems, database schema creation and modification, and database object access control management
Some companies, such as Microsoft (not "windows"), make products, such as SQL server, with "SQL" in the title. But SQL Server is:
Microsoft SQL Server 2005 is a comprehensive, integrated data management and analysis software
http://www.microsoft.com/sql/prodinfo/overview/default.mspx

Troubles with timeout using java.nio.channels and non-blocking sockets

Hello.
I have a server application that employs java.nio.channels with non-blocking sockets.
The server waits for connections. The client should connect and be first in sending data.
Timeouts are significant! If client exceeds the allowed time to send data, the server should break the connection.
The huge trouble I've discovered that I cannot control the timeout when client connects but remains silent.
My code looks as follows:
<pre>
Selector oSel;
SocketChannel oSockChan;
Socket oSock;
SelectionKey oSelKey;
Iterator<SelectionKey> oItSelKeys;
int iCurrState, iMask, iCount;
iCurrState = INT_SERVER_WORKING;
iMask = SelectionKey.OP_ACCEPT | SelectionKey.OP_CONNECT | SelectionKey.OP_READ | SelectionKey.OP_WRITE;
while ( iCurrState == INT_SERVER_WORKING )
try
*// retrieving next action*
iCount = oSel.select();
if ( iCount > 0 )
oItSelKeys = oSel.selectedKeys().iterator();
while ( oItSelKeys.hasNext() )
oSelKey = oItSelKeys.next();
oItSelKeys.remove();
if ( oSelKey.isValid() )
switch ( oSelKey.readyOps() & iMask ) {
case SelectionKey.OP_ACCEPT :
oSockChan = oSSockChan.accept();
oSockChan.configureBlocking(false);
oSock = oSockChan.socket();
oSock.setKeepAlive(true);
oSockChan.register(oSel,SelectionKey.OP_READ,new MyPacket(oSock.getInetAddress(),oSock.getPort()));
break;
case SelectionKey.OP_READ :
oSelKey.interestOps(0);
((MyPacket) oSelKey.attachment()).inRequest(); *// preparing request*
this.getReader().add(oSelKey); *// sending key to reading thread*
break;
case SelectionKey.OP_WRITE :
oSelKey.interestOps(0);
((MyRequest) oSelKey.attachment()).inResponse(); *// preparing response*
this.getWriter().add(oSelKey); *// sending key to writing thread*
break;
case SelectionKey.OP_CONNECT :
default :
*// nothing to do*
catch ( IOException oExcept )
*// do some actions*
</pre>
Timeouts are easily controlled by reading and writing threads (see OP_READ and OP_WRITE ).
But when a client just connects without consequent data send, the state of this connection remains as OP_ACCEPT. The connection remains open for arbitrarily large time and I cannot control it!
Please help with idea how can I terminate such connections!

How can I process the keys that weren't selected at the bottom of the loop? Should I use the method keys() ?Yes. Form a new set from keys() and removeAll(selectedKeys()). Do that before you process selectedKeys().
And the second moment: as I understood a single key may contain several operations simultaneously? Thus I should use several if's (but not if/else 'cause it's the equivalent of switch ... case ).If there is anything unclear about 'your switch statement is invalid. You need an if/else chain' I fail to see what it is. Try reading it again. And if several ifs were really the equivalent of "switch ... case", there wouldn't be a problem in the first place. They're not, and there is.

Xml, xsl, java and database

I have several xml documents that uses different tags to describe the same type of information. In my case they describe a row in a database table.
file1.xml
<therow>
<name>Jones</name>
<title>manager</title>
</therow>
file2.xml
<arow>
<employee>King</employee>
<job>manager</job>
</arow>
I want to use a technology for reading the content of the two files and generate a new document for each file that will use generic tags so the content of the new files could look like:
file1
<row>
<employeename>Jones</employeename>
<employeetitle>manager</employeetitle>
</row>
file2
<row>
<employeename>King</employeename>
<emplyeetitle>manager</employeetitle>
</row>
An then I can read this information by using a generic Java class and insert it into the database table..
I was told that the technology used for doing this was XSLT. I have been searching the web for examples and also tried to read some of the content in the Steve Muench book 'Building Oracle XML Applications'.
I have managed to create XSL files, but I would really appreciate if someone could show me a real example of how to do this, with the xml file, the xsl file, and a small java program to read this. Can anyone help?

Post your XSL files, otherwise we'd have to write them again.

Selecting between java.io and java.nio

Hi,
I'm a bit confused between java.io and java.nio. What sre the major differences between these two?
In areas are these best applicable?

The java.nio package improves on the basic Java I/O that was available prior to JDK 1.4.
It is designed to interoperate more natively with the underlying file handles (sockets, open files etc) to achieve better performance.
The improvements include true non-blocking I/O, better buffer management, character-set support, channels (similar to Occam's channels) and selectors, and some other ancillery stuff. Most of these classes are designed to be inherently threadsafe.
There are some examples here:
http://java.sun.com/j2se/1.4.2/docs/guide/nio/example/index.html
However, if you are still a beginner with file or network I/O, its better if you start with the simple I/O first. You'll appreciate the NIO improvements better afterwards.

Java.nio.BufferOverflowException on big Strings ( 87MB)

Hi!
When calling getBytes() on a big String the following exception is thrown:
java.nio.BufferOverflowException
     at java.nio.charset.CoderResult.throwException(Unknown Source)
     at java.lang.StringCoding$CharsetSD.decode(Unknown Source)
     at java.lang.StringCoding.decode(Unknown Source)
     at java.lang.StringCoding.decode(Unknown Source)
     at java.lang.String.<init>(Unknown Source)
     at java.lang.String.<init>(Unknown Source)
     at com.db.janus.Janus.main(Janus.java:99)
This happens as soon as the size of the String gets bigger than 92003848 bytes.
I can reproduce it with the following code:
String x = new String(new byte[92003848]); // --> OK
String x = new String(new byte[92003849]); // --> Exception
JDK Version: 1.4.2_06
OS: Windows XP
Is this a known bug / limitation? I found a similar bug related to Java on Linux / Solaris, but this one was about getBytes() with Strings > 16MB.
Thanks for your help!

Is there any alternative to solve this problem.I am getting the error
java.nio.BufferOverflowException .This happens in my mail server when the file six\ze of xml where i have stored data about spam/nonspam grows upto 22mb.I have stored this xml in database as blob.at server startup uploading in in my class.Keeps througout in JVM after every 50 th request saving information back to database(about spam/nonspam).Now if xml size grows i am storing this in string to parse etc.it gives this bug error.Sholud i cahnge my datatype or use something as XML Parser to split my xml in pieces(mechanism i am not aware of at all) so that string just takes some size of xml (but here can i use string builder etc.). Now is there any proper solution to this kind of problem whenever i removes some daya from xml spam mail goes to inbox and vice versa.
Please suggest me some solution if possible.
complete error is:
Servlet initialization failed:
java.nio.BufferOverflowException
at java.nio.charset.CoderResult.throwException(CoderResult.java:259)
at java.lang.StringCoding$CharsetSD.decode([BII)[C(Unknown Source)
         at java.lang.StringCoding.decode(Ljava.lang.String;[BII)[C(Unknown Source)
         at java.lang.String.<init>([BIILjava.lang.String;)V(Unknown Source)
         at java.lang.String.<init>([BLjava.lang.String;)V(Unknown Source)
         at zerocode.core.ByteBlock.toString(ByteBlock.java:89)
         at zcCollab.mailProcessing.NaiveBayesClassifier._getInstanceData(NaiveBayesClassifier.java:275)
         at zcCollab.mailProcessing.NaiveBayesClassifier.initialize(NaiveBayesClassifier.java:86)
         at zcCollab.mailProcessing.SimpleDeliveryManager._setupClassifiers(SimpleDeliveryManager.java:409)
         at zcCollab.mailProcessing.SimpleDeliveryManager.<init>(SimpleDeliveryManager.java:62)
         at zcCollab.zcApp.InfoManagerApp._createDeliveryProcessor(InfoManagerApp.java:307)
         at zcCollab.zcApp.InfoManagerApp.initialize(InfoManagerApp.java:294)
         at zerocode.udm.Application._createInstance(Application.java:168)
         at zerocode.udm.Application.createInstanceForServlet(Application.java:111)
         at zerocode.servlet.UdmServlet._createApplication(UdmServlet.java:67)
         at zerocode.servlet.UdmServlet._initialize(UdmServlet.java:52)
         at zerocode.servlet.GenericServlet.init(GenericServlet.java:116)
         at org.mortbay.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:292)
         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:329)
         at org.mortbay.jetty.servlet.ServletHandler.dispatch(ServletHandler.java:657)
         at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:567)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1808)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1758)
         at org.mortbay.http.HttpServer.service(HttpServer.java:879)
         at org.mortbay.http.handler.ForwardHandler.handle(ForwardHandler.java:130)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1808)
         at org.mortbay.http.HttpContext.handle(HttpContext.java:1758)
         at org.mortbay.http.HttpServer.service(HttpServer.java:879)
Thanks
Vijendra

Oracle JDBC driver throwing java.nio.BufferUnderflowException exception

Hi,
We have a an application for which the middleware(SAR file) is running on Jboss.
We have configured a DCN (data change notification) listener (As part of SAR file) to listen for the databases changes on port 56090.
A weekly machine security scan has been scheduled. As part of the system scan, it scans the port 56090. Following exception has been thrown by the JDBC driver at the time of scanning.
2009-05-24 16:29:37,874 INFO [STDOUT] java.nio.BufferUnderflowException
2009-05-24 16:29:37,874 INFO [STDOUT] at java.nio.Buffer.nextGetIndex(Buffer.java:398)
2009-05-24 16:29:37,874 INFO [STDOUT] at java.nio.HeapByteBuffer.get(HeapByteBuffer.java:117)
2009-05-24 16:29:37,875 INFO [STDOUT] at oracle.jdbc.driver.NTFConnection.readByte(NTFConnection.java:616)
2009-05-24 16:29:37,875 INFO [STDOUT] at oracle.jdbc.driver.NTFConnection.readInt(NTFConnection.java:650)
2009-05-24 16:29:37,875 INFO [STDOUT] at oracle.jdbc.driver.NTFConnection.unmarshalNSDataPacket(NTFConnection.java:498)
2009-05-24 16:29:37,875 INFO [STDOUT] at oracle.jdbc.driver.NTFConnection.unmarshalOneNSPacket(NTFConnection.java:404)
2009-05-24 16:29:37,875 INFO [STDOUT] at oracle.jdbc.driver.NTFConnection.run(NTFConnection.java:181)
And during scanning time very high CPU usage (90%) is encountered. Normal CPU usage of the application server system will be around 20%.
Can you please give any information on the event/data which is causing the above exception. and how it can be handled, so as to minimize CPU usage back to 20%?
Please give more information about oracle.jdbc.driver.NTFConnection class.
Thanks in Advance
Abdur Rahman
Edited by: user11269361 on Jun 15, 2009 8:57 AM

Hi,
The DCN protocol is synchronous so if the security system scan sends a byte to the port 56090 the protocol will break. I recommend you configure the system scan to not interfere with DCN.
Regards,
Jean

Java.nio and databases

Similar Messages

Maybe you are looking for