InputStream - OutputStream: Reader - Writer
What was not possible by InputStream & OutputStream that JAVA had to provide Reader and Writer. I have found out that if I use InputStream to read a Chinese HTML page, it reads it. I can create a copy of the same HTML page by using OutputStream, the data remaining un-affected. Then, why do I go for Reader and Writer?
What happens if InputStream encounters some unicode charchters to be read? How does InputStream and OutputStream function internally?
Well, you can go look at the source of InputStream/OutputStream.
The reason we have Reader/Writer as well is because the streams don't deal with character encoding (or at least, they tried to and did it very badly). So you would normally use streams for dealing with binary data and reader/writer for dealing with character data. If you're using streams to copy a file and this file happens be be HTML with Chinese characters, this will work fine because you're just duplicating bytes. But if you try and use a stream to read this file into memory, you will have problems because it won't understand the Unicode characters in the file. In this case, you need to use a Reader and tell it what the character encoding of the file is.
Similar Messages
-
When to use BufferedInputStream/OutputStream/Reader/Writer
I'd be very grateful if someone could explain when to use BufferedInputStream, etc., and what the dis/advantages are.
TIA
Shaundata channels such as files sockets, etc, tend to be
a) slow
b) block oriented
BufferedOutputStream attempts to mitigate this by presenting data in block format. Rather than a bunch of little segments of bytes. -
File transfer, read write through sockets in client server programming java
Hello All, need help again.
I am trying to create a Client server program, where, the Client first sends a file to Server, on accepting the file, the server generates another file(probably xml), send it to the client as a response, the client read the response xml, parse it and display some data. now I am successful sending the file to the server, but could not figure out how the server can create and send a xml file and send it to the client as response, please help. below are my codes for client and server
Client side
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.BufferedReader;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.net.InetAddress;
import java.net.Socket;
import java.net.UnknownHostException;
public class XMLSocketC
public static void main(String[] args) throws IOException
//Establish a connection to socket
Socket toServer = null;
String host = "127.0.0.1";
int port = 4444;
try
toServer = new Socket(host, port);
} catch (UnknownHostException e) {
System.err.println("Don't know about host: localhost.");
System.exit(1);
} catch (IOException e) {
System.err.println("Couldn't get I/O for the connection to host.");
System.exit(1);
//Send file over Socket
//===========================================================
BufferedInputStream fileIn = null;
BufferedOutputStream out = null;
// File to be send over the socket.
File file = new File("c:/xampp/htdocs/thesis/sensorList.php");
// Checking for the file to be sent.
if (!file.exists())
System.out.println("File doesn't exist");
System.exit(0);
try
// InputStream to read the file
fileIn = new BufferedInputStream(new FileInputStream(file));
}catch(IOException eee)
System.out.println("Problem, kunne ikke lage fil");
try
InetAddress adressen = InetAddress.getByName(host);
try
System.out.println("Establishing Socket Connection");
// Opening Socket
Socket s = new Socket(adressen, port);
System.out.println("Socket is clear and available.....");
// OutputStream to socket
out = new BufferedOutputStream(s.getOutputStream());
byte[] buffer = new byte[1024];
int numRead;
//Checking if bytes available to read to the buffer.
while( (numRead = fileIn.read(buffer)) >= 0)
// Writes bytes to Output Stream from 0 to total number of bytes
out.write(buffer, 0, numRead);
// Flush - send file
out.flush();
// close OutputStream
out.close();
// close InputStrean
fileIn.close();
}catch (IOException e)
}catch(UnknownHostException e)
System.err.println(e);
//===========================================================
//Retrieve data from Socket.
//BufferedReader in = new BufferedReader(new InputStreamReader(toServer.getInputStream()));
DataInputStream in = new DataInputStream(new BufferedInputStream(toServer.getInputStream()));
//String fromServer;
//Read from the server and prints.
//Receive text from server
FileWriter fr = null;
String frn = "xxx_response.xml";
try {
fr = new FileWriter(frn);
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
try{
String line = in.readUTF(); //.readLine();
System.out.println("Text received :" + line);
fr.write(line);
} catch (IOException e){
System.out.println("Read failed");
System.exit(1);
in.close();
toServer.close();
public class XMLSocketS
public static void main(String[] args) throws IOException
//Establish a connection to socket
ServerSocket serverSocket = null;
try {
serverSocket = new ServerSocket(4444);
} catch (IOException e) {
System.err.println("Could not listen on port: 4444.");
System.exit(1);
Socket clientLink = null;
while (true)
try
clientLink = serverSocket.accept();
System.out.println("Server accepts");
BufferedInputStream inn = new BufferedInputStream(clientLink.getInputStream());
BufferedOutputStream ut = new BufferedOutputStream(new FileOutputStream(new File("c:/xampp/htdocs/received_from_client.txt")));
byte[] buff = new byte[1024];
int readMe;
while( (readMe = inn.read(buff)) >= 0)
{ //reads from input stream, writes the file to disk
ut.write(buff, 0, readMe);
// close the link to client
clientLink.close();
// close InputStream
inn.close();
// flush
ut.flush();
// close OutputStream
ut.close();
//Sending response to client
//============================================================
//============================================================
System.out.println("File received");
}catch(IOException ex)
{System.out.println("Exception.");}
finally
try
if (clientLink != null) clientLink.close();
}catch(IOException e) {}
clientLink.close();
//serverSocket.close();
}SERVER
import java.net.*;
import java.io.*;
public class XMLSocketS
public static void main(String[] args) throws IOException
//Establish a connection to socket
ServerSocket serverSocket = null;
try {
serverSocket = new ServerSocket(4545);
} catch (IOException e) {
System.err.println("Could not listen on port: 4444.");
System.exit(1);
Socket clientLink = null;
try
clientLink = serverSocket.accept();
System.out.println("Server accepts the client request.....");
BufferedInputStream inn = new BufferedInputStream(clientLink.getInputStream());
BufferedOutputStream ut = new BufferedOutputStream(new FileOutputStream(new File("c:/xampp/htdocs/received_from_client.txt")));
byte[] buff = new byte[1024];
int readMe;
while( (readMe = inn.read(buff)) >= 0)
{ //reads from input stream, writes the file to disk
ut.write(buff, 0, readMe);
ut.flush();
//Sending response to client
//============================================================
BufferedInputStream ftoC = null;
BufferedOutputStream outtoC = null;
// File to be send over the socket.
File file = new File("c:/xampp/htdocs/thesis/user_registration_response.xml");
try
// InputStream to read the file
ftoC = new BufferedInputStream(new FileInputStream(file));
}catch(IOException eee)
{System.out.println("Problem reading file");}
// OutputStream to socket
outtoC = new BufferedOutputStream(clientLink.getOutputStream());
byte[] buffer = new byte[1024];
int noRead;
//Checking if bytes available to read to the buffer.
while( (noRead = ftoC.read(buffer)) >= 0)
// Writes bytes to Output Stream from 0 to total number of bytes
outtoC.write(buffer, 0, noRead);
outtoC.flush();
//============================================================
System.out.println("File received");
}catch(IOException ex)
{System.out.println("Exception.");}
finally
try
if (clientLink != null) clientLink.close();
}catch(IOException e) {}
clientLink.close();
//serverSocket.close();
}CLIENT SIDE
import java.io.*;
import java.net.*;
public class XMLSocketC
@SuppressWarnings("deprecation")
public static void main(String[] args)
// Server: "localhost" here. And port to connect is 4545.
String host = "127.0.0.1";
int port = 4545;
BufferedInputStream fileIn = null;
BufferedOutputStream out = null;
// File to be send over the socket.
File file = new File("c:/xampp/htdocs/thesis/sensorList.xml");
try
// InputStream to read the file
fileIn = new BufferedInputStream(new FileInputStream(file));
}catch(IOException eee)
{System.out.println("Problem");}
try
System.out.println("Establishing Socket Connection");
// Opening Socket
Socket clientSocket = new Socket(host, port);
System.out.println("Socket is clear and available.....");
// OutputStream to socket
out = new BufferedOutputStream(clientSocket.getOutputStream());
byte[] buffer = new byte[1024];
int numRead;
//Checking if bytes available to read to the buffer.
while( (numRead = fileIn.read(buffer)) >= 0)
// Writes bytes to Output Stream from 0 to total number of bytes
out.write(buffer, 0, numRead);
// Flush - send file
out.flush();
//=======================================
DataInputStream in = new DataInputStream(new BufferedInputStream(clientSocket.getInputStream()));
BufferedWriter outf = new BufferedWriter(new FileWriter("c:/xampp/htdocs/received_from_server.txt",true));
String str;
while(!(str = in.readLine()).equals("EOF")) {
System.out.println("client : Read line -> <" + str + ">");
outf.write(str);//Write out a string to the file
outf.newLine();//write a new line to the file (for better format)
outf.flush();
//=======================================
// close OutputStream
out.close();
// close InputStrean
fileIn.close();
// close Socket
clientSocket.close();
}catch (IOException e)
{System.out.println("Exception.");}
Could you please point where am I doing the stupid mistake, client to server is working properly, but the opposite direction is not.
Thanks -
Inputstream & outputstreams in network programming
Hi
I have been experimenting with inputstreams & outputstreams through sockets, and have been having some annoying problems.
First off, is it possible to have multiple outputstream types going through the same socket, eg a printwriter to send strings, and an OutputStream to send bytes (files), so that first i can send the file name, then file size, using the writers println methods, then send a byte array via the OutputStreams write method. Do i need to do some clean up etc after each transmission through the socket.
My client is set up to recieve two strings, then a byte array.
Sometimes it works, sometimes it doesnt.
I have even tried setting the buffer sizes of the sockets etc, but still to no success. I think the outputstreams are getting cluttered, but to be honest I dont know. Has anyone successfully and reliably done this type of thing before, if so can you help.
SteveAt least say flush() after having finihhed with one and before turning to the other.
-
Hello
For my socket communications, I am thinking of using InputStream/OutputStream to read/write data. Just realized that InputStream and OutputStream are abstract classes. In particular, the read() and write () are abstract methods
with no actual implementation.
=================================
public abstract int read() throws IOException
Reads the next byte of data from the input stream. The value byte is returned as an int in the range 0 to 255. If no byte is available because the end of the stream has been reached, the value -1 is returned. This method blocks until input data is available, the end of the stream is detected, or an exception is thrown.
A subclass must provide an implementation of this method.
Returns:
the next byte of data, or -1 if the end of the stream is reached.
Throws:
IOException - if an I/O error occurs.
============================================
However, when I call these methodes, they actually work!
What is going on here?
What is your recommendation for socket I/O? should i stick with my InputStream/OutputStream, or I should use DataInputStream/DataOutputStream?
Thanks for your help,
AlexHello
For my socket communications, I am thinking of using
InputStream/OutputStream to read/write data. Just
realized that InputStream and OutputStream are
abstract classes. In particular, the read() and write
() are abstract methods
with no actual implementation.
=================================
public abstract int read() throws IOException
Reads the next byte of data from the input stream. The
value byte is returned as an int in the range 0 to
255. If no byte is available because the end of the
stream has been reached, the value -1 is returned.
This method blocks until input data is available, the
end of the stream is detected, or an exception is
thrown.
A subclass must provide an implementation of this
method.
Returns:
the next byte of data, or -1 if the end of the stream
is reached.
Throws:
IOException - if an I/O error occurs.
============================================
However, when I call these methodes, they actually
work!
What is going on here?There is nothing strange about that. The type of the object that Socket.getInputStream() returns is a subclass of class InputStream. Read a book about Java interfaces and abstract classes (or look it up in the tutorial: http://java.sun.com/docs/books/tutorial/java/index.html).
What is your recommendation for socket I/O? should i
stick with my InputStream/OutputStream, or I should
use DataInputStream/DataOutputStream?That depends on what data you want to communicate over the socket. DataInputStream and DataOutputStream make it easy for you to send and receive Java primitive types over the socket. You can create a DataInputStream that wraps the input stream of your socket as follows:
Socket s = ...; // create the socket etc.
InputStream sis = s.getInputStream();
DataInputStream dis = new DataInputStream(sis);
// ... now use dis to read dataJesper -
Problems with Buffered Read() - write() cycle
URGENT!!!
I have two network hosts communicating through Socket.
i need a cycle of read/write until an escape string is written ("QUIT").
BUT after first readLine() it stucks on blocking method, doesn't read anything!
why?
so my code look like this (i cut out initialization and boring parts):
BufferedReader input;
BufferedWriter output;
String readString = "";
while(read!="QUIT") {
output.write("mySentence");
readString = input.readLine();
}THANKSyou are all right, but I do use newLine() just to have the line separator.
the exception is always the same (this afternoon it didn't result in an exception... ):
java.net.SocketException: Software caused connection abort: recv failedI really can't figure it out...
moreover, I don't understand which side generates the problem, since one side is an Applet.
Again, the curious thing is that before the cycle they read and write (just once) normally!
I post the actual part of the code, in order not to create misunderstandings...
of course I used EITHER the "\n" OR the newLine() but not togheter...
server side the method is called within a thread class...
APPLET:
do {
String letto = listaParametri.poll();
os.write(letto);
os.newLine();
while( !letto.equals("LAST") );
os.flush();
while( this.isActive() ) {
received = is.readLine();
if ( received.equals("CHANGE") )
getAppletContext().showDocument(reloadURL, "_main");
break;
else if( received.equals("ALIVE") )
{ os.write("YES\n");
os.newLine();
os.flush();
}SERVER:
while(!CLOSED)
{if(CHANGE) {
outputStream.write("CHANGE");
outputStream.close();
System.out.println("sent RELOAD comand - now Closed - );
break;
else {
outputStream.write("ALIVE\n");
//outputStream.newLine();
outputStream.flush();
System.out.println("inputStream");
appletStatus = inputStream.readLine();
System.out.println(" - appletStatus=" + appletStatus);
if( appletStatus == null ) CLOSED = true;
else if( appletStatus.equals("YES") ) CLOSED = false; -
Pls How do I read/write a xml file using java
I want to read/write trough a server/client a file.xml from evt. the same computer, this is the code for the client
import java.io.*;
import java.net.*;
class client
public static void main (String args[]) throws Exception
final String msg = "<beg>note</beg>";
// BufferedReader in = new BufferedReader(new FileReader("note.xml"));
final Socket clientSocket = new Socket ("localhost", 8000);
final OutputStream clientStream = clientSocket.getOutputStream ();
clientStream.write (msg.getBytes ());
// clientStream.write (in.getBytes ());
clientSocket.close ();
// BufferedReader in = new BufferedReader(new FileReader("note.xml"));
// System.out.println(in.readLine());
I want to read it and put it in a buffer...
please somebody help me!!!
regards
HarryPls read this:
http://forum.java.sun.com/thread.jsp?forum=31&thread=271751
And don't post the same message twice. It's annoying. -
How do you create default Read/Write Permissions for more than 1 user?
My wife and I share an iMac, but use separate User accounts for separate mail accounts, etc.
However, we have a business where we both need to have access to the same files and both have Read/Write permissions on when one of us creates a new file/folder.
By default new files and folders grant Read/Write to the creator of the new file/folder, and read-only to the Group "Staff" in our own accounts or "Wheel" in the /Users/Public/ folder, and read-only to Everyone.
We are both administrators on the machine, and I know we can manually override the settings for a particular file/folder by changing the permissions, but I would like to set things up so that the Read/Write persmissions are assigned for both of us in the folder for that holds our business files.
It is only the 2 of us on the machine, we trust each other and need to have complete access to these many files that we share. I have archiveing programs running so I can get back old versions if we need that, so I'm not worried about us overwriting the file with bad info. I'm more concerned with us having duplicates that are not up to date in our respective user accounts.
Here is what I have tried so far:
1. I tried to just set the persmissions of the containing folder with us both having read/write persmissions, and applied that to all containing elements.
RESULT -> This did nothing for newly created files or folders, they still had the default permissions of Read/Write for the creating User, Read for the default Group, Read for Everyone
2. I tried using Sandbox ( http://www.mikey-san.net/sandbox/ ) to set the inheritance of the folder using the methods laid out at http://forums.macosxhints.com/showthread.php?t=93742
RESULT -> Still this did nothing for newly created files or folders, they still had the default permissions of Read/Write for the creating User, Read for the default Group, Read for Everyone
3. I have set the umask to 002 ( http://support.apple.com/kb/HT2202 ) so that new files and folders have a default permission that gives the default group Read/Write permissions. This unfortunately changes the default for the entire computer, not just a give folder.
I then had to add wife's user account to the "Staff" group because for some reason her account was not included in that. I think this is due to the fact that her account was ported into the computer when we upgraded, where as mine was created new. I read something about that somewhere, but don't recall where now. I discovered what groups we were each in by using the Terminal and typing in "groups username" where username was the user I was checking on.
I added my wife to the "Staff" group, and both of us to the "Wheel" group using the procedures I found at
http://discussions.apple.com/thread.jspa?messageID=8765421�
RESULT -> I could create a new file using TextEdit and save it anywhere in my account and it would have the permissions: My Username - Read/Write, "Staff" or "Wheel" (depending on where I saved it) - Read/Write, Everyone - Read Only, as expected from the default umask.
I could then switch over to my wife's account, open the file, edited it, and save it, but then the permissions changed to: Her Username - Read/Write, (unknown) - Read/Write, Everyone - Read Only.
And when I switch back to my account, now I can open the file, but I can't save it with my edits.
I'm at my wits end with this, and I can believe it is impossible to create a common folder that we can both put files in to have Read/Write permissions on like a True Shared Folder. Anyone who has used windows knows what you can do with the Shared folder in that operating system, ie. Anyone with access can do anything with those files.
So if anyone can provide me some insight on how to accomplish what I really want to do here and help me get my system back to remove the things it seems like I have screwed up, I greatly appreciate it.
I tried to give as detailed a description of the problem and what I have done as possible, without being to long winded, but if you need to know anything else to help me, please ask, I certainly won't be offended!
Thanks In Advance!
SteveThanks again, V.K., for your assistance and especially for the very prompt responses.
I was unaware that I could create a volume on the HD non-destructively using disk utility. This may then turn out to be the better solution after all, but I will have to free up space on this HD and try that.
Also, I was obviously unaware of the special treatment of file creation by TextEdit. I have been using this to test my various settings, and so the inheritance of ACLs has probably been working properly, I just have been testing it incorrectly. URGH!
I created a file from Word in my wife's account, and it properly inherited the permissions of the company folder: barara - Custom, steve - Custom, barara - Read/Write, admin - Read Only, Everyone - Read Only
I tried doing the chmod commands on $TMPDIR for both of us from each of our accounts, but I still have the same behavior for TextEdit files though.
I changed the group on your shared folder to admin from wheel as you instructed with chgrp. I had already changed the umask to 002, and I just changed it back to 022 because it didn't seem to help. But now I know my testing was faulty. I will leave it this way though because I don't think it will be necessary to have it set to 002.
I do apparently still have a problem though, probably as a result of all the things I have tried to get this work while I was testing incorrectly with TextEdit.
I have just discovered that the "unknown user" only appears when I create the a file from my wife's account. It happens with any file or folder I create in her account, and it exists for very old files and folders that were migrated from the old computer. i.e. new and old files and foders have permissions: barara - Read/Write, unknown user - Read Only, Everyone - Read Only
Apparently the unknown user gets the default permissions of a group, as the umask is currently set to 022 and unknown user now gets Read Only permissions on new items, but when I had umask set to 002, the unknown user got Read/Write permissions on new items.
I realize this is now taking this thread in a different direction, but perhaps you know what might be the cause of this and how to correct or at least know where to point me to get the answer.
Also, do you happen to know how to remove users from groups? I added myself and my wife to the Wheel group because that kept showing up as the default group for folders in /Users/Shared
Thanks for your help on this, I just don't know how else one can learn these little "gotchas" without assistance from people like you!
Steve -
An alert message pops up upon opening saying could not initiate application security component, and it says to check to see if profile has no read/write restrictions. Than when it opens all of my saved passwords are gone, I use a master password and its disabled. When I try to enter in a new on e it says can't change password. I can't even open yahoo e-mail says that my ssl security is down but when I check it its clicked. I'm just very confused as to whats going on.
== This happened ==
Every time Firefox opened
== 5/14/2010 ==
== User Agent ==
Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.1.249.1064 Safari/532.5See [[Could not initialize the browser security component]]
Rename (or delete) secmod.db (secmod.db.old) in the [http://kb.mozillazine.org/Profile_folder_-_Firefox Profile Folder] in case there is a problem with the file. -
System Folder errors after I changed all permissions on HD to read & write
Hi,
Two things may have caused probs on my new 2010 iMac (Snow Leopard), and Applecare is shut so I would really appreciate some help as I have urgent work.
1) INCORRECT PERMISSIONS
I have been stupid. I clicked on Macintosh HD and changed all permissions to read & write because I wanted to be sure I could open and edit all documents on other computers.
I ran Disk Utility Repair Permissions from the install disc, but I am still getting system error messages, and my HP printer won't work.
The first message, in Repair Permissions, said: Warning: SUID file System/Library/Cores has been modified and will not be repaired. I have read a support doc on this which says no need to worry but I don't like it and would like to fix this.
More importantly, my HP printer won't work, displays error beside the document in print dialogue box.
Deleting the printer and readding it didn't work, so I downloaded new drivers and tried to install them, which is when I got the second system error message: System extension System/Library/Extensions/BJUSBLoad.kext was installed improperly and cannot be used. Please try reinstalling it or contact product's vendor for an update.
I checked the permissions on the file and they were still wrong despite Repair Permissions, allowing everyone to read & write. So I have now clicked on the entire System folder and changed the permissions to: System read & write, admin read only, everyone read only.
Will this fix it or do I need to do something else, such as check ownership, to make sure all permissions on the computer are now correct?
2) MEMORY STICK SHUT DOWN MY IMAC
Additionally (though I don't think this had anything to do with my problems), I inserted a Sandisk USB memory stick the other day and it immediately shut down the computer. When I inserted it into my Macbook it initially rejected it and gave me a message saying the device wanted too much power so it had ejected it to prevent damage to my computer. When I tried again it was OK. I totally reformatted the stick in case there was something harmful on it, but should I now bin the stick as faulty? Scared to use it again.
3) IS IT BEST TO REINSTALL ENTIRE SOFTWARE?
If I do a reinstall of all the software from the install disc, will it wipe out all my data, such as Mail, documents, bookmarks and other apps?
I would back-up, but if I try and back up files on my external drive it will automatically do a Time Machine back-up and I don't want to do that in case it backs-up all the corrupted files. Otherwise, I wouldn't mind starting again just to be sure all is well.
Expert advice would be very much appreciated.
Thank you
SarahOh, silly really. I was in a hurry and working on docs that I needed to take to the office and open on another computer there.
But when I checked the permissions on the doc it said I could read & write but everyone else was read only.
I thought if I opened it on another machine I'd be stuck with read only access and not be able to work on there. I think I couldn't change it, so I thought to avoid any future problems like that I would change everything on machine!
Yikes. Won't do that again
Sarah -
We have a problem with one of our deployments of Windows Server 2012 Hyper-V with a 2 node cluster connected to a iSCSI SAN.
Our setup:
Hosts - Both run Windows Server 2012 Standard and are clustered.
HP ProLiant G7, 24 GB RAM, 2 teamed NIC dedicated to Virtual Machines and Management, 2 teamed NIC dedicated to iSCSI storage. - This is the primary host and normaly all VMs run on this host.
HP ProLiant G5, 20 GB RAM, 1 NIC dedicated to Virtual Machines and Management, 2 teamed NIC dedicated to iSCSI storage. - This is the secondary host that and is intended to be used in case of failure of the primary host.
We have no antivirus on the hosts and the scheduled ShadowCopy (previous version of files) is switched of.
iSCSI SAN:
QNAP NAS TS-869 Pro, 8 INTEL SSDSA2CW160G3 160 GB i a RAID 5 with a Host Spare. 2 Teamed NIC.
Switch:
DLINK DGS-1210-16 - Both the network cards of the Hosts that are dedicated to the Storage and the Storage itself are connected to the same switch and nothing else is connected to this switch.
Virtual Machines:
3 Windows Server 2012 Standard - 1 DC, 1 FileServer, 1 Application Server.
1 Windows Server 2008 Standard Exchange Server.
All VMs are using dynamic disks (as recommended by Microsoft).
Updates
We have applied the most resent updates to the Hosts, WMs and iSCSI SAN about 3 weeks ago with no change in our problem and we continually update the setup.
Normal operation
Normally this setup works just fine and we see no real difference in speed in startup, file copy and processing speed in LoB applications of this setup compared to a single host with 2 10000 RPM Disks. Normal network speed is 10-200 Mbit, but occasionally
we see speeds up to 400 Mbit/s of combined read/write for instance during file repair
Our Problem
Our problem is that for some reason all of the VMs stops responding or responds very slowly and you can for instance not send CTRL-ALT-DEL to a VM in the Hyper-V console, or for instance start task manager when already logged in.
Symptoms (i.e. this happens, or does not happen, at the same time)
I we look at resource monitor on the host then we see that there is often an extensive read from a VHDX of one of the VMs (40-60 Mbyte/s) and a combined write speed to many files in \HarddiskVolume5\System Volume Information\{<someguid and no file extension>}.
See iamge below.
The combined network speed to the iSCSI SAN is about 500-600 Mbit/s.
When this happens it is usually during and after a VSS ShadowCopy backup, but has also happens during hours where no backup should be running (i.e. during daytime when the backup has finished hours ago according to the log files). There is however
not that extensive writes to the backup file that is created on an external hard drive and this does not seem to happen during all backups (we have manually checked a few times, but it is hard to say since this error does not seem leave any traces in event
viewer).
We cannot find any indication that the VMs themself detect any problem and we see no increase of errors (for example storage related errors) in the eventlog inside the VMs.
The QNAP uses about 50% processing Power on all cores.
We see no dropped packets on the switch.
(I have split the image to save horizontal space).
Unable to recreate the problem / find definitive trigger
We have not succeeded in recreating the problem manually by, for instance, running chkdsk or defrag in VM and Hosts, copy and remove large files to VMs, running CPU and Disk intensive operations inside a VM (for instance scan and repair a database file).
Questions
Why does all VMs stop responding and why is there such intensive Read/Writes to the iSCSI SAN?
Could it be anything in our setup that cannot handle all the read/write requests? For instance the iSCSI SAN, the hosts, etc?
What can we do about this? Should we use MultiPath IO instead of NIC teaming to the SAN, limit bandwith to the SAN, etc?Hi,
> All VMs are using dynamic disks (as recommended by Microsoft).
If this is a testing environment, it’s okay, but if this a production environment, it’s not recommended. Fixed VHDs are recommended for production instead of dynamically expanding or differencing VHDs.
Hyper-V: Dynamic virtual hard disks are not recommended for virtual machines that run server workloads in a production environment
http://technet.microsoft.com/en-us/library/ee941151(v=WS.10).aspx
> This is the primary host and normaly all VMs run on this host.
According to your posting, we know that you have Cluster Shared Volumes in the Hyper-V cluster, but why not distribute your VMs into two Hyper-V hosts.
Use Cluster Shared Volumes in a Windows Server 2012 Failover Cluster
http://technet.microsoft.com/en-us/library/jj612868.aspx
> 2 teamed NIC dedicated to iSCSI storage.
Use Microsoft MultiPath IO (MPIO) to manage multiple paths to iSCSI storage. Microsoft does not support teaming on network adapters that are used to connect to iSCSI-based storage devices. (At least it’s not supported until Windows Server 2008 R2. Although
Windows Server 2012 has built-in network teaming feature, I don’t article which declare that Windows Server 2012 network teaming support iSCSI connection)
Understanding Requirements for Failover Clusters
http://technet.microsoft.com/en-us/library/cc771404.aspx
> I have seen using MPIO suggests using different subnets, is this a requirement for using MPIO
> or is this just a way to make sure that you do not run out of IP adressess?
What I found is: if it is possible, isolate the iSCSI and data networks that reside on the same switch infrastructure through the use of VLANs and separate subnets. Redundant network paths from the server to the storage system via MPIO will maximize availability
and performance. Of course you can set these two NICs in separate subnets, but I don’t think it is necessary.
> Why should it be better to not have dedicated wireing for iSCSI and Management?
It is recommended that the iSCSI SAN network be separated (logically or physically) from the data network workloads. This ‘best practice’ network configuration optimizes performance and reliability.
Check that and modify cluster configuration, monitor it and give us feedback for further troubleshooting.
For more information please refer to following MS articles:
Volume Shadow Copy Service
http://technet.microsoft.com/en-us/library/ee923636(WS.10).aspx
Support for Multipath I/O (MPIO)
http://technet.microsoft.com/en-us/library/cc770294.aspx
Deployments and Tests in an iSCSI SAN
http://technet.microsoft.com/en-US/library/bb649502(v=SQL.90).aspx
Hope this helps!
TechNet Subscriber Support
If you are
TechNet Subscription user and have any feedback on our support quality, please send your feedback
here.
Lawrence
TechNet Community Support -
could not initialize the application's security component .the most likely cause is problems with files in your application's probile directory.please check that this directory had no read/write restrictionsand ur hard disk is not full or close to full .It is recommended that you exit the application and fix the problem.if you continue to use this session ,you mightsee incorrect application behaviour when accessing security features....THIS IS THE ALERT I GET WHENEVER I TRY TO OPEN FIREFOX.
See:
*https://support.mozilla.com/kb/Could+not+initialize+the+browser+security+component -
How can I make sure files I transfer stay at read/write instead of read?
My employer recently got six brand new Mac Pros for our a/v guys. Well I wasn't able to get migrate to work (it wouldn't detect the slave drives, only the master) so I had to put files onto an external drive (a LACIE 1TB) and put them on the new Mac Pro OS 10.5.
We're finding out that files that we're moving, their permissions are being changed to read only instead of remaining at read/write or so it seems.
We've also got these machines on a closed network so we can share files between different areas of the office. Each computer has an administrator account and allows sharing and gives those it is sharing with, read/write permissions. But if I transfer a file from machine a to machine b, it comes out as read only one the file.
I'm thinking we jacked up the file sharing somewhere somehow. Curious if anyone might be able to take a crack at this. We're all new to leopard.
Screenshots:
Thanksusually though; they are techs; and so geeky; that in windows they set themselves as admins with unlimited privaleges. So what they do is go to account setting and just delete your account; and when they delete; they will delete everything; and all your person files.
Most though; find that too troublesome; so they just reformat the drive with a backup they built; for that computer.
Most of the things you can do is deleting temporary internet files; and cookies; and any saved passwords on your account. Usually all found in options/tools in most web browsers. For windows; most files are on desktop; my music; my downloads; my documents; and etc. -
How to read, write and save a file?
Hello everybody,
I have a question about reading , writing and save something into a file (etc text file)
For example: I want to create a file of football in which there are many key words as: "football", "touch down", "quarterback", "ball", "linebacker", "coach", "player"...
Now i want to open to read, write one more word "head" ( synonym with "coach") and store it for using next time. How could i do that?
Could anybody help me about that?
Thank you very much in advance.
still_learntake a look in the API about FileInputStream and FileOutputStream
-
SharePoint read/write list item to another DB/file
Hi All,
I am working on MOSS 2007. I have the following requirement.
There are around 50 document librararies on a site collection. I need to store all the items from these libraries in a locally (it could be DB/CSV or excel or access) and read/write them periodically for some calculations/comparisions.
Note: 1) the VM that I am working doesnt have MS Office installed to try for Access or Excel.
2) Is there a way to use access without needing to install MS Office.
Please let me know the best approach for the above task.
Any help is greatly appreciated.
Regards.Hi All,
Can any one help me with this..?
Thanks.
Maybe you are looking for
-
Copy of the filled in PDF Form
If a person fills out the PDF form and clicks submit, the responses will populate the table in FormsCentral, but is there a way that upon clicking submit a PDF version of the completed form can be emailed to a specific address?
-
Late 2013 Mac Pro wake from sleep crashes - which logs to look at?
I started encountering wake-from-sleep crashes with my late 2013 Mac Pro a couple of weeks ago. Prior to that the system had been pretty solid. Other than app and OS updates installed from the App Store, I can't think of any software changes that m
-
Separate only one screen on mission control?
Is there a way to seperate just 1 of my screens from the rest in Maverick? I have 4 screens. 3 monitors are essential meant to be one large work area. I will often use a program with a canvas spread across 2 screens and one for control panels. My 4th
-
Hi everybody, does anyone know how to make a long click valuable on a JButton because MousePressed doesn't seem to manage it . If you have an idea , please tell me.
-
Wanted to know how to install player on kindle hd 7" with linux, safari/silk when all your downloads support Firefox?