Reader/Writer.close() is slow
I've written a multi-threaded application that does a lot of disk I/O using Buffered and File Reader/Writers. Each thread is reading and writing it's OWN file. Everything works great except the performance of the Reader/Writer.close() method. When I'm running say 10 threads at a time and my code calls close() it can take anywhere from 1 - 3 seconds for the call to return. If I only run 5 threads then close() returns in a respectable timeframe.
Can someone please explain what could be causing close() to take so long to return?
Thanks!
Each thread has to vie for CPU time with other processes on your system. More threads means more scheduling of processes, the threads do not run at the same time--it's just an illusion created by scheduling. When you have 10 threads, you have more scheduling going on--hence more wait when you want to close out: you have wait for each to gain enough active CPU time to flush and close, but there can only be up to as many threads running at one time as your system has CPUs*Threads/CPU. Just because you have more Java threads, does not mean that you're processing faster--it just means there is seperate processing going on as the system can schedule resources for your threads to work in with the rest of the system processes all of them essentially competing for many of the same resources.
Similar Messages
-
Hello,
I notice that in labview, I can:
read/write to a SPREADSHEET
This requires FILE_PATH and FORMAT
read/write to a FILE
This requires a FILE_PATH and NO format
(because it is binary data)
I can read lines... BUT I CANNOT write lines
First question: why can I not write lines?
Continuing... there are other ways to read/write.
I can OPEN a file
And, then I can read/write
So: how is THIS read/write different from the ones above.
I can see that THIS read/write requires a refnum from the
open... OK... one structural difference...
This one does not allow for formatting... but yields a BROWN data.
Now, what is this DATA... is it binary?
What is the purpose of the OPEN/read/write/Close
on the second lines of the FIL
E/IO vi's
Can someone describe these and how they relate to each other?
Thanks,
Tom
Hello,Tom Impelluso wrote:
> Hello,
>
> I notice that in labview, I can:
>
> read/write to a SPREADSHEET
> This requires FILE_PATH and FORMAT
>
This is called a TAB delimited file that most spreadsheet applications
can read
there is a TAB between columns and and EOL between rows. Writes in ASCII
(I should say ASCII printable since all chars are ASCII)
>
> read/write to a FILE
> This requires a FILE_PATH and NO format
> (because it is binary data)
Yep
>
>
> I can read lines... BUT I CANNOT write lines
>
> First question: why can I not write lines?
>
I found this strange also but, you can use the Write Char or generate
your own VI from
the lower level VIS. Which would be using the Write Char and adding EOL
as the last character in the line
There is a VI called Write + (String) that does not show up in the
palette but is used in some of the other VIs.
>
> Continuing... there are other ways to read/write.
>
> I can OPEN a file
>
> And, then I can read/write
>
> So: how is THIS read/write different from the ones above.
It is a byte stream which could be almost anything from ASCII to G
datatypes
>
>
> I can see that THIS read/write requires a refnum from the
> open... OK... one structural difference...
> This one does not allow for formatting... but yields a BROWN data.
>
> Now, what is this DATA... is it binary?
Not always but it can be. If you open an Excel file in a text editor you
will see
what a complex byte stream file looks like. The data is interspersed
with formatting bytes.
>
>
> What is the purpose of the OPEN/read/write/Close
> on the second lines of the FILE/IO vi's
these are the primitives that are used in all of the other file
functions.
This is the most basic level of what file I/O does.
>
>
> Can someone describe these and how they relate to each other?
The Open/Read/Write/Close are the primitves. The others are built up
from these
for special file types (like spreadsheet). They use the primitives and
add some formatting
and other options.
Dont reply to this e-mail address. My account is down and I am using
someone elses.
Kevin Kent -
Can TestStand open, read, write and close binary files?
From a TestStand sequence, I need to open, read, write and close binary files. Does TestStand support this capability?
Christine -
In the past I have used the C/C++ Adapter to call directly into the CVI RTE functions. The CVI functions I have used are OpenFile, WriteFile and CloseFile. The functions are exported from the DLL as CVI_OpenFile, CVI_WriteFile, and CVI_CloseFile. I just had to make sure that on termination of the sequence, that the integer handle was properly released by calling the close function.
If you are using LabVIEW, you could call directly into the low level VIs to do a similar set of operations.
Scott Richardson (NI)
Scott Richardson
National Instruments -
Hard drive read write speeds are too slow
I was having an issue with 17inch macbook pro. was getting 4.0mb/s read write speed tops. and smart status failure.
So i replaced the drive. The new drive is a 7200 RPM 500GB WD drive. I'm getting 12.0mb/s on top speed. Computer feels a little quicker but still slow overall.
I ran this test on a fresh install of snow leopard.
Is it just me or do those read/write speeds on the new drive still seem slow?
2007 17inch macbook pro with 2gigs of ram.That does seem slow - on the other hand, what are you using to measure it?
My 1Tb 2.5" SATA drive performance shows as follows: http://grab.by/7zau
Xbench Stats: http://grab.by/7zau
Anything in System Profiler? http://grab.by/7zaR
Link Speed at 3 Gb? -
Formatted on XP - EXTREMELY SLOW read/write on the Mac !!!!
Man this iPod is wack!! Im using the iPod mini in Disk Mode to transfer bits (updaters since my XP machine is NEVER connected to the net) back and forth between my PowerBook and my XP Intel based machine. Obviously i set up and formatted the iPod Mini on the XP machine.
Observation, file read/write times are unacceptably SLOW on Mac and the PC. Did Apple deliberately make this thing unusable in this situation? This thing has a transfer speed as if it were connected to a USB 1.1 port but it is NOT, on any of my machines. I format my USB Flash Drive (SanDisk Cruzer Titanium) on the same XP machine, use the same ports and it's read/write times are at least 5 times faster than the Mini !!!
When i format and setup the iPod Mini on the Mac then install MacDrive on the PC astonishingly the read/write times significantly increase on BOTH Mac and the PC. As i suggested i think there is something drastically wrong with Apple's PC iPod setup, there is NO way my other devices formatted on the PC have such abysmal performance . Man i click on a folder in the Finder of the PC formatted iPod and it take about 3 seconds until the files inside are displayed...this is also the case on the PC.
Anyone else experienced this?
I was going to buy another iPod specifically for transferring data between my Mac's and my colleagues PC's, but there is no way i will with such severely degraded read/write performance.
BestApparently this is not a single isolated case.
Scott Aronian Posts :
"On Mac FAT32 Format Slower Than HFS "
My 60GB iPod came in Windows FAT32 format which I liked since I could use it in disk mode on both Mac & Windows. However it seemed slow to start playing video and when switching between video files.
I did some timing tests and on the Mac FAT32 was much slower than HFS.
FAT32
Start playing movie = 12s
Stop movie and restart at same location = between 12-30s
HFS
Start playing movie = 5s
Stop movie and restart at same location = between 6s
If you have a Mac and your new 5G iPod is formatted for Windows, you may want to use the iPod Updated to restore and make the drive HFS.
There is NO such performance problems with USB Flash drives when formatted to Fat32 on PC's i have tested many and they perform similar on BOTH platforms.
Something fishy goin on here -
Socket read write on Solaris 8 too slow.
Hi
We have an application which consists of several server instances.
The front-end is web-based , using JSP/Servlets. We are using Tomcat 4.0.
The servlet makes several connections to the underlying servers. But the read/write operation is too slow. Same setup runs much quicker on Windows.
We are running jdk1.4.1_02 (stable ?). Would much appreciate any help.
cheers
Projyaltomcat version --3.2.3
j2sdk version----1.4
plarform---------solaris
solaris version--8
using the aboue configuration i need to deploy JSP codes on the solaris platfrom..can any technical expert guide me on how to do this..would be grateful if anyone could suggest a good site where i can download free tuorials of "JSP" for development in HTML files format or HELP format]
thank you
regards
brijesh -
Sir,
I have installed MSMQ in my Windows server 2008 R2 operation system but the queue reading and writing is very slow .It gives me 150 to 160 messages per second for write or read and the speed decreases if the read and write operation is performed at
a time .
Earlier I have MSMQ on Windows server 2003 .Read/write operations is fine and performance is good.It gives me 1000 to 2000 messages per second .
Regards,
SandeepHi,
Thanks for posting here.
Regarding the current issue, please try to refer to the following article to see if it could improve the performance.
How to improve MSMQ disk performance
http://blogs.msdn.com/b/johnbreakwell/archive/2008/02/13/msmq-disk-performance.aspx
Hope this helps.
Best Regards,
Andy Qi
Andy Qi
TechNet Community Support -
I have an old (~6 yrs) HP laptop running Win XP. I've scanned for viruses, deleted temp files, etc, and defraged the hard drive. After doing a fresh boot, all works OK until I start Firefox (3.6.6), and then every mouse click sends the hard drive into read/write mode for up to a minute. Just writing this memo has taken eight minutes; going to my bank site and setting up a transfer (about 12 mouse clicks) took 16 minutes (!).
Things other than Firefox continue to be plagued by the same problems, after starting and closing Firefox. Opening an Explorer window can take up to three minutes. Closing Firefox doesn't seem to solve the problem - I need to reboot to do that.
Any thoughts?Start Firefox in Safe Mode (above). Wait a few minutes, then close Firefox.
Is there still a problem?
If you are still having issues;
Type '''about:support''' in the address bar and press '''Enter.'''
Under the main banner, press the button; '''Copy Text To Clipboard.'''.
Then in the reply box at the bottom of this page,
do a right click in the box and select '''Paste.'''
This will show us your system details.
'''No Personal Information Is Collected.'''
Start your '''Computer''' in safe mode with networking. Then start Firefox.
'''[http://encyclopedia2.thefreedictionary.com/Linux+Safe+Mode Starting The Computer In Safe Mode;<br>Free Online Encyclopedia]''' -
Slow performanc​e to read/write shared variables programati​cally
We are using datasocket read and write functions to read and write shared variables programatically (in the same machine) but we only achieve a performance of aprox. 200 reads/writes per second. We are using Labview 8.6 with DSC.
Is possible to get better results? That performance is normal?
Any help would be appreciated. Thank you in advance.Hi MMCDAT,
I think this value can
be normal as you can see in this link:
http://zone.ni.com/devzone/cda/tut/p/id/5037
As you can see, the
limit for datasocket depends on your Ethernet limitations, even if you as using
it just in one PC:
http://digital.ni.com/public.nsf/websearch/6AC9E65734E53F9A8625672400637ECC?OpenDocument
You can improve the
performance changing the update mode or Vis configurations:
http://digital.ni.com/public.nsf/allkb/F8F7DE98856B50588625672400648045?OpenDocument
http://digital.ni.com/public.nsf/allkb/2D9C6D73A160537986256B290076456E?OpenDocument -
This is an Alert when I open the browser. It isn't functioning at all. I've cleared ample space on my C drive, but I cannot find the application's profile directory to check the read/write restrictions.
See https://support.mozilla.com/kb/Could+not+initialize+the+browser+security+component
-
Slow read/write over Thunderbolt and SSD
Hey all,
I recently picked up a Seagate GoFlex Thunderbolt sled and a 256GB Toshiba SSD for it.
I am wanting to use it as my Lightroom 5 catalog and Smart Preview drive, and leave the main HD clean.
When transfering all the info over from my main HDD, (158GB) I checked on Activity Monitor to see the real time read/write to the drive, and it only topped out at 150MB/sec. What gives?
I tested with an external FW800 and got 50-60MB/sec
Then tested with esata drive and got 80MB/sec
Both of these are platter drives.
I was expecting WAAAYYY more from the Seagate TB and SSD combo.
Did I set something up wrong? Have I been reading data wrong?
2011 i7 MBP
16GB ram
Toshiba SSD
Seagate GOflex Thunderbolt Adapter
Thank youHey,
I have the Seagate GoFlex Thunderbolt. And C300 SSD connected to it. The speed at which I am getting the SSD write at 196.5 MB/s and to read 323.4 MB/s.
You can use something call blackmagic disk speed to monitor how fast your drive is actually reading and writing.
The speed limitation you see may be from the actual SSD or copying from HDD to SSD. This will test direct reading and writing on the SSD, and will give you a better/truer speed.
Hope that helps
Ed -
could not initialize the application's security component .the most likely cause is problems with files in your application's probile directory.please check that this directory had no read/write restrictionsand ur hard disk is not full or close to full .It is recommended that you exit the application and fix the problem.if you continue to use this session ,you mightsee incorrect application behaviour when accessing security features....THIS IS THE ALERT I GET WHENEVER I TRY TO OPEN FIREFOX.
See:
*https://support.mozilla.com/kb/Could+not+initialize+the+browser+security+component -
Error 33172 occurred at Read & Write data transfer between two or more PF2010 controller
Hi,i need to do data transfer between two or more FP2010 controller.e.g. FP2010(A) & FP2010(B).
FP2010(A) need to transfer the measurement (from its I/O module) to FP2010(B) to do the data analysis.These data transfer should be synchronous btw two controller to prevent data lost.
From the vi used in the attachment,i encountered some problems at:
(1) Error 33172 occurred while publishing the data.Can i create and publish data under different item name?
(2) How to synchronies the read & write btw contorller?
All controller are communicating with each other directly without the need of a host computer to link them together
Is there any other method to do fast data transfer betwe
en controller?Hi YongNei,
You were succesful in omiting enough information to make it very difficult to answer!
Please post your example.
Please tell us what version of LV-RT you are using.
Please define what you concider "fast data transfer".
Have you concidered mapping the FP tags of FP2010(A) to FP2010(B) and vise versa?
WHat exactly has to be syncronized?
If you have something that is close to working, share that.
Well, that as far as I can go with the info you have provided. Depending on the details, what you are asking could be anything from trivial to impossible with the currently available technology. I just can't say.
It would probably be a good idea to start over with a fresh question (sorry) because not many people are going to know what a a "
PF2010" is and I can not guarentee that I will be able to get back to you personally until next week-end.
Trying to help you get an answer,
Ben
Ben Rayner
I am currently active on.. MainStream Preppers
Rayner's Ridge is under construction -
To export query from Access to Excel in Read/Write mode in VBA
Below is the code which exports the query named 'LatestSNR' from Access to Excel;
Public Sub Expdata()
Dim rst As DAO.Recordset
Dim Apxl As Object
Dim xlWBk, xlWSh As Object
Dim PathEx As String
Dim fld As DAO.Field
PathEx = Forms("Export").Text14 'path comes from the directory given in form
Set Apxl = CreateObject("Excel.Application")
Set rst = CurrentDb.OpenRecordset("LatestSNR")
Set xlWBk = Apxl.Workbooks.Open(PathEx)
'xlWBk.ChangeFileAccess xlReadWrite
Set xlWBk = Workbook("PathEx")
Apxl.Visible = True
Set xlWSh = xlWBk.Worksheets("Metadatasheet")
xlWSh.Activate
xlWSh.Range("A2").Select
For Each fld In rst.Fields
Apxl.ActiveCell = fld.Name
Apxl.ActiveCell.Offset(0, 1).Select
Next
rst.MoveFirst
xlWSh.Range("A2").CopyFromRecordset rst
xlWSh.Range("1:1").Select
' selects all of the cells
Apxl.ActiveSheet.Cells.Select
' selects the first cell to unselect all cells
xlWSh.Range("A2").Select
rst.Close
Set rst = Nothing
' Quit excel
Apxl.Quit
End Sub
After the execution of code, the query is transferred to excel sheet and is viewed in 'Read only' mode. If I try to save it, a copy of the excel file is produced. Can the Excel be opened in Read/Write mode ? so as to save the workbook and also to transfer
the query to same workbook repeatedly.
If in case the change of mode is not possible, then is there any alternative method?Try this version:
Public Sub Expdata()
Dim rst As DAO.Recordset
Dim Apxl As Object
Dim xlWBk As Object, xlWSh As Object
Dim PathEx As String
Dim i As Long
PathEx = Forms("Export").Text14 'path comes from the directory given in form
Set Apxl = CreateObject("Excel.Application")
Set xlWBk = Apxl.Workbooks.Open(PathEx)
Set xlWSh = xlWBk.Worksheets("Metadatasheet")
Set rst = CurrentDb.OpenRecordset("LatestSNR")
For i = 1 To rst.Fields.Count
xlWSh.Cells(1, i).Value = rst.Fields(i - 1).Name
Next i
rst.MoveFirst
xlWSh.Range("A2").CopyFromRecordset rst
xlWBk.Close SaveChanges:=True
Apxl.Quit
rst.Close
Set rst = Nothing
End Sub
or else
Public Sub Expdata()
Dim PathEx As String
PathEx = Forms("Export").Text14 'path comes from the directory given in form
DoCmd.TransferSpreadsheet TransferType:=acExport, _
SpreadsheetType:=acSpreadsheetTypeExcel12Xml, _
TableName:="LatestSNR", _
Filename:=PathEx, _
HasFieldNames:=True, _
Range:="Metadatasheet!"
End Sub
Regards, Hans Vogelaar (http://www.eileenslounge.com) -
File read/write in Oracle JVM
Are there any known issues around file read/write in Oracle JVM. I finally got around the issue of getting the error that says no permission is there by giving <<ALL FILES>> option, but still I don't see a file being created. Any help is appreciated.
Code:
try{
fwriter = new PrintWriter(new FileOutputStream("debugLog.txt",true));
} catch (IOException ioe) {
System.err.println("IO Exception for Output File" + ioe.getMessage());
Thanks.
-MujibA couple of suggestions:
1. (Please don't be offended...) I assume you have some stuff like:
fwriter.println("hello");
fwriter.flush();
fwriter.close();
in your try block?
2. Try catching FileNotFoundException and SecurityException also.
3. Try providing a full path on the file just to make sure it's not buried somewhere odd.
John H.
Maybe you are looking for
-
Writing a Standalone Client for EJB 3.0 Bean For Weblogic 10
Steps for Writing a Standalone Client Client Code package com.client; import java.util.Properties; import javax.naming.Context; import javax.naming.InitialContext; import javax.naming.NamingException; import javax.rmi.PortableRemoteObject; import com
-
Ipad restore functions are not working
I just tried to update my ipad2 to iOS 8, and it crashed while the update was being downloaded onto the ipad. Now when I plug it in, it says to go to restore settings, and then says "the iPad update server could not be contacted". My network setting
-
Cannot restore profile pictures on incoming messages in Facebook. At a recent point Firefox flashed a note saying they were "deleting or removing" something from my FB and I thought it was for one thing only. But now ALL profile pictures on incoming
-
Hi, OS = HP Unix Datbase = 10.2.0.1 My management want me to enable auditing at database level.....It means, once i enabled the auditing at db level...it captures all the uses login,logout, what they are doing at database..everything in sys.aud$ tabl
-
my airport extreme has connected to a Tplink router, i set my AE in Create network, so i get a problem of Double NAT. How can i repair this probleme because i have to use TPlink for my IP TV and i want to use AE to creat my network and the guest NW.