Limiting the number of threads that are created

I am trying to write a web crawler as a way of learning about multi-threading and I have hit a stumbling block (probably my design). The way the application I have started works is that I provide a single URL, which is passed to a thread and that thread then parses the web page building a list of links (If the links are not already held in a 'visited' list then they are saved in a list in the Runnable object that is parsing the html page).
When the page has been parsed the list of links are then passed to a method in a Utilities class that creates a new group of threads, 1 for each link and each of these threads then does the same thing. The problem I have is that as more and more links are captured more and more threads are created eventually I either get a out of memory exception or I get an operating system 'cannot create native thread' message. I am not sure if the threads are staying in memory after they have done their tasks, I am not sure why I am running out of memory so quickly.
What I would like to try and do is to set a limit for the maximum number of threads created and then only create new threads if the limit is not breached. If someone could point me in the right direction that would be good, I have googled around but cant find an example of what I want to do.
Thanks

Thanks for that, that has given me a lot to read up on and I can already see where I went wrong.... I think implementing a thread pool and also a work queue is probably the way I will go from now on.

Similar Messages

Maximum number of aggregates that are created on infocube

Hi,
Can anyone tell how many Maximum number of aggregates that are created on infocube.
Regards@thanks

HI,
I don't think there is any limit to the no. of aggregates. But it basically depends upon the volume of data and the reporting needs.
Regards
Rajesh

After upgrading to Firefox 8.0.1, now on at least one website I use, it limits the number of characters that I can type in a text box to 275, is there a way to fix this?

After upgrading to Firefox 8.0.1, now on at least one website I use, it limits the number of characters that I can type in a text box to 275, is there a way to fix this?

That may be part of the coding that the web page is using.
*https://developer.mozilla.org/en/HTML/element/input

How do you regulate the number of emails that are kept on your phone before they are deleted?

HOW DO YOU CHANGE THE NUMBER OF EMAILS THAT ARE KEPT ON YOUR PHONE BEFORE YOU DELETE THEM?

Your carrier sets this by time, not the number of "rings" since different ring tones take different times/ring. The typical setting is 30 seconds, sometimes up to 40 seconds. Contact your carrier to shorten it.

What are the .fm.sp files that are created when using SharePoint?

What are the .fm.sp files that are created when using SharePoint? When I'm working with SharePoint, I've noticed that duplicate files ending in .fm.sp are created. I've been unable to find any reference or documentation about them so far.

Whe you use sharepoint as a CMS connections in Framemaker it creates the folder where Sp is installed and also a file ending .sp is created that let SP know that its the file associated with it.
.fm.sp indicates that its the framemaker type SP file.
Dont worry about it as its not creating any mess in the system.
Harpreet

How to limit the number of emails that are shown for each account

Hi, I'm going from iOS 6.1 to my new 5s iOS 7 and I wanted to know if there was a way to limit the number of emails shown for each of my accounts? In iOS 6 there is a setting "show" where you can choose how many you would like to be loaded at a time from 50-1,000 , is there a setting similar to this and I am just not seeing it? Please help me asap. I have only logged in 3 of my 12 email accounts and already have over 13k emails loaded...

You can't limit it to the last 50 messages, but you can limit it to the last 1000 messages in each "folder" (label).
Go to the Gmail website and click on the gear icon to the top-right side.
Click on Settings.
Click on the "Forwarding and IMAP/POP" tab across the top of the Settings area.
Towards the bottom of the IMAP section, change the "Folder Size Limit" setting to "Limit IMAP folders to contain no more than this many messages". By default 1000 is selected, though you can also choose 2000, 5000, or 10000.
Click Save Changes at the bottom.

Is there a function to send window id's to TS to use for placement of the standard panels/windows that are created during normal TS operation?

I have asked this question previously in a different manner and have not received any useful hints. I want to manage the placement of all of the standard TS windows/panels during normal operation of TS.
Thanks in advance,
Paul

Paul,
I am assuming that you are talking about the File Dialog when you select "Open Sequence File..." from the menu. The code for the dialog is encapsulated as a command in the Operator Interface. If you look in the TestStand help, under Command Kinds, you can see all the coded commands that will cause the OI to respond. The CommandKind_OpenSequenceFile command is what causes the File Dialog to popup. When the user selects the proper file and chooses to Open the dialog, the OpenSequenceFile command triggers a DisplaySequenceFile event. This event is handled by the Application Manager with a callback function (or VI if you are using LabVIEW). The SequenceFile gets updated in the SequenceFileView manager. Unfortunately, much of this code is handled internally by the Operator Interface manager controls. The File Dialog is simply a function that gets called, and the Window handle is never really accessed by the OI developer.
You may have a few options, however. There may be a Windows kernel function, that accesses child Window handles. I would search MSDN for any functions that will help access Window handles.
The other option is to implement the File Open dialog yourself. I believe, the File Chooser is a standard dialog popup, that you can call from most programming languages that are accessing Windows functionality. However, you should trigger the DisplaySequenceFile event after the user selects a file. This will cause the SequenceFileView manager to be udpated with the proper sequence file.
I have not tried either of these implementations.
If you have any questions, please let me know.
Thanks,
Tyler T.
NI

Is there any way to limit the number of Threads running in Application(JVM)

Hello all,
is there any way to limit the number of Threads running in Application(JVM)?
how to ensure that only 100 Threads are running in an Application?
Thanks
Mohamed Javeed

You should definitely use a thread pool for this. You can specify maximum number of threads that can be run. Be note that the thread pool will only limit the number of threads that are submitted to it. So donot call "Thread"s start() method to start thread on your own. Submit it to the pool. To know more, study about executor service and thread pool creation. Actually it will not be more than 20 line code for a class and you might need maximum of 2 such classes, one for threadPool and other one for rejection handler (if you want).
Just choose the queue used carefully, you just have to pass it as a parameter to the pool.
You should use "Bounded" queue for limiting threads, but also be careful in using queues like SynchronizedQueue as the queue will execute immediately the threads submitted to it if maximum number of threads have not been running. Otherwise it will reject it and you should use rejection handler. So if your pool has a synchronized queue of size 100, if you submit 101th thread, it will be rejected and is not executed.
If you want some kind of waiting mechanism, use something like LinkedBlockingQueue. What this will do is even if you want 100 threads, you can specify the queue's size to be 1000, so that you can submit 1000 threads, only 100 will run at a time and the remaining will wait in the queue. They will be executed when each thread already executing will complete. Rejection occurs only when the queue oveflows.

How to find out the number of threads created under java vm at runtime

our application seems to have hit the max number of threads that can be created under vm and the vm will just hang after that. that behavior seems to be consistent from an article that I read earlier.
I wonder any way that java app can find out the number of threads currently created under vm?
thanks in advance!

If you are not starting an extra thread group you can use the Thread.activeCount method. But this willnot return all the threads, as there are some jvm threads, i.e. garbage collector. Also if you are running on a Unix type operating system then you can send a signal to the jvm to give you a thread dump, i think the signal is sighup.
If the application is hanging i would of thought that you have a deadlock situation rather then the JVM not being able to create new threads, roughly how many threads should be running ??

Is there a limit to the number of groups that can be created in BusinessObjects?

Hi all,
We are planning to implement a group structure where there a quite a lot of groups that has to be created. that's why i'm wondering if there is a limit to the number of groups we can create in CMC.
Thanks in advance for your answers..

Damien,
I have NOT come across any limits but you need to be very careful on how many nested layers you create and if you are also going to use 3rd party security like AD &/or LDAP. Then you will see some performance issues.
How many groups are looking to create 2,000 / 4,000 / 6,000 ?
Regards,
Ajay

When I receive and answer e-mail, the correspondence is usually "nested" with the original message, and the number of replies, etc. are indicated with a number next to the original message. How can I change the mail settings so that every reply is listed

When I receive and answer e-mail, the correspondence is usually "nested" "behind" the original message, and the number of replies, etc. are indicated with a number next to the original message in the inbox. How can I change the mail settings so that every reply is listed individually in the inbox instead of having them "stacked" behind the original message. I'm afraid I might accidentially overlook messages the way they are handled now.

Somebody here will correct me if I'm wrong, but what you have to do is go into Settings, Mail/Contacts/Calendars and under Mail, turn off "Organize by Thread". I think that will make each message show individually.

Is there a limitation to the number of calendars that iCal can manage?

My wife and I share calendars. Recently she found that one of my calendars was not showing up on her iMac and any dates or appointments were automatically sent to Notifications. However, all of my calendars and hers as well show up on her iPhone and iPad. With that we took her iMac to a Genius Bar. The tech after trying many things determined that because the age of her iMac (a 24", dual core) was limiting iCal to 10 calendars. I find this incredible. I can't image that this could be a hardware problem. She and both are running under Mountain Lion. Has anyone ever heard of this issue?

Kim331 wrote:
Is there a limit to the number of devices that I can run off my iTunes account?
According to this Support Article > http://support.apple.com/kb/HT4627
Your Apple ID can have up to 10 devices and computers (combined) associated with it.

HT201774 How do I delete excess email messages from my iPhone. There are 300 messages listed, and every time I delete some, more are added from old messages. How do I limit the number of messages that can appear on my phone? How can I delete messages i

How do I delete excess email messages from my iPhone. There are 300 messages listed, and every time I delete some, more are added from old messages. How do I limit the number of messages that can appear on my phone? How can I delete messages in bulk,

You can't.
All photos transferred from your computer are stored in the Photo Library. The photos in the album or albums below are not duplicates - they include a pointer to the original photos stored in the Photo Library. This way you can view the photos in a particular album only by selecting the album, or you can view all photos available in all albums by selecting Photo Library.
Just as with an iTunes playlist. A song cannot be in an iTunes playlist unless the song is in the main iTunes library. Placing a song in a playlist does not duplicate the song. Remove the main iTunes library or the songs from the main iTunes library that are in a playlist and the songs will no longer be in the playlist either. This way you can listen to the songs in the playlist only by selecting the playlist, or all songs in your entire iTunes library by selecting Music under Library in the iTunes source list.
The same with iPhoto on a Mac. A photo cannot be in an in iPhoto Event or Album unless the photo is in the main iPhoto library. Placing a photo in an Event or Album does not duplicate the photo. This way you can view the photos in an iPhoto Event or Album only by selecting the Event or Album, or all photos in all Events or Albums.

Is there a way to find the temporary table that are created or read during?

Hi All,
I'm working on performance optimization where I have to find the temporary tablespaces that are created or read during runtime? Is there any way to find it?
Can you please also tell me how the temporary tables are created? And after each run time whether the data is getting deleted or the whole table is getting deleted?
Whether these tables are created only during runtime? What are the naming conventions for all the temporary tables that are created?
Is the table creation has anything to do with Delta or Full load?
Regards,
Kartik

Stephen Tyler Bloom wrote:
When they come out with the next garageband, they should add that feature .
be sure to let Apple know:
http://www.bulletsandbones.com/GB/GBFAQ.html#sendfeedback
(Let the page FULLY load. The link to your answer is at the top of your screen)

Get total number of comps that are used in the opened project?

Hi everyone!
Does anyone know how i can get the total number of comps in the project-panel or the total number of comps that are used in the opened project?
This is what i have so far, however it's not exactly what I am looking for:
for (var i = 1; i <= app.project.numItems; i++) {
    if (app.project.item(i) instanceof CompItem) {
        alert(i);
JesB

You can get the total number of comps with pretty much what you have
var numComps = 0;
for (var i = 1; i <= app.project.numItems; i++) {
    if (app.project.item(i) instanceof CompItem) {
        numComps++;
You could look at the usedIn attribute of each comp to see how many comps appear in other comps, but that won't identify your top-level comp. I'm not sure how you would do that unless you just bump the count by 1:
var numUsedComps = 0;
var numComps = 0;
for (var i = 1; i <= app.project.numItems; i++) {
    if (app.project.item(i) instanceof CompItem){
        numComps++;
        if (app.project.item(i).usedIn.length > 0) {
            numUsedComps++;
if (numComps > 0) numUsedComps++;
alert("number of comps = " + numComps + "\rnumber of used comps = " + numUsedComps);
Dan

Limiting the number of threads that are created

Similar Messages

Maybe you are looking for