Full text index searching in large document sets

Question

I have been placed in charge of a digital PDF document library for a small biotech company. The library consists of about 1000 100-300 page .pdf documents which have been scanned and OCRed. In order to facilitate the full text searching of the documents a PDX catalog has been created. In theory, the PDX catalog would seem to be an excellent means of quickly accessing the data, but due the sheer volume of text that is contained in the documents this does not seem to be the case.
Any given search may take hours to complete and many computers in the department have been known to lock up due to the load of running a search. Obviously, this has made using the PDX search more of a hassle than it is worth.
I do not know exactly how the index searches work, but from what I gather they somehow search within each document in turn and return to you all the instances in all the documents that contain a certain term. If this is the case, than it would make sense that the searches would take a long time because the search would have to search each of the 1000 documents in sequence.
The thing is: we really do not need to know the context and placement of every instance that a word appears in a document. All we need to know is IF it appears, and perhaps how many times. Is there a way to make an index that will simply give us this information without having to search the actual document?
Heres an example of what I am trying to achieve:
Note: I know almost nothing about full text indexes so please forgive me if any of this sounds insane
Lets say we have a document called "word count.pdf" which contained the following text:
"blah blah yadda yadda text Recombinant human insulin more text still texting and so on"
And another called "word count 2.pdf" with the following text
"Recombinant human insulin and la la la dee do"
The indexes for these files could be condensed and stored like this:
"Word count.pdf"
Blah 2
yadda 2
recombinant 1
human 1
insulin 1
text 2
texting 1
and 1
so 1
on 1
"Word count 2.pdf"
recombinant 1
human 1
insulin 1
and 1
la 3
dee 1
do 1
In this example, if we were to run a search on "text" the index would return "word count.pdf, 3 instances (2 of text and 1 of texting" whereas if we were to search for "recombinant" it would return both "word count.pdf, 1 instance" and "word count 2.pdf, 1 instance".
This way, I could quickly weed out all documents that do not have the word that I am looking for and get an idea about which documents should be searched more in depth without scanning every single instance of the term in every document.
Is there any way to accomplish something similar to this using acrobat? (Or anything else, for that matter)
My specifications: (similar to specs of all computers searching the pdx):
Windows XP,
intel celeron CPU 2.6GHz, 1G of ram
Adobe Acrobat 8 Professional

Answer 1

Look at dTSearch. We used the publisher version for a CD with large files sets (with hundreds of pages per file/thousands of PDF pages of multicolumn index data - text heavy), and it does a great job. The desktop version would provide the type of searching you are looking for. Indexing is also very fast. Our customer complained, like yourself, about the speed of searches in Acrobat 6 and higher - most of the delay is due to the population of the results window.
http://www.dtsearch.com/