Verity Collection Bloat

I recently had a problem where the ‘parts’
directory in my search collection began to bloat with tons of files
resulting in a HUGE and unexpected bill from my hosting company (a
shared host).
I had the following set to run every hour
<cfindex
collection="my_collection"
action="update"
type="custom"
body="article_body,
title" key="id" title="title"
query="my_query"
custom1="path"
custom2="section"
>
This resulted in new files (.ddd and .did files) being added
to the search collection’s “parts” directory. I
saw problems in my search results too: Articles would show up two
and three times in search results when there should be only one
entry.
After I noticed the problem I changed the action to
“refresh”. Things now seem OK, but as I understand it,
using “action=update” shouldn’t cause the
behavior I’m seeing.
Can anyone help me with this?

I am having a similar problem but only with the UNI language
pack. It is indexing fine with the English or Englishx packs. It
also worked fine until we updated to the latest patches (7.0
worked, 7.02 does not)
I'd posted the following in the 'getting started' section but
maybe it should be here instead.
I have two verify collections. The English collection is
working ok. The French collection using the 'uni' multilanguage
does not index using the Cold Fusion code, and if I try to index it
on the server directly, I wait and wait and eventually it will give
me a page not found.
When I was at 7.0, this index worked fine. Now after
upgrading to 7.02, I cannot create the index.
I have tried deleting the collection and starting over but I
get the same problem.
This is the code that had been working:
<CFLOCK NAME="CFINDEXSEILOCK" TYPE="exclusive"
TIMEOUT="90">
<CFINDEX ACTION="refresh" COLLECTION="sei"
KEY="E:\UTEWebsite\Francais\" TYPE="path" URLPATH="
http://www.ute-sei.org/Francais/"
EXTENSIONS=".htm, .html, .cfm, .pdf, .doc" RECURSE="yes"
language="uni">
</CFLOCK>
Any thoughts?

Similar Messages

CF8 Verity Collection Problem

I am running the test code shown below.
The result is:
query
CATEGORIES CHARSET CREATED DOCCOUNT EXTERNAL LANGUAGE
LASTMODIFIED MAPPED NAME ONLINE PATH REGISTERED SIZE
1 No 1252 {ts '2008-12-11 15:51:16'} 81 NO english {ts
'2008-12-11 15:52:05'} NO testcontent YES
/opt/coldfusion8/verity/collections/testcontent CF 220
struct
Detail The collection you specified does not exists or is not
registered with the ColdFusion Search Service.
Message The collection testcontent does not exist.
[snip rest of dump]
When I use the CLI rcvdk program, the collection works fine
and I can search against the collection.

Dessimis wrote:
> I am running the test code shown below.
>
> The result is:
>
> query
> CATEGORIES CHARSET CREATED DOCCOUNT EXTERNAL LANGUAGE
LASTMODIFIED
> MAPPED NAME ONLINE PATH REGISTERED SIZE
> 1 No 1252 {ts '2008-12-11 15:51:16'} 81 NO english {ts
'2008-12-11
> 15:52:05'} NO testcontent YES
> /opt/coldfusion8/verity/collections/testcontent CF 220
>
> struct
> Detail The collection you specified does not exists or
is not registered with
> the ColdFusion Search Service.
> Message The collection testcontent does not exist.
> [snip rest of dump]
>
> When I use the CLI rcvdk program, the collection works
fine and I can search
> against the collection.
>
> <cfcollection action = "list" name = "verity" >
>
> <cfoutput>List of all known verity
collections.<br><br></cfoutput>
> <CFDUMP var = "#verity#">
> <br>
> <cftry>
> <cfsearch name = "quick"
> collection="southernrefecontent"
You're searching the "southernrefecontent" collection here,
not
testcontent (unless you anonymized the name before sending).
Mack

Verity Collection Errors in CF9

Hello,
We are trying to create verity collections in CF9 and have successfully done so in the past. For some reason we are now getting errors and have not been able to find a resolution. Here are the errors:
Error E0-0720 (I/O Filter): Could not load filter 'flt_kv -recognize -bifmime' which is named in your style.uni file.
Warn E2-0527 (Document Index): Document 1 (140128011): Stream error (-2) - SKIPPING
We can see in the debug info that verity is reading the database because the read record counts seem to be correct. We have played with file permissions, deleted the collections, uninstalled/re-installed Verity, still to no avail.
Coldfusion 9
CentOS 6
Thanks in advance for any help offered.
Best regards,
Shannon

Seems you installed apache server into the same $ORACLE_HOME as oracle DB home. It's pretty weird since there are many subdirectories (and files inside them) in both servers with same names that interfere (conflict) with each other, such as assistants, bin, diagnostics, install, inventory, jdbc, jdk, jlib, ldap, lib, network, nls, opmn, etc. Therefore, whichever server software was installed first, it was badly overwritten by the second. For instance:
"Cannot load /u01/app/oracle/11g/ohs/modules/mod_onsint.so into server: /u01/app/oracle/11g/lib/libons.so: undefined symbol: _intel_fast_memcpy"
Note that there are two /lib/libons.so shared files - one shipped with apache server and one shipped with oracle server, and they differ. You obviously have only one libons.so (which one?) in the /u01/app/oracle/11g/lib/ that is now shared by both servers.
Reinstall 11g and apache again. Apache server must go into its own home, different from oracle DB home. My installation is as follows:
ORACLE_HOME for database: /opt/oracle/product/11.1.0/db_1
ORACLE_HOME for apache: /opt/oracle/product/10.1.3.3/OracleAS_1
NJ

Verity Collection dropped during scheduled task.

I have a verity collection that I have scheduled to refresh every night at 2am. Over the past week, when it is scheduled to kick off, it is dropped from the "Verity Collections" window and in the error report, it states that the collection in question does not exist.
I've gone back to the directory on the server and the said collection folder is still there, but does not show in the "Verity Collections" in CF Admin.
I've created another "version" of the collection and manually kicked off the task and the collection does refresh. When the scheduled task kicks off again at the 2am mark, the same error happens again and the colection does not show in the Verity Collections window.
Any ideas?
Thanks
DarrellA

I'll restate this in simpler fashion.
Can you retrieve MX 7 verity search results on a MX 6.1
server?
Can this be done via CFHTTP or any other similar method?
Thanks

CF7 Verity collection not compatible on CF6.1?

Hi,
Does anyone know if CF7 Verity collections are supposed to be
back-compatible with CF6.1?
My Internet site is 7 but Intranet is 6.1. This upgrade has
just happened recently, and since Intranet searches over Internet
collection
crashes server (JRUN restarts). That's what I am pinning it
down to anyway, coz nothing else has changed that I know of.
We have a cron job that copies our Internet collection down
to the Intranet nightly. We copy the collect over so that staff can
search over bother Internet/Intranet in one search, rather than
separately (as they often don't know which site a particular
document is on).
Our Intranet search app uses the same code as Internet
(different servers, but same script) so the problem can't be
that...
Anyone else have this problem? Know any work arounds?
I am thinking of setting up some sort of web service on
Internet that hopefully my Intranet can tap into (my luck there
will be firewall issues).
I can create two separate searches - Intranet only and
Internet only (which can forward staff to Internet site) but as I
said, they would not to search over the two sites...
Thanks in advance for any help!

Here's a patch which fixes the problem:
http://www.adobe.com/cfusion/knowledgebase/index.cfm?id=tn_19339
This question was asked a dozen times before.
Everybody seeking support for this problem missed there's a
patch and no one bothered to tell there's a patch...

Copying Verity Collection

Since my shared hosting provider has the timeout set so low I
can't index my verity collection of pdf files before it times out.
So, I'm told I can index it locally, then FTP the
index/collection up to my site at the hosting provider.
But when I do this, the "url" variable in my search results
is messed up - actually reflects the absolute path on my local
machine!?
Is there a best practice way of copying a verity collection?
OR, an efficient way to index pdf files - I only have 118 of
them.
Thanks.

You can index locally on your machine and then upload the
collection and the files to your hosting, but the directory
structure (drive, folders, etc) must be the same in you local
machine and in your hosting provider for the files being indexed,
if not it won't work.
regards

Verity collection keep disappearing

i have CF MX7 and i do have a verity collection it keep
disappearing some time....i am wondering is this a bug or am i
missing somthing.....please help..thanks..

I am having a similar problem but only with the UNI language
pack. It is indexing fine with the English or Englishx packs. It
also worked fine until we updated to the latest patches (7.0
worked, 7.02 does not)
I'd posted the following in the 'getting started' section but
maybe it should be here instead.
I have two verify collections. The English collection is
working ok. The French collection using the 'uni' multilanguage
does not index using the Cold Fusion code, and if I try to index it
on the server directly, I wait and wait and eventually it will give
me a page not found.
When I was at 7.0, this index worked fine. Now after
upgrading to 7.02, I cannot create the index.
I have tried deleting the collection and starting over but I
get the same problem.
This is the code that had been working:
<CFLOCK NAME="CFINDEXSEILOCK" TYPE="exclusive"
TIMEOUT="90">
<CFINDEX ACTION="refresh" COLLECTION="sei"
KEY="E:\UTEWebsite\Francais\" TYPE="path" URLPATH="
http://www.ute-sei.org/Francais/"
EXTENSIONS=".htm, .html, .cfm, .pdf, .doc" RECURSE="yes"
language="uni">
</CFLOCK>
Any thoughts?

Error when updating a verity collection

Does anyone know what this error means? Happens when updating
a verity collection.....
Error Code: File locking/unlocking error
[VdkError_FileLocking]. (-146)
Thanks, Joe

I was able to reproduce this issue in the beta also. The issue has since been fixed. For the time being, it looks like you can just clear (delete) that particular value and re-add it w/ the updated binary.

Backup/archive/export verity collection

Does anyone know how to backup/archive/export a verity
collection?
The CF Help files say to go to admin/server settings/archives
but this is not an option in my version of CFMX.
Anyone?? thank you so much.
Nicko

Does anyone know how to backup/archive/export a verity
collection?
The CF Help files say to go to admin/server settings/archives
but this is not an option in my version of CFMX.
Anyone?? thank you so much.
Nicko

Possible Bug in Adminstrator - Verity Collections page

Am running 7.0.2 Enterprise on Win2k in J2EE mode (Jrun
updater6) and have been working on a cf5 to cfmx migration. HAd to
delete all the migrated collections, and recreate and reindex
everything, no big deal. Since this is done programatically, I
ocassionally took a peek at the Verity page in the CF adminstrator
to keep track of progress. We have over 200 collections indexing
~75,000 files on the server. Once all the indexing was completed,
started testing the search on the site, and all worked as expected.
BUT, when checking the verity Admin page, I cannot get a
listing of the collections. Last view had about 90 collections
showing (estimate). Am getting the "Unable to retrieve collections
from the Search Service.
Please verify that the ColdFusion MX Search Server is
installed and running."
I know it is running, because the search works, and I can add
a doc to a folder and re-index and it gets picked up. Is there a
limit to how many collections can be displayed in the Adminstrator.
And on the same note, this code <cfcollection
action="LIST" name="allcoll"> will throw a timeout error
"com.verity.api.administration.ConfigurationException:
java.io.IOException: Read timed out "
which it did not do with less than 50 collections.
Then, when deleting collections (using a cfdirectory to get
the names from the directory since I cannot list them) I get this
error
"The collection you specified does not exists or is not
registered with the ColdFusion Search Service." Thsi I know is
wrong because I can search against it...
I am fairly confident that the problem is related to the
number of collections I have registered, but since I cannot delete
them, I am kinda stuck.

OK, so its not a bug. What it is is that verity on CF7 cannot
list collections that were migrated from CF5. Once I recreated
them, everything works fine. Deleting them was tough tough. I had
to move the folders with the collection info out of thier location,
then move them back in 10 at a time and run a cfcollection action =
list and loop thru to cfcollection action = delete routine in order
to clear them out. Once that was done and I recreated and
re-indexed, everything is happy now.

CF & Verity Collection Categories

I've recently upgraded to CF7 (7.0.2), and I'm having issues
with category support in Verity.
I've converted many of our collections to a single collection
with category support. This method works fine until the collection
is optimized. When I optimize the collection several documents will
no longer return in a search of their original category, but will
on a search of a different category.
The problem has made the collections unusable, as even
re-indexing the collection does not correct the issue.
Any advice would be appreciated. Thanks

Ballocks.
For those using the newsfeed, this line:
<cfset sCategory =
"ÀÞááØï">
Might come out with question marks instead of the word
"Russia" in Russian.
It reads OK on the web-UI.
Adam

Verity Collection Unabled to Create Anything

Why am I getting this error?
Unable to create collection sts.
Unable to create collection sts.
An error occurred while creating the collection:
com.verity.api.administration.ConfigurationException: Fail to
create the index. (-6220)

I am having a similar problem but only with the UNI language
pack. It is indexing fine with the English or Englishx packs. It
also worked fine until we updated to the latest patches (7.0
worked, 7.02 does not)
I'd posted the following in the 'getting started' section but
maybe it should be here instead.
I have two verify collections. The English collection is
working ok. The French collection using the 'uni' multilanguage
does not index using the Cold Fusion code, and if I try to index it
on the server directly, I wait and wait and eventually it will give
me a page not found.
When I was at 7.0, this index worked fine. Now after
upgrading to 7.02, I cannot create the index.
I have tried deleting the collection and starting over but I
get the same problem.
This is the code that had been working:
<CFLOCK NAME="CFINDEXSEILOCK" TYPE="exclusive"
TIMEOUT="90">
<CFINDEX ACTION="refresh" COLLECTION="sei"
KEY="E:\UTEWebsite\Francais\" TYPE="path" URLPATH="
http://www.ute-sei.org/Francais/"
EXTENSIONS=".htm, .html, .cfm, .pdf, .doc" RECURSE="yes"
language="uni">
</CFLOCK>
Any thoughts?

Problems connecting to verity collection creation form

1. Am running CF MX7. Am following directions in CF
documentation to create a form by which a user can create a CF
Collection for a Verity search. I'm having several problems.
2. I created a form and an action page, that it's suggested
that I put in the MyApps directory in the cf_root. Well, Since I'm
running IIS, I don't have a MyApps folder or a cf_root folder, the
root folder of the default Website, is wwwroot/. And I've created a
MyApps folder and put both pages in the folder.
3. It suggests that to access the Create Collection form, one
should type
http://Hostname:Port/myapps/name
of file.cfm
So I've tried many purmutations of that. I continue to get a
Page not found error. No CF errors, just a page not found error.
4. One other item may be affecting this problem, but I don't
know how to get around it. The Home page of the site is in a
subfolder of the wwwroot folder as apposed to being dumped directly
into the wwwroot folder.
Can anyone assist me? WCW

Thanks, I did do that, and also changed the URL I was using
to access the Create Collection form. I am running IIS, also, my
Website root folder is not actually inside the wwwroot or Inet Pub.
But the Home Page is set to the actual location of the Website
folder. So, the MyApps folder is inside the root folder of the
Website, even though it's in a different location.
Don't understand why you need a port number. And what port
number would that be anyway? If I put port 80, which is the port of
the Web site, hit enter, it just disappears when the browser
resolves the address. Don't really need it.
Anyway, I can get to the creation form, create a collection,
and have written pages to index the collection.
Now my problem is that the results I get when executing a
search is a list of Paths, not URLs. Very strange. Don't know if
I'm filling out the form wrong or whether my coding is wrong, but
Search results should NOT show the PATH to the page, but the URL.
Does anyone recognize this incorrect result? Any idea of what
I could be doing wrong? If so, I could really use some assistance.
Thanks, WCW

Verity collection and French Accents

I have a problem I have this french website I'm trying to
manage but I'm having trouble with the stupid french accents. If I
get the data from the database with a cfquery no problem I jsut add
a cfcontent tag with the corect local and bingo all is good.
Now when I ptu the data from the database in a collection
then start the problem. I tried with meta tag and cfcontent and a
bunch load of other stuff but I still get ??? in stead of the
éèçà and so forth. Now my database is mysql on
french local, collection was created with the language french
option.
I'm not sure what else to try to get those weird char to go
up.

I folowed this link
http://www.adobe.com/cfusion/knowledgebase/index.cfm?id=tn_18163
and I did try to go in the style.uni and change all the
charset form 1252 to 8859 then reindex the collection but it did
not work. I tried diferent language for the create collection but
still no luck.
I still cant find a way to change the charset for my
collection. My datasource has the right charset since with a
cfsearch I get what I want, but it is not a very efective way to
search.

Verity collection date ranges

I have a collection of documents differentiated by topic and
date. The search result looks something like this,
"/Supply/Supply2004/23Jan06". I use a calendar function that will
input a date into a textbox in the following format: MM-DD-YYYY. I
am not sure how to get the two date formats to understand each
other. The cfsearch doesn't seem to have an attribute that will
manipulate a date format in the results. Any ideas

Debi is right... You aren't very specific about what you want to do here.
If you're looking the last 4 months of data to show on the report and then distinguish the records for the last 8 weeks, then do something like this...
In the Select Expert use a formula like this...
{TableName.DateField} >= DateAdd("m", DateDiff("m", #1/1/1900#, CurrentDate) -4, #1/1/1900#)
This will bring in all of the records for the last 8 months.
Then create a formula like this...
IF {TableNmae.DateField} >= DateAdd("ww", DateDiff("ww", #1/7/1900#, CurrentDate) -8, #1/7/1900#)
THEN "Last 8 weeks"
ELSE "Last 4 months"
You could then group on this formula if you so choose.
HTH,
Jason

Verity Collection Bloat

Similar Messages

Maybe you are looking for