TREX initial Indexing

Hi, I am trying to index about 12mio datasets.
I changed the queue parameters according to a PerformanceHowTo:
Parameter
Schedule Type: Count
Schedule Max Documents: 100000
Index Bulk Size: 50000
Optimize Bulk Size: 400
Initial Indexing Mode: On
From [THIS|http://help.sap.com/saphelp_nw70/helpdata/en/46/ceb96feb7e2708e10000000a11466f/content.htm] Page.
However If I watch the queue I see that all documents arre currently "Optimizing".
The Problem is now that the Memory usage is very I high. I guess it tries to get all documents into memory first for optimizing?
Well, I watched the memory usage and now it's not rising anymore and the log says:
[5472] 2010-12-07 11:05:00.392 e indexmgr     IndexMgr.cpp(01690) : IndexHandle 310815136 index ses:sap_qmd_mdt100_bs_zcer_find__1 acquire search_delta timeout -1 waking up our self
[5472] 2010-12-07 11:05:00.392 e indexmgr     IndexMgr.cpp(01690) : index "ses:sap_qmd_mdt100_bs_zcer_find__1"; search_delta 1; write_delta 1; finish_delta_merge 1/0/0(1); handles: thread 5472 (310815136) state search_delta acquired 2010-12-07 11:05:00.392, thread 7940 (327607456) state free to finish_delta_merge acquired 2010-12-07 09:30:18.819, thread 1356 (319214768) state write_delta acquired 2010-12-07 08:54:02.568
[2376] 2010-12-07 11:06:58.782 e memwatch     MemWatch.cpp(01048) : waiting for unload/memfree: 5800 seconds
[2376] 2010-12-07 11:06:58.782 e memwatch     MemWatch.cpp(01067) : ses:sap_qmd_mdt100_bs_zcer_find__1 is locked: 5800 seconds
[2376] 2010-12-07 11:06:58.782 e memwatch     MemWatch.cpp(01067) : ses:sap_qmd_mdt100_bs_zcer_find__1 is locked: 7976 seconds
[2376] 2010-12-07 11:08:39.094 e memwatch     MemWatch.cpp(01048) : waiting for unload/memfree: 5901 seconds
[2376] 2010-12-07 11:08:39.094 e memwatch     MemWatch.cpp(01067) : ses:sap_qmd_mdt100_bs_zcer_find__1 is locked: 5901 seconds
[2376] 2010-12-07 11:08:39.094 e memwatch     MemWatch.cpp(01067) : ses:sap_qmd_mdt100_bs_zcer_find__1 is locked: 8077 seconds
Does this mean that it's not possible to optimize 12mio in documents in one package?
Edited by: Daniel Winter on Dec 7, 2010 11:11 AM

Hi Daniel,
you have choosen following queue parameter:
Schedule Type: Count
Schedule Max Documents: 100000
Index Bulk Size: 50000
Optimize Bulk Size: 400
Initial Indexing Mode: On
That means 400 * 50.000 Business Objects / Documents will be optimized in one step.
Now it depends on your amount of main memory if this settings are fine or not.
But if you detect that this do not work that of cause you can decrease the values.
For example you can use instead of 400 of Bulk Size only 200 or any other value.
You have to try it....
But ony way the question is do you get any error messages? Or is this only a question in general?
Best regards
Frank

Similar Messages

ESH_ADM_INDEX_ALL_SC cannot perform initial indexing for all search connectors

Dear SAP Gurus,
We are implementing TREX version 7.10.50 for Talent Management ECC 6.0 - EHP 5.
I'd like to ask you question regarding ESH_ADM_INDEX_ALL_SC program
which used to create search connector for TREX and perform initial indexing for all search connectors.
As we know we can perform indexing using ESH_COCKPIT transaction code or use ESH_ADM_INDEX_ALL_SC.
If I try to perform indexing using ESH_COCKPIT, all search connectors can be indexed ("searchable" column are "checked" and status are changed to "Active" for all search connectors).
However, if I try to perform indexing using ESH_ADM_INDEX_ALL_SC, not all search connectors are indexed.
I've traced the program ESH_ADM_INDEX_ALL_SC using ST01 transaction code and found these error:
- rscpe__error 32 at rscpu86r.c(6;742) "dest buffer overflow" (,)
- rscpe__error 32 at rscpc (20;12129) "convert output buffer overflow"
- rscpe__error 128 at rstss01 (1;178) "Object not found"
Please kindly help me to solve this issue,
Thank you very much
Regards,
Bobbi

Hi Luke,
Please find below connectors and the status after running ESH_ADM_INDEX_ALL_SC:
HRTMC AES Documents Prepared
HRTMC AES Elements Prepared
HRTMC AES Templates Prepared
HRTMC Central Person Prepared
HRTMC Functional Area Prepared
HRTMC Job Prepared
HRTMC Job Family Prepared
HRTMC Org Unit Prepared
HRTMC Person Active
HRTMC Position Prepared
HRTMC Qualification Active
HRTMC Relation C JF 450 Active
HRTMC Relation C Q 031 Active
HRTMC Relation CP JF 744 Active
HRTMC Relation CP P 209 Active
HRTMC Relation CP Q 032 Active
HRTMC Relation CP TB 743 Active
HRTMC Relation FN Q 031 Active
HRTMC Relation JF FN 450 Active
HRTMC Relation JF Q 031 Active
HRTMC Relation P Q 032 Active
HRTMC Relation S C 007 Active
HRTMC Relation S CP 740 Active
HRTMC Relation S JF 450 Active
HRTMC Relation S O 003 Active
HRTMC Relation S O Area of Responsibility Active
HRTMC Relation S P 008 Active
HRTMC Relation S Q 031 Active
HRTMC Relation S S Manager Active
HRTMC Relation SC JF FN Active
HRTMC Structural authority Active
HRTMC Talent Group Prepared
As suggested by OSS, we implement SAP Note 1058533.
Kindly need your help.
Thank you very much
Regards
Bobbi

TREX not indexing Excel files

Hi all,
We are getting the following error when trying to index some MS Excel files:
14011 - Filter: The file is password protected or encrypted
We are using TREX 7.10 rev 34.
We have found that the cause of this is when the Excel file has been set to protect the worksheet with a password (in Excel, go to Tools -> Protection -> Protect Workbook -> set password). It seems odd to me that TREX cannot index a file like this because it should be able to open it. I would think that it only would have problems if it were trying to write to the file (which it is not trying to do).
Does anyone have any ideas as to why it's like this or what can we do about this other than remove the workbook password? We have dozens of files like this, and I need to find a solution other than removing the password.
Thanks!
-StephenS

Hi Stephen,
I couldn't find any release note specific to this issue however I do know that there are newer filter versions available in the most recent TREX revisions so the upgrade is usually a good option to resolve such issues. Of course I cannot say for certain if Revision 39 will resolve it without trying to index your specific document on that version myself.
Anyway I know there are some Oracle filter issues ongoing with TREX 7.1 so I think an upgrade to a newer revision may be inevitable at some stage, but perhaps in this scenario it would be best to wait for confirmation of the exact revision that contains the fix from the TREX experts in your message.
Kind regards,
Lorcan.

Initial Indexing Mode ( On / Off ) ?

Hello ALL
We are trying to configure the queue parameters for initial indexing and Daily update of index .
We are not able to see the Initial Indexing Mode parameter and for that we can set it up :
 On for initial indexing
off for Daily update index.
Can somebody tell us where we can find this parameter in order to edit it on or off .
How do we have to proceed.
Many Thanks

Hi Deepti
I have already checked that link and read its content.
The problem is that I don´t know how to edit this parameter ( Initial Indexing Mode ) . Have you done it already ? How ?
Regards
Stanley

RFC Communication error when doing initial indexing

Hello Everyone,
I have a quick question, when doing the initial index rollup for a BIA index the job is failing with RFC communication error.
We currently have the setting related to " TREXRfcServer threads" set to "Automatic Changes" in order to avoid the RFC Communication errors when loading big loads in BI system.
We currently are on Revision 49.
I also have referred to OSS Note 1138603.
The issue is when I am rolling up the BIA Index the rollup job is failing with RFC Communication errors.
The Global parameters within BIA are:
BATCHPARA - 2
NUMPROC - 5
PKGSIZE - 10000000
SUBPKGSIZE - 20000
We have two app servers with 52 Dialog processes and 25 Background processes.
Any help would be really appreciated.
Thanks
Dharma.

Removing references to systems and ports
Issue:
When the initial indexing is being carried out for an InfoCube to
create the BIA Index and there is more than one application server in
the system the job is failing with the http error.
Hello Everyone,
Thanks for the input, after the automatic changes was configured for concurrent requests the error is different right now.
The error message is,
"Remote communication failure with partner http://<Removed>:<removed>/indexCellTable"
Overview/Background:
When the indexing job is triggered as a background job the failure happens when the S table index is being filled.
In our system where we are trying to do the stress testing we currently have two app servers and one central instance.
When I limit the RFCGROUP in RSDDTREXADMIN table to one App server in terms of logon group the job finishes fine.
Whereas, when I do not limit the RFCGROUP and all app servers are open for access the background job finishes with the error message
Remote communication failure with partner http://<Removed>:<removed>/indexCellTable.
We have looked into the OSS Note 1102652 and our BASIS team confirmed that they were able to ping from BI App servers into BIA blades and vice versa.
We also confirmed if the local gateway was used for connectivity.
The above issue was not in other systems where we only had one server.
We currently are at Revision 49 and SP 13 in BI.
We have E5345 Clowertown blades (2 x 4) 16GB blades.
Please let us know if you need additional information.
Thanks in advance for the input.
Dharma.
Edited by: Arun Varadarajan on Apr 24, 2009 1:36 AM

Use TREX to index /search archive on IXOS?

I wonder whether TREX can index/search archives on IXOS.
Could anybody share experience here?
Thanks!

Hi Eric,
Using SAP Netweaver Enterprise Search you can search content from ERP, BI, KM, File Search, Business Object Search, Search in Webservice etc., for more information please go through this URL
https://websmp209.sap-ag.de/nwes70
Please provide points if this helps you

Using TREX to index R3(WAS) tables

Hi,
I am developing a bsp application on WAS 6.40 and I will like to know if there is a way to use TREX in order to index table contents. Of course, I need also a way to use TREX for searching. Can anyone help?

Hi,
I order to have a quicker response, I think you should post this question in KM forum.
Meanwhile, have a look at things like
http://help.sap.com/saphelp_nw04/helpdata/en/22/89ab46e2674648aec9c0c7c0c040b0/frameset.htm
https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/81c1ca90-0201-0010-a2a1-b0412ec9cd1a
and https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/ec4ee290-0201-0010-6c8d-98db9abff2cc
Eddy

TREX Monitor / Index Monitor are hanging

Hi Experts ,
We are running on EP 2004s SP 12 and TREX 7.0 rev 34.
Our issue is TREX Monitor (System Admin -> Monitoring -> KM -> TREX Monitor) is hanging and not able to display anything.
Same thing is happening with Index Monitor also .
We have checked both TREX and EP are running fine , I can see other iViews in monitor view eg. - Crawler Monitor .
Can anybody help ..
Thaks & Regards,
Amit

Hi Amit,
I have had this issue before as well but unfortunately wasn't able to find out what exactly the problem is. I believe it is some type of communication problem between TREX and your portal. Try restarting both TREX and the portal and see if it helps.
-Esther

TREX Active Index - Which index is really being used?

We are using ISA E-Commerce 5.0 for ERP 6.0. We run the ISA_CATALOG_REPLICATION program as an hourly job to create indexes in TREX. We add items to our catalog several times a day. Our variant is set to keep 5 indexes.
1. Where do I verify which index is REALLY active? They are all green in TREX Admin and they all have an Active flag in SRMO. (I am pretty certain the older indexes are not being used by ISA because the new items show up in B2B after the ISA_CATALOG_REPLICATION completes)
2. When the ISA_CATALOG_REPLICATION cancels due to an SRET error, I want to go back to the last valid index in TREX and force the system to use that index. How do I do that?
Thank you
Jon Sells

SAP told that this functionality is not available in TREX. So there really is no point in having more than 1 index at a time.

TREX not indexing document 10 KB

hi...
TREX search is working fine for documents < 10 KB but if i try indexing documents > 10 KB it gives prperation failed error.Please let me know what as to be done so that it indexes document > 10 KB
Please help me on this as it is very urgent
Thanks in Advance,
Shanshank

I installed an EP7/TREX SP10 two years ago. All worked fine until I decided to apply windows SP2 and some security patch. My machine is a win2003 x64.
Now EP7 works fine, but TREX in not able to reindex documents > 10kb.
Trex preprocessor trace says:
Preprocessor.cpp(03550) : HANDLE: DISPATCH - Processing Document with key '/documents/Segreterie/Documenti PCTP/Doc. in Arrivo/Anno 2009/2009_03123.pdf' failed, returning PREPROCESSOR_ACTIVITY_ERROR (Code 6401)
In the portal security log file I found the corresponding error:
System/Security/Authentication#sap.com/irj#com.sap.engine.services.security.authentication.logincontext#Guest#2####764602d0407311dea83600188b77747b#SAPEngine_Application_Threadimpl:3_17##0#0#Info#1#com.sap.engine.services.security.authentication.logincontext#Plain###LOGIN.FAILED
User: N/A
Authentication Stack: ticket
It seems to me that the portal is not able to pass user name to Trex (User: N/A) and then It is not authorized to retrive the documents.
I tried to change the user of indexmanager service "index_service"; to set the alternative host in url generator service. Nothing changed.
Any suggestion?
Giorgio Peressin

TREX not indexing

hi,
I have installed TREX 6.1 server which is working fine. But when i configure it to a EP 7 for indexing its repositories it is not responding. Can i use a TREX6.1 server for pointing to a EP7 system in the first hand? If yes, why is it not indexing the documents in it? please clarify.
Regards,
Ganesh N

Hi Akhilesh,
I am also having some problem in TREX search ,documents are giving lots of error:
8005,time out reached
preparation failed.
When i go to KM-> component monitor->repository managers two of my repository are having below errors:
Name :Company_SAN
Type: Repository manager
status: Start up failed
Des: Startup Error: getting mapped math - Invalid argume
Name :Web repository
Type: Repository manager
status: Start up failed
Des: Startup Error: Exception during start up of sub-manager: W2kSecurityManager: problems getting config parameters from Repository Manager
Please come up with some solution to it.
Thanks in Advance
Amit kumar koyal

TREX system index

Hi - Any idea on what metadata TREX indexes are built ?
Regards
mx

Hi,
You can check this in -Forums->KnowledgeManagement and COllaboration to get more idea as there is separate section
/thread/145844 [original link is broken]
/thread/49043 [original link is broken]
Regards,
Moorthy

KM TREX not indexing

Hi
I have a problem regarding the TREX.
I have this folder indexed.
I create resources in this folder by a custom WDP application.
Basically i will upload a .pdf or .doc file from desktop to this folder.
Now the problem is some of the documents are not getting indexed.
There are very few documents though, but i cant find a reason why it is not getting indexed.
Now, when i check the properties of the documents which gets failed, i dont see any problem.
If i simple press "Save" button, the document starts gettign indexed and thus it gets indexed.
Here, i am not changing any property, but simply pressing the save button.
Till date, i couldnt reproduce the error.
But since this error was reported from Prod, we want to know why this error comes and how it can be avoided.
Please give ur thoughts...
Regards
BP

Hi Piyush,
My case is like some docuemtns didnt get indexed.
Well some others working perfectly fine.
Even now if i upload some docuemtns, it is getting indexed.
And as i stated earlier, i goto details view and press "Save", it starts indexing the doc.
So communication to the TREX server is working fine.
The main problem is i cannot replicate the issue.
Here is the code sample i use to upload the document to KM
public boolean uploadDocument(
          final String documentName,
          final byte[] documentContent,
          final String location,
          final String fileExtension,
          final String company,
          final String brokerInstitution,
          final Date publishedDate,
          final String title)
          throws UserManagementException, ParseException, WcmException {
          final IContent fileContent;
          final RID rid;
          final ICollection collection;
          final IResource createdRes;
          fileContent = this.getResContent(documentContent, fileExtension);
          rid = RID.getRID(location);
          collection =
               (ICollection) this.resourceFactory.getResource(
                    rid,
                    this.resourceContext);
          //Check for duplicate
          String tempStr =
               (new StringBuffer()
                    .append(location)
                    .append('/')
                    .append(documentName))
                    .toString();
          final IResource reportResource =
               this.resourceFactory.getResource(
                    RID.getRID(tempStr),
                    this.resourceContext);
          if (reportResource == null && collection != null) {
               final IPropertyMap propertyMap =
                    this.getMetadatapropertyMap(
                         company,
                         brokerInstitution,
                         publishedDate,
                         title);
               createdRes =
                    collection.createResource(
                         documentName,
                         propertyMap,
                         fileContent);
               return true;
          } else {
               return false;
     public IPropertyMap getMetadatapropertyMap(
          final String company,
          final String brokerInstitution,
          final Date publishedDate,
          final String title)
          throws ResourceException, ParseException {
          final IMutablePropertyMap propertyMap = new MutablePropertyMap();
          final String propertyNamespace = getproperty("PROPERTIES_NAMESPACE");
          final String companyNameID = getproperty("COMPANY_NAME");
          final String brokerInstitutionID = getproperty("BROKER_INSTITUTION");
          final String publishDateID = getproperty("PUBLISHED_DATE");
          final String createdDateID = getproperty("CREATED_DATE");
          final String titleID = getproperty("TITLE");
          IPropertyName propertyName =
               new PropertyName(propertyNamespace, companyNameID);
          IProperty property = new MutableProperty(propertyName, company);
          propertyMap.put(property);
          propertyName = new PropertyName(propertyNamespace, brokerInstitutionID);
          property = new MutableProperty(propertyName, brokerInstitution);
          propertyMap.put(property);
          propertyName = new PropertyName(propertyNamespace, publishDateID);
          property = new MutableProperty(propertyName, publishedDate);
          propertyMap.put(property);
          //Calendar cal = Calendar.getInstance();
          //Date today = new Date(cal.getTime().getTime());
          java.util.Date date = new java.util.Date();
          Date today = new Date(date.getTime());
          propertyName = new PropertyName(propertyNamespace, createdDateID);
          property = new MutableProperty(propertyName, today);
          propertyMap.put(property);
          propertyName = new PropertyName(propertyNamespace, titleID);
          property = new MutableProperty(propertyName, title);
          propertyMap.put(property);
          return propertyMap;
Regards
BP

TREX: No index available

Hello,
i have a problem with the TREX (6.1) and the EP 6(SR 1).
The TREX Monitor show that the HTTP, Name, Index and Queue Server are green.
I can create an index (Index Administrator) but after saving the index isn't in the Index Administrator visible.
And he doesn't work, too.
With the TREX Monitor I can see the queue of the index (empty).
Can anybody help me?
regards,
Jochen

Hi Jochen,
did you go through the TREX post installation tasks?
Is the service user assigned that TREX needs for indexing?
Cheers, Karsten

Trex Server Indexing

Hi all DMS Guys
I am Carrying out implementation of DMS , Content Server, and Trex server for the first time.
I have completed DMS , and Content Server configuration.
I want Information on Trex server configuration or some documentation on TREX server configutation and installation.
If any body can pls help me urgently it would be great help.
Thanks
With Warm Regards

Hi Mangesh,
If you are looking for a Search Option to be activated in DMS then its very simple,
Go To T.code SKPR07 (KPRO IMS: Monitoring), select Tools tab,
and in Document Class Field Select : Select "DMS:Physical Object for main Originals"
2. In Main menu under Edit select Administration, you get an POP-Up menu of Administration, fill "Document Class" (DMS_PCD1), Document area (DMS), Management table (DMS1DIDSTA)
Check (tick) "Create Indexing Readiness"
Click on the "Indexable Attributes" you get the sub-screen select all or required one.
You can schedule the indexing job by two ways i.e. by ABAP program RSTIRIDX with a T.Code SM36. click on job wizard & Proceed.
After doing all above steps you can see in CV04N SearchTxt Field
Regards
Rehman
Reward Your Points If Useful

TREX initial Indexing

Similar Messages

Maybe you are looking for