A question for Tokenizer in a search engine project.

I am doing a search engine project in Java. The project will need two parts: indexer and query Processer. The class I have includes IndexBuilder.java, InvertedIndex.java, Parser.java, Tokenizer.java, StopWordList.java, Lovinstemmer.java.
Among above class files, I am not quite understand how Tokenizer works. The following is what I am understanding the scenario of index part works. Please check what I talk about how Tokenizer works, it might be wrong.
I think Parser object will get the input file, then will deal with the words in the file one by one. FOr example, for the first word in the input file, Parser will contact StopWordList.java and Lovinstemmer.java to get rid of unuseful characters, then pass this changed word to Tokenizer, Tokenizer will then do sth to this words, then return a token( here, token is the words) to the Parser, then Parser will put this token into InvertedIndex.
Am I right? If I misunderstood sth, please let me know. Thanks,
James

It's quite possible that what you say is correct, but why are you asking us? Those classes are not part of the standard Java API, but were written (or at least) specified by somebody else. So ask that person.

Similar Messages

Question on Wlp81 OOTB autonomy search engine

HI all:
I am an BEA SE come from china.when use Wlp81 OOTB autonomy search engine, I have
download autonomy_langfiles_win32.zip from dev2dev.bea.com and config it. But
I found which still cannot search Chinese correctly. I have four question on Wlp81
OOTB autonomy search engine.need your help. Thank you very much!
please see the attachment for the detail.
1.Auto Indexer cannot search Chinese correctly
2.http fetch cannot search Chinese correctly
3.how to open the search result on another machine
4.how to config the odbc fetch
[Question on Wlp81 OOTB autonomy search engine.rar]

Didn't get any bites on this, but thought I'd follow up on my original post. I looked around a bit more and opted for the ZOOM search engine for my site.
The authors appear on their own support forum quite a bit and have helped with the learning curve.
All in all I'm pleased with ZOOM.
John

How do I substitute Google for Ask as my search engine?

Ask substituted itself today and I can't get rid of it. Nothing I found on the menu button has been any help. I really would appreciate anything that helps.

Hi, please read [https://support.mozilla.org/en-US/kb/search-bar-add-change-manage-search-engines-firefox this article] on how to set your preferred search engine.
If you are unable to set it, you probably have a PUP (Potentially Unwanted Program), so please see [https://support.mozilla.org/en-US/kb/remove-toolbar-has-taken-over-your-firefox-search this article] for instructions on how to remove it.
Hope that will help.

How to improve the performance for integrating third party search engine

hi,
I have been working on integrating verity search engine with KM. the performance for retrieving search result totally depend on how many search result returned, for example if there is less than 10 records, it only takes 3 seconds, but if there are 200 records, it takes about 3 minutes, is it normal? anyway to improve it? Thanks!
T.J.

Thilo,
thanks for the response, would you recommend some document for configuring KM cache service, I did change memory cache, and also dynamic web reposity, whatelse out there that I can change? right now, I have one instance(EP6.4 sp11) works well, it returns 200 records from Stellent within 6s. But when I put this KM global service on EP6.0 sp2 (our current system) it takes about 15s. I am not sure is this because of different EP version, or something else. I have tried my best to slim down SOAP component from Stellent. I don't think anything else can be done from that side anymore. before I changed the SOAP, it tooks about 60s. just wonder what else I can do on KM side to improve it performance? Thanks!
T.J.

Question for scripting gurus: GREP search, change case make Smallcaps

I have no knowledge of scripting at all, but this question keeps coming up during training sessions: is it possible to (java)script this:
- Do a GREP search \u\u+
- Change case to lowercase
- Apply SmallCaps (or: apply character style)
this would allow to search for acronyms and change them to smallcaps (or, even better: apply a character style with small caps and tracking)
I know it is easy for OpenType smallcaps (do a GREP search, change to OT smallcaps) but this doesn't really change case. And some fonts used aren't OT.
Anyone?
Would be VERY apreciated!!

But Harbs is a seasoned scripter who knows he'll get flamed if one of his scripts "just does not work" ;)
Well, now that you mention it, the script is not really foolproof. It's a quick and dirty script which I threw together very quickly. It's missing any error checking, some of the variables global, and it's not in a private namespace. These are all things which could cause it to "just not work" ;-)
Here's a more foolproof construct... (and it'll work on the current story if selected, or the whole document if there's no story selected) It will create a new character style if one does not exist and work on character styles within style groups as well. I wrapped the whole script in an anonymous function to give it a unique namespace as well.
(function()
if(app.documents.length==0){return}
var doc=app.documents[0];
// Change the following to your style name!
var character_style_name = 'Small Caps';
try{var range = app.selection[0].parentStory}
catch (err){var range = doc}
//comment out next line if you do not want styles.
var charStyle = GetCharacterStyle(character_style_name,doc);
app.findGrepPreferences = null;
app.findGrepPreferences.findWhat="\\u\\u+";
var finds=range.findGrep();
for (var i=0;i<finds.length;i++){
finds[i].changecase(ChangecaseMode.lowercase);
//comment out next line if you do not want styles.
finds[i].applyCharacterStyle (charStyle)
//uncomment next line if you do not want styles.
//finds[i].capitalization=Capitalization.smallCaps;
function GetCharacterStyle(styleName,doc){
var charStyles=doc.allCharacterStyles;
for(var i=0;i<charStyles.length;i++){
if(charStyles[i].name==styleName){
return charStyles[i];
return doc.characterStyles.add({name:styleName,capitalization:Capitalization.smallCaps});

SAF for Web Client without search engine?

Hi!
Is it necesary to implement SAF to have IC Web cliente, if we do not need Search Tools?
Best regards,
Cristina

Hello Cristina,
You don't necessarily need the Software Agent Framework (SAF) for the Interaction Center if you don't need to the Knowledge Search (e.g., solution search) or ERMS Categorization and Solution Auto-Suggest.
Warm regards,
John

Creating a Search engine question

HI,
I have a question. I built a search engine for my site to just rip through my DB and show the records.
Currently if someone enters "big white dog", my app does an sql statement using LIKE and searches for '%big white dog%'. This is ok but I have a few problems.
1) It should search for big AND white AND dog. So how do i break up the string into separate words.
2) Currently if someone types in ring, things that say bearing are coming up. I tried doing LIKE '% ring %' (with the spaces) but if the word ring is the first word in the record that im scanning, it wont come up.
Basically I need some help from someone that has done this before. Im not looking for entire search engine code. Just need someone to explain the technique and then I can get it done.
Thanks so much for your time.

ok, so now i have the string broken apart but this is my last and final problem.
record1 = 'Wheel bearing';
record2 = 'ring - diamond';
Lets say the above two records are in my db. If i search for like %ring% both records come up, which is wrong. ONly the second record should come up. If i change it to like % ring % it doesnt bring up anything because there is no space before ring in record2. I have tested this on many search engines and they get this to work, but I dont know how they do it.
Any suggestions???
THanks

Safari 6 dropdown menu for switching search engines gone? Nooo!

Safari 6 no longer has dropdown menu for switching quickly between search engines? I know how to set/change the default search engine but I like/use some features on Google and others on Bing. I do research all day - this stinks! (They got rid of the snapback feature too. Are they nuts? The Reader feature is nice, but doesn't make up for losing these other things - would go back to previous version if it weren't such a PITA to do it)
Is there a quick workaround - something faster than loading the alternative search engine main page and inputting search field info all over again?

download the latest version of Glims... now works with Mountain Lion and Safari 6
http://wiki.machangout.com/howdoi/glims-development-build
haven't had any problems with it .... brings back favicons, too

Provide "Suggestions" feature for all searchbox search engines.

Is there a hack that can force Bing/Yahoo/Google/Wikipedia suggestions for Searchbox engines that do not provide suggestions themself.
I used Suggestthemall extension before for this purpose, but it does not work in Firefox 4.0.

You can still disable the search engines if you have root access to your device. Specifically, you need to pull and modify the file search.json. On my device, I have:
adb pull /data/data/org.mozilla.firefox/files/mozilla/rbn69ru0.default/search.json
Now, by default, this file doesn't format everything nicely, so you can use something like
http://jsonformat.com
In order to clean things up. In any case, you'll see bits such as
"_id": "[app]/bing.xml",
"_name": "Bing",
"_hidden": false,
"description": "",
"__searchForm": "http://www.bing.com",
You need to take the "_hidden" flag and change it to true. Do this for all of the search engines to hide all of them. Or, frankly, just delete everything under engines. Then, upload the file back and restart firefox with a command like:
adb push search.json /data/data/org.mozilla.firefox/files/mozilla/rbn69ru0.default/
Until an application restart, the changes won't be seen.
As a slight editorial note, it's dumb that we have to do this by hand. Either we should be able to disable things under Settings->Customize->Search settings or we should have some way of turning things off in about:config. While I understand that removing menu items that cause too many support issues, completely removing the option even from about:config is counterproductive.

With the current warning regarding internet explorer, is your "google" search engine safe? If not, what would be the safest search engine?

Same as above. I'm concerned about using internet explorer at this time. I would appreciate your response.

What is the warning you are reading about ?
This is probably not really a Firefox support question if it relates to Search engines.
The Google Search used is an HTTPS version and so that helps with security. Firefox itself has some security features built in. You may find these articles of interest.
* http://www.mozilla.org/en-US/firefox/desktop/trust/#secure
** https://blog.mozilla.org/blog/2013/01/28/privacy-day-2013/
* [[How does built-in Phishing and Malware Protection work?]]

I cannot change the search engine to google from yahoo

I have read and tried several suggestions on how to change my search engine to google and it will not change to google. I have deleted anything yahoo from my computer, I did the troubleshoot suggestions, I used malawarebytes, and microsoft essentials. Additionally, I deleted the spigot/youtube downloader and toolbar, that includes the add-ons, ms config, the registry, and in my programs. Does anyone know, outside what has already been mentioned what I can do to change the yahoo search engine to google. I would like to add, I still have the same issue in safe mode.
https://support.mozilla.org/en-US/questions/919083?s=change+search+engine+to+google&r=11&as=s
https://support.mozilla.org/en-US/questions/920396?s=change+search+engine+to+google&r=3&as=s
https://support.mozilla.org/en-US/questions/915358?s=change+search+engine+to+google&r=4&as=s

Very Easy no need to install any thing.
1. In address bar Type in about:config (press enter)
2. Press enter again to accept I will be careful...
3. Type '''browser.startup.homepage_override.buildID''' in search bar
4. right click on it and click reset.
5. Type '''browser.startup.homepage_override.mstone''' in search bar
6. Right click on it and click rest.
close Firefox and restart it. Good to go

Support validity question about installing and configuring SP13 on same machine as FAST to use both as search engines for SP2010

Hi,
I have a SharePoint 2010 farm which is using its own servers.
This farm has a FAST Search for SharePoint server which is a standalone on a dedicated server.
This farm is my testing staging environment and not a Production environment, eventhough its almost an exact copy of the Production farm.
I would like to replace the FAST server with SharePoint 2013 as a search engine for my SharePoint 2010 farm.
Because i have a standalone deployment of FAST on a dedicated server and i dont have any more servers to add to this farm,
I would like to install SharePoint 2013 on same server as FAST while FAST is still active, connect it to the 2010 farm and do some tests before i will remove the FAST content and query SSA's.
I would like to know if installing a SP2013 on same machine as FAST is supported by Microsoft as long as the FAST is not connected to the SP2013? (performance is not an issue)
What about All in one machines like development servers, is it supported by Microsoft to install FAST on same machine as SP2010 farm? (again, performance is not an issue)
thanks,
Guy

I would like to know if this scenario is supported by microsoft.
This installation will be for testing purposes only, if there are any issues with installing SharePoint2013 on a FAST search machine, i would like to know what kind of problems i may run into.
I'd also like to clear out that this FAST and SharePoint2013 are not comunicating with eachother but with a SharePoint2010 farm that is configured on different servers.
I would really appreciate direct answers to these questions.
thanks for your answers,
Guy

Adding keywords and phrases for high search engine hits

I'm new to iWeb and most things seem to be working. I have one question and one problem, hopefully you guys can help out.
Question: how do I add html code that is not visible on the page but that will help Google and other search engines find my page using keywords, phrases, etc.? Please assume I know very little about this and need the holding hand explanation.
Problem: I have a media file portion on my website that displays pictures (working very well) and one that is supposed to show a movie (not working at all). The movie was made with iMovie and has an .m4v extension. When you click on the link for the movie it shows a blank screen on Firefox and a Quicktime logo with a big question mark on Safari. I'm attempting to view the website on my Mac with the latest Quicktime software on it but it doesn't work. Any ideas on how to fix it?
Some additional info: the hosting provider is 1and1.com and I have a Windows server package (unfortunately). Also, if I publish the site to a folder the movie area works flawlessly, it just does not work on the hosted site.
Thanks!!

You don't need to do post publication manual editing of your site to add keywords and other metadata. You can use the freeiWeb SEO Tool. If you're publishing to MobileMe you can add them after publication. If you're publishing to a commercial hosting server you'll need to publish to a folder on the desktop, run SEO and then use a 3rd party ftp client like the free Cyberduck to upload your site.
If you have to republish SEO remembers what you added and can quickly add them again before uploading.
OT

I like to use Google for my search engine but it has been replaced with Amazon. How do I get google back?

In the upper right corner of the Firefox screen, there is a box for a search engine search. I used to have google there, but recently it somehow changed to Amazon.com. How do I switch it back to google?

OK, might want to make that change in your profile to get faster answers to future questions. If you leave the "iOS6" that's now in your profile, helpers will have to ask the OS question every time you post, slowing your getting help you deserve. Note the level of detail I was able to cram in my Equipment profile (at the bottom of this post).
I don't have 10.7 on any of our Macs but I suspect a system update may have removed that iPhoto option. However, you can still get to your iPhoto files if 10.7 is anything like 10.8.
In System Preferences > Desktop and Screensaver > Screensaver tab, there is a "Source" dropdown menu direct under the preview pane. Click it and select "Choose Folder..":
That gives you a dialog box that should point to your "Pictures" folder. Within that is your iPhoto library. Select it and see if that gives you access to your images.

How to make my site top in the search engines for a high ranking?

I am new at this, and I am planning to build a big site on iweb, but my question is where can I get information on how to make my site with high rankings? Would I have to post my site in lots of search engines, or have the hosting company help out? how do i do this?

As SEO (Search Engine Optimization) is important to you, you may want to consider alternatives to iWeb as "iWeb websites are not made to be search engine friendly" — read more here:
SEO For iWeb: How to get your iWeb Websites into Google & Other Major Search Engines
...iWeb is designed for personal sites and, as such, Apple apparently thinks there's no need for Google — you tell grandma the URL of your personal iWeb site, she painstakingly types it in and then gazes admiringly at the photos of her grandchildren.
Apple hasn't changed this approach in three iterations of iWeb but perhaps, with all the feedback it receives on this issue, there's still hope. Meanwhile, web-design apps for business sites tend to integrate search engine optimization — Sandvox Pro is an example:
http://www.karelia.com/sandvox/getyour_site_noticed_andb.html
Another advantage of Sandvox is that, when viewing your site in Safari, the text increase & decrease commands will work — they don't work for an iWeb-created site. ...For business use, it's important that customers don't turn away from your site because they consider your text too small to read.
However, if you still really want to use iWeb, this article may help:
http://www.iwebformusicians.com/SearchEngines/SEO.html
By the way, for more info about SEO in general, this blog of a web-design app (Yola) recently started a multi-part series on the subject — the first is here:
http://blog.yola.com/2009/06/15/seo-101
The rest of the series (in no particular order) can be found here:
http://blog.yola.com/category/seo/
And a video:
_Using Title Tags Effectively For Search Engine Optimization (SEO)_

A question for Tokenizer in a search engine project.

Similar Messages

Maybe you are looking for