Search Engine in Java

Hi All,
I just wonder anyone can lend me a hand please. I am using Java to develop a search engine at the moment, I need to store all the words from a single html page, and I have done this, but the thing is how can I get all pages from a certain intranet? Thank you in advance.
Best Regards,
Ray

As per my knowledge any search engine while use the meta data of a html page first then go for the whole contents.
Ans it is always better to use google or any other searching tool instead of developing it from scratch. it is good only for practice.

Similar Messages

How to interact with google search engine using java HTTP POST

I am trying to develop an desktop app that acts as an online radio tuner. I want to use google as an index site for actually searching for radio stations online. So I was wandering if it is possible for me to use java to actually interface with the google search engine servers. For example, I have built a gui that allows you to enter search text. This gui will connect to the web. I would like it to connect to the Google site and post the query to the google web servers. Then i would like to began to parse the response( html page) that i get back from the google servers. Like using google without actually typing in the text field on the google site. Can anyone tell me if it is possible to do this with java. If so, what would be the best way to go about doing this. Thanks.

Thanks for the response. It was actually alot easier to do than I anticipated. All I had to do was create a URL object initialized with the google url(search text embedded in the url - http://www.google.com/search?q=" + searchText + "&hl=en&ie=UTF-8&oe=UTF8).
Looks like this:
String google = http://www.google.com/search?q=" + searchText + "&hl=en&ie=UTF-8&oe=UTF8;
URL url = new URL(google); // creating Google Url object
URLConnection connection = url.openConnection();
       connection.setDoOutput(true);
       connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)" ); // fooling google servers into thinking the request is coming from a real browser
       BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));The content returned by the google server can be accessed calling the BufferedReader class methods.

How to implement a search engine in java ?

Hi All,
1. Can anyone explain me with java code how to connect to a excel database residing on a
different(I know that we do it thro' dsn) machine in my network. I know that we do it with a dsn.
I created a network map but when I execute the statement
"Connection conn = (Connection)DriverManager.getConnection("jdbc:odbc:accesscarddsn");"
then I am getting the error:
SQLException:[Microsoft][ODBC Microsoft Access Driver] Too few parameters. Expected 1.
2. I am doing a website where some pages of website are jsp, servlets. I want to implement a search functionality in my site so that if the user types a keyword then I need to show him the links to the pages(whether jsp,html) which contain this keyword. How do I do that other than using property files .
Thanks & Regards,
Kumar.

With regard to your first question,you need to supply user name and passwrd in the connection statement.I guess that's problem when it saya..too few parameters. IF there is no user name and passwd, sne dnull strings.
With regard to the search functionality, there are many ways you can implement it.I am not sure about any readymade products available on the web, however you can code it yourself, it might be time consuming but I feel it is better.You can do it this way:
1. Parse all the html docs in ur website, ignore all the html tags and maintain a HASH table of all words and the locations you found the word in.You can optioanlly remove all stopwords, i.e. words that occur commonly like 'a', 'an' etc..
2. The next phase is to take a query through a cgi program and search the hash table for all keywords,make sure you seach for each keyword in the query seperately.Retreieve the docs, display the results on a html page.
Cheers!

Search Engine - Need One

Can anyone recommend a search engine, with java api access that will run
with weblogic 8.1?
I want to be able to index, html, pdf, doc and query the results via a
java api.
Thanks

Any documentation?
Wes Hewatt wrote:
Jay Zimmett <[email protected]> wrote:
Can anyone recommend a search engine, with java api access that will
run
with weblogic 8.1?
I want to be able to index, html, pdf, doc and query the results via
a
java api.
ThanksWebLogic Portal includes Autonomy's DRE.

Search engine

Hi there!
I am writing a simple search engine in java. i want it to read from given urls, search for specific keywords and then return strings from the particular pages, which are to be displayed on a standard output.
please help.

http://www.google.com/search?hl=en&safe=off&q=search+engine+java&btnG=Search

[HELP] Search engine

Hi all, I have just joined this forum.
I am currently writing on my thesis about implementing NLP (natural language processing) for an intelligent search engine. The fact turns to be the NLP in Java is not available yet, so in conclussion of my thesis i want to demo the search engine using Java in servlet. Does anyone know where i can get full code that i can use? i asked my advisor and he is ok with this, as long as i do not publish it.
I do not use the code for commercial use, only for demo purpose until i found the java NLP library.
Thank you all in advance. I am really looking forward to have one. thank you very much.

Hi all, I have just joined this forum.
I am currently writing on my thesis about
implementing NLP (natural language processing) for an
intelligent search engine. The fact turns to be the
NLP in Java is not available yet, so in conclussion
of my thesis i want to demo the search engine using
Java in servlet. Does anyone know where i can get
full code that i can use? i asked my advisor and he
is ok with this, as long as i do not publish it.
I do not use the code for commercial use, only for
demo purpose until i found the java NLP library.
in other words you have discovered that it doesn't exist and then go on to ask the usual homework question "giv mi ze koduz", assuming that what you just stated you found doesn't exist does in fact exist and that someone will just give it to you for free, which would of course if it happened void your earlier conclusions.
So what will it be?
Either you conclude that it doesn't exist in which case there's nothing to demonstrate, OR you admit your conclusions were wrong, it does exist, and you just couldn't find it.
Or (3rd option) you decide that the fact it doesn't exist is something that can and should be corrected and fund a project to work towards that.
And no, we're not here to do your homework for you.

I am working on serarch engine in java want help from you

Hi I am making some kind of search engine in java.In which i want to search initials from database(ms access)
suppose to search united states of america
we type usoa
so how can i do that
from where to i start
can anybody plz.........
Edited by: Karamjeet on Jul 2, 2009 12:06 PM

First of all, don't make your own search engine from scratch. An example of a good usable base for many kinds of search engines is Lucene
Second, use a real database and not ms access. If all you need to do is to search for some patterns in some db columns, indexing and sql queries is probably all you need.

Web search engine

Hi All,
My Name is Lakshitha. I am developing a web search engine using Java. I am doing this as my BCS project. I know java little bit . Can any body Please help me on this. First thing i want to know what platform i have to use.
(JSP,Servlets,) please help me where should i start
any suggestions please send to [email protected]

Hi All,
My Name is Lakshitha. I am developing a web search
engine using Java. I am doing this as my BCS project.
I know java little bit . Can any body Please help me
on this.What's your problem?
First thing i want to know what platform i
have to use.
(JSP,Servlets,) please help me where should i startThat depends on your specifications. You should be the one to know which technology to use. A websearch is usually created as a website itself, though, so JSP will be it.
any suggestions please send to [email protected]

Two issues: Mail dotcom is now telling me to enable java in my browser and firefox has issues w/duckduckgo search engine. I am not allowed 2 instal DDG n search

2 issues: After a windows update on 5 10 2012, maildotcom says I need to enable java in my browser, when as far as I can tell, it is.
Second issue is, I am not able to add duckduckgo as my preferred search engine. I get a pop up in search tool add-ons that says "sorry, you need a Mozilla based browser to install a search plug in."
Why and how can these irritations be fixed?

'''''NoScript''''' 
*To add a site to the NoScript "whitelist" ('''always allow''') --> when on the site, click the NoScript icon and choose "Allow ''site''"; the site will be placed on the NoScript Options > Whitelist (non-italicized).
*If you click "Temporarily allow ''site''", the site will be allowed during the Firefox current session (until you close/exit Firefox) and will be placed on the NoScript Options > Whitelist (italicized) and will be removed from the whitelist when you close/exit Firefox.
*NoScript Whitelist --> click the NoScript icon, choose Options, click the Whitelist tab
'''''DuckDuckGo''''' (you probably would want 1 & 3 or 2 & 3) 
#This adds a DDG '''toolbar''' and says it adds the "''encrypted (HTTPS / SSL) version of DuckDuckGo in your '''search bar'''"'' --> https://addons.mozilla.org/en-US/firefox/addon/duckduckgo-ssl/
#You can '''add only an item to the Search Bar''' (without a toolbar) by choosing one of the items here --> http://mycroft.mozdev.org/search-engines.html?name=duckduckgo&opensearch=yes
#You can '''change the Location Bar search''' (replacing the default Google search) by following these instructions --> http://ilias.ca/blog/2012/03/how-to-make-firefox-use-duckduckgo/
This DDG page says there is some problem installing DDG plugins but does not explain the problem or who is responsible for fixing the issue
*http://help.duckduckgo.com/customer/portal/articles/216441-firefox
'''If this reply solves your problem, please click "Solved It" next to this reply when signed-in to the forum.'''
Not related to your question, but...
You may need to update some plug-ins. Check your plug-ins and update as necessary:
*Plug-in check --> http://www.mozilla.org/en-US/plugincheck/
*Adobe Shockwave for Director Netscape plug-in: [https://support.mozilla.org/en-US/kb/Using%20the%20Shockwave%20plugin%20with%20Firefox#w_installing-shockwave Installing ('''''or Updating''''') the Shockwave plugin with Firefox]
*'''''Adobe PDF Plug-In For Firefox and Netscape''''': [https://support.mozilla.org/en-US/kb/Using%20the%20Adobe%20Reader%20plugin%20with%20Firefox#w_installing-and-updating-adobe-reader Installing/Updating Adobe Reader in Firefox]
*Shockwave Flash (Adobe Flash or Flash): [https://support.mozilla.org/en-US/kb/Managing%20the%20Flash%20plugin#w_updating-flash Updating Flash in Firefox]
*Next Generation Java Plug-in for Mozilla browsers: [https://support.mozilla.org/en-US/kb/Using%20the%20Java%20plugin%20with%20Firefox#w_installing-or-updating-java Installing or Updating Java in Firefox]

Search engine machine in java

I would like to give me some general ideas about how to implement a search engine machine implemented in Java. Please help me by giving some standard algorithms or methods or a site tha contains similar source code

String.indexOf(String);
Questions to ask yourself:
Are you interested in speed or quality of the search?
Do you need an intelligent search or a simple search?
How is the data being stored and how can you access that data?

Java Search Engine?

Does anyone know whether it would be possible to write a web based search engine such as google in java? I don't want detailed information, but does anyone know what kind of equipment/software I would need, and the difficulty it poses. Any relevent URLs appreciated.
cheers

The reason why I am enquiring about this is that I'm thinking of carrying out this task for my CS project. Is it sufficiently difficult? Or even possible in a 6 month or so period, part time?I want all the code to be my own and not using any current engines, if that makes any sense.
Does any1 know how the current engines on www such as google, ask.com are written?Relevent info appreciated.
Cheers

Whenever I use a search engine, I keep getting the error message "The connection was reset while the page was loading".

I am able to access the internet, but none of the search engines like Google, Yahoo or Bing will work. the page will not load. I get the error message:
The connection to the server was reset while the page was loading.
* The site could be temporarily unavailable or too busy. Try again in a few moments.
* If you are unable to load any pages, check your computer's network connection.
* If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

Your plugins list shows two Flash plugins and other outdated plugin(s) with known security and stability risks.
# Shockwave Flash 10.0 r45
# Shockwave Flash 10.1 r53
# Adobe Shockwave for Director Netscape plug-in, version 11.0
# Next Generation Java Plug-in 1.6.0_19 for Mozilla browsers
Flash Player uninstall: http://www.adobe.com/go/tn_14157 (this will remove the Firefox Flash plugin and the ActiveX control for IE)
Update the [[Flash]] and [[Shockwave|Shockwave for Director]] plugin to the latest version.
*http://www.adobe.com/software/flash/about/
*http://www.adobe.com/shockwave/welcome/
*http://www.adobe.com/downloads/
Update the [[Java]] plugin to the latest version.
*http://java.sun.com/javase/downloads/index.jsp (Java Platform: Download JRE)
Do a malware check with a few malware scan programs. 
You need to use all programs because each detects different malware. 
Make sure that you update each program to get the latest version of the database.
*http://www.malwarebytes.org/mbam.php - Malwarebytes' Anti-Malware
*http://www.superantispyware.com/ - SuperAntispyware
*http://www.safer-networking.org/en/index.html - Spybot Search & Destroy
*http://www.lavasoft.com/products/ad_aware_free.php - Ad-Aware Free
*http://www.microsoft.com/windows/products/winfamily/defender/default.mspx - Windows Defender: Home Page
See also "Spyware on Windows": http://kb.mozillazine.org/Popups_not_blocked and [[Searches are redirected to another site]]

Search Engine issue in Windows 7 32 bit

Hi,
Configuring ATG Search 10.0.1 on Windows 7 (32 bit) & getting following error while indexing.I have a 55 GB free memory space so I don't think any issues with disk space.
I am aware that ATG Search 10.x is not supported for Windows 7 32 bit system.Search with CRS is working on the same environment but erroring out for my application. Any guess on this issue?
Thanks!
Mani
20:06:43,310 WARN [IndexDeploymentService] Bad search engine 26600001 on krv for partition 26500020
20:06:43,316 ERROR [IndexDeploymentService] Message from engine on krv: Fri Jan 16 08:26:56 CST 1970 error Could not create the physical partition kernel o
bject. GetLastError returned 0.
20:06:51,537 ERROR [IndexDeploymentService] Swap failed for index 26500002; aborting swap and tearing down staging. Swap rollback Policy Description: All physic
al partitions must have at least one successfully initialized search engine.
20:06:51,562 ERROR [IndexLoader] Error occurred in step partition_step for sync task 26600004 of type complete. Error:
atg.searchadmin.core.IndexLoaderException: atg.search.SearchAdminException: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'in
dexing environment (Production)' failed. Check logs.
atg.searchadmin.core.IndexLoaderException: atg.search.SearchAdminException: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'in
dexing environment (Production)' failed. Check logs.
at atg.searchadmin.core.IndexLoader.prepareEnvironment(IndexLoader.java:676)
at atg.searchadmin.core.IndexLoader.executeSession(IndexLoader.java:1992)
at atg.searchadmin.core.IndexLoader$IndexingThread.run(IndexLoader.java:2427)
Caused by: atg.search.SearchAdminException: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'indexing environment (Production)'
failed. Check logs.
at atg.searchadmin.core.stage.PartitionDispatchStage.doInitializeSession(PartitionDispatchStage.java:322)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:143)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:149)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:149)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:149)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:149)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:149)
at atg.searchadmin.core.stage.IndexingPipelineSessionImpl.initialize(IndexingPipelineSessionImpl.java:242)
at atg.searchadmin.core.stage.IndexingPipelineSessionService.createSession(IndexingPipelineSessionService.java:121)
at atg.searchadmin.core.IndexLoader.prepareEnvironment(IndexLoader.java:666)
... 2 more
Caused by: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'indexing environment (Production)' failed. Check logs.
at atg.search.routing.IndexDeploymentService.deployIndexSynchronously(IndexDeploymentService.java:326)
at atg.search.routing.RoutingSystemService.deployIndexSynchronously(RoutingSystemService.java:2132)
at atg.searchadmin.core.stage.PartitionDispatchStage.doInitializeSession(PartitionDispatchStage.java:315)
... 11 more
Edited by: user11263665 on Sep 19, 2012 7:38 AM
Edited by: user11263665 on Sep 19, 2012 7:40 AM

I tried the ATG Search indexing in ATG 10.1.1 and I faced the same issue.
My CRS is up and running and I am able to complete an order in it. But search is not working with it.
atg.searchadmin.core.IndexLoaderException: atg.search.SearchAdminException: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'indexing environment' failed. Check logs.
     at atg.searchadmin.core.IndexLoader.prepareEnvironment(IndexLoader.java:623)
     at atg.searchadmin.core.IndexLoader.executeSession(IndexLoader.java:2069)
     at atg.searchadmin.core.IndexLoader$IndexingThread.run(IndexLoader.java:2542)
Caused by :atg.search.SearchAdminException: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'indexing environment' failed. Check logs.
     at atg.searchadmin.core.stage.PartitionDispatchStage.doInitializeSession(PartitionDispatchStage.java:326)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:169)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineSessionImpl.initialize(IndexingPipelineSessionImpl.java:296)
     at atg.searchadmin.core.stage.IndexingPipelineSessionService.createSession(IndexingPipelineSessionService.java:144)
     at atg.searchadmin.core.IndexLoader.prepareEnvironment(IndexLoader.java:613)
     at atg.searchadmin.core.IndexLoader.executeSession(IndexLoader.java:2069)
     at atg.searchadmin.core.IndexLoader$IndexingThread.run(IndexLoader.java:2542)
Caused by (#2):atg.search.routing.IndexDeploymentException: Deployment of index for environment 'indexing environment' failed. Check logs.
     at atg.search.routing.IndexDeploymentService.deployIndexSynchronously(IndexDeploymentService.java:354)
     at atg.search.routing.RoutingSystemService.deployIndexSynchronously(RoutingSystemService.java:2266)
     at atg.searchadmin.core.stage.PartitionDispatchStage.doInitializeSession(PartitionDispatchStage.java:319)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:169)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineSessionImpl.initialize(IndexingPipelineSessionImpl.java:296)
     at atg.searchadmin.core.stage.IndexingPipelineSessionService.createSession(IndexingPipelineSessionService.java:144)
     at atg.searchadmin.core.IndexLoader.prepareEnvironment(IndexLoader.java:613)
     at atg.searchadmin.core.IndexLoader.executeSession(IndexLoader.java:2069)
     at atg.searchadmin.core.IndexLoader$IndexingThread.run(IndexLoader.java:2542)

Web Site Search Engine

Hi,
I was wondering if someone could help me locate a code for a search engine of my web site. My dilemma is, most of the search engines I've located search the web. This's not what I want. I'm looking for a search engine where you can select from a menu display and then hit search and it will return results in a text format from my web site only.
If you're not sure what I'm looking for, an example can be found at www.westernvirtualairlines.com
They use the Quick Flight Search which is exactly what I would like to use.
If you enter Salt Lake City in the "from" box, and Los Angeles in the "to" box it will return the results in the same format I wish to use.
Any help would be awsome and greatly appreciated.
Thanks for any help and assistance.
jak62562

Well, you can use Google to search your own web pages, but I don't think that's quite what you're asking.
That site you pointed to undoubtedly has its own database set up, so when you click search, it searches through their database and returns any relevant data to your search. If this is what you're attempting to accomplish, there's not going to be any script you can just cut and paste and have everything magically work for you.
While you may not see it, those aren't simply HTML pages you're looking at. That site uses Active Server Pages, orASP, Microsoft's parallel to JSP (Java Server Pages). This pages query the database, organize the result and display it on the page. If you want something this complex, you'll either need to invest some serious time into learning a technology that can handle it, or hire a developer to do it for you.

Search engin, JSP or swing Application

Hello,
i want to build a search engin(like Google) on a special archive ,the question: is swing endurance the load of all search results to display it or there will be any problems in displaying, thus ,it is preferable to using ordinary web page.
thanks

A search engine usually has two pieces. One piece gathers the data and organizes it so it can be searched. The other piece queries that data and presents the results. Many times a database is used to store the searchable data. For a fairly complex set of searchable data, the limiting factor is usually the speed of the data search. It really doesn't matter if the user interface is C, Java or Fortran.

Search Engine in Java

Similar Messages

Maybe you are looking for