Java Search Engine?

Does anyone know whether it would be possible to write a web based search engine such as google in java? I don't want detailed information, but does anyone know what kind of equipment/software I would need, and the difficulty it poses. Any relevent URLs appreciated.
cheers

The reason why I am enquiring about this is that I'm thinking of carrying out this task for my CS project. Is it sufficiently difficult? Or even possible in a 6 month or so period, part time?I want all the code to be my own and not using any current engines, if that makes any sense.
Does any1 know how the current engines on www such as google, ask.com are written?Relevent info appreciated.
Cheers

Similar Messages

Search Engine in Java

Hi All,
I just wonder anyone can lend me a hand please. I am using Java to develop a search engine at the moment, I need to store all the words from a single html page, and I have done this, but the thing is how can I get all pages from a certain intranet? Thank you in advance.
Best Regards,
Ray

As per my knowledge any search engine while use the meta data of a html page first then go for the whole contents.
Ans it is always better to use google or any other searching tool instead of developing it from scratch. it is good only for practice.

Two issues: Mail dotcom is now telling me to enable java in my browser and firefox has issues w/duckduckgo search engine. I am not allowed 2 instal DDG n search

2 issues: After a windows update on 5 10 2012, maildotcom says I need to enable java in my browser, when as far as I can tell, it is.
Second issue is, I am not able to add duckduckgo as my preferred search engine. I get a pop up in search tool add-ons that says "sorry, you need a Mozilla based browser to install a search plug in."
Why and how can these irritations be fixed?

'''''NoScript''''' 
*To add a site to the NoScript "whitelist" ('''always allow''') --> when on the site, click the NoScript icon and choose "Allow ''site''"; the site will be placed on the NoScript Options > Whitelist (non-italicized).
*If you click "Temporarily allow ''site''", the site will be allowed during the Firefox current session (until you close/exit Firefox) and will be placed on the NoScript Options > Whitelist (italicized) and will be removed from the whitelist when you close/exit Firefox.
*NoScript Whitelist --> click the NoScript icon, choose Options, click the Whitelist tab
'''''DuckDuckGo''''' (you probably would want 1 & 3 or 2 & 3) 
#This adds a DDG '''toolbar''' and says it adds the "''encrypted (HTTPS / SSL) version of DuckDuckGo in your '''search bar'''"'' --> https://addons.mozilla.org/en-US/firefox/addon/duckduckgo-ssl/
#You can '''add only an item to the Search Bar''' (without a toolbar) by choosing one of the items here --> http://mycroft.mozdev.org/search-engines.html?name=duckduckgo&opensearch=yes
#You can '''change the Location Bar search''' (replacing the default Google search) by following these instructions --> http://ilias.ca/blog/2012/03/how-to-make-firefox-use-duckduckgo/
This DDG page says there is some problem installing DDG plugins but does not explain the problem or who is responsible for fixing the issue
*http://help.duckduckgo.com/customer/portal/articles/216441-firefox
'''If this reply solves your problem, please click "Solved It" next to this reply when signed-in to the forum.'''
Not related to your question, but...
You may need to update some plug-ins. Check your plug-ins and update as necessary:
*Plug-in check --> http://www.mozilla.org/en-US/plugincheck/
*Adobe Shockwave for Director Netscape plug-in: [https://support.mozilla.org/en-US/kb/Using%20the%20Shockwave%20plugin%20with%20Firefox#w_installing-shockwave Installing ('''''or Updating''''') the Shockwave plugin with Firefox]
*'''''Adobe PDF Plug-In For Firefox and Netscape''''': [https://support.mozilla.org/en-US/kb/Using%20the%20Adobe%20Reader%20plugin%20with%20Firefox#w_installing-and-updating-adobe-reader Installing/Updating Adobe Reader in Firefox]
*Shockwave Flash (Adobe Flash or Flash): [https://support.mozilla.org/en-US/kb/Managing%20the%20Flash%20plugin#w_updating-flash Updating Flash in Firefox]
*Next Generation Java Plug-in for Mozilla browsers: [https://support.mozilla.org/en-US/kb/Using%20the%20Java%20plugin%20with%20Firefox#w_installing-or-updating-java Installing or Updating Java in Firefox]

Search engine machine in java

I would like to give me some general ideas about how to implement a search engine machine implemented in Java. Please help me by giving some standard algorithms or methods or a site tha contains similar source code

String.indexOf(String);
Questions to ask yourself:
Are you interested in speed or quality of the search?
Do you need an intelligent search or a simple search?
How is the data being stored and how can you access that data?

How to interact with google search engine using java HTTP POST

I am trying to develop an desktop app that acts as an online radio tuner. I want to use google as an index site for actually searching for radio stations online. So I was wandering if it is possible for me to use java to actually interface with the google search engine servers. For example, I have built a gui that allows you to enter search text. This gui will connect to the web. I would like it to connect to the Google site and post the query to the google web servers. Then i would like to began to parse the response( html page) that i get back from the google servers. Like using google without actually typing in the text field on the google site. Can anyone tell me if it is possible to do this with java. If so, what would be the best way to go about doing this. Thanks.

Thanks for the response. It was actually alot easier to do than I anticipated. All I had to do was create a URL object initialized with the google url(search text embedded in the url - http://www.google.com/search?q=" + searchText + "&hl=en&ie=UTF-8&oe=UTF8).
Looks like this:
String google = http://www.google.com/search?q=" + searchText + "&hl=en&ie=UTF-8&oe=UTF8;
URL url = new URL(google); // creating Google Url object
URLConnection connection = url.openConnection();
       connection.setDoOutput(true);
       connection.setRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)" ); // fooling google servers into thinking the request is coming from a real browser
       BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));The content returned by the google server can be accessed calling the BufferedReader class methods.

Whenever I use a search engine, I keep getting the error message "The connection was reset while the page was loading".

I am able to access the internet, but none of the search engines like Google, Yahoo or Bing will work. the page will not load. I get the error message:
The connection to the server was reset while the page was loading.
* The site could be temporarily unavailable or too busy. Try again in a few moments.
* If you are unable to load any pages, check your computer's network connection.
* If your computer or network is protected by a firewall or proxy, make sure that Firefox is permitted to access the Web.

Your plugins list shows two Flash plugins and other outdated plugin(s) with known security and stability risks.
# Shockwave Flash 10.0 r45
# Shockwave Flash 10.1 r53
# Adobe Shockwave for Director Netscape plug-in, version 11.0
# Next Generation Java Plug-in 1.6.0_19 for Mozilla browsers
Flash Player uninstall: http://www.adobe.com/go/tn_14157 (this will remove the Firefox Flash plugin and the ActiveX control for IE)
Update the [[Flash]] and [[Shockwave|Shockwave for Director]] plugin to the latest version.
*http://www.adobe.com/software/flash/about/
*http://www.adobe.com/shockwave/welcome/
*http://www.adobe.com/downloads/
Update the [[Java]] plugin to the latest version.
*http://java.sun.com/javase/downloads/index.jsp (Java Platform: Download JRE)
Do a malware check with a few malware scan programs. 
You need to use all programs because each detects different malware. 
Make sure that you update each program to get the latest version of the database.
*http://www.malwarebytes.org/mbam.php - Malwarebytes' Anti-Malware
*http://www.superantispyware.com/ - SuperAntispyware
*http://www.safer-networking.org/en/index.html - Spybot Search & Destroy
*http://www.lavasoft.com/products/ad_aware_free.php - Ad-Aware Free
*http://www.microsoft.com/windows/products/winfamily/defender/default.mspx - Windows Defender: Home Page
See also "Spyware on Windows": http://kb.mozillazine.org/Popups_not_blocked and [[Searches are redirected to another site]]

Search Engine issue in Windows 7 32 bit

Hi,
Configuring ATG Search 10.0.1 on Windows 7 (32 bit) & getting following error while indexing.I have a 55 GB free memory space so I don't think any issues with disk space.
I am aware that ATG Search 10.x is not supported for Windows 7 32 bit system.Search with CRS is working on the same environment but erroring out for my application. Any guess on this issue?
Thanks!
Mani
20:06:43,310 WARN [IndexDeploymentService] Bad search engine 26600001 on krv for partition 26500020
20:06:43,316 ERROR [IndexDeploymentService] Message from engine on krv: Fri Jan 16 08:26:56 CST 1970 error Could not create the physical partition kernel o
bject. GetLastError returned 0.
20:06:51,537 ERROR [IndexDeploymentService] Swap failed for index 26500002; aborting swap and tearing down staging. Swap rollback Policy Description: All physic
al partitions must have at least one successfully initialized search engine.
20:06:51,562 ERROR [IndexLoader] Error occurred in step partition_step for sync task 26600004 of type complete. Error:
atg.searchadmin.core.IndexLoaderException: atg.search.SearchAdminException: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'in
dexing environment (Production)' failed. Check logs.
atg.searchadmin.core.IndexLoaderException: atg.search.SearchAdminException: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'in
dexing environment (Production)' failed. Check logs.
at atg.searchadmin.core.IndexLoader.prepareEnvironment(IndexLoader.java:676)
at atg.searchadmin.core.IndexLoader.executeSession(IndexLoader.java:1992)
at atg.searchadmin.core.IndexLoader$IndexingThread.run(IndexLoader.java:2427)
Caused by: atg.search.SearchAdminException: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'indexing environment (Production)'
failed. Check logs.
at atg.searchadmin.core.stage.PartitionDispatchStage.doInitializeSession(PartitionDispatchStage.java:322)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:143)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:149)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:149)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:149)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:149)
at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:149)
at atg.searchadmin.core.stage.IndexingPipelineSessionImpl.initialize(IndexingPipelineSessionImpl.java:242)
at atg.searchadmin.core.stage.IndexingPipelineSessionService.createSession(IndexingPipelineSessionService.java:121)
at atg.searchadmin.core.IndexLoader.prepareEnvironment(IndexLoader.java:666)
... 2 more
Caused by: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'indexing environment (Production)' failed. Check logs.
at atg.search.routing.IndexDeploymentService.deployIndexSynchronously(IndexDeploymentService.java:326)
at atg.search.routing.RoutingSystemService.deployIndexSynchronously(RoutingSystemService.java:2132)
at atg.searchadmin.core.stage.PartitionDispatchStage.doInitializeSession(PartitionDispatchStage.java:315)
... 11 more
Edited by: user11263665 on Sep 19, 2012 7:38 AM
Edited by: user11263665 on Sep 19, 2012 7:40 AM

I tried the ATG Search indexing in ATG 10.1.1 and I faced the same issue.
My CRS is up and running and I am able to complete an order in it. But search is not working with it.
atg.searchadmin.core.IndexLoaderException: atg.search.SearchAdminException: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'indexing environment' failed. Check logs.
     at atg.searchadmin.core.IndexLoader.prepareEnvironment(IndexLoader.java:623)
     at atg.searchadmin.core.IndexLoader.executeSession(IndexLoader.java:2069)
     at atg.searchadmin.core.IndexLoader$IndexingThread.run(IndexLoader.java:2542)
Caused by :atg.search.SearchAdminException: atg.search.routing.IndexDeploymentException: Deployment of index for environment 'indexing environment' failed. Check logs.
     at atg.searchadmin.core.stage.PartitionDispatchStage.doInitializeSession(PartitionDispatchStage.java:326)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:169)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineSessionImpl.initialize(IndexingPipelineSessionImpl.java:296)
     at atg.searchadmin.core.stage.IndexingPipelineSessionService.createSession(IndexingPipelineSessionService.java:144)
     at atg.searchadmin.core.IndexLoader.prepareEnvironment(IndexLoader.java:613)
     at atg.searchadmin.core.IndexLoader.executeSession(IndexLoader.java:2069)
     at atg.searchadmin.core.IndexLoader$IndexingThread.run(IndexLoader.java:2542)
Caused by (#2):atg.search.routing.IndexDeploymentException: Deployment of index for environment 'indexing environment' failed. Check logs.
     at atg.search.routing.IndexDeploymentService.deployIndexSynchronously(IndexDeploymentService.java:354)
     at atg.search.routing.RoutingSystemService.deployIndexSynchronously(RoutingSystemService.java:2266)
     at atg.searchadmin.core.stage.PartitionDispatchStage.doInitializeSession(PartitionDispatchStage.java:319)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:169)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineStage2Impl.initializeSession(IndexingPipelineStage2Impl.java:175)
     at atg.searchadmin.core.stage.IndexingPipelineSessionImpl.initialize(IndexingPipelineSessionImpl.java:296)
     at atg.searchadmin.core.stage.IndexingPipelineSessionService.createSession(IndexingPipelineSessionService.java:144)
     at atg.searchadmin.core.IndexLoader.prepareEnvironment(IndexLoader.java:613)
     at atg.searchadmin.core.IndexLoader.executeSession(IndexLoader.java:2069)
     at atg.searchadmin.core.IndexLoader$IndexingThread.run(IndexLoader.java:2542)

Web Site Search Engine

Hi,
I was wondering if someone could help me locate a code for a search engine of my web site. My dilemma is, most of the search engines I've located search the web. This's not what I want. I'm looking for a search engine where you can select from a menu display and then hit search and it will return results in a text format from my web site only.
If you're not sure what I'm looking for, an example can be found at www.westernvirtualairlines.com
They use the Quick Flight Search which is exactly what I would like to use.
If you enter Salt Lake City in the "from" box, and Los Angeles in the "to" box it will return the results in the same format I wish to use.
Any help would be awsome and greatly appreciated.
Thanks for any help and assistance.
jak62562

Well, you can use Google to search your own web pages, but I don't think that's quite what you're asking.
That site you pointed to undoubtedly has its own database set up, so when you click search, it searches through their database and returns any relevant data to your search. If this is what you're attempting to accomplish, there's not going to be any script you can just cut and paste and have everything magically work for you.
While you may not see it, those aren't simply HTML pages you're looking at. That site uses Active Server Pages, orASP, Microsoft's parallel to JSP (Java Server Pages). This pages query the database, organize the result and display it on the page. If you want something this complex, you'll either need to invest some serious time into learning a technology that can handle it, or hire a developer to do it for you.

Search engin, JSP or swing Application

Hello,
i want to build a search engin(like Google) on a special archive ,the question: is swing endurance the load of all search results to display it or there will be any problems in displaying, thus ,it is preferable to using ordinary web page.
thanks

A search engine usually has two pieces. One piece gathers the data and organizes it so it can be searched. The other piece queries that data and presents the results. Many times a database is used to store the searchable data. For a fairly complex set of searchable data, the limiting factor is usually the speed of the data search. It really doesn't matter if the user interface is C, Java or Fortran.

Our MAC Books have issues loading search engines like Google, when on the Corp. network, all other browsers work fine.

Our MAC users have trouble loading search engines on our Corp. network when going through our FWs. The browser works fine when they are on our guest network or out side the office.

system details are incorrect.... i am using FF4.0.1 as on the about firefox tab
Application Basics
Name
Firefox
Version
4.0.1
User Agent
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.13) Gecko/2009073022 Firefox/3.0.13 (.NET CLR 3.5.30729)
Profile Directory
Open Containing Folder
Enabled Plugins
about:plugins
Build Configuration
about:buildconfig
Extensions
Name
Version
Enabled
ID
Microsoft .NET Framework Assistant
1.2.1
false
{20a82645-c095-46ed-80e3-08825760534b}
Java Console
6.0.20
true
{CAFEEFAC-0016-0000-0020-ABCDEFFEDCBA}
Java Quick Starter
1.0
true
[email protected]
BullGuard Safe Browsing Toolbar
1.0
true
antiphishing@bullguard
PC Sync 2 Synchronisation Extension
1.0.0.713
false
[email protected]
Modified Preferences
Name
Value
accessibility.typeaheadfind.flashBar
0
browser.history_expire_days.mirror
180
browser.history_expire_days_min
5
browser.places.importBookmarksHTML
false
browser.places.importDefaults
false
browser.places.leftPaneFolderId
-1
browser.places.migratePostDataAnnotations
false
browser.places.smartBookmarksVersion
2
browser.places.updateRecentTagsUri
false

3rd Party Search Engine Integration - some property not rendering

Hi,
I have tried to integrate a 3rd Party Search Engine into TREX following the published example.
I have successfully integrated and search results from the 3rd party engine are showing up.
However, some properties like cm_description, cm_contentlength, cm_modifedby are not rendering; their value are null; the search layout are specified correctly.
I only managed to set
1) cm_displayname
2) cm_contenttype
3) cm_id
and are showing up.
The setting of the rest of the properties are done in the same way and put into the propertymap.
Any idea why are the not rendering?? Any help appreciated. Thanks!

Hello Thilo,
I tried to deploy the original package com.customer.search.par, but unfortunately I got also an error with this package.
The disadvantage is, that I don't get an error message during start-up in one of the logger - for cause analysis.
I got the following error message within KM:
java.lang.NullPointerException
     at com.sapportals.wcm.repository.runtime.CmSystem.getResourceImpl (CmSystem.java:774)
     at com.sapportals.wcm.repository.runtime.CmSystem.getResource (CmSystem.java:325)
     at com.sapportals.wcm.repository.runtime.CmAdapter.getResource (CmAdapter.java:81)
     at com.sapportals.wcm.portal.service.KMServiceImpl.afterInit (KMServiceImpl.java:260)
     at com.sapportals.portal.prt.core.broker.PortalServiceItem.__initServiceInstanceStep2(PortalServiceItem.java:736)
What can I do at this point?
Best regards,
Walter

How to create search engine in JSF page with AJAX

I want to create Java Server Faces page with search engine using AJAX. I'm interested can I do this with PL/SQL statement which searches strings into all Oracle tables at database level? What are the best practices in this case?

user10484841 wrote:
I want to create Java Server Faces page with search engine using AJAX.
I'm interested can I do this with PL/SQL statement which searches strings into all Oracle tables at database level? I give up.
Can you?
What are the best practices in this case?Best practice is to know where data is stored to avoid search of all Oracle tables.

Offline search engine based on Oracle Text and XML SDK?

Hi,
I am pretty new to Oracle Text, so I am not shure if the
question is correct.
We have an intranet, with lots of MS Word, PDF and HTML files.
The site is starting to get out of hand, there is to many
information distributed all over it.
We would need a search engine for these files. The front would
be HTML or XML based on XSQL. The backend, that is the search
engine would look like this: I would store the HTML, PDF and
Word files inf the file system and regularly reload them into
Oracle into BFILE columns. I could user Oracle Text to index the
content and use XDK to make the searching part for it.
Is the Oracle Text feasable?
Every hint would be highly appreciated.
Tamas Szecsy

We would need a search engine for these files. The front would
be HTML or XML based on XSQL. The backend, that is the search
engine would look like this: I would store the HTML, PDF and
Word files inf the file system and regularly reload them into
Oracle into BFILE columns. I could user Oracle Text to index the
content and use XDK to make the searching part for it.
Is the Oracle Text feasable?You can store HTML, PDF and Word documents in the file system or
in a database column and make it searchable using Oracle Text.
You can write a JSP or PSP that executes the SQL statement and
then displays the result. Take a look at the Indagine sample
code (Java - it's a little bit old: version 8.1.5) or the "Text
search with PL/SQL Server Pages" notes. All available from
otn.oracle.com/products/text -> sample code.

Online search engine code using Spring+JDBCTemplate

Hi ,
I have a requirement to create the search engine using Spring and JDBCDao Support.
So, any one can help me in coding.
hope for help,
thanks and regards
vasan

Hi,
I think to export the logic to the beans will not help a lot, because the .jsp and the beans are compiled together to a servlet anyway. And so, it doesn`t really matter, where the logic has been befor, after compilation it will be in the servlet.. But to have java code logic in the bean and not in the jsp is a good idea anyway. It is always nice to have as few scriptlet code as possible, makes the application more readable..
I think you should better have a closer look to the search logic, and the database. For such huge data amount, the Oracle database would be the best. And then let the database do as many of the search work as possible, because java is a little slow, and the database will be allways faster for this things of work..
lexip

6.1 search engine customization doc error

I'm not sure who to report this to, but there seems to be an error in the documentation for the search engine customization examples.
The page is at http://docs.sun.com/source/817-1831-10/agsearch.html
The section in question says, "The following sample code lists the top ten articles on Java Web Services on a site"
The first line of HTML/JSP says:
<s1ws:search Collection="Articles" Query="Java Web Services" />
Note that "Collection" and "Query" are uppercase.
If you copy that whole example block into a JSP and run it, you get server errors.
On my machine (Win 2000), the error messages say the attributes are invalid according to the TLD. In the TLD, they're lowercase and if I change them to lowercase, the page runs (after having changed the collection to match my local config.)
If that saves anybody any headache time, so be it.
Dave

Thanks for the note, David. It's been forwarded along.

Java Search Engine?

Similar Messages

Maybe you are looking for