Site could not be crawled and connect in server

Question

I have multiple web applications/sites on this server/farm. I cannot crawl one site. I got the error message in the crawl log as below after performing the Full Crawl:
“This item could not be crawled because the repository did not respond within the specified timeout period. Try to crawl the repository at a later time, or increase the timeout value on the Proxy and Timeout page in search administration.
You might also want to crawl this repository during off-peak usage times.”
I have followed the suggestion on the web do the following
(1)    
Select the "Ignore SSL errors" on the Farm Search Administration page.
(2)    
No Proxy server. I did increase the Time-out to 120.
(3)    
Internet Options>Connections>LAN settings. Uncheck ‘Automatically detect settings’.
(4)    
DisableLoopbackCheck in the registry
After the above changes, I still get the same problem.
 Although the service account has full access to anything, I cannot connect this web site via IE from the server (while I can connect to other sites). The only different is SSL is applied to this site. I will appreciate if someone can help
me out.

Answer 1

Hi,
According to your post, my understanding is that you get an error message in the crawl log  after performing the Full Crawl.
I recommend that you can try to check and test the following things in your environment.
  1.  The first investigation should be to check if you can login to the page via your Brower or with Telnet.
    Open a Command Prompt and hit Telnet gokanx.wordpress.extranet:80. I received a “connection failed” on my black box. This was meaning that your couldn’t reach your site from your internal server.
http://gokanx.wordpress.com/2013/06/19/how-to-bypass-the-firewall-tmg-and-proxy-during-a-content-crawl-in-sharepoint-publishing-farm/
  2.  Go to SharePoint 2013 Central Administration, click “Search Service Application” in “Manage service application”, click the “Index Reset” to reset all crawled content, then start Full Crawl again.
Opening IE with Search Service Account on Service and try to open the Sites to crawl.
https://social.technet.microsoft.com/Forums/en-US/0e1f7601-b4b3-446a-9eab-ed6c79faa8e2/sharepoint-search-crawl-not-working-this-item-could-not-be-crawled-because-the-repository-did-not?forum=sharepointadminprevious
  3.  In Command prompt run “netsh winhttp show proxy”. If you see there is a proxy server, which can be old or not working, we need to reset this setting.
  Run the command “netsh winhttp reset proxy” to Reset the proxy and again do a Show command, you should get “No Proxy Server” screen.
  In Order these are the steps.
  a. netsh winhttp show proxy
  b. netsh winhttp reset proxy
  c. Net stop Osearch
  d. Net start Osearch
You can also check if there are any Proxy Settings in IE Configuration, try to remove the Proxy Settings configured in the IE.
http://sharepoint-works.blogspot.com/2012/10/this-item-could-not-be-crawled-because.html
http://sharepointmadeeasy.blogspot.com/2012/07/search-crawl-time-out-or-proxy-issue.html
  4.  Try to import the certificate to the Certificate store of the crawler (and the SharePoint Cert Store [Central Admin > Security > Manage Trust and upload the root certifcate of the repository])
http://sharepoint.stackexchange.com/questions/58946/could-not-establish-trust-relationship-for-the-ssl-tls-secure-channel
  5.  Add the following rules to the proxy bypass list and reboot the servers.  To do this run command prompt as an admin and use the following command:
netsh winhttp set proxy xxx.xxx.xxx.xxx:portnumber bypass-list=”<local>,sharepoint”
http://andrewroth1.wordpress.com/2012/09/11/sharepoint-2010-search-service-local-sharepoint-sites-cannot-be-crawled-error/
If this issue still exists, you can check the ULS log to find more information about this issue.
For SharePoint 2013, by default, ULS log is at C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\LOGS.
Thanks,
Yumi Fu
Forum Support
Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Subscriber Support, contact
[email protected].