CRS 10.1 with Endeca 3.1.1 -home page doesn't show image and promo products

Question

Thanks to this forum and especially cmore I'm close to setting up my local dev environment, but still have something to resolve.
I have ATG 10.1.2 + CRS + Endeca 3.1.1. I installed BCC with Staging and Production (switchable) stores.
The problem is with Production CRS - both US Store and Home Store don't show image and promoted products on Home page.
What I see below Top Category pane is empty square. Unlike Production, home page on Stage works fine - I see homepage_promo_storeSiteUS_en_anonymous.png and promoted products.
I think the problem is not related to Endeca, as I currently connect both Stage and Prod to the same Endeca CRS, I also believe that issue is not related to configuration - I tried to run production with Stage configuration, and it changed nothing. I believe the issue is somewhere in data (DB) - but I did full deployment for Prod and it didn't help. I verified targeters and slots on Prod through ACC, and they looks fine. So don't know at this point what to do next.
While doing full deploy I see some errors in the log which might be related to the issue, But the same errors I see on Stage as well and still Stage works without issues:
10:17:41,805 ERROR [nucleusNamespace.] Invalid attempt to resolve component /atg/registry/Slots/HomeTheme in scope global. It is defined in scope request
10:17:41,811 ERROR [nucleusNamespace.] Invalid attempt to resolve component /atg/registry/Slots/PromotedProduct1 in scope global. It is defined in scope request
10:17:41,816 ERROR [nucleusNamespace.] Invalid attempt to resolve component /atg/registry/Slots/PromotedProduct2 in scope global. It is defined in scope request
10:17:41,819 ERROR [nucleusNamespace.] Invalid attempt to resolve component /atg/registry/Slots/PromotedProduct3 in scope global. It is defined in scope request
10:17:41,822 ERROR [nucleusNamespace.] Invalid attempt to resolve component /atg/registry/Slots/FeaturedProduct2 in scope global. It is defined in scope request
10:17:41,826 ERROR [nucleusNamespace.] Invalid attempt to resolve component /atg/registry/Slots/FeaturedProduct3 in scope global. It is defined in scope request
10:17:41,829 ERROR [nucleusNamespace.] Invalid attempt to resolve component /atg/registry/Slots/FeaturedProduct4 in scope global. It is defined in scope request
10:17:41,832 ERROR [nucleusNamespace.] Invalid attempt to resolve component /atg/registry/Slots/FeaturedProduct5 in scope global. It is defined in scope request
10:17:41,836 ERROR [nucleusNamespace.] Invalid attempt to resolve component /atg/registry/Slots/FeaturedProduct1 in scope global. It is defined in scope request
10:17:41,839 ERROR [nucleusNamespace.] Invalid attempt to resolve component /atg/registry/Slots/CategoryPromotionContent2 in scope global. It is defined in scope request
10:17:41,842 ERROR [nucleusNamespace.] Invalid attempt to resolve component /atg/registry/Slots/CategoryPromotionContent1 in scope global. It is defined in scope request
Last update: I compared Stage and Prod home pages HTML code, and notice that the first difference between two pages is absence of endeca-slot Div, the below code present on Stage, and is Not present on Prod:
It is the first difference only , there are a bunch more.
<div class="endeca-slot" zone="/content/Shared/Global Search Configuration/Search Box">
<script type="text/javascript">
if(Endeca.Site) {
Endeca.Site.Audit.contents["/content/Shared/Global Search Configuration/Search Box"] = [{"@type":"SearchBox","endeca.internal.created.by":"admin","rule.msg":"Fired","endeca.internal.last.modified.by":"admin","rule.msgcode":"13","rule.state":"fired","endeca.internal.resource.path":"\/sites-live\/NYCoCRS\/content\/Shared\/Global Search Configuration\/Search Box\/Search Box","endeca.internal.workflow.state":"ACTIVE","type":"DGraph.MerchDebug","Title":"Search Box","rule.id":"1","endeca.internal.last.modified":"2013-03-20T11:43","endeca.internal.date.created":"2013-03-20T11:43","Zone":"\/content\/Shared\/Global Search Configuration\/Search Box"}
</script>
Edited by: PKU on Apr 1, 2013 8:32 AM
Edited by: PKU on Apr 1, 2013 8:35 AM

Answer 1

After some analysis and experiments:
Most likely the issue is with Scenarios. HomeTheme Slot is populated by Theme scenario.
For some reason the slot remains empty on "bad" environment. On "good" environment the slot contain promo content item after the first call of the slot. I checked respective targeter and promo content item, and they look OK, so I believe something wrong with Scenarios.