Incoming Email not working for one web app, but IS working for others in same farm.

Question

I have enabled incoming email in a large multi-web application farm running SharePoint 2013 SP1 and the Feb 2015/March 2015 CUs that were pushed down in Windows Updates.  It works just fine for two web applications, but it will not work for one particular
web application. I see the email land in the /drop folder. If I remove the email address from the library in http://nonworkingapp web
application and use it on a library in a http://working web application, then the email is processed successfully. Conversely,
if I take an address from a library in http://working and move it to a library inhttp://nonworkingapp it
does not work. I've tried multiple site collections on http://nonworkingapp. This rules out any SMTP issues, etc. and means
the issue is bound to this web application. Permissions are the same between the "non-working" and "working" locations: domain users are members on the site, and the library inherits permissions from the site.  The App Pool and Timer
Service accounts have rights to the folder and to the library.
The error in the ULS log is the typical error processing message, but it gives no additional information even though I have enabled Verbose logging.
The Incoming E-Mail service has completed a batch. The elapsed time was 00:00: 00.0156294. The service processed 2 message(s) in total. Errors occurred processing 2 message(s): Message ID: Message
ID:
Typically, you see "alias not found" or some permissions or quota issue here, but this issue is NOT the same as those. In this case NO additional information is provided in ULS.
Edit: running stsadm.exe -o refreshdms -url http://one/failing/site fixed some similar issues where "Unknown alias" was
reported even though the list did have an alias (see here:http://blogs.technet.com/b/vinitt/archive/2009/07/15/e-mail-enabled-list-alias-information-is-not-synchronized-between-configuration-database-and-content-database.aspx).
This did not fix the issue for the case mentioned above.
What could possibly be the issue preventing incoming email from working on this one web application?

Answer 1

Thanks for your reply.  Incoming E-Mail was running on all 3 WFE, however I do not think this was the cause.  As noted, _other_ emails send to http://workingapp are working just fine.  I can send two emails - one to a library on http://workingapp
and another on http://notworkingapp.  The email to http://notworkingapp will sit in the drop folder, while the one to http://workingapp will be processed correctly.  In addition, the ULS log on WFE1 indicates an attempt to process the file fails.
 So clearly, the timer job on WFE1 is running and attempting to process the file.  For kicks, I disabled the service on WFE2 and WFE3, but with no change in the behavior.
WRT to #1, I have done this already (as noted in the original post).  I can remove an alias from http://workingapp and place it on http://nonworkingapp and it is not processed.  If I return it to http://workingapp it is processed.  In other
words, I go into Library Settings -> Incoming Email for a working library, and copy the alias, then disable incoming email for that library.  I then go to Library Settings -> Incoming Email for a non-working library, enable incoming email, and paste
the alias.  Once done, emails sent to the alias are not processed.  I then do the reverse, and it works again.
On #2, I have already tried multiple libraries on http://nonworkingapp (also as noted in the original post).
In my environment, incoming email does not work for any library on http://nonworkingapp, but does work for any library on http://workingapp.  Troubleshooting listed above has ruled out any problem with basic mail server configuration or library settings
- the problem is bound to http://nonworkingapp
I'm not sure what you mean by "steps to reproduce the issue".  If I knew that, I would probably be able to solve the issue.  The best I could come up with to describe the situation would be:
Set up SP 2010 in a three-node configuration with multiple web apps
Enable incoming email and confirm it works
Migrate to a SP2013 SP1 farm in a three node configuration.
Enable a new email alias for a library
Observe that it doesn't work for one web application.
Obviously, it's pretty unlikely that attempting that in a lab would actually have the same result.