IIS Application Pool “randomly” stopping
April 21, 2011
Posted by on
This issue has just sent me crazy for the past day. On my development machine the application pool hosting one of our webservices would randomly stop with nothing more than the following error in the System Windows Log
Application pool ‘xx’ is being automatically disabled due to a series of failures in the process(es) serving that application pool.
Which resulted in subsequent calls to the service returning
HTTP Error 503. The service is unavailable.
Any attempt to attach a debugger to the service was no help, with the debugger becoming detached every 10 seconds when w3wp.exe died.
The automatic disabling of the app pool turned out to be due to the Rapid-Fail Protection that will stop the application pool after a default of 5 worker process failures. Several suggestions say to just disable Rapid-Fail protection, but this just masks the problem to stop the 503 from occurring.
With some help from Chris Forster I dived into the world of Windows Debugging, looking for any kind of IIS exception that could be causing the worker process to keep failing, which turned up nothing.
The culprit – NServiceBus
To cut a long and boring debugging story short, NServiceBus was the culprit. Or rather the user account of the application pool not having the permission to Read or Peek messages from the message queue. On application startup NServiceBus attempts to send a message and read it from the message queue in the background, killing the process after 10 seconds if it is not able to read it back from the queue.