Investigating Stuck Messages in RabbitMQ
Overview
When sending messages and provisioning requests through RabbitMQ, they are placed in a queue. This guide demonstrates the actions you should take to investigate and resolve stuck messages within the queue.
Unacked Messages in RabbitMQ refer to unacknowledged messages (Messages that have been sent but have not successfully completed and are stuck in processing).
Investigating and Releasing Unacked Messages
-
Stop the AtriaProvisioningService: Navigate to Windows Services or Task Manager and stop the AtriaProvisioningService. This will change the stuck message to have a Ready status.
-
Run SQL Profiler on the Database Server: This will help us to identify what will be processed.
-
Start the AtriaProvisioningService. Navigate to Windows Services or Task Manager and start the AtriaProvisioningService. This will try to re-run the message that was stuck.
-
Review the SQL Profiler Tracing: This will display information about the action that was trying to be executed. this might help to identify the reason why it's not being processed. Take note of this and we will skip the stuck message on the next step.
-
Stop the AtriaProvisioningService
-
Locate the Stuck Message: In RabbitMQ, open the queue where the stuck message is located.
-
Configure the Settings: Set the Ack Mode to
Automatic ack
, and the Messages field to1
:
-
Click Get Message(s): This will automatically acknowledge the first message in the queue which is the one getting stuck.
-
Start the AtriaProvisioningService: This will continue to process the other messages:
- Review and Provision: For the manually acknowledged message(s), the Provisioning Request Log will still show it as In Progress. This allows for further investigation as needed.
Once the review on the provisioning requests are done, you will need to re-provision for any skipped messages.
if there are multiple stuck messages, repeat the steps again for each stuck message.