How to optimize the Magento 2 queue consumers

If your are having problem with the magento 2 cron jobs and it gets stuck or long-running you can try to optimize the way magento handles the queue consumers.

By default consumers should continue polling for messages if the number of processed messages is less than the max_messages value (which can be specified in the env.php file). This is actually only recommended for large merchants where a constant message flow is expected.

For smaller merchants it is recommended that the consumers process closes the TCP connection and terminate without waiting for additional messages to enter the queue. To changes the default behavior add/change your env.php with the following instructions

'queue' => [
    'consumers_wait_for_messages' => 0,
]

Additionally, if your shops runs on magento 2.4.1 or higher, it is possible to reduce the CPU and memory usage even more by adding the only_spawn_when_message_available parameter to the queue configuration array. If this parameter is set to true or 1 the consumer will only run when a message is available in the queue and it will be terminated when there are no more messages to process. Configuration would be as follow:

'queue' => [
    'consumers_wait_for_messages' => 0,
    'only_spawn_when_message_available' => 1
]

Leave a Reply

Your email address will not be published. Required fields are marked *