Processing large volumes of data in a long-running task is neither performant nor scalable. If you've ever had to bulk update entities with an update hook and Batch API, or with a script run via drush, you know what I'm talking about. If you've ever had to send thousands of emails with Drupal and a hook_cron() implementation, you know how tricky that can be to process them all in a reasonable amount of time.

What if there was another way to process lots of jobs in a fast, safe, and scalable manner? In this session, I will introduce you to Drupal's Queue API and the database queue, which are baked into Drupal core. We'll discuss:

When to use queues
Types of database queues in Drupal
Queue workers, including how to build your own scalable, dynamic queue workers
Automated testing of queues and queue workers

Erich Beyrent of SciShield

https://www.drupalasheville.com/2024/session/getting-stuff-done-working-queues-drupal