Queue basics — moving heavy work out of the request
The previous chapter separated the processing that follows order confirmation out of place(). The wait has not changed. The listener runs synchronously, so no response comes back until the mail is sent.
This chapter moves everything on the far side of that line outside the request. The last problem raised in chapter 1 is solved here.
What changes when you move it out of the request
The current flow looks like this.
Receive the request → create the order → send the mail → return the response
~~~~~~~~~~~~~~ the user is waiting
Put a queue in the middle and it becomes this.
Receive the request → create the order → record what to do → return the response
↓
a worker picks it up → sends the mail
Just recording what to do finishes in an instant. The actual sending is picked up and executed later by a separate process. The user does not wait for it.
The other problem from the previous chapter is solved here too. With the mail server down, a synchronous listener turned the order API into a 500. With a queue, a failed job is left on record and can be re-run later. The order stays successful and only the mail is retried.
Putting the listener on a queue
The change is one line.
use Illuminate\Contracts\Queue\ShouldQueue;
class SendOrderConfirmation implements ShouldQueue
{
public function handle(OrderPlaced $event): void
{
$event->order->user->notify(new OrderConfirmed($event->order));
}
}
Implement ShouldQueue and Laravel sends this listener to a queue instead of calling it directly1. The body of handle() is unchanged. Because we drew the line in the previous chapter, moving it takes only this.
Put SerializesModels back on the event
When sending to a queue, Laravel converts the event into a string and stores it. Putting the whole Order model in there causes trouble.
The values as of the moment of storage are recorded wholesale, so if the order's state changes before the worker picks it up, it is processed with the stale values. The data also gets large.
Put back the SerializesModels we removed in the previous chapter.
use Illuminate\Queue\SerializesModels;
class OrderPlaced
{
use SerializesModels;
public function __construct(
public readonly Order $order,
) {}
}
This trait collapses a model down to its primary key for storage and re-reads it on retrieval2. The values as of when the worker processes it are what get used.
Because it is re-read by primary key, relationships loaded before storage are lost. Write $event->order->user and a fresh query fires inside the worker at that point.
Chapter 8 enabled Model::preventLazyLoading(! $this->app->isProduction()). Run a worker locally or in staging and that line throws.
Tests do not hit it. That check is written to skip a model that was just created, and Order::factory()->create() produces exactly that. It is the same state that makes chapter 14's wasRecentlyCreated true. A restored model is a "row that was read," so it is not skipped. The result is tests passing while only the worker fails.
Load the relationships you need with $order->load('user') before creating the event, or fetch them explicitly on the worker side.
Because it re-reads, if the target row is gone by the time the job runs, the model is not found and the job fails. For something rarely deleted, like an order, this is not a problem.
For temporary data, or something users can delete, declaring public $deleteWhenMissingModels = true; means it is quietly discarded rather than recorded as a failure2. It suits processing where "if the target is gone, there is nothing to do."
Write it on the class that goes on the queue. That is inside the Job class we build next, or inside the queued listener. Not the event class.
Running a worker
Jobs are being recorded now. Nobody is executing them yet.
php artisan queue:work
That command is the process that watches the queue and processes jobs in order2. Leave it running.
This is easy to forget during development. Without a worker running, the order API gets faster and not a single email arrives. Nothing errors. It is "recorded, but nobody picked it up."
Choosing where to record
The recording destination is set by the driver, switched with QUEUE_CONNECTION in .env.
| Driver | Destination | When to use it |
|---|---|---|
sync | Nowhere | Executes on the spot. The same as not using a queue |
database | The jobs table | Small to medium scale. No extra middleware required |
redis | Redis | Larger scale. Faster retrieval |
Laravel 13's .env.example has QUEUE_CONNECTION=database. This series stays with that. The jobs table was created back when you ran the migrations in chapter 3; that is the "we use it in chapter 16" note from then.
sync is the "do not use a queue" setting. The name suggests asynchrony; it is the opposite. It executes on the spot when you dispatch. It is the default in the test environment, which matters for the test behavior covered below.
Choosing Redis comes down to job volume and latency requirements. database reads and writes a table for every job it takes, so as the count grows it becomes database load. You do not need to start with Redis. You can move once it hurts.
Creating a Job class
A listener reacts to an event. Sometimes you want to say "do this later" with no event involved. For that you create a Job class.
Let us write "send a shipping instruction to the warehouse system" from the previous chapter.
php artisan make:job NotifyWarehouse
<?php
namespace App\Jobs;
use App\Models\Order;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
use Illuminate\Support\Facades\Http;
class NotifyWarehouse implements ShouldQueue
{
use Queueable;
public function __construct(
public readonly Order $order,
) {}
public function handle(): void
{
Http::post('https://warehouse.example.com/shipments', [
'order_id' => $this->order->id,
'items' => $this->order->items->map(fn ($item) => [
'product_id' => $item->product_id,
'quantity' => $item->quantity,
]),
])->throw();
}
}
You put it on the queue with dispatch. You have to decide where to dispatch from. A shipping instruction also reacts to "an order was confirmed," so as in the previous chapter we make a listener.
<?php
namespace App\Listeners;
use App\Events\OrderPlaced;
use App\Jobs\NotifyWarehouse;
class RequestShipment
{
public function handle(OrderPlaced $event): void
{
NotifyWarehouse::dispatch($event->order);
}
}
This listener does not implement ShouldQueue, because it does not need to. dispatch only records the job and finishes instantly. The heavy work is inside NotifyWarehouse, which is already asynchronous. Put the listener on a queue too and you are queueing a job to queue a job.
That is the difference from SendOrderConfirmation in the previous chapter. That one sends mail directly inside handle(), so the listener itself had to be asynchronous.
There is no SerializesModels written here. It is included in the Queueable trait that make:job adds. Events do not include it, which is why we added it ourselves in the previous section. That is why the syntax differs between jobs and events.
Note the ->throw(). Even if the external API returns 500, Http::post() does not throw on its own. Without it, the job vanishes as a success while having failed. A job's success or failure is decided by whether it throws.
Listener or Job?
The call comes down to what the trigger is.
- Reacting to "an order was confirmed" → a listener. The announcer does not know its listeners
- Asking by name for "do this later" → a Job. The caller knows the target
A shipping instruction is the former, so a listener receives it and dispatches the Job. The listener stays light while the heavy work is carved off into its own unit.
Dispatching a Job directly is for a clear trigger. Calling NotifyWarehouse::dispatch($order) from a "resend" button in an admin screen needs no event in between.
Do not dispatch inside a transaction
Chapter 14 said not to dispatch to a queue inside a transaction. Here is why.
DB::transaction(function () use ($user, $items) {
$order = Order::create([...]);
NotifyWarehouse::dispatch($order); // not committed yet
// ... creating the lines ...
});
The worker is a separate process. It cannot see inside this transaction. If the job is recorded before the commit, the instant the worker picks it up it goes looking for an order that does not exist yet. SerializesModels re-reads by primary key, so it fails with "not found."
Depending on timing it succeeds or fails. That shape means it does not reproduce locally and only fails during production's busy hours.
A rollback makes it worse. The order never happened, and only the shipping instruction job remains on the queue.
The explanation above applies when the queue's storage is separate from the application's database. Redis and SQS are such cases.
It does not happen with the database driver's defaults. When connections.database.connection in config/queue.php is unset (DB_QUEUE_CONNECTION empty), the INSERT into the jobs table uses the same connection and therefore joins the same transaction3. A job before the commit is invisible to other processes, and a rollback removes the job along with everything else.
So with this series' default configuration, the Bad example above does not actually break. We still move it outside the transaction, because it breaks the instant you switch the driver to Redis. Code that breaks depending on configuration is safer fixed while it is not broken. Otherwise you get to investigate "for some reason jobs only fail in production" in the middle of a queue migration.
Two solutions
One is, as in the previous chapter, to dispatch after leaving the transaction. The position is visible, so this series takes that route.
The other is a setting that defers dispatch until after the commit. It can be specified per connection2.
'redis' => [
'driver' => 'redis',
// ...
'after_commit' => true,
],
It is written on the Redis side. On the database connection the job joins the transaction anyway, so setting it changes nothing. The setting only means something when the storage is outside the database.
You can also specify it per call.
NotifyWarehouse::dispatch($order)->afterCommit();
Conversely, with after_commit enabled, use beforeCommit() when you want one particular job dispatched immediately2.
When you enable it globally in configuration, the whole team has to know. Reading the code does not tell you dispatch is deferred. It is the same trade-off as ShouldDispatchAfterCommit in the previous chapter.
Deciding destinations in one place
As jobs accumulate, you will want to decide which job goes to which queue: mail is light and should turn over fast, PDF generation is heavy and belongs to a dedicated worker.
You can specify it at dispatch time.
NotifyWarehouse::dispatch($order)->onQueue('warehouse');
In Laravel 13, you can declare those mappings all together2.
use Illuminate\Support\Facades\Queue;
public function boot(): void
{
Queue::route(NotifyWarehouse::class, queue: 'warehouse');
}
The dispatching side does not write the destination. When operational needs make you want to split queues, you do not have to go hunting for every dispatch call. You can specify the connection too, and set it for interfaces and parent classes.
The worker side starts up naming the queues it handles.
php artisan queue:work --queue=warehouse
A worker stays up and keeps running
The singleton caveat mentioned in chapter 6 becomes real here.
A normal web request throws away its process state when processing finishes. A worker does not. It does not exit after one job; it goes to fetch the next. It is faster for not rebooting the framework each time, but state survives.
If a class registered as a singleton still holds information from the previous job, the next job reads it. The invoice number example from chapter 6's exercise is exactly this situation. Another customer's data gets mixed in.
Not carrying state across jobs is the baseline. Take the values you need as job properties and leave nothing behind when processing ends.
A worker loads the framework at startup and keeps running. Rewriting a file has no effect.
If during development you feel "I fixed it but the behavior is the same," restart the worker. The same problem occurs at production deployment. Chapter 17 covers how to handle it in detail.
Tests
As promised in chapter 10, we use Queue::fake().
Confirm a job was dispatched
use App\Jobs\NotifyWarehouse;
use Illuminate\Foundation\Testing\RefreshDatabase;
use Illuminate\Support\Facades\Queue;
use Laravel\Sanctum\Sanctum;
pest()->use(RefreshDatabase::class);
test('placing an order queues the warehouse notification job', function () {
Queue::fake();
Sanctum::actingAs(User::factory()->create());
$product = Product::factory()->create(['price' => 1200]);
Stock::factory()->create(['product_id' => $product->id, 'quantity' => 10]);
$this->postJson('/api/v1/orders', [
'items' => [['product_id' => $product->id, 'quantity' => 3]],
])->assertStatus(201);
Queue::assertPushed(NotifyWarehouse::class);
});
Queue::fake() does not execute jobs. It only records that they were pushed. assertPushed confirms what got pushed.
Beware that fake changes what a test means
Call Queue::fake() and the job's body never runs in that test. The test above confirms only that it was dispatched. It stays green even if the code calling the warehouse API is broken.
The job's body gets checked separately by calling it directly.
use App\Jobs\NotifyWarehouse;
use Illuminate\Foundation\Testing\RefreshDatabase;
use Illuminate\Http\Client\RequestException;
use Illuminate\Support\Facades\Http;
pest()->use(RefreshDatabase::class);
test('a failing warehouse API fails the job too', function () {
Http::fake([
'warehouse.example.com/*' => Http::response('', 500),
]);
$order = Order::factory()->create();
expect(fn () => (new NotifyWarehouse($order))->handle())
->toThrow(RequestException::class);
});
Http::fake() stops outbound communication4. This is what chapter 10 promised for "when chapter 16 writes a job calling an external API." A test that really fires a request out bothers the other service, and your test fails when theirs is down.
Remove ->throw() and this test goes red. It pins down whether an external API failure is treated as a job failure.
phpunit.xml sets QUEUE_CONNECTION=sync. Without Queue::fake(), jobs execute on the spot.
So a test without fake() is in the state of "meant to be asynchronous, passing synchronously." Nothing fails, so nobody notices. Decide whether you want to confirm the dispatch only or the execution too before you write it.
What matters in production
If the worker dies, nobody processes anything
queue:work is just a process. It stops on a server restart or an abnormal exit from an exception. If nobody notices it stopped, jobs pile up.
You need something that monitors the process and restarts it automatically. Chapter 17 covers it.
Failed jobs are recorded, but nobody is looking
A job that exhausts its attempts is recorded in the failed_jobs table. The failed setting in config/queue.php does that by default3. It is not thrown away, so it can be re-run later.
The problem is that if nobody is looking, it is merely recorded. Finding out only when a "my email never arrived" ticket comes in means you are paying the cost of going asynchronous with none of the benefit.
Failure notifications, the re-run procedure, and retry count design are covered in chapter 17. Do not ship to production in this chapter's state.
Order is not guaranteed
Run several workers and jobs are not necessarily processed in the order they were pushed. A "shipped" email can arrive before the "order confirmed" one.
When order matters, either combine the jobs into one, or dispatch the follow-on work from inside the preceding job.
Watch out for long-running jobs
A worker has a time limit on execution. Exceed it and the job is stopped partway. Processing that got partway through is left half-done.
Split heavy processing into small jobs. For 10,000 updates, 100 jobs of 100 records each limits the damage from a failure.
Do not ship to production on sync
QUEUE_CONNECTION=sync means executing on the spot without a queue. Leave that in production and the entire point of going asynchronous is gone.
And nothing errors. The response is just slow. Check the environment variables on your deployment target.
Summary
- A queue records "what to do" and a separate process executes it later
- A listener becomes asynchronous just by implementing
ShouldQueue.handle()is unchanged - Without
queue:workrunning, nobody processes anything. Jobs pile up with no error SerializesModelscollapses a model to its primary key and re-reads it at execution. It fails if the row is gonesyncis the "do not use a queue" setting. Despite the name, it executes synchronously- Do not
dispatchinside a transaction. The worker is a separate process and goes looking for a row that does not exist yet - But that failure depends on the driver. The
databasedefault shares the connection and does not break; switch to Redis and it does - To defer, use the
after_commitsetting orafterCommit(). It does make the code harder to follow - Failed jobs are recorded in
failed_jobs. They are not lost, but without something to look at them you never notice Queue::route()gathers per-job destinations into one place- A worker stays up and keeps running. Do not carry state across jobs. Restart after fixing code
Queue::fake()confirms the dispatch only. Check the job's body by calling it directly
What to read next
It is asynchronous now, but not in a state you can operate in production. Nobody notices when the worker dies, and nobody looks at what piles up in failed_jobs. The same job can run twice. The next chapter, Queues in production — idempotency, retries, and failed_jobs, covers failure notifications and re-runs, designing retry counts and backoff, and monitoring and restarting workers. It also collects on the execution side of the idempotency touched on in chapters 13 and 14.
Exercises
The following code sometimes fails a job with "order not found." Explain the cause
public function place(User $user, array $items, ?string $idempotencyKey = null): Order
{
return DB::transaction(function () use ($user, $items, $idempotencyKey) {
// ... decrementing stock ...
$order = Order::create([...]);
NotifyWarehouse::dispatch($order);
foreach ($items as $item) {
$order->items()->create([...]);
}
return $order;
}, attempts: 3);
}
Sample answer
The job is dispatched before the commit.
The worker runs in a separate process. It cannot see inside this transaction. The row created by Order::create() is invisible to other processes until the commit.
If the worker picks the job up before the commit, SerializesModels re-reading the order by primary key finds nothing, and the job fails. The result depends on timing, so it does not reproduce locally and only fails in production.
Here, whether the symptom appears depends on QUEUE_CONNECTION.
- Redis or SQS — the storage is separate from the application's database, so it happens as described. A rollback leaves only the job behind
- The
databasedefault — the INSERT intojobsjoins the same transaction, so it does not happen. A rollback removes the job too
You cannot say "it does not happen now, so we need not fix it." It breaks the instant you switch the driver to Redis, and you would be hunting the cause in the middle of a queue migration. Code that breaks depending on configuration gets fixed while it is not broken.
Dispatch after leaving the transaction. If circumstances prevent moving the position, use afterCommit().
How do you respond to "the Queue::fake() test passed, so mail sending works"?
You cannot say it works. Queue::fake() does not execute jobs, so what was confirmed is the dispatch alone.
The test stays green even if the code composing the email is wrong, the recipient is wrong, or the connection settings for the external service are wrong. Dispatch and execution have to be confirmed separately.
Split it in two. That it is dispatched goes to Queue::fake() and assertPushed. The job's body gets checked by calling handle() directly. The test using Notification::fake() from the previous chapter is the latter kind.
There is one more thing to confirm: did that test even go through the asynchronous path? The test environment's QUEUE_CONNECTION is sync, so if Queue::fake() was forgotten, the job executed on the spot and passed. Whether it passed synchronously or only confirmed the dispatch is not visible without reading the test code.
And finally, whether a worker is running is a separate matter. Tests do not confirm that queue:work is up. If no worker is running on the deployment target, every test can pass and not a single email arrives.