Queue System

The Volare application implements a robust queue infrastructure for handling asynchronous tasks using PostgreSQL for development and Amazon SQS for production.

Overview

Key Features:

• Database-driven queues for development (PostgreSQL)
• Automatic worker management via Docker Compose
• Production-ready configuration for Amazon SQS
• Built-in monitoring and debugging commands
• Graceful restarts and failure handling

Architecture

Components

Application Code -> Queue Connection -> PostgreSQL Jobs Table
                                              |
                                              v
                    Queue Worker Container -> Job Handler -> Success/Failure

Infrastructure

Development Environment:

• Queue Driver: PostgreSQL database driver
• Worker Process: Dedicated Docker container (queue service)
• Management: Docker Compose (no Supervisor needed)
• Persistence: Database tables (jobs, failed_jobs, job_batches)

Production Environment:

• Queue Driver: Amazon SQS
• Worker Process: EC2 instances or ECS containers
• Management: AWS infrastructure

Queue Priority

The worker processes queues in priority order:

1. aerticket-tickets - Ticket issuance (highest priority)
2. aerticket-bookings - Booking creation
3. default - General tasks

Quick Start

Dispatching Jobs

use App\Jobs\TestQueueJob;

// Simple dispatch
TestQueueJob::dispatch('Hello, Queue!');

// Delayed dispatch (5 minutes)
TestQueueJob::dispatch('Delayed message')
    ->delay(now()->addMinutes(5));

// Custom queue
TestQueueJob::dispatch('High priority')
    ->onQueue('high-priority');

Monitoring

# Check queue status
./vendor/bin/sail artisan queue:monitor

# Watch logs in real-time
./vendor/bin/sail artisan tail

# Check failed jobs
./vendor/bin/sail artisan queue:failed

Configuration

Environment Variables

# Queue driver: database (dev) or sqs (production)
QUEUE_CONNECTION=database

# Database queue settings
DB_QUEUE_CONNECTION=pgsql
DB_QUEUE_TABLE=jobs
DB_QUEUE=default
DB_QUEUE_RETRY_AFTER=90

# Job retry settings
QUEUE_RETRY_AFTER=90
QUEUE_MAX_ATTEMPTS=3

Docker Compose Configuration

queue:
  build:
    context: './vendor/laravel/sail/runtimes/8.4'
    dockerfile: Dockerfile
  volumes:
    - '.:/var/www/html'
  networks:
    - sail
  depends_on:
    - pgsql
    - redis
  command: php artisan queue:work --queue=aerticket-tickets,aerticket-bookings,default --sleep=3 --tries=3 --max-time=3600

Worker Behavior:

• Polls every 3 seconds for new jobs
• Max 3 retry attempts per job
• Max 1 hour runtime before graceful restart
• Auto-restart on container failure

Common Operations

Restarting Workers

# Signal workers to restart after current job
./vendor/bin/sail artisan queue:restart

# Or restart the Docker container
./vendor/bin/sail restart queue

Handling Failed Jobs

# List all failed jobs
./vendor/bin/sail artisan queue:failed

# Retry a specific failed job
./vendor/bin/sail artisan queue:retry {job_id}

# Retry all failed jobs
./vendor/bin/sail artisan queue:retry all

# Flush all failed jobs
./vendor/bin/sail artisan queue:flush

Monitoring Workers

# Check worker status
docker ps | grep queue

# View worker logs
./vendor/bin/sail logs queue

# Follow worker logs in real-time
./vendor/bin/sail logs -f queue

AerTicket Queue Jobs

Ticket Issuance Queue

Queue Name: aerticket-tickets

Purpose: Process ticket issuance requests with highest priority

Jobs:

• AerticketIssueTicketJob - Issues tickets for confirmed bookings

Configuration:

public int $tries = 3;
public int $timeout = 120;
public array $backoff = [30, 60, 180];

Booking Creation Queue

Queue Name: aerticket-bookings

Purpose: Process flight booking creation asynchronously

Jobs:

• AerticketCreateBookingJob - Creates flight bookings
• AerticketRetrieveBookingJob - Retrieves PNR details

Security Best Practices

Validate Job Data

public function __construct(
    public int $userId,
    public string $action
) {
    if (!User::find($userId)) {
        throw new \InvalidArgumentException('Invalid user ID');
    }
}

Encrypt Sensitive Data

use Illuminate\Contracts\Queue\ShouldBeEncrypted;

class ProcessPayment implements ShouldQueue, ShouldBeEncrypted
{
    public function __construct(
        public string $creditCardNumber
    ) {}
}

Rate Limiting

use Illuminate\Queue\Middleware\RateLimited;

public function middleware(): array
{
    return [new RateLimited('api-calls')];
}

Troubleshooting

Worker Not Processing Jobs

1. Check worker is running:

docker ps | grep queue

2. Check worker logs:

./vendor/bin/sail logs queue

3. Restart worker:

./vendor/bin/sail restart queue

Jobs Failing Repeatedly

1. Check failed job details:

./vendor/bin/sail artisan queue:failed

2. Review exception:

./vendor/bin/sail artisan tinker
>>> DB::table('failed_jobs')->latest()->first()->exception;

3. Fix issue and retry:

./vendor/bin/sail artisan queue:retry all

Connection Timeouts

1. Increase timeout in job:

public $timeout = 120; // 2 minutes

2. Increase retry_after:

DB_QUEUE_RETRY_AFTER=180