Servers and Containers

Every Volāre server is an EC2 instance started by an Auto Scaling group (ASG) from a prebuilt server image (AMI). At boot the instance downloads its configuration from AWS SSM Parameter Store, pulls the Docker images from ECR and starts the containers. Nothing is changed by hand on a running server: to change something, you change the source and replace the instances.

Source: infra-volare/terragrunt/live/us-east-2/{staging-1,production-1}/autoscaling/, infra-volare/terragrunt/templates/

Auto Scaling groups

ASG	Instance types	Pricing	min / max / desired	Health check
api-production-1	t3.medium, t3a.medium	on-demand	2 / 10 / 2	load balancer
front-production-1	t3.medium, t3a.medium	on-demand	2 / 10 / 2	load balancer
ops-runner-production-1	t3.medium, t3a.medium	on-demand	2 / 10 / 2	EC2
hermes-production-1	t4g.large, m7g.large (ARM)	on-demand	1 / 1 / 1	EC2
api-staging-1	t3.small, t3a.small	spot	0 / 10 / 1	load balancer
front-staging-1	t3.micro, t3a.micro	spot	0 / 10 / 1	load balancer
ops-runner-staging-1	t3.small, t3a.small	spot	0 / 10 / 1	load balancer
hermes-staging-1	t4g.medium, m7g.medium (ARM)	spot	1 / 1 / 1	EC2

• There are no scaling policies: capacity only changes when someone changes desired.
• All servers sit in private subnets; only the load balancers are public.
• Production ops-runner uses EC2 health checks on purpose: it also hosts the Analytics MCP, and a failing MCP must not get the whole ops-runner terminated.
• Staging uses spot instances, so a staging server can disappear with two minutes’ notice. The ASG replaces it automatically.

What runs on each server

api — Arkana and the API

Container	Image	Purpose
web	volare/backend:<backend tag>	nginx on port 80, forwards PHP to php-fpm
php-fpm	volare/php-fpm:<backend tag>	The Laravel application
nightwatch	volare/nightwatch-agent	Laravel Nightwatch agent

ops-runner — everything that is not a web request

Container	Image	Purpose
queue-worker	volare/queue-worker:<backend tag>	One worker that consumes all queues (see Queue System)
scheduler	volare/php-fpm:<backend tag>	php artisan schedule:work — runs routes/console.php
php-fpm, web	same as api	Serves Telescope and is where you run Artisan commands
nightwatch	volare/nightwatch-agent	Nightwatch agent
analytics-mcp	volare/analytics-mcp	Production only — the Analytics MCP at analytics.byvolare.com

Migrations run when an ops-runner boots

The ops-runner start-up script runs php artisan migrate --force (with retries) every time an ops-runner instance starts, in both staging and production. Deploying a new backend version therefore applies its migrations. Write migrations that are safe while the previous version is still serving traffic.

The containers are named app-<service>-1 (for example app-php-fpm-1), because the compose project lives in /opt/app. To run Artisan, see Server Access.

front — the Web

A single container, front-<env>, running volare/frontend:<frontend tag> on port 4321 (Astro server-side rendering). The load balancer uses sticky sessions (24 h) for it.

hermes — the AI agent

Hermes runs directly under systemd, not in Docker. See Hermes.

Docker images

Application images are built by build-images.yml in the volare repo and pushed to one ECR registry shared by both accounts:

ECR repository	Built from	Used by
volare/php-base	backend/docker/common/php-base	Base for php-fpm and queue-worker (PHP 8.5, minimum patch pinned with PHP_MIN_VERSION)
volare/php-fpm	backend/docker/common/php-fpm	api, ops-runner (php-fpm, scheduler)
volare/queue-worker	backend/docker/common/queue-worker	ops-runner
volare/backend	backend/docker/production/nginx	api, ops-runner (web)
volare/frontend	frontend/Dockerfile	front
volare/analytics-mcp	analytics-mcp/	ops-runner (production)
volare/nightwatch-agent	mirror of laravelphp/nightwatch-agent	api, ops-runner

Tags: vX.Y.Z for releases (created on merge to master; also tagged latest), sha-<commit> for every build, and the branch name for PR builds (ready-to-test label). Never deploy :master — it is an old image left in the registry and missing the php-fpm health check. Old images are cleaned up by ECR lifecycle policies (keep v*, the last 10 sha-*, expire untagged after a day).

Configuration and secrets

Servers read their configuration from SSM Parameter Store at boot only:

Parameter	Becomes	Who writes it
/volare/<env>/backend/env	/opt/app/.env on api and ops-runner (the Laravel .env)	A person, by hand (SecureString)
/volare/<env>/frontend/env	.env of the front container	A person, by hand (SecureString)
/volare/<env>/autoscaling/{api,ops-runner}/docker-compose	/opt/app/docker-compose.yml	Terraform (rendered from templates/docker-compose.*.yml.tmpl)
/volare/production-1/analytics-mcp/env	analytics-mcp.env	A person, by hand
/volare/<env>/hermes/env	Hermes .env	A person, by hand

Database credentials, API keys for AerTicket, Stripe and so on all live in the backend .env parameter. The Laravel Passport keys for the Arkana MCP are added to it with scripts/passport-keys-to-ssm.sh.

To change an environment variable in staging or production:

1. Edit the parameter in the AWS console (Systems Manager › Parameter Store) or with aws ssm put-parameter, using the right account (volare or volare-production profile).
2. Run the Instance Refresh ASGs workflow in infra-volare for that environment. Running servers never re-read SSM; only new instances pick the change up.

Server images (AMIs)

AMI	Base	Contents	Used by
golden-ubuntu-docker-ansible-x86_64-*	Ubuntu 24.04	Docker + compose, AWS CLI, Grafana Alloy (metrics and logs to Grafana Cloud)	api, front, ops-runner
golden-ubuntu-hermes-arm64-*	Ubuntu 26.04, ARM	Hermes Agent pinned to a stable release, rootless Docker, Caddy, Alloy	hermes

AMIs are built with Packer + Ansible (infra-volare/packer/) through the Packer Build workflow. Every deploy picks the newest matching AMI, so after a new AMI is built the next deploy replaces all instances of that environment.