1.17.0 — Role-aware fleets, content-node monitoring, fleet-once cron

2026-08-23

A fleet with more than one kind of node can now be deployed entirely through the framework: a project declares which nginx, cron and systemd files each node role owns, and every box converges exactly its own set. MojoSec gains a content-serving profile whose tenant directories are genuinely watched — a customer publish announces itself through a root-owned broker and produces no alert, while an unexplained change still alarms. Two changes require action before upgrading: the retired Elastic-IP handoff keys must leave brownfield manifests, and scheduled functions now run once per fleet unless declared per-node.

Breaking

• The eip-handoff and eip-rollback provisioning commands are removed, with no alias. A brownfield fleet manifest still carrying eip_handoff_role_arn or eip_handoff_canaries now fails closed naming the key — delete both before upgrading. nlb_eip_allocations survives on its own and no longer requires them.
• Scheduled @schedule functions now run once per fleet per minute instead of once per node. A function that is genuinely node-local must declare @schedule(..., per_node=True) or it will silently run on only one node. Audit your scheduled functions before upgrading.

Added

• Node roles. A project ships aws/node_roles.conf declaring which role owns which installed conf.d/, cron.d/ or systemd/ file; deploy installs the node's own set and removes foreign ones. The role is resolved on the node from a root-sealed authority (provision-written labels are promoted into it automatically), an undeclared project keeps today's behavior exactly, and check_node gains a roles section that grades the node against its role.
• Content-node monitoring. The new al2023-content-v1 MojoSec profile watches enrollment-declared tenant content roots. A publish opens an annotation window through a root-owned broker (publish_window(...) context manager): the caller declares scope, root derives and digests every path, an annotated publish produces no alert, and an unannotated change under the same root still alarms. An application annotation can only ever excuse a change the application was already authorized to make.
• add_node accepts optional source_instance and subnet_id placement on single and batch operations, validated fully before any mutation; the packaged Admin gains matching placement controls on all three Add Node surfaces.
• Nodes recover their kernel firewall on boot through an engine-startup hook.

Changed

• A release no longer aborts when an already-installed vhost fails inspection — it warns and installs the repository's own conf. A node that previously blocked a release now proceeds with a deploy warning, so watch the vhost warning phases; a missing repository certificate is newly fatal.
• Capacity operations whose progress cannot be durably recorded are refused up front (503) instead of running unrecorded; add_node's later legs stay tolerant so a Redis blip cannot orphan a paid node.
• A named add_node subnet must share its source's availability zone — cross-zone placement is refused (a current one-zone limit, not a design position).
• Unregistered OAuth protected-resource paths fall through to the application's URL tree instead of being claimed with a JSON 404, so an app can publish its own document there; registered paths are byte-identical.
• The rendered deploy contract (var/deploy) is pruned to exactly the current set, so a template retired by a framework upgrade is now actually removed from nodes on the next deploy instead of surviving indefinitely.
• The hourly firewall reconcile is broadcast so every node repairs its own kernel; cron heartbeats move to version 2, adding node and executed/skipped counts (existing readers unaffected).

Fixed

• Brownfield AWS discovery reads every provider page; a partial read blocks apply instead of silently under-reporting collisions and ownership.
• Managed-path NLB re-creation adopts its existing Elastic IPs again instead of allocating fresh public addresses (a regression since 1.15.27 — if you re-created an NLB on an affected version, re-check upstream IP allowlists).
• Concurrent WebApp alias attaches converge on one alias instead of surfacing a 500.
• Fleet readiness reports an unreadable roster as one failure naming the likely cause, instead of blaming every node.

Security

• Fleet fan-out resolves its roster with a fail-closed reader, so a Redis fault can no longer silently collapse a broadcast into a unicast.
• Firewall reconciliation is per-node with host-scoped locking; concurrent reconciles can no longer race and install an incomplete address set.
• The role manifest refuses to claim MojoSec's own units and the framework's package-owned configs, so a manifest typo cannot uninstall the sensor as a trusted change.
• Removing the Elastic-IP handoff engine removes the only privileged bypass of the destructive-prefix guard.

Upgrade notes

• Delete eip_handoff_role_arn and eip_handoff_canaries from brownfield manifests first; the upgrade fails closed on either.
• Audit scheduled functions for node-local work and add per_node=True where it applies.
• Adopting node roles: label the nodes (or let existing provision labels seal on the next deploy), verify with check_node --section roles, and only then commit aws/node_roles.conf — a manifest deployed to an unlabeled node refuses to converge it, before anything is touched.
• Enrolling a content node changes nothing about how it serves; keep TLS material outside the enrolled content root or its changes will (correctly) alarm.