Deploy

Build and deploy the runtime binary with embedded frontend support, automatic migrations, and rolling updates.

The Build

# Build frontend (SvelteKit)
cd frontend && bun install && bun run build && cd ..

# Or build frontend (Dioxus)
cd frontend && dx build --web --release && cd ..

# Build release binary (frontend embedded by default)
cargo build --release

Output: target/release/my-app

Single binary containing backend, optional frontend, and migrations.

What Happens

Forge bundles the frontend build directory into the binary at compile time using rust-embed. On startup, the runtime connects to PostgreSQL, acquires an advisory lock for migrations, runs any pending schema changes, then starts serving HTTP traffic. On SIGTERM, the node stops accepting new work, drains in-flight requests (30 second timeout), deregisters from the cluster, and exits.

Embedded Frontend

The embedded-frontend feature bundles the compiled frontend into the binary.

SvelteKit templates embed frontend/build
Dioxus templates embed frontend/dist

// src/main.rs
#[cfg(feature = "embedded-frontend")]
mod embedded {
    use axum::{body::Body, http::{header, Request, StatusCode}, response::{IntoResponse, Response}};
    use rust_embed::Embed;
    use std::{future::Future, pin::Pin};

    #[derive(Embed)]
    #[folder = "frontend/build"]
    pub struct Assets;

    async fn serve_frontend_inner(req: Request<Body>) -> Response {
        let path = req.uri().path().trim_start_matches('/');
        let path = if path.is_empty() { "index.html" } else { path };

        match Assets::get(path) {
            Some(content) => {
                let mime = mime_guess::from_path(path).first_or_octet_stream();
                ([(header::CONTENT_TYPE, mime.as_ref())], content.data).into_response()
            }
            None => {
                // SPA fallback for client-side routing
                match Assets::get("index.html") {
                    Some(content) => ([(header::CONTENT_TYPE, "text/html")], content.data).into_response(),
                    None => (StatusCode::NOT_FOUND, "not found").into_response(),
                }
            }
        }
    }

    pub fn serve_frontend(req: Request<Body>) -> Pin<Box<dyn Future<Output = Response> + Send>> {
        Box::pin(serve_frontend_inner(req))
    }
}

#[tokio::main]
async fn main() -> Result<()> {
    let builder = Forge::builder();

    #[cfg(feature = "embedded-frontend")]
    let builder = builder.frontend_handler(embedded::serve_frontend);

    builder.config(config).build()?.run().await
}

# Cargo.toml
[features]
default = ["embedded-frontend"]
embedded-frontend = ["dep:rust-embed", "dep:mime_guess"]

[dependencies]
rust-embed = { version = "8", optional = true }
mime_guess = { version = "2", optional = true }

During development, disable the feature to serve frontend separately:

cargo run --no-default-features

Auto Migrations

Migrations run automatically on startup with distributed locking.

-- migrations/0001_create_users.sql
-- @up
CREATE TABLE users (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    email TEXT UNIQUE NOT NULL,
    created_at TIMESTAMPTZ DEFAULT NOW()
);

-- @down
DROP TABLE users;

Forge acquires a PostgreSQL advisory lock before running migrations:

SELECT pg_advisory_lock(0x464F524745)

The lock ID (0x464F524745) is derived from "FORGE" in hex. Only one node holds this lock at a time. Other nodes block until migrations complete. The lock releases automatically when the connection closes.

Multiple nodes can start simultaneously. First node runs migrations. Others wait, which avoids race conditions.

Health Endpoints

Endpoint	Purpose	Response
`/_api/health`	Liveness probe	`200 { status: "healthy", version: "0.1.0" }`
`/_api/ready`	Readiness probe	`200/503 { ready: bool, database: bool, reactor: bool, version: "0.1.0" }`

The readiness probe queries the database:

SELECT 1

Returns 503 Service Unavailable if the database connection fails. Use this for load balancer health checks.

# kubernetes
livenessProbe:
  httpGet:
    path: /_api/health
    port: 8080
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /_api/ready
    port: 8080
  periodSeconds: 5

Docker Deployment

FROM rust:1-slim-bookworm AS dev

WORKDIR /app

RUN apt-get update && apt-get install -y \
    pkg-config libssl-dev curl && rm -rf /var/lib/apt/lists/*

RUN cargo install cargo-watch --locked

# Development: watch for changes, no frontend embedding
CMD ["cargo", "watch", "-x", "run --no-default-features"]

FROM oven/bun:1-alpine AS frontend-builder

WORKDIR /app/frontend
COPY frontend/package.json frontend/bun.lock* ./
RUN bun install --frozen-lockfile || bun install
COPY frontend ./
RUN bun run build

FROM rust:1-alpine AS builder

WORKDIR /app
RUN apk add --no-cache musl-dev openssl-dev openssl-libs-static pkgconf

COPY Cargo.toml Cargo.lock* ./

# Cache dependencies
RUN mkdir -p src && \
    echo "fn main() {}" > src/main.rs && \
    cargo build --release --no-default-features && \
    rm -rf src

COPY src ./src
COPY migrations ./migrations
COPY --from=frontend-builder /app/frontend/build ./frontend/build

# Build with embedded frontend
RUN touch src/main.rs && cargo build --release

FROM alpine:3.21 AS production

WORKDIR /app
RUN apk add --no-cache ca-certificates libgcc

COPY --from=builder /app/target/release/my-app /app/my-app
COPY --from=builder /app/migrations /app/migrations

EXPOSE 8080

ENV RUST_LOG=info

CMD ["/app/my-app"]

Multi-stage build: frontend compiled with Bun, backend with Rust, final image is Alpine (~30MB base).

Docker Compose

services:
  app:
    build:
      context: .
      target: production
    ports:
      - "9081:9081"
    environment:
      - DATABASE_URL=postgres://postgres:password@db:5432/app
      - RUST_LOG=info
    depends_on:
      db:
        condition: service_healthy
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9081/_api/ready"]
      interval: 5s
      timeout: 5s
      retries: 30

  db:
    image: postgres:18
    environment:
      POSTGRES_PASSWORD: password
      POSTGRES_DB: app
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 5s
      retries: 5

volumes:
  postgres_data:

Graceful Shutdown

On SIGTERM, Forge performs an orderly shutdown:

Stop accepting work: shutdown_requested flag set, new requests rejected
Drain in-flight requests: Wait up to 30 seconds for active requests to complete
Release leadership: Advisory locks released, standby nodes can acquire
Deregister from cluster: Node removed from forge_nodes table
Close connections: Database pool drained

// Shutdown sequence (internal)
pub async fn shutdown(&self) -> Result<()> {
    self.shutdown_requested.store(true, Ordering::SeqCst);

    // Set status to draining
    self.registry.set_status(NodeStatus::Draining).await?;

    // Wait for in-flight requests (30s timeout)
    self.wait_for_drain().await;

    // Deregister from cluster
    self.registry.deregister().await?;

    Ok(())
}

In-flight requests tracked with atomic counter. RAII guard ensures accurate counting:

// Each request increments on entry, decrements on exit
// Returns None if the node is draining (rejects new work)
if let Some(_guard) = InFlightGuard::try_new(shutdown.clone()) {
    // Process request; guard decrements on drop
}

Rolling Updates

Rolling update pattern:

Start new nodes (migrations run once, others wait)
New nodes register in cluster, begin accepting traffic
Send SIGTERM to old nodes
Old nodes drain (stop accepting, complete in-flight)
Load balancer routes to new nodes via readiness probe
Old nodes exit after drain completes

# Kubernetes rolling update
kubectl set image deployment/app app=myregistry/my-app:v2

# deployment.yaml
spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    spec:
      terminationGracePeriodSeconds: 45  # > drain timeout (30s)
      containers:
        - name: app
          readinessProbe:
            httpGet:
              path: /_api/ready
              port: 8080
            periodSeconds: 2
          lifecycle:
            preStop:
              exec:
                command: ["sleep", "5"]  # Allow LB to deregister

Key settings:

Setting	Value	Reason
`terminationGracePeriodSeconds`	45	Exceeds drain timeout (30s) to allow completion
`maxUnavailable`	0	Always maintain capacity during rollout
`preStop` sleep	5s	Time for load balancer to stop routing

Under the Hood

Migration Locking

Forge uses a PostgreSQL advisory lock to serialize migrations across the cluster:

-- Acquire (blocks until available)
SELECT pg_advisory_lock(0x464F524745)

-- Run migrations...

-- Release
SELECT pg_advisory_unlock(0x464F524745)

Advisory locks are session-scoped. If the node crashes mid-migration, the lock releases automatically when the connection terminates, so manual cleanup is not required.

Connection Draining

The drain timeout (30 seconds) balances two concerns:

Too short: Active requests terminated mid-flight, user sees errors
Too long: Deploys take forever, resource waste

Each handler increments an atomic counter on entry:

pub fn try_new(shutdown: Arc<GracefulShutdown>) -> Option<Self> {
    if shutdown.should_accept_work() {
        shutdown.increment_in_flight();
        Some(Self { shutdown })
    } else {
        None  // Reject new work during shutdown
    }
}

The counter decrements on drop (RAII). Shutdown waits until counter reaches zero or timeout expires.

Cluster Deregistration

Before exit, nodes deregister from the cluster table:

DELETE FROM forge_nodes WHERE id = $1

This allows:

Other nodes to claim orphaned jobs immediately
Load balancers to stop routing (if using database-based service discovery)

Write-Ahead Logging

Forge inherits PostgreSQL's durability guarantees. All state changes (jobs, workflows, migrations) go through WAL. Committed transactions survive crashes, and point-in-time recovery is possible to minimize data loss on unexpected termination.

TLS

By default the gateway serves plain HTTP and expects a load balancer, reverse proxy, or CDN in front of it to terminate public TLS. That is still the recommended pattern for internet-facing deployments — you get HSTS, OCSP stapling, ACME, and cert rotation for free.

Enable direct TLS on the gateway by setting both cert_path and key_path under [gateway.tls] when:

You need encrypted traffic between the load balancer and the app (ALB backend HTTPS, compliance requirements).
You're running without a fronting proxy and want HTTPS on the app directly.
You're running in a cluster where pods or nodes speak HTTPS to each other.

Configuration

Provide a PEM-encoded certificate chain and private key on disk:

[gateway.tls]
cert_path = "${GATEWAY_TLS_CERT_PATH}"
key_path = "${GATEWAY_TLS_KEY_PATH}"

Both paths set → TLS is on. Both omitted → plain HTTP. Setting only one is a startup error. Use a certificate issued by a CA (cert-manager, ACM Private CA, mounted Kubernetes secret, etc.) for production.

Certificate rotation requires a process restart — there is no hot reload in this release. Use a rolling deployment to swap mounted secrets with zero downtime.

Generating a cert quickly

For backend-only use behind a load balancer that doesn't validate the backend chain — or for internal services and local development — a throwaway self-signed cert is fine. openssl produces one in a single command:

openssl req -x509 -newkey rsa:2048 -nodes -days 365 \
  -keyout key.pem -out cert.pem \
  -subj "/CN=app.internal" \
  -addext "subjectAltName=DNS:app.internal,DNS:localhost,IP:127.0.0.1"

Mount the resulting cert.pem and key.pem into the app container or ship them alongside the binary, then point cert_path / key_path at them. Rotate by regenerating and restarting — there is no hot reload.

Health checks with TLS

When TLS is enabled, load balancer and Kubernetes probes must target HTTPS. For ALBs, set the target group protocol to HTTPS and point the health check to /_api/ready. For Kubernetes probes, set the probe scheme: HTTPS. If the certificate is self-signed or internally issued, the LB / kubelet must be willing to skip chain validation (ALB target group HTTPS health checks do this automatically; httpGet probes do too).

What TLS on the gateway does NOT do

No HSTS or OCSP stapling — do that at the edge.
No ACME / auto-renewal — certificate management is external.
No mTLS / client certificate verification — the gateway only terminates TLS, it does not authenticate peers.
No hot reload — rotating certificates requires a restart.

Do not use a self-signed cert as the public TLS terminator for an internet-facing deployment. Browsers and API clients will not trust it. Put a load balancer or CDN with a real CA-issued cert in front.

Environment Variables

Variable	Required	Description
`DATABASE_URL`	Yes	PostgreSQL connection string
`RUST_LOG`	No	Log level (`info`, `debug`, `trace`)
`HOST`	No	Bind address (default: `0.0.0.0`)
`PORT`	No	HTTP port (default: `8080`)

Additional variables can be accessed in handlers via ctx.env("VAR_NAME").

Observability

OTLP Transport

Forge exports telemetry via OTLP over HTTP (port 4318), not gRPC (port 4317). Configure your collector's HTTP receiver accordingly. If your collector only accepts gRPC, you will need an HTTP-capable proxy or receiver.

Emitted Metrics

Metric	Type	Labels	Description
`http_requests_total`	counter	`method`, `path`, `status`	Total HTTP requests
`http_request_duration_seconds`	histogram	`method`, `path`, `status`	Request latency
`fn.executions_total`	counter	`function`, `kind`	Function call count
`fn.duration_seconds`	histogram	`function`, `kind`	Function execution latency
`job_executions_total`	counter	`job_type`, `status`	Job execution count
`job_duration_seconds`	histogram	`job_type`	Job execution latency
`active_connections`	up-down counter	`type`	Current active connections

Tracing Headers

Every response includes these headers for request correlation:

Header	Description
`x-request-id`	UUID for the request
`x-trace-id`	OpenTelemetry trace ID
`x-span-id`	Current span ID

W3C Trace Context

Forge extracts the traceparent header from inbound requests per the W3C Trace Context spec. If present, the request joins the existing distributed trace.

The Build​

What Happens​

Embedded Frontend​

Auto Migrations​

Health Endpoints​

Docker Deployment​

Docker Compose​

Graceful Shutdown​

Rolling Updates​

Under the Hood​

Migration Locking​

Connection Draining​

Cluster Deregistration​

Write-Ahead Logging​

TLS​

Configuration​

Generating a cert quickly​

Health checks with TLS​

What TLS on the gateway does NOT do​

Environment Variables​

Observability​

OTLP Transport​

Emitted Metrics​

Tracing Headers​

W3C Trace Context​

Production Checklist​