# Persistent Daemon

_Mothership and peer, running in the background, surviving reboots and network changes._

## Overview

A Mothership started with `aura mothership start` in a terminal is fine for development. For production — or for any team that expects the Mothership to be reachable on Monday morning — it needs to run as a persistent daemon. Same applies to the peer side: developers should not have to remember to run `aura` after every reboot.

This page covers daemonization on the three platforms we support (Linux with systemd, macOS with launchd, and Windows — via WSL or NSSM), network-change handling, log locations, and rotation.

## Linux: systemd

Systemd is the default on every Linux distribution Aura supports. The idiomatic setup uses a single unit file with a user-level service for peers and a system-level service for Motherships.

### Mothership as a system service

Create `/etc/systemd/system/aura-mothership.service`:

```text
[Unit]
Description=Aura Mothership
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=aura
Group=aura
ExecStart=/usr/local/bin/aura mothership start --foreground
Restart=on-failure
RestartSec=5s
LimitNOFILE=65536

# Sandboxing
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/aura /var/log/aura
PrivateTmp=true
ProtectKernelTunables=true

[Install]
WantedBy=multi-user.target
```

Then:

```bash
sudo useradd --system --home /var/lib/aura --shell /usr/sbin/nologin aura
sudo mkdir -p /var/lib/aura /var/log/aura
sudo chown aura:aura /var/lib/aura /var/log/aura
sudo systemctl daemon-reload
sudo systemctl enable --now aura-mothership
```

Verify:

```bash
systemctl status aura-mothership
journalctl -u aura-mothership -f
```

The `--foreground` flag tells Aura not to fork; systemd wants to own the process. `Restart=on-failure` ensures a crash is not fatal.

### Peer as a user service

Peers should run under the developer's own account, not as a system daemon. Create `~/.config/systemd/user/aura-peer.service`:

```text
[Unit]
Description=Aura peer sync
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStart=/usr/local/bin/aura sync --foreground
Restart=on-failure
RestartSec=10s

[Install]
WantedBy=default.target
```

Enable:

```bash
systemctl --user daemon-reload
systemctl --user enable --now aura-peer
loginctl enable-linger "$USER"
```

`loginctl enable-linger` ensures the user service keeps running when the developer is not logged in. Without it, the service stops when they log out.

## macOS: launchd

On macOS, launchd is the equivalent of systemd. There are two scopes: _LaunchDaemons_ (system-wide, run as root or a specified user) and _LaunchAgents_ (per-user).

### Mothership as a LaunchDaemon

Create `/Library/LaunchDaemons/com.aura.mothership.plist`:

```text
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.aura.mothership</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/local/bin/aura</string>
    <string>mothership</string>
    <string>start</string>
    <string>--foreground</string>
  </array>
  <key>RunAtLoad</key>
  <true/>
  <key>KeepAlive</key>
  <true/>
  <key>UserName</key>
  <string>_aura</string>
  <key>StandardOutPath</key>
  <string>/var/log/aura/mothership.log</string>
  <key>StandardErrorPath</key>
  <string>/var/log/aura/mothership.err</string>
</dict>
</plist>
```

Load:

```bash
sudo launchctl bootstrap system /Library/LaunchDaemons/com.aura.mothership.plist
sudo launchctl enable system/com.aura.mothership
```

Create the `_aura` user via System Settings or `dscl`. This is more fiddly on macOS than Linux; many teams run their Mothership on a Mac mini as a regular user with a LaunchAgent instead, which is fine for small teams.

### Peer as a LaunchAgent

Create `~/Library/LaunchAgents/com.aura.peer.plist`:

```text
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>com.aura.peer</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/local/bin/aura</string>
    <string>sync</string>
    <string>--foreground</string>
  </array>
  <key>RunAtLoad</key>
  <true/>
  <key>KeepAlive</key>
  <true/>
</dict>
</plist>
```

Load:

```bash
launchctl bootstrap gui/$UID ~/Library/LaunchAgents/com.aura.peer.plist
launchctl enable gui/$UID/com.aura.peer
```

## Windows

Windows is not a first-class deployment target for Mothership hosts — we recommend Linux or macOS for any Mothership. But developers on Windows can run peers.

Easiest path: use WSL2 and follow the Linux instructions. Aura's WSL install works out of the box.

If you must run native Windows, NSSM (the Non-Sucking Service Manager) wraps `aura sync` as a Windows service:

```bash
nssm install AuraPeer "C:\Program Files\Aura\aura.exe" "sync --foreground"
nssm set AuraPeer AppStdout "C:\ProgramData\Aura\logs\peer.log"
nssm set AuraPeer AppStderr "C:\ProgramData\Aura\logs\peer.err"
nssm start AuraPeer
```

This works. It is not our happiest path. File paths, signals, and the WAL's fsync semantics are subtly different on Windows. Test carefully.

## Auto-Reconnect on Network Change

A laptop that moves from WiFi to Ethernet to WiFi-again should not need manual intervention to stay connected. Aura handles this by subscribing to OS-level network change notifications.

- **Linux**: netlink socket. Reacts to interface up/down, IP changes, route changes.
- **macOS**: SystemConfiguration framework notifications. Reacts to WiFi association changes, VPN up/down.
- **Windows**: `NotifyIPInterfaceChange`. Similar semantics.

When a network change is detected:

1. Aura drops existing peer and Mothership connections (their underlying sockets may already be dead but haven't timed out).
2. It waits up to 2 seconds for the new network to stabilize.
3. It re-resolves the Mothership's hostname (in case DNS answers differently on the new network).
4. It reconnects, going through the normal peer certificate authentication.
5. It triggers a WAL reconcile (see [offline mode](/offline-mode)).

The whole dance usually completes in under 5 seconds. You will see:

```text
[netchange] interface state changed
[sync] connection dropped, reconnecting
[sync] reconnected to mship_7fcd21a9
[sync] reconcile: pulled 3 events, pushed 0 events
```

> **Gotcha.** Captive portals (hotel WiFi) will pass the network-change check but return HTTPS redirects for the Mothership hostname. Aura detects this and surfaces a `captive portal suspected` warning in the log. Open a browser, sign in, and Aura will reconnect on the next retry cycle.

## Log Locations

| Platform | Location |
|---|---|
| Linux (systemd) | `journalctl -u aura-mothership`, `journalctl --user -u aura-peer` |
| Linux (non-systemd) | `~/.local/state/aura/log/` |
| macOS (daemon) | `/var/log/aura/` |
| macOS (agent) | `~/Library/Logs/aura/` |
| Windows | `%ProgramData%\Aura\logs\` or `%LOCALAPPDATA%\Aura\logs\` |

Log format is structured JSON by default. Toggle to plain text:

```toml
[logging]
format = "text"
level  = "info"  # trace, debug, info, warn, error
```

## Log Rotation

Aura does not rotate its own logs when using systemd or launchd — it delegates to the platform.

**systemd / journald** already rotates. To bound size:

```text
# /etc/systemd/journald.conf
SystemMaxUse=500M
MaxRetentionSec=30d
```

**launchd** does not rotate plain files. Pair with `newsyslog`:

```text
# /etc/newsyslog.d/aura.conf
/var/log/aura/mothership.log   _aura:_aura   644  5  10000  *  GZ
```

**Plain files** on Linux without systemd: use `logrotate`:

```text
# /etc/logrotate.d/aura
/var/log/aura/*.log {
  daily
  rotate 14
  compress
  delaycompress
  missingok
  notifempty
  copytruncate
}
```

## Health Checks for Monitoring Systems

Plug Mothership into your monitoring stack by hitting the health endpoint:

```bash
curl -sf https://mothership.acme.internal:7777/healthz || exit 1
```

Every 30 seconds from Prometheus / Datadog / your monitoring tool of choice. A non-200 response or a connect failure means the Mothership is not answering.

For deeper metrics:

```bash
curl -k https://mothership.acme.internal:7777/metrics
```

Returns Prometheus-format metrics: connected peers, WAL size, push/pull rates, handshake counts by result, TLS cert expiry seconds remaining. Scrape every 60s.

> **Security callout.** `/metrics` is unauthenticated by default for local scraping. Bind Mothership to a non-public interface, or put it behind your monitoring VPN. You can require bearer-token auth for metrics:

```toml
[metrics]
require_token = true
token_file    = "/etc/aura/metrics-token"
```

## Upgrading a Running Daemon

Aura binaries are designed for drop-in replacement. On systemd:

```bash
sudo cp aura-new /usr/local/bin/aura
sudo systemctl restart aura-mothership
```

Restart takes a few seconds. During the gap:

- Peers briefly enter offline mode.
- Existing peer-to-peer direct connections keep working.
- On reconnect, peers reconcile normally.

There is no in-place reload of the Mothership binary yet. Plan restarts outside critical windows, or run a [mesh topology](/team-topology#mesh) where a sibling Mothership covers the gap.

## Startup Ordering and Dependencies

A Mothership that starts before the network is up will fail on bind and respawn. A peer that starts before the VPN has come up will fail to reach the Mothership and respawn. In both cases systemd/launchd handle the respawn for you, but it makes for noisy logs and a minute-long cold start.

Avoid this with explicit dependencies.

**systemd.** Both units above already include `After=network-online.target`. If you're on Tailscale:

```text
[Unit]
After=network-online.target tailscaled.service
Wants=network-online.target tailscaled.service
```

**launchd.** Does not have a dependency system in the systemd sense, but `KeepAlive = true` combined with a short retry is effectively equivalent. If you need precise ordering, wrap the start in a script that blocks on a network reachability check.

## Running as Non-Root

Treat the Mothership host like any other server. The Aura binary does not need root to do its job, and running as a dedicated user limits blast radius. The systemd example above creates an `aura` user; the macOS example uses `_aura`. Keep doing this.

The one exception is binding to privileged ports (below 1024) on Linux. We covered `setcap` above. Capability preferred over running as root.

Home directory for the Mothership user should not be `/home/aura` but something like `/var/lib/aura`. This signals "service, not person" to anyone reading your `/etc/passwd`.

## Observability in Production

A Mothership without monitoring is a Mothership waiting to surprise you. The minimum:

- Scrape `/metrics` into Prometheus or equivalent. Alert on `aura_sync_backlog > 1000` sustained, `aura_tls_cert_not_after_seconds < 30d`, and `aura_connected_peers` dropping by more than 20% from baseline.
- Ship `journalctl` output to a log aggregator. Alert on any `ERROR`-level message.
- Health check `/healthz` every 30s from an external monitor.
- Daily back up of the data directory. Verify restores monthly.

These are not Aura-specific practices; they are server operation table stakes. Mothership is a server. Operate it like one.

## Next Steps

- [Scale your Mothership hardware appropriately](/scaling-mothership)
- [Set up health checks and monitoring](/mothership-troubleshooting)
- [Harden TLS for production](/tls-and-jwt)