docs/operations/build-and-operations.md
1 # Build and Operations
2
3 ## Scope
4
5 This runbook describes local single-machine operation using the Rust workspace and `moraine` as the primary lifecycle interface.
6
7 ## Build
8
9 ```bash
10 cd ~/src/moraine
11 cargo build --workspace
12 ```
13
14 This produces binaries for:
15
16 - `moraine`
17 - `moraine-ingest`
18 - `moraine-monitor`
19 - `moraine-mcp`
20
21 ## Install Runtime Binaries From Source
22
23 ### Cargo install
24
25 ```bash
26 git clone https://github.com/eric-tramel/moraine.git ~/src/moraine
27 cd ~/src/moraine
28 for crate in moraine moraine-ingest moraine-monitor moraine-mcp; do
29 cargo install --path "apps/$crate" --locked
30 done
31 ```
32
33 Or install directly from GitHub without cloning:
34
35 ```bash
36 for bin in moraine moraine-ingest moraine-monitor moraine-mcp; do
37 cargo install --git https://github.com/eric-tramel/moraine.git \
38 --package "$bin" \
39 --bin "$bin" \
40 --locked
41 done
42 ```
43
44 ### Prebuilt release binary
45
46 ```bash
47 curl -fsSL https://raw.githubusercontent.com/eric-tramel/moraine/main/scripts/install.sh \
48 | bash
49 export PATH="$HOME/.local/bin:$PATH"
50 ```
51
52 The installer fetches a full bundle (`moraine`, `moraine-ingest`, `moraine-monitor`, `moraine-mcp`) and overwrites binaries in place in a single bin directory.
53
54 Install directory precedence:
55
56 1. `MORAINE_INSTALL_DIR`
57 2. `XDG_BIN_HOME`
58 3. `$(dirname "$XDG_DATA_HOME")/bin`
59 4. `~/.local/bin`
60
61 The installer writes a receipt at `${XDG_CONFIG_HOME:-~/.config}/moraine/install-receipt.json`.
62
63 Installer environment configuration:
64
65 - `MORAINE_INSTALL_REPO` (default `eric-tramel/moraine`)
66 - `MORAINE_INSTALL_VERSION` (default `latest`)
67 - `MORAINE_INSTALL_ASSET_BASE_URL` (requires `MORAINE_INSTALL_VERSION` to be a non-`latest` tag)
68 - `MORAINE_INSTALL_SKIP_CLICKHOUSE` (`1|true|yes|on` skips managed ClickHouse install)
69
70 ## Publish prebuilt binaries
71
72 Tag-driven GitHub Actions release workflow:
73
74 1. Push a semantic tag (example: `v0.3.1`).
75 2. Workflow `.github/workflows/release-moraine.yml` builds:
76 - `x86_64-unknown-linux-gnu`
77 - `aarch64-unknown-linux-gnu`
78 - `aarch64-apple-darwin`
79 3. Uploads `moraine-bundle-<target>.tar.gz` plus `moraine-bundle-<target>.sha256` to the tag release.
80
81 Each bundle includes `manifest.json` with target/version metadata, per-binary checksums, and build metadata.
82
83 Multiplatform functional CI (`.github/workflows/ci-functional.yml`) also packages per-target bundles and validates `scripts/install.sh` by installing from a local artifact server before running the stack + MCP smoke test.
84
85 ## Config model
86
87 Use one shared config schema at `config/moraine.toml`.
88
89 Resolution precedence:
90
91 1. `--config <path>`
92 2. env override (`MORAINE_CONFIG`, plus `MORAINE_MCP_CONFIG` for MCP)
93 3. `~/.moraine/config.toml` (if present)
94 4. repo default `config/moraine.toml`
95
96 ## Start stack
97
98 ```bash
99 cd ~/src/moraine
100 bin/moraine up
101 bin/moraine up --output rich
102 ```
103
104 `moraine up` does the following:
105
106 1. Starts ClickHouse process.
107 2. Waits for DB health.
108 3. Applies versioned migrations through `moraine-clickhouse` (`schema_migrations` ledger).
109 4. Starts ingest and optional services from `runtime` config.
110
111 It auto-installs managed ClickHouse when missing and `runtime.clickhouse_auto_install=true`.
112
113 `moraine clickhouse status` reports managed install state, active binary source (`managed` vs `PATH`), installed version, and checksum state.
114
115 ## DB lifecycle
116
117 ```bash
118 cd ~/src/moraine
119 bin/moraine db migrate
120 bin/moraine db doctor
121 bin/moraine db doctor --output json
122 ```
123
124 `db doctor` checks:
125
126 - ClickHouse health/version.
127 - Database existence.
128 - Applied vs pending migrations.
129 - Required table presence.
130
131 ## Service entrypoints
132
133 ```bash
134 cd ~/src/moraine
135 bin/moraine run ingest
136 bin/moraine run monitor
137 bin/moraine run mcp
138 bin/moraine run clickhouse
139 ```
140
141 ## Replay latency benchmark
142
143 Use the replay benchmark to measure current MCP `search` latency against recent worst-case telemetry:
144
145 ```bash
146 python3 scripts/bench/replay_search_latency.py --config config/moraine.toml
147 ```
148
149 For workload inspection only, run with `--dry-run`.
150 Detailed options, output fields, and troubleshooting are in `operations/replay-search-latency-benchmark.md`.
151
152 ## Status, logs, shutdown
153
154 ```bash
155 cd ~/src/moraine
156 bin/moraine status
157 bin/moraine status --output rich --verbose
158 bin/moraine status --output json
159 bin/moraine logs
160 bin/moraine logs ingest --lines 200 --output plain
161 bin/moraine down
162 ```
163
164 Status includes process state, DB health/schema checks, and latest ingest heartbeat metrics.
165 `bin/moraine logs clickhouse` reads ClickHouse's internal rotating log at
166 `~/.moraine/clickhouse/log/clickhouse-server.log`.
167
168 All subcommands support output control:
169
170 - `--output auto|rich|plain|json` (default `auto`, rich on TTY).
171 - `--verbose` for expanded diagnostics in rich/plain output.
172
173 ## Legacy scripts
174
175 Legacy lifecycle aliases remain as fail-fast migration stubs with a `moraine` replacement hint:
176
177 - `bin/start-clickhouse`
178 - `bin/init-db`
179 - `bin/status`
180 - `bin/stop-all`
181
182 Legacy wrappers remain only as fail-fast stubs:
183
184 - `bin/start-ingestor` -> `bin/moraine up`
185 - `bin/run-codex-mcp` -> `bin/moraine run mcp`
186 - `bin/moraine-monitor` -> `bin/moraine run monitor`
187
188 ## Failure triage
189
190 1. If `up` fails before migration, run `bin/moraine db doctor` and inspect ClickHouse logs via `bin/moraine logs clickhouse`.
191 2. If ingest stalls, run `bin/moraine status` and confirm heartbeat recency/queue depth.
192 3. If monitor APIs degrade, run `bin/moraine run monitor` in foreground for direct error output.
193 4. If MCP retrieval degrades, verify `search_*` tables in doctor output and rerun `db migrate`.