# CEM Event Journal

The Phase 5 consumer reads validated canonical CEM JSONL and writes one durable,
append-only journal record for each unique `event_id`. It has no network, AMI,
database, CRM, WebSocket, archive, or broker dependency.

Related architecture: MPBX-002 and MPBX-006. Release: `cem-event-journal-v0.5`.

This is a durable local audit/event journal implementation. It does not claim broker-level, distributed or transport-level exactly-once delivery. Journal-first append plus checkpoint reconciliation provides practical local idempotency under the documented filesystem assumptions and stable `event_id` semantics.

The journal, Consumer SDK, CRM Shadow and Parity paths are validated local/shadow capabilities. The legacy listener remains authoritative for production CRM effects, and journaling an event does not create a CRM side effect.

## Usage

```sh
php tools/cem_event_journal.php \
  --input /tmp/cem-live-stream/cem-events.jsonl \
  --journal /tmp/cem-journal/journal.jsonl \
  --checkpoint /tmp/cem-journal/checkpoint.json \
  --once
```

Use `--follow` to wait for complete appended lines. `--poll-ms` defaults to 250.
`--max-events` limits complete source lines handled by this invocation. With
neither mode option, the safe default is a single pass. `--once` and `--follow`
cannot be combined.

## Durability and recovery

Each accepted record contains the canonical identity and routing fields,
consumer metadata, the source line offset, a SHA-256 checksum, and the parsed
canonical event without modification. The consumer flushes and synchronizes the
journal append before atomically replacing its checkpoint. Checkpoint files are
mode `0600`; runtime directories are mode `0700` where permissions permit.

At startup the complete journal is scanned to reconstruct processed `event_id`
values before the checkpoint offset is resumed. If a crash occurs after journal
append but before checkpoint replacement, replay sees the existing event ID,
does not append it again, and commits the newer offset. A partial or invalid
final journal record is not repaired automatically: startup fails and requires
operator inspection. Partial final input lines remain unread until completed.

Input truncation, replacement, or a changed committed boundary restarts reading
at byte zero and increments `rotations_detected`; journal idempotency still
prevents repeated event IDs. Input, journal, and checkpoint paths must differ.
An exclusive nonblocking lock derived from the journal/checkpoint pair prevents
concurrent consumers.

The checkpoint schema version is `1` and stores the input path, device/inode,
committed byte offset, last complete-line boundary, last journaled event ID,
processed event identities, boundary checksum data, and update timestamp.

## Limitations

The processed identity set and startup journal scan grow linearly with journal
size. Automatic journal rotation, retention, compaction, and repair are outside
Phase 5. The journal and checkpoint cannot form one filesystem transaction;
journal-first ordering plus startup reconciliation prevents duplicate records,
but durability ultimately depends on filesystem guarantees.