# CEM shadow follow

The Phase 3 follower consumes only the sanitized JSONL written by `tools/ami_capture.php`. It does not connect to AMI or any other PBX interface.

Related architecture: MPBX-001, MPBX-002 and MPBX-006. Releases: `cem-runtime-shadow-v0.3` and `cem-aggregate-persistence-v0.4`.

This guide describes the local PHP runtime implementation. MPBX documents own canonical event, runtime-responsibility and security requirements.

The follower is a validated shadow capability. It does not replace the legacy listener, which remains authoritative for production CRM effects, and it invokes no production CRM side effect.

## Manual DEV validation

In Terminal A, start a bounded sanitized passive capture using an ignored local configuration:

```bash
php tools/ami_capture.php \
  --config ami/config/ami_capture_config.inc.php \
  --output-dir runtime/ami-capture \
  --duration 120 \
  --max-events 500
```

In Terminal B, resolve and verify the newly created `*.ami-events.jsonl` filename in `runtime/ami-capture/` (do not select a metadata file or an older capture), then start the follower against that exact path:

```bash
php tools/cem_shadow_follow.php \
  --input runtime/ami-capture/<verified-name>.ami-events.jsonl \
  --output var/events/cem-live-shadow.jsonl \
  --state var/runtime/cem-shadow-follow.state.json \
  --tenant-id dev-local \
  --customer-id <reviewed-dev-customer> \
  --deployment-id <reviewed-dev-deployment> \
  --environment DEV \
  --poll-ms 250
```

Make one controlled DEV call. Observe canonical events incrementally with `tail -f var/events/cem-live-shadow.jsonl`, then stop both tools with `Ctrl+C`.

The follower commits only complete newline-terminated source records. State is atomically replaced after every complete input line. Restart resumes at the saved byte boundary and scans existing CEM output for idempotency and causation continuity.

Source identity options are explicit non-secret runtime scope. Existing invocations retain the historical shadow defaults, but controlled live runs should always supply tenant, customer, deployment and environment. The committed follower state binds these values to the byte offset and rejects a restart under a different identity.

At the Grandstream adapter boundary, a structurally valid `DialBegin.DestChannel` such as `PJSIP/3000-...` is normalized to canonical `agent_extension=3000`. Caller digits, arbitrary channel text and queue names are not used for this derivation. Raw `DestChannel` remains controlled provenance; downstream CRM consumers should use the canonical payload.

Explicit Grandstream/Asterisk `Queue` fields on queue-family events are normalized as canonical PBX `queue_id`. `QueueCallerJoin` or `AgentCalled` commonly supplies this evidence before `DialBegin`, allowing the aggregate to include the retained queue in the ringing event even when `DialBegin` has no `Queue` field. `Exten`, `Context`, `Application`, `ConnectedLineNum`, caller data and channel names are never queue-identity fallbacks. If authoritative queue evidence arrives only after ringing, the follower does not rewrite the historical event or emit a synthetic lifecycle transition.

Inbound root-leg `CallerIDNum` is normalized at the PBX adapter boundary and becomes canonical `caller_identity` for ringing. Formatting separators are removed, an optional leading `+` is preserved, and only 7–15 digit telephone identities are accepted. Queue/agent extensions, `ConnectedLineNum`, secondary-leg caller fields and arbitrary channel digits are rejected as caller evidence. Downstream CRM consumers use this canonical fact and contain no Grandstream parsing.

State schema version 1 records the input path, device and inode when available, committed byte offset, last complete-line boundary, boundary checksum, isolated capture-session identity, processed source identities and update time. The boundary checksum detects an in-place truncate-and-regrow even when the replacement content has already exceeded the previous offset.

Phase 4 added persistent active aggregate state. When aggregate persistence is enabled, a follower restart restores the active call's correlation and subject identity, lifecycle state and emitted flags, timing, known call legs, source identities, direction, queue/agent enrichment, bridge identifiers, recording identities and last emitted event identity used for causation. Truncation or inode replacement creates a new isolated capture-session context so unrelated capture sessions are not merged.

Terminal aggregate state is removed only after the terminal event is flushed and the follower offset is committed. The aggregate store, CEM output and follower checkpoint are separate files rather than one transactional journal. Recovery uses deterministic identities and reconciliation, but durability still depends on filesystem semantics. Parent directories are not explicitly `fsync`ed, and a torn JSONL append or abrupt storage failure can require operator inspection. See `CEM_AGGREGATE_STATE` for the implementation-specific write order and recovery limits.