Commit graph

1 commit

Author SHA1 Message Date
Trevon
111988c6e4
feat(topology): typed event time and session for topology inputs (#305)
* feat(topology): typed event time and session for topology inputs

Add `eventTime` (epoch seconds, UTC) and `sessionId` to
`TopologyDocumentInput`. The extractor fills them from two documented
metadata keys: `event_time` (integer epoch seconds, or ISO-8601
`YYYY-MM-DD` / `YYYY-MM-DD[T ]HH:MM[:SS[.frac]][Z|±HH:MM]`, naive
datetimes are UTC) takes precedence over the document createdTime; an
unparsable value is skipped with a debug log, never an error, and a
zero createdTime leaves the field unset. `session_id` is copied
verbatim and empty when absent. The keys are only consulted when the
extraction loads metadata; otherwise createdTime is the only source.

`parseTopologyEventTime` is public for tests and future callers; the
calendar math is Hinnant's days_from_civil with range-checked fields.
Extraction stats count the event time source and invalid values.

Nothing downstream reads the fields yet. Artifacts, codecs and both
routing fingerprints are unchanged, which the new test pins.

Tests (tests/unit/topology/topology_input_event_time_catch2_test.cpp):
parser accept/reject tables, metadata precedence over createdTime,
invalid ignored, session_id, includeMetadata=false fallback, and
byte-identical snapshot payload plus equal construction and road
fingerprints with and without the typed fields.

Signed-off-by: trvon <me@trevon.dev>

* bench(longmemeval): event_time and session_id metadata, round docs, haystack sidecar

Every exported corpus document now carries `metadata.event_time`
(ISO-8601, normalized from the LongMemEval `YYYY/MM/DD (Day) HH:MM`
haystack date, first-seen date per session) and `metadata.session_id`.
Document ids, titles, texts, queries and qrels are unchanged in the
default mode.

`--round-docs` adds one document per session round (`<sid>__r<n>`): a
user turn opens a round and the replies that follow join it; a leading
assistant turn is its own round. Round docs share the session's
`event_time`/`session_id` and add `round_index`; qrels stay at the
session level.

`--full-haystack` also writes `haystacks.jsonl`, one line per question
with `haystack_session_ids` and the normalized `haystack_dates`, so
per-question membership survives the global corpus deduplication
(the 2026-10-10 preflight lost it).

The converter is split into `convert` / `write_outputs`; `--self-test`
runs both on a three-session synthetic fixture without a download and
is the regression check for the default output.

Signed-off-by: trvon <me@trevon.dev>

* bench(retrieval): carry BEIR corpus metadata onto ingested documents

`retrieval_quality_bench` ingests the BEIR corpus as a directory, which
cannot carry per-file metadata, so `event_time`, `session_id` and
`round_index` from corpus.jsonl never reached the topology extractor.

`BEIRDocument::metadata` now holds the corpus line's `metadata` object
(strings as is, numbers and booleans stringified; both the generic and
the SciFact loader read it), and `applyBenchmarkDocumentMetadata`
copies it onto the stored documents matched by exact file path through
the metadata repository (`ON CONFLICT DO UPDATE`, so warm caches can
repeat it). The bench runs it after the ingest wait, for cold and
reused corpora alike, and logs how many documents were updated or not
found.

Tests: loader parses and coerces metadata; apply helper writes the
keys, skips documents without metadata, counts missing ones and is
idempotent against a real MetadataRepository.

The portability allowlist entries for the two touched bench files are
re-pointed at their shifted lines; no raw environment read was added.

Signed-off-by: trvon <me@trevon.dev>

---------

Signed-off-by: trvon <me@trevon.dev>
2026-10-14 08:47:04 -06:00