Commit graph

4 commits

Author SHA1 Message Date
Trevon
6d8877a6d4
feat(search): add Simeon bandit tuning (#33)
* feat(search): per-profile simeon bandit with training-free self-tuning

* fix(zyp): free parallel buffers correctly

* fix: code cleanup and simeon version bump

* fix(search): guard Simeon bandit state
2026-05-17 04:08:50 -04:00
trvon
65564158a9
perf(zyp): true parallel PDF extraction + throughput benchmark
- Implement Document::extractAllParallelized() using std::async
  to distribute pages across threads with per-thread Document
  instances, achieving true parallelism (not the old fake alias)

- zyp_plugin.cpp now uses extractAllParallelized() by default
  for PDF extraction during ingestion

- Add zyp_extraction_bench standalone benchmark measuring
  sequential vs 2/4/N-thread extraction throughput (pages/s, MB/s)

Build: ninja bench_zyp
Run:   build/debug/plugins/zyp/zyp_extraction_bench <pdf>
2026-05-15 12:13:33 -04:00
trvon
66c21c0b25
perf(ingest): bypass GLiNER for zyp PDFs, preserve plugin metadata
- Add IContentExtractor::extractTextAndMetadata() to carry plugin metadata
- Add extractDocumentContent() in extraction_util for text+metadata
- PIQ uses plugin-provided title from PDF metadata, skipping GLiNER
  entirely when a title already exists (e.g. from zyp PDF parsing)
- Fix misleading 'parallel' naming in zyp wrapper/comments
  (zpdf_extract_all_parallel is actually sequential)

This removes the dominant slow-write source for paperbridge PDF
ingestion — title metadata batches that previously backed up the
WriteCoordinator with 34s queue waits.
2026-05-15 11:52:28 -04:00
trvon
bc71d36534
migrated pdf plugin, daemon shutdown logic fix 2025-12-31 14:46:07 -05:00