trvon
1f9bc767a0
feat(search): reduce graph noise, add edge-creation diagnostics
...
- PostIngestQueue: halve co-occurrence edge budgets (entities 12→8, edges 36→16),
skip edges where neither entity is biomedical high-value, raise the
confidence floor for biomedical pairs (0.10→0.20) and non-biomedical
pairs (0.05→0.08), bump base multiplier (0.50→0.65). This cuts total
edge volume by over 50% while retaining signal quality.
- search_tuner.h: raise graphRerankMinSignal to 0.08 (from default 0.01)
to match the elevated edge-confidence floor.
- New atomic graph-construction counters (gs_processed_, gs_totalEntities_,
gs_highValueEntities_, gs_totalEdges_, gs_totalPrimaryTopicEdges_) log
entity → edge pipeline every 100 documents via [PIQ-graph].
- Periodic [graph_rerank] log (every 50 queries) reports reranker
window size, matched candidates, whether boost was applied, concept
count, and elapsed time — enabling end-to-end traceability from
ingestion to retrieval.
2026-05-05 20:56:12 -04:00
trvon
6704e0bb76
fix(search): wire GLiNER concept extractor, fix PQ dirty race, add KG NER awareness
2026-05-05 20:06:06 -04:00
trvon
cbb86506d0
feat: add RM3 pseudo-relevance feedback + adopt research-tuned config
2026-05-05 16:13:18 -04:00
trvon
73fbe4c842
feat: add BM25 variant RRF fusion to lexical backend
2026-05-05 16:13:18 -04:00
trvon
102d5a1e50
feat: integrate simeon concept mining into lexical backend
2026-05-05 16:13:18 -04:00
trvon
dc6aaa13ce
feat: add FAISS HNSW vector backend with ConfigResolver policy
2026-05-05 15:38:22 -04:00
trvon
8b7bfe575d
feat: improved search fusion logic and tuning parameters. CI improvements, reverting to push with wrangler in CI. Experimenting with vector rebuild to address additional lost overhead in search
2026-05-05 08:58:20 -04:00
trvon
ea6cd51da3
fix: removed reduant corpus adapter
2026-05-04 16:33:57 -04:00
trvon
7ae52e8b8b
fix: test fixes and search fusion update to correctly expose simeontext fusion so that it does not drop ft5s candidates. Still explore fusion optimizations
2026-05-04 15:54:57 -04:00
trvon
2d5c6625eb
fix: download logic improvements and ci test fixes. Stagging download fixes seperate from search features
2026-05-04 13:29:17 -04:00
trvon
5adb8f153f
security: dev scripts and hooks update, perf update, security fixes for quality issues
2026-05-02 21:40:47 -04:00
trvon
0ef3f8aeac
perf: improve compress behavior for r2. Compress locally and transmit and retrieve compressed data
2026-05-02 00:57:35 -04:00
trvon
3dd0e2b1f6
security: adding opengrep config to githook with rules. Addressing additional concerns
2026-05-01 20:25:22 -04:00
trvon
e5eaaf8292
fix: fixing fragiling SocketServer loop
2026-05-01 12:07:40 -04:00
trvon
0204072397
fix(asan): Addressing asan test regressions and suite failures from recent changes
2026-05-01 09:18:39 -04:00
trvon
9b2c1b8631
docs: benchmark updates, small regression fixes in multiclient communication, ci fixes for windows build for pinned boost resolution, additional writer coordinator migrations
2026-04-30 14:35:21 -04:00
trvon
9b69495367
feat: migrating additional subsystems to write coordinator. Addressing symspell holding write pool and blocking ingestion
2026-04-30 10:19:37 -04:00
trvon
2f505b009e
fix: mcp router fix
2026-04-30 09:42:52 -04:00
trvon
dae95f4267
feat: write coordinator
2026-04-30 09:16:20 -04:00
trvon
438572cc1a
fix: ci build fix for pinned version change. checkpoiting write improvements before move all writes to single component
2026-04-30 08:59:05 -04:00
trvon
c3db49ffdd
test: test fixes for flaky tests and bug fix for single db writing for kg. Contemplating swapping backend
2026-04-29 22:41:45 -04:00
trvon
ae1f0c10d2
perf: cleaning up search experiments, moving to new routing system for search for larger corpus improvements, repair system now triggers semantic / graph repair
2026-04-29 20:36:02 -04:00
trvon
ff7696a0ff
ci: addressing warm issues with ci. staging bug fixes and search research work
2026-04-29 17:22:20 -04:00
trvon
3919e92c8e
refactor: sandbox improvements and docs updates
2026-04-29 14:51:36 -04:00
trvon
9ac45139a3
perf: startup optimizations
2026-04-28 20:14:42 -04:00
trvon
f4ffb3d8c5
feat(daemon): adaptive physical-walk TTL + statvfs fast-path, hot-tier getenv→TuningSnapshot, RepairScheduling/RepairTuning slice extractions
2026-04-28 15:39:11 -04:00
trvon
c55ce54b87
fix: ci fixes and experiment for search / cluster routing
2026-04-28 01:29:48 -04:00
trvon
611d77054c
perf: instrumentation and ingestion optimizations
2026-04-27 11:01:59 -04:00
trvon
e2332a8440
optimizations: throughput optimizations, bug fixes, docs updates and benchmark cleanup
2026-04-26 21:43:00 -04:00
trvon
39576186ee
test: instrumentation expansions for ingestion speed experiments and tuning improvements
2026-04-26 19:55:29 -04:00
trvon
eae3c5afbb
feat: adding storage check, db corruption check and tests
2026-04-26 18:54:17 -04:00
trvon
f825bc3cf5
feat: ci/test regressions updates, reinstated onnxruntime (simeon will still be default), expanding topology instrumentation in search, newsletter update with throughput comparison
2026-04-26 17:03:37 -04:00
trvon
42ee44cef4
fix(ci): ci test and bug fixes. Improvements for testing topology construction
2026-04-26 13:08:55 -04:00
trvon
84015b39b7
build(ci,tests): cleaning build system and updates related to ci test failures and compat
2026-04-25 14:48:44 -04:00
trvon
49b40d5471
fix(ci): ci build fixes for windows, linux, macos tests lanes
2026-04-25 12:55:02 -04:00
trvon
8772cc8e54
fix(docs,ci): addressing breaking changes in zig v0.16.x in Dockerfile, docs update and code cleanup
2026-04-25 11:27:18 -04:00
trvon
0b61c0818a
docs: clean up docs and ci fixes for tests
2026-04-25 00:08:18 -04:00
trvon
9b774d6073
feat: profiling improvements
2026-04-24 23:23:02 -04:00
trvon
1f9387e5aa
fix: topology fixes improvements, alignment in daemon readiness checkpoint
2026-04-24 20:56:22 -04:00
trvon
082c381687
feat: cleaning up request path and fixing checkpoint logic underload
2026-04-24 18:42:00 -04:00
trvon
8de19b84f0
feat: adding instrumentation and config options for build and repair system. Bumping simeon and improving topology system with experimental findings
2026-04-24 17:38:24 -04:00
trvon
7de4d95a43
feat: repair logic improvements, expanding config options to disable rebuilding/repair for memory constrained systems
2026-04-24 10:53:15 -04:00
trvon
8417340289
feat: extending instrumentation for memory and cluster profiling. COntinue UX improvements for grep and search cli commands and mcp
2026-04-24 08:31:16 -04:00
trvon
1fcb7cf6df
feat: expanding topology instrumentation to expose quality of clusters, cli ux improvements and expanded modifiers
2026-04-23 21:28:35 -04:00
trvon
06b4992c10
feat: improving tuning and expanding instrumentation for memory allocation
2026-04-23 20:24:43 -04:00
trvon
926ef8a640
feat: memory optimizations, and topology tuning improvements
2026-04-23 20:06:45 -04:00
trvon
2ced7a7e5c
feat: reinforcing proxy path for quick ux, adding topology tuning engine for corpus cluster usage in search, simeon memory management improvements
2026-04-23 17:33:47 -04:00
trvon
e88578a9fc
feat: adding support for phss vector search, test fixes, daemon status regression fix, ci fixes for docker builds with new third-party library
2026-04-23 11:02:18 -04:00
trvon
127bc9e9e0
fix: fixing leakage of build artifacts in release, unifying repair path, adding cache to tuning path and bumping simeon version
2026-04-23 10:20:25 -04:00
trvon
df0c78cf29
feat: test fixes, metric expansion, build improvements for gliner
2026-04-23 08:48:33 -04:00
