Commit graph

1,371 commits

Author SHA1 Message Date
trvon
c64a0b239d
feat: adding fuzzing updates and fixes in ipc 2026-02-08 22:20:47 -05:00
trvon
b2c6121990
Merge branch 'main' of github.com:trvon/yams 2026-02-08 22:19:55 -05:00
trvon
1e0ca1566c
fix: fixing bind issue when generating large batches in repair 2026-02-08 22:19:15 -05:00
trvon
fa3d15eedf
Merge branch 'main' of github.com:trvon/yams 2026-02-08 22:00:46 -05:00
trvon
3101780fe2
feat: adding additional benchmark and validating onnxruntime model usage for embedding generations. Should be much faster 2026-02-08 21:59:14 -05:00
trvon
f4f3c055fe
feat: benchmark expansion and normalization. Imrpoving metrics for repair and models 2026-02-08 20:35:39 -05:00
trvon
de4cb653e2
Merge branch 'main' of github.com:trvon/yams 2026-02-08 18:49:11 -05:00
trvon
907cddf6d3
feat: fuzzing tooling 2026-02-08 18:48:54 -05:00
trvon
c83110319d
feat: benchmark, model testing / enhancements, insertion performance updates 2026-02-08 18:33:29 -05:00
trvon
90dc507951
feat: bump submodule 2026-02-08 15:16:12 -05:00
trvon
6fe735f10c
Merge branch 'main' of github.com:trvon/yams 2026-02-08 14:52:39 -05:00
trvon
1a0667d28f
fix: improving graph repair 2026-02-08 14:51:57 -05:00
trvon
d229d9f20b
chore: update submodules 2026-02-08 14:29:48 -05:00
trvon
1c800e086b
fix: removing claude/skills 2026-02-08 14:28:56 -05:00
trvon
fe003bcfeb
build: update symspell submodule 2026-02-08 14:26:33 -05:00
trvon
fd95eb44f7
Merge branch 'main' of github.com:trvon/yams 2026-02-08 14:18:59 -05:00
trvon
27881ecfbd
docs: update changelog; ignore local artifacts 2026-02-08 14:09:57 -05:00
trvon
ac843d1e93
Merge branch 'main' of github.com:trvon/yams 2026-02-08 14:06:09 -05:00
trvon
a3522b8b22
fix: expose gpu usage, add command imrpovments and other bug fixes 2026-02-08 14:04:57 -05:00
trvon
c4a7992122
fix: repairmanager, post ingest and pipeline fixes 2026-02-08 14:02:57 -05:00
trvon
430057f4bf
fix: SocketServer slots for proxy and cli requests, fixed tests and increase doctor request handling 2026-02-08 13:02:08 -05:00
trvon
3c09becde5
fix: repair improvements and socketserver hardening 2026-02-07 23:37:32 -05:00
trvon
448f5e47b4
fix: fixing repairmanager logic, windows build issues, linting, resource scaling 2026-02-07 22:07:56 -05:00
trvon
c638d9aad4
fix: addressing test compilation issues 2026-02-06 10:37:08 -05:00
trvon
567ee32a99
fix: fixing macos coreml detection 2026-02-06 10:16:35 -05:00
trvon
3fd243f687
fix: repair command improvements and optimizations 2026-02-06 10:15:50 -05:00
trvon
90d28d722e
fix: docker build fixes addition clamping logic for onbound memory group 2026-02-06 09:27:34 -05:00
trvon
df1348f3c7
fix: tuning improvements and better tests. Now working address memory issue of huge queue when ingesting allot of files 2026-02-06 04:31:09 -05:00
trvon
d960839e54
fix: embeddings optimizations 2026-02-05 21:27:12 -05:00
trvon
ab06614505
fix(repair): fix SIGSEGV crash, embedding failures, and runaway repair loops
- Fix writer_drain() SIGSEGV by setting connection_closing_ on write failure
- Add InternalError/ResourceExhausted to embedding retry list
- Fix document-level embedding chunk_id (0→999999) and level mismatch
- Add circuit breaker to bail out after consecutive/rate-threshold failures
- Add semaphore concurrency limiter (max 2) for repair embed requests
- Replace silent catch-all with error-logging catch blocks
2026-02-05 21:15:26 -05:00
trvon
fc4b5ceffb
fix: fixing metrics system and removing deadcode 2026-02-05 19:41:50 -05:00
trvon
e0612298d3
fix: fixing rocm builds, removing dead code and some optimizations nightly-20260206-e0612298 2026-02-05 18:33:54 -05:00
trvon
af6fdbd5c3
docs: updating changelog 2026-02-05 12:52:00 -05:00
trvon
5b06ca7a2c
fix(onnx): ensure GPU provider defines are passed to plugin
The ONNX plugin was not receiving -DYAMS_ONNX_COREML_ENABLED=1 because
the CMake dependency method doesn't propagate COMPILE_DEFINITIONS from
Conan's pkg-config.

Changes:
- Try pkg-config first to get compile definitions with GPU flags
- Fall back to CMake method if pkg-config not available
- Add explicit platform detection fallback for macOS CoreML
- Support all GPU providers: CoreML, CUDA, DirectML, MIGraphX

Fixes GPU acceleration not being enabled despite setup.sh detecting it.
2026-02-05 12:48:07 -05:00
trvon
357720407b
fixing rocm gpu build logic 2026-02-05 12:07:10 -05:00
trvon
ded16c11d4
improvements of parallel ingestion 2026-02-05 11:51:24 -05:00
trvon
8a06575a0c
fix: reverted bad change. moved embedding generation to repair coordinator. Still tuning resource usage and governors 2026-02-05 10:21:25 -05:00
trvon
a9f8a3dd63
Revert "reverting resource changes"
This reverts commit b35c84cb2e.
2026-02-05 00:18:42 -05:00
trvon
1af0356419
adding back governor code 2026-02-05 00:16:01 -05:00
trvon
b35c84cb2e
reverting resource changes 2026-02-05 00:11:56 -05:00
trvon
f2a9adbd42
revert: test fixes 2026-02-05 00:09:31 -05:00
trvon
4b2185dbcf
feat(governance): implement Netflix Gradient2 adaptive concurrency limiters
Replace CPU-threshold-based scaling with latency-feedback-based adaptive
concurrency control using the Netflix Gradient2 algorithm.

Core Changes:
- Add GradientLimiter class implementing Netflix Gradient2 algorithm
- Integrate limiters into all 6 post-ingest stages (extraction, kg, symbol,
  entity, title, embed)
- Add TuneAdvisor configuration knobs for gradient limiter tuning
- Simplify ResourceGovernor to emergency-only backstop (CPU > 95%, mem > 95%)
- Update TuningManager to read dynamic limits from gradient limiters
2026-02-04 23:38:33 -05:00
trvon
b071112836
feat: test coverage improvements. Scaling bug addresses 2026-02-04 20:20:18 -05:00
trvon
14c35926f1
test improvements and bug fixes in parser, scaling logic, and mcp tools 2026-02-04 14:42:28 -05:00
trvon
de5abde337
fix: test improvements nightly-20260204-de5abde3 2026-02-03 12:49:05 -05:00
trvon
6b6bd6ba4b
feat: staging plugin and brew update and mcp connection bug fix 2026-02-03 12:47:42 -05:00
trvon
2a14a75aa6
staging v0.8.3 for brew formula update nightly-20260203-2a14a75a 2026-02-02 11:05:11 -05:00
trvon
29de453020
fix: mcp improvements for grep and search, improving helper usage v0.8.2 2026-02-02 03:28:52 -05:00
trvon
79ea83eac8
fix: test fixes annd windows build downgrade to c++20 2026-02-02 02:15:09 -05:00
trvon
a8a9c74b31
fix: ci fixes for test compilation 2026-02-01 19:24:19 -05:00