Commit graph

1,146 commits

Author SHA1 Message Date
trvon
3fd243f687
fix: repair command improvements and optimizations 2026-02-06 10:15:50 -05:00
trvon
90d28d722e
fix: docker build fixes addition clamping logic for onbound memory group 2026-02-06 09:27:34 -05:00
trvon
df1348f3c7
fix: tuning improvements and better tests. Now working address memory issue of huge queue when ingesting allot of files 2026-02-06 04:31:09 -05:00
trvon
d960839e54
fix: embeddings optimizations 2026-02-05 21:27:12 -05:00
trvon
ab06614505
fix(repair): fix SIGSEGV crash, embedding failures, and runaway repair loops
- Fix writer_drain() SIGSEGV by setting connection_closing_ on write failure
- Add InternalError/ResourceExhausted to embedding retry list
- Fix document-level embedding chunk_id (0→999999) and level mismatch
- Add circuit breaker to bail out after consecutive/rate-threshold failures
- Add semaphore concurrency limiter (max 2) for repair embed requests
- Replace silent catch-all with error-logging catch blocks
2026-02-05 21:15:26 -05:00
trvon
fc4b5ceffb
fix: fixing metrics system and removing deadcode 2026-02-05 19:41:50 -05:00
trvon
e0612298d3
fix: fixing rocm builds, removing dead code and some optimizations nightly-20260206-e0612298 2026-02-05 18:33:54 -05:00
trvon
af6fdbd5c3
docs: updating changelog 2026-02-05 12:52:00 -05:00
trvon
5b06ca7a2c
fix(onnx): ensure GPU provider defines are passed to plugin
The ONNX plugin was not receiving -DYAMS_ONNX_COREML_ENABLED=1 because
the CMake dependency method doesn't propagate COMPILE_DEFINITIONS from
Conan's pkg-config.

Changes:
- Try pkg-config first to get compile definitions with GPU flags
- Fall back to CMake method if pkg-config not available
- Add explicit platform detection fallback for macOS CoreML
- Support all GPU providers: CoreML, CUDA, DirectML, MIGraphX

Fixes GPU acceleration not being enabled despite setup.sh detecting it.
2026-02-05 12:48:07 -05:00
trvon
357720407b
fixing rocm gpu build logic 2026-02-05 12:07:10 -05:00
trvon
ded16c11d4
improvements of parallel ingestion 2026-02-05 11:51:24 -05:00
trvon
8a06575a0c
fix: reverted bad change. moved embedding generation to repair coordinator. Still tuning resource usage and governors 2026-02-05 10:21:25 -05:00
trvon
a9f8a3dd63
Revert "reverting resource changes"
This reverts commit b35c84cb2e.
2026-02-05 00:18:42 -05:00
trvon
1af0356419
adding back governor code 2026-02-05 00:16:01 -05:00
trvon
b35c84cb2e
reverting resource changes 2026-02-05 00:11:56 -05:00
trvon
f2a9adbd42
revert: test fixes 2026-02-05 00:09:31 -05:00
trvon
4b2185dbcf
feat(governance): implement Netflix Gradient2 adaptive concurrency limiters
Replace CPU-threshold-based scaling with latency-feedback-based adaptive
concurrency control using the Netflix Gradient2 algorithm.

Core Changes:
- Add GradientLimiter class implementing Netflix Gradient2 algorithm
- Integrate limiters into all 6 post-ingest stages (extraction, kg, symbol,
  entity, title, embed)
- Add TuneAdvisor configuration knobs for gradient limiter tuning
- Simplify ResourceGovernor to emergency-only backstop (CPU > 95%, mem > 95%)
- Update TuningManager to read dynamic limits from gradient limiters
2026-02-04 23:38:33 -05:00
trvon
b071112836
feat: test coverage improvements. Scaling bug addresses 2026-02-04 20:20:18 -05:00
trvon
14c35926f1
test improvements and bug fixes in parser, scaling logic, and mcp tools 2026-02-04 14:42:28 -05:00
trvon
de5abde337
fix: test improvements nightly-20260204-de5abde3 2026-02-03 12:49:05 -05:00
trvon
6b6bd6ba4b
feat: staging plugin and brew update and mcp connection bug fix 2026-02-03 12:47:42 -05:00
trvon
2a14a75aa6
staging v0.8.3 for brew formula update nightly-20260203-2a14a75a 2026-02-02 11:05:11 -05:00
trvon
29de453020
fix: mcp improvements for grep and search, improving helper usage v0.8.2 2026-02-02 03:28:52 -05:00
trvon
79ea83eac8
fix: test fixes annd windows build downgrade to c++20 2026-02-02 02:15:09 -05:00
trvon
a8a9c74b31
fix: ci fixes for test compilation 2026-02-01 19:24:19 -05:00
trvon
1fdb890825
fix: additional update optimizations for plugin 2026-02-01 18:43:52 -05:00
trvon
bb6c4b1f34
fix: fixing update logic for matches based on plugin 2026-02-01 18:29:33 -05:00
trvon
f6251f6982
fix: performance updates, v0.8.2 pre-staging 2026-02-01 17:48:19 -05:00
trvon
747adbc62c
fix: mcp update 2026-02-01 14:11:35 -05:00
trvon
73b7d1eabb
fix: compiler issues addressed and homebrew updates 2026-01-31 21:52:13 -05:00
trvon
0049dbe900
fix: windows build fixes v0.8.1 2026-01-31 18:23:15 -05:00
trvon
8835d6704e
fix: removing mobile bindings from builds 2026-01-31 17:15:25 -05:00
trvon
d6d926801b
fix: nightly fixes for homebrew, daemon fixes and install logic improvements 2026-01-31 13:46:37 -05:00
trvon
2d5f491aa2
windows build fixes 2026-01-31 09:05:22 -05:00
trvon
e12b299fca
Merge branch 'experimental' 2026-01-31 01:31:41 -05:00
trvon
7ca1afb954
fix: extract bug fixes and cleaning up unused session logic 2026-01-31 01:31:08 -05:00
trvon
46aa12c0fd
fix: performance tuning improvements, windows build fixes and other improvements 2026-01-31 01:01:52 -05:00
trvon
4707e18119
committing agent file 2026-01-30 17:56:24 -05:00
trvon
12c039c33a
mcp tool exposure 2026-01-30 17:55:50 -05:00
trvon
2946b231dd
fix: license header file updates, mcp improvements, windows build fixes 2026-01-30 17:23:33 -05:00
trvon
33b83aeb53
fix: fixing windows head ordering, reducing complexity in mcp server (debugging opencode, claude, droid failed to list tools issues) 2026-01-30 09:57:27 -05:00
trvon
0614229387
fix: onnx pool model scaling improvements, test fixes and UX improvements 2026-01-29 23:18:50 -05:00
trvon
96f19e2cd2
feat: mcp ui work and stabiliizations 2026-01-29 16:55:21 -05:00
trvon
1e50b0c531
Improve daemon shutdown reliability and timeout handling
SocketServer:
- Increase connection handler wait timeout from 100ms to 2s
- Increase accept loop wait timeout from 500ms to 3s
- Prevents race conditions where handlers are cancelled mid-operation

WorkCoordinator:
- Replace thread detach with extended join attempt
- Prevents use-after-free from detached threads accessing freed resources

Daemon Command:
- Increase shutdown wait timeout from 3s to 6s
- Increase default waitForDaemonStop timeout from 5s to 10s
- Allows time for graceful SocketServer shutdown

These changes address SIGABRT and SIGTERM issues during daemon shutdown
by ensuring async operations complete before resources are freed.
2026-01-29 11:55:07 -05:00
trvon
38ec4336bf
refactored memory scaling logic 2026-01-29 10:36:33 -05:00
trvon
b81040bffa
Update ONNX tuning tests for new profile scale values
Profile scale values changed to:
- Efficient: 0.0 (minimum 2 slots)
- Balanced: 0.5
- Aggressive: 1.0

Updated all test expectations accordingly.
2026-01-29 10:35:13 -05:00
trvon
c012adc03a
Fix client timeout recovery tests - add missing error message keywords
The tests were checking for specific error message keywords that didn't
cover all possible error scenarios:
- Added 'pipe' and 'shutdown' for cancellation errors
- Added 'establish' for connection refused errors

These keywords cover 'Broken pipe' and 'Failed to establish connection'
error messages that occur during daemon shutdown.
2026-01-29 09:39:08 -05:00
trvon
7416224ec6
Fix ONNX tuning integration tests - correct profile scale factor expectations
The tests had incorrect expectations for profile scale factors:
- Efficient: 0.40 (not 0.5)
- Balanced: 0.75 (not 1.0)
- Aggressive: 1.0 (not 1.5)

Updated all test assertions to match actual TuneAdvisor::profileScale() values.
2026-01-29 09:23:08 -05:00
trvon
49aa12fa90
improved tests and mcp improvements 2026-01-29 00:27:07 -05:00
trvon
41db2b1fb7
reverting metadata cache patch and fixing list metadata pathing for speed. cleaning up ux add and list commands and remove legacy metadata path 2026-01-28 16:51:03 -05:00