* fix(ci): size the onnx resource budget for aarch64 page alignment
libyams_onnx_resource.so is 67,624 bytes on aarch64 against a 64,000-byte budget, which failed the v0.20.4 linux-arm64 release job. It holds 23 KB of sections; aarch64 links with 64 KiB segment alignment (LOAD align 0x10000), which pads the file. The budget was measured on x86_64. 81,000 is the aarch64 size plus 20% headroom. Every other aarch64 payload is within 86% of its budget.
Signed-off-by: trvon <me@trevon.dev>
* fix(plugins): declare the S3 legacy factory with the plugin export macro
93f24394 marked the S3 plugin's yams_plugin_create/destroy_object_storage definitions YAMS_PLUGIN_API so they stay exported under hidden visibility, but include/yams/plugins/object_storage.h still declared them without it. MSVC rejects a definition whose linkage differs from the first declaration (C2375), so the v0.20.4 Windows release job failed to compile s3_plugin.cpp. GCC and Clang accept the mismatch, which is why the Linux and macOS lanes passed.
The header now includes yams/plugins/abi.h and declares both functions YAMS_PLUGIN_API, matching how abi.h declares the rest of the plugin C ABI. Checked with MSVC 14.51 (x64 target): a translation unit with the s3_plugin.cpp definitions fails with C2375 against the old header and compiles against the new one.
Signed-off-by: trvon <me@trevon.dev>
* fix(build): drop install_tag from the mobile install_headers call
324be33a passed install_tag: 'devel' to install_headers(), which accepts no such keyword. Meson 1.12 fails the configure with "install_headers got unknown keyword arguments install_tag" whenever -Denable-mobile-bindings=true, which broke the Tests linux-x86_64-mobile lane on experimental. Meson tags installed headers 'devel' by itself, so the header still ships only with the devel set (checked with meson introspect --install-plan and meson install --tags devel|runtime --dry-run on grim).
Signed-off-by: trvon <me@trevon.dev>
* fix(packaging): install the Windows onnx resource DLL next to the executables
yams-cli.exe, yams-daemon.exe and yams-mcp-server.exe import
yams_onnx_resource-0.dll at load time, but Meson installed it to lib\ and
yams.wxs never packaged it. The shipped v0.20.3 MSI therefore installs a
yams.exe that cannot start: extracting it (msiexec /a) and running
`yams.exe --version` exits 0xC00004BC with no output, and dumpbin
/dependents lists yams_onnx_resource-0.dll. The release job never runs the
staged CLI on Windows, so CI stayed green.
Drop the explicit Windows install_dir so Meson installs the DLL to bindir
(the import library stays in libdir) and add it to the MSI. With the change
`meson install` stages bin\yams_onnx_resource-0.dll and the staged
`yams.exe --version` prints the version.
Signed-off-by: trvon <me@trevon.dev>
* ci(release): smoke-test the staged Windows CLI before building the MSI
The v0.20.3 MSI shipped a yams.exe that exits 0xC00004BC because a load-time DLL was not packaged, and nothing in the release job ran the staged binary. Run the staged yams.exe --version with a 60 s timeout after meson install and fail the job on a non-zero exit. Checked on a Windows host against the old stage layout (exits 0xC00004BC, step fails) and the fixed one (prints 0.20.5, step passes).
Signed-off-by: trvon <me@trevon.dev>
* fix(cli): keep init vector tests hermetic and the sentinel tied to disk
cli_init_command and cli_commands failed on every platform since #247.
The Tests workflow runs unit lanes with YAMS_VDB_IN_MEMORY=1 and vectors
disabled, so the vector store init creates is in memory: no vectors.db
was written (init_command_catch2_test.cpp:350) and a forced re-init,
seeing no file, overwrote the 384 sentinel with 1024 (:455).
The two cases assert on the on-disk store, so they now pin that
environment with ScopedEnvVar instead of inheriting the CI one.
Init also stamped a dimension sentinel for the in-memory store even
though nothing was written to disk. Stamp it only when this init created
vectors.db, and cover the in-memory case.
Signed-off-by: trvon <me@trevon.dev>
* fix(test): ignore worker-thread stages in memory sync serialization test
daemon_memory_sync_apply_serialization failed on Windows with
contentStages 3 == 2 and "memory_sync stage observer threw at
apply.after_content". The memory_sync worker reconciles once as soon as
it starts and runs the apply callback on its thread. On a slow host that
first cycle lands after the harness reports ready, inside the test's
observer window, so a third apply hit the observer and set an already
satisfied promise.
Reproduced on Linux under CPU load (1 of 192 runs, same signature).
The observers now count only stages reported from the test's own caller
threads, and the attempts check allows an extra worker attempt. 0 of 384
runs fail under the same load.
Signed-off-by: trvon <me@trevon.dev>
* fix(vector): do not mmap vectors.db on Windows
repair_maintenance failed on Windows only: the live vectors.db VACUUM
reported "4 MB -> 4 MB" and the file never shrank
(repair_maintenance_catch2_test.cpp:301). The standalone VACUUM case,
with no other connection open, passes there.
The vector backend connection sets mmap_size=256MB. On Windows,
SetEndOfFile from another handle fails with ERROR_USER_MAPPED_FILE while
a view of the file is mapped, and SQLite's winTruncate ignores that
error, so the truncate at the end of the VACUUM connection's checkpoint
is silently skipped. Linux and macOS can truncate a mapped file.
Skip the mmap pragma on Windows so the idle VACUUM in RepairService can
reclaim disk.
Signed-off-by: trvon <me@trevon.dev>
* ci: name the red Tests streak when a nightly is refused
The nightly gate refused with only "no successful Tests run for SHA",
so a run of red Tests on experimental stayed silent. On refusal, also
report the Tests conclusion for that SHA, the newest failing Tests run on
experimental with its SHA and failing jobs, and the last green run, as
annotations and in the job summary. The gate itself is unchanged.
Signed-off-by: trvon <me@trevon.dev>
* fix(daemon): claim the clean-shutdown stamp exclusively on Windows
db_recovery failed on Windows with "clean shutdown stamp has only one
consumer" (db_recovery_test.cpp:108): both concurrent consumers
rejected a valid stamp.
The claim renamed the stamp with MoveFileExW. That opens the source
sharing read, write and delete and renames through the handle, so two
callers can both open the stamp and both succeed, the second moving the
first's claim file away; their opens also fail each other's reads with
ERROR_SHARING_VIOLATION. Driving the real stamp code from two threads on
Windows x64, 205 of 2000 rounds ended with both consumers rejecting it
("stamp missing or malformed"); never with both trusting it.
On Windows, open the stamp with no sharing, retrying briefly while
another opener holds it, and rename through that handle. A concurrent
consumer then finds no stamp. Same driver: 2000 of 2000 rounds with
exactly one trusted consumer and no leftover claim files. The dropped
MOVEFILE_WRITE_THROUGH only affects cross-volume copy moves; the claim is
a same-directory rename. POSIX keeps rename(2).
The test now captures both consumers' reasons on failure.
Signed-off-by: trvon <me@trevon.dev>
---------
Signed-off-by: trvon <me@trevon.dev>
The clean-shutdown stamp is written through yams.db.integrity.tmp and
consumed by renaming it to yams.db.integrity.claim.<pid>.<n>. Both are
removed on the normal path, but a crash between the rename and the unlink
leaves them behind, and nothing ever cleaned them up, so they accumulate in
the data directory across crashes.
sweepStaleDbIntegrityStampArtifacts() removes them conservatively:
- only the exact .tmp name and .claim.<pid>.<n> names are considered; the
live stamp and any other file are never touched;
- a file must be older than a minimum age, so an in-flight publish or
consume is never raced;
- a claim is removed only when its pid is provably not running (kill(pid, 0)
reports ESRCH / OpenProcess reports no such process); an unparsable owner
keeps the file.
It runs from RepairService::runRecoveryArtifactMaintenance() alongside the
corrupt-DB retention cleanup, so it only happens in the background when
maintenance is allowed, with stampArtifactMinAge (1h) in
RepairService::Config.
Signed-off-by: trvon <me@trevon.dev>
Corrupt-DB handling could destroy the only copy of documents:
- removeCorruptDbFiles() deleted every readable yams.db.corrupt-* after a
startup or `repair --orphans --remove-corrupt` salvage, including DBs whose
salvageFromCorruptDb() had failed or only partly copied, and DBs that were
never salvaged because their document count was <= the live DB's.
- A corrupt DB whose count was <= the live DB's was otherwise kept forever
with only a "skipping" log, with no path that ever decided it was safe to
drop.
- quarantineAndRecreate() renamed onto yams.db.corrupt-<second>, so two
quarantines in the same second overwrote the first copy.
Deletion is now decided by evidence, not by which code path ran:
corruptDbSalvageConfirmed() opens the corrupt DB read-only, attaches the
live DB and checks that every document hash in it is present in the live
DB (salvage copies documents keyed by hash). removeSalvagedCorruptDbs()
removes an artifact only when that check succeeds and the artifact is older
than a retention window (quarantine timestamp from its name, else mtime);
unreadable, unconfirmed or young artifacts are kept and reported with the
reason.
Startup salvage no longer deletes anything. RepairService owns cleanup:
runRecoveryArtifactMaintenance() runs from the background loop when
maintenanceAllowed(), every recoveryArtifactSweepInterval (6h), with
corruptDbRetention (7 days) in RepairService::Config. An explicit
`repair --orphans --remove-corrupt` waives the retention window but still
requires confirmation. yams doctor explains the retention policy next to
the artifacts it lists.
The "yams.db" / ".corrupt-" naming is centralized in db_recovery.h
(kMetadataDbFileName, kCorruptDbMarker, corruptDbPrefix(),
isCorruptDbFileName(), listCorruptDbs()) and used by salvage, recovery,
RepairService and the doctor checks instead of four hand-rolled scans.
Signed-off-by: trvon <me@trevon.dev>
When a stale-WAL checkpoint at startup (ServiceManager::recoverStaleWalIfPresent)
or the shutdown checkpoint (DatabaseManager::shutdown) failed with a
"malformed"/"corrupt" error, both paths deleted yams.db-wal and yams.db-shm.
A WAL that SQLite cannot checkpoint may still contain committed transactions
that never reached the main file, so deleting it discarded them with no copy
left for manual recovery.
Both paths now call quarantineSqliteSidecars(), which renames the sidecars
next to the database as yams.db-wal.quarantine-<UTC ts> (and -shm), never
over an earlier quarantine, and logs where each file went. SQLite no longer
treats them as sidecars, so the database still continues from its last
checkpoint, but the bytes are preserved. The WAL is moved first; if it cannot
be moved the SHM is left in place so SQLite sees a consistent pair.
yams doctor now lists quarantined sidecars alongside corrupt DB artifacts so
they are not forgotten.
Signed-off-by: trvon <me@trevon.dev>
+ cell tail, not free space), torn WAL, truncation, crash snapshot,
salvage row-by-row fallback, writer promise draining, pool behavior.
Runs green on libsql and stock sqlite.
