yams/plugins/zyp/meson.build
Trevon 0dd3985db4
refactor: simplify runtime paths and harden retrieval (#61)
* refactor(tuning): prune dormant runtime knobs

Remove unused tuning controls and adaptive search budget state. Extract testable ONNX slot reconciliation and search telemetry hydration seams, and correct the Apple Silicon sysctl value width.

* refactor(vector): retire dormant backends

Remove unused FAISS, model-infrastructure, VectorIndexManager, and TurboQuant paths while keeping Simeon PQ and float-vector storage.

Migrate unused V2.2 quantized schemas transactionally to V2.3. Databases containing non-default quantized formats or packed-code blobs fail closed with recovery guidance and no mutation.

* refactor(plugins): remove legacy SDK tooling

Remove the Ghidra and SDK submodules, their bespoke tests, and stale scan exclusions. Retain generic PluginProcess coverage against the repository mock fixture.

* build(vector): remove stale FAISS overrides

* refactor(ipc): remove dormant FSM snapshots

* refactor(ingest): unify bounded KG enqueue

* fix(plugins): harden allocation contracts

Keep ABI outputs null or fully initialized when allocations fail, and represent empty object payloads without relying on malloc(0).

* fix(storage): define empty object reads

Return a null buffer with zero length for successful empty-object reads instead of treating malloc(0) as an allocation contract.

* refactor(storage): trim through string views

* test: prune dormant harness helpers

Remove unused readiness and fixture helpers, and keep benchmark client-count doubling in a wide integer domain.

* chore(cpp): initialize analyzer-facing locals

* build(simeon): advance analyzer cleanup

* refactor(config): drop dead daemon client keys

* refactor(ipc): inject daemon client transport

Socket and in-process execution now share DaemonClient request dispatch, so CLI fallbacks do not need alternate service paths.

* refactor(retrieval): route CLI through daemon

Search and grep now use the same daemon handlers over socket or in-process transport. Live grep remains an explicit freshness inspection.

* refactor(graph): move CLI scope into daemon

Graph CLI requests now share daemon handlers across socket and in-process transports. CWD filtering occurs before pagination so scoped totals and pages stay correct.

* build: isolate test-only runtime controls

* refactor(metadata): decouple lock telemetry

* fix(services): return on request timeout

* fix(retrieval): enforce workspace scope

* fix(graph): scope nodes before pagination

* fix(grep): drop binary result lines

* fix(daemon): bound checkpoint wait

* fix(repair): use KG health checks

* fix(storage): always verify compressed CRC

* perf(vector): skip unchanged PQ snapshots

* build(zyp): exclude bench from default build

* build: protect Meson files from formatter

* refactor: remove unreachable production APIs

* refactor(daemon): remove dormant runtime state

* fix(vector): harden daemon provider calls

Timed-out coroutines must own their client and requests. Provider discovery must also fail instead of returning synthetic embeddings in production.

* refactor(storage): remove unusable adapters

* fix(cli): unify in-process add requests

* fix(tsan): preserve async worker progress

* test(cli): isolate daemon-dependent suites

* fix(repair): expose queue backoff progress
2026-07-31 14:16:45 -04:00

223 lines
7 KiB
Meson

#zyp - Zig YAMS PDF Extractor
#High - performance PDF text extraction using zpdf(https: // github.com/Lulzx/zpdf)
if not get_option ('plugin-zyp')
subdir_done()
endif
if host_machine.system() == 'darwin' and get_option('b_coverage')
warning('Skipping zyp plugin in macOS coverage builds: Clang gcov runtime cannot link with the zyp bundle/benchmark.')
subdir_done()
endif
#Find Zig compiler(requires 0.15.x; zpdf is not Zig 0.16 - compatible yet)
zig = find_program('zig', required: false)
if not zig.found()
warning('Zig compiler not found, skipping zyp plugin. Install Zig 0.15.x to enable.')
subdir_done()
endif
zig_version_result = run_command(zig, 'version', check: false)
if zig_version_result.returncode() != 0
warning('Unable to determine Zig version, skipping zyp plugin.')
subdir_done()
endif
zig_version = zig_version_result.stdout().strip()
if not zig_version.version_compare('>=0.15.0')
warning('Zig @0@ is unsupported by zyp; zpdf requires Zig 0.15+. Skipping zyp plugin.'.format(zig_version))
subdir_done()
endif
#Platform - specific library name and rpath
if host_machine.system() == 'darwin'
zpdf_lib_name = 'libzpdf.dylib'
#@loader_path is used for shared libraries / modules to find libs in same directory
zyp_install_rpath = '@loader_path'
zyp_link_args = ['-fno-sanitize=thread']
elif host_machine.system() == 'windows'
zpdf_lib_name = 'zpdf.dll'
zpdf_import_lib_name = 'zpdf.lib'
zyp_install_rpath = '' # Windows doesn't use rpath
#Link against dynamic CRT libraries; avoid static CRT conflict
zyp_link_args = [
'/NODEFAULTLIB:libcmt',
'/NODEFAULTLIB:libcmtd',
'msvcrt.lib',
'vcruntime.lib',
'ucrt.lib',
]
else
zpdf_lib_name = 'libzpdf.so'
#$ORIGIN refers to directory containing the loading module
zyp_install_rpath = '$ORIGIN'
zyp_link_args = ['-fno-sanitize=thread']
endif
#Build zpdf shared library via Zig and copy to build directory
#Zig outputs DLLs to zpdf / zig - out / bin / on Windows, shared libs to zig - out / lib / on Unix
zpdf_src_dir = meson.current_source_dir() / 'zpdf'
if host_machine.system() == 'windows'
#Windows : DLL goes to bin /, import lib goes to lib /
zpdf_lib_src = zpdf_src_dir / 'zig-out' / 'bin' / zpdf_lib_name
zpdf_import_lib_src = zpdf_src_dir / 'zig-out' / 'lib' / zpdf_import_lib_name
else
zpdf_lib_src = zpdf_src_dir / 'zig-out' / 'lib' / zpdf_lib_name
endif
zpdf_build_dir = meson.current_build_dir()
fs = import('fs')
if not fs.exists(zpdf_src_dir / 'build.zig')
warning('zpdf source not found (missing build.zig), skipping zyp plugin build.')
subdir_done()
endif
#Platform - specific build command
if host_machine.system() == 'windows'
#Windows : Use PowerShell for build
#Build zpdf and copy both DLL and import lib to build directory
zpdf_lib = custom_target('zpdf_shared',
command: [
'powershell', '-NoProfile', '-Command',
'Push-Location "@0@"; & "@1@" build -Doptimize=ReleaseFast; Copy-Item "@2@" "@3@\\@4@"; Copy-Item "@5@" "@3@\\@6@"; Pop-Location'.format(
zpdf_src_dir,
zig.full_path(),
zpdf_lib_src,
zpdf_build_dir,
zpdf_lib_name,
zpdf_import_lib_src,
zpdf_import_lib_name
)
],
output: [zpdf_lib_name, zpdf_import_lib_name],
build_by_default: true,
console: true,
)
#Extract just the import library for linking
zpdf_import_lib = zpdf_lib[1]
elif host_machine.system() == 'darwin'
zpdf_build_command = (
'cd "@0@" && "@1@" build -Doptimize=ReleaseFast -Dtarget=@2@-macos.${MACOSX_DEPLOYMENT_TARGET:-26.0} && cp "@3@" "@4@/@5@"'
).format(
zpdf_src_dir,
zig.full_path(),
host_machine.cpu_family(),
zpdf_lib_src,
zpdf_build_dir,
zpdf_lib_name
)
zpdf_lib = custom_target('zpdf_shared',
command: [
'sh', '-c', zpdf_build_command
],
output: [zpdf_lib_name],
build_by_default: true,
console: true,
)
else
#Unix(Linux / macOS) : Use shell
zpdf_lib = custom_target('zpdf_shared',
command: [
'sh', '-c',
'cd "@0@" && "@1@" build -Doptimize=ReleaseFast && cp "@2@" "@3@/@4@"'.format(
zpdf_src_dir,
zig.full_path(),
zpdf_lib_src,
zpdf_build_dir,
zpdf_lib_name
)
],
output: [zpdf_lib_name],
build_by_default: true,
console: true,
)
endif
#Create a dependency object that includes the zpdf library
#Use build_rpath for build - time, install_rpath handles installed location
if host_machine.system() == 'windows'
#On Windows, link against the import library only(not the DLL)
#Use sources to establish build dependency, but link via explicit path
zpdf_dep = declare_dependency(
link_args: [zpdf_build_dir / zpdf_import_lib_name],
sources: [zpdf_import_lib], # Build dependency on just the import lib
)
else
zpdf_dep = declare_dependency(
link_args: ['-L' + zpdf_build_dir, '-lzpdf'],
sources: [zpdf_lib], # This creates the build dependency
)
endif
#Compiler - specific flags for this plugin.
zyp_cpp_args = []
if meson.get_compiler('cpp').get_id() != 'msvc'
zyp_cpp_args = ['-std=c++20', '-fno-sanitize=thread']
endif
#Plugin sources
zyp_sources = files(
'zyp_plugin.cpp',
'zpdf_wrapper.cpp',
'pdf_metadata.cpp',
)
#Build the plugin(minimal dependencies - no spdlog to avoid TSan conflicts)
zyp_plugin = shared_module('yams_zyp',
zyp_sources,
include_directories: [
include_directories('../../include'),
include_directories('.'),
],
dependencies: [zpdf_dep],
cpp_args: zyp_cpp_args,
link_args: zyp_link_args,
#build_rpath ensures the plugin can find libzpdf during build / test
build_rpath: zpdf_build_dir,
#install_rpath ensures the plugin can find libzpdf after installation
install_rpath: zyp_install_rpath,
build_by_default: true,
install: true,
install_dir: plugins_dir,
name_prefix: '',
)
#Install zpdf library alongside the plugin
#Use platform - specific install script
if host_machine.system() == 'windows'
meson.add_install_script('powershell', '-NoProfile', '-Command',
'$dest = if ($env:DESTDIR) { Join-Path $env:DESTDIR "@1@" } else { "@1@" }; New-Item -ItemType Directory -Force -Path $dest | Out-Null; Copy-Item -Path "@0@" -Destination $dest'.format(
zpdf_build_dir / zpdf_lib_name,
get_option('prefix') / plugins_dir
)
)
else
meson.add_install_script('sh', '-c',
'mkdir -p "$DESTDIR@1@" && cp "@0@" "$DESTDIR@1@/"'.format(
zpdf_build_dir / zpdf_lib_name,
get_option('prefix') / plugins_dir
)
)
endif
#-- -- zyp extraction benchmark -- --
#Measures sequential vs parallel PDF extraction throughput.
#Usage : zyp_extraction_bench < pdf_path>[repeat_count]
zyp_extraction_bench_sources = files(
'../../tests/benchmarks/zyp_extraction_bench.cpp',
'zpdf_wrapper.cpp',
)
zyp_extraction_bench = executable('zyp_extraction_bench',
zyp_extraction_bench_sources,
include_directories: [
include_directories('../../include'),
include_directories('../..'),
include_directories('.'),
],
dependencies: [zpdf_dep],
cpp_args: zyp_cpp_args,
build_rpath: zpdf_build_dir,
build_by_default: false,
install: false,
override_options: ['b_sanitize=none'],
)