trevon/yams
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2
yams/setup.sh
Trevon 6248c87bcc
Some checks are pending
Fuzz Smoke / libFuzzer smoke (push) Waiting to run
Details
test(daemon): replay startup schedules under a fuzzer (#335) 
Startup ordering defects were only observable by waiting minutes on a live
daemon, and the one that aborted it took a live restart loop to find. Give the
startup contracts a deterministic harness that runs in milliseconds.

+ startup_fuzz_support.h drives the real ServiceManagerFsm and the real
  DeferralGate with virtual time: legal phase advances, injected failure,
  shutdown and terminal reset, idle/busy, pressure/calm, starvation deadlines
  and cancellation
+ oracles fail closed on shutdown resurrection without reset, failure-to-ready,
  eligibility disagreeing with fsm.isReady(), admission outside the virtual
  deadline, and ineligible work touching the admission gate; fuzzed IEEE-754
  components must never yield non-finite or out-of-range scores
+ catch2_startup_replay runs named regressions plus 1024 seeded traces per run
  and reports the root seed and action offset for replay
+ fuzz_startup builds the same harness for libFuzzer/AFL, with a seed generator
  that refuses to overwrite different existing files
- the fuzzing lane no longer defaults to TSan: harnesses link
  -fsanitize=fuzzer,address,undefined and the two runtimes cannot share a
  process; combining them now fails with an explanation instead of producing a
  binary that cannot start
- documented the lane invocation, target naming and the coverage limits,
  including that governor call ordering inside the poller is not covered

Scope: contract-valid single-owner schedules. Executor interleavings, late
callbacks, concurrent FSM instances and filesystem/model faults are not modeled.

Signed-off-by: trvon <git@trevon.dev>
2026-10-22 16:48:50 -06:00

1220 lines
45 KiB
Bash
Executable file
Raw Permalink Blame History

#!/bin/bash
# Unified build script for YAMS
#
# Usage: ./setup.sh [Debug|Release|Profiling|Fuzzing] [--coverage] [--tsan] [--no-tsan] [--asan] [--no-asan]
# ./setup.sh Release --with-tests [--with-onnx] [--offline] [--system-deps]
# --with-onnx: Build the deprecated ONNX Runtime integration (off by default)
# build_type: Release (default), Debug, Profiling, or Fuzzing (TODO)
# --coverage: Enable code coverage instrumentation (Debug builds only)
# --tsan: Enable ThreadSanitizer for race detection (default for Debug builds)
# --no-tsan: Disable ThreadSanitizer (overrides default)
# --asan: Enable AddressSanitizer for memory error detection
# --no-asan: Disable AddressSanitizer (overrides default)
#
# Environment variables (for CI/advanced use):
# YAMS_OFFLINE - Disable network access for dependency resolution
# (Conan cache only, no wrap downloads)
# YAMS_USE_SYSTEM_DEPS - Skip Conan entirely and resolve dependencies via
# pkg-config/CMake/system prefixes
# YAMS_CONAN_HOST_PROFILE - Path to Conan host profile (bypasses auto-detection)
# YAMS_CONAN_ARCH - Target architecture (x86_64, armv8, etc.)
# YAMS_BUILD_DIR - Override the Meson/Conan build directory
# YAMS_EXTRA_MESON_FLAGS - Additional Meson setup flags
# YAMS_PKG_CONFIG_PATH - Extra host pkg-config search path passed to Meson
# YAMS_BUILD_PKG_CONFIG_PATH - Extra build-machine pkg-config path for Meson
# YAMS_CMAKE_PREFIX_PATH - Extra host CMake prefix path passed to Meson
# YAMS_BUILD_CMAKE_PREFIX_PATH - Extra build-machine CMake prefix path for Meson
# YAMS_MESON_NATIVE_FILE - Use this Meson native file instead of Conan output
# YAMS_MESON_CROSS_FILE - Use this Meson cross file instead of Conan output
# YAMS_MESON_WRAP_MODE - Override Meson wrap mode (default, nofallback,
# nodownload, forcefallback, nopromote)
# YAMS_ENABLE_MOBILE_BINDINGS - Build mobile C ABI shared library (true/false)
# YAMS_DISABLE_RE2 - Disable RE2 dependency and use std::regex fallback (true/false)
# YAMS_COMPILER - Force compiler (clang or gcc)
# YAMS_CPPSTD - C++ standard (17, 20, 23)
# YAMS_ENABLE_MODULES - Enable C++20 modules (true/false, auto-detected if not set)
# YAMS_LIBCXX_HARDENING - libc++ hardening mode (none, fast, extensive, debug)
# YAMS_INSTALL_PREFIX - Installation prefix (default: /usr/local or Homebrew)
# YAMS_ONNX_GPU - GPU provider for ONNX (auto|cuda|coreml|migraphx|none, default: auto)
# auto: macOS=coreml, Linux+NVIDIA=cuda, Linux+ROCm=migraphx, otherwise=none
# migraphx: AMD ROCm GPU acceleration (builds ONNX Runtime from source)
# YAMS_USE_SYSTEM_ONNX - Use system-installed onnxruntime (auto|true|false, default: auto)
# auto: detect system onnxruntime and use it if found
# true: require system onnxruntime (fail if not found)
# false: always download/build via Conan recipe
# YAMS_ONNX_PREFIX - Override prefix path for system onnxruntime
# (e.g., /opt/homebrew/opt/onnxruntime)
# YAMS_ROCM_PATH - ROCm installation path for MIGraphX builds (default: /opt/rocm)
# YAMS_ROCM_GPU_TARGETS - Semicolon-separated AMD GPU architectures for MIGraphX builds
# (default: gfx900;gfx906;gfx908;gfx90a;gfx1030;gfx1100)
# Common targets:
# gfx900 = Vega 10 (MI25, Vega 56/64)
# gfx906 = Vega 20 (MI50/60, Radeon VII)
# gfx908 = CDNA (MI100)
# gfx90a = CDNA2 (MI200 series: MI210, MI250, MI250X)
# gfx942 = CDNA3 (MI300 series: MI300A, MI300X, MI325X)
# gfx1030 = RDNA2 (RX 6800/6900 series)
# gfx1100 = RDNA3 (RX 7900 series)
# gfx1101 = RDNA3 (RX 7700/7800 series)
# gfx1200 = RDNA4 (RX 9070 series)
#
# The script prefers Clang when available, falling back to GCC otherwise. It
# ensures Conan is given a concrete C++ standard so dependencies resolve cleanly
# and keeps Meson in sync with the generated toolchain file.
set -euo pipefail
is_true() {
case "${1:-}" in
1 | true | TRUE | yes | YES | on | ON)
return 0
;;
*)
return 1
;;
esac
}
command_path_or_empty() {
command -v "$1" 2>/dev/null || true
}
ENABLE_OFFLINE=false
USE_SYSTEM_DEPS=false
ENABLE_COVERAGE=false
ENABLE_TSAN="${ENABLE_TSAN:-}" # Preserve environment variable if set
ENABLE_ASAN="${ENABLE_ASAN:-}" # Preserve environment variable if set
BUILD_TYPE_INPUT=""
ENABLE_RELEASE_TESTS=false
ENABLE_ONNX=false
TSAN_EXPLICIT=false
ASAN_EXPLICIT=false
if is_true "${YAMS_OFFLINE:-false}"; then
ENABLE_OFFLINE=true
fi
if is_true "${YAMS_USE_SYSTEM_DEPS:-false}"; then
USE_SYSTEM_DEPS=true
fi
INITIAL_MESON_BIN="$(command_path_or_empty meson)"
INITIAL_NINJA_BIN="$(command_path_or_empty ninja)"
while [[ $# -gt 0 ]]; do
case "$1" in
--coverage)
ENABLE_COVERAGE=true
shift
;;
--tsan)
ENABLE_TSAN=true
TSAN_EXPLICIT=true
shift
;;
--no-tsan)
ENABLE_TSAN=false
TSAN_EXPLICIT=true
shift
;;
--asan)
ENABLE_ASAN=true
ASAN_EXPLICIT=true
shift
;;
--no-asan)
ENABLE_ASAN=false
ASAN_EXPLICIT=true
shift
;;
--with-tests)
# Force building tests/benchmarks even in Release builds.
ENABLE_RELEASE_TESTS=true
shift
;;
--with-onnx)
# ONNX Runtime is deprecated and opt-in: it adds a heavy onnxruntime
# dependency (built from source for some GPU providers).
ENABLE_ONNX=true
shift
;;
--offline)
ENABLE_OFFLINE=true
shift
;;
--system-deps)
USE_SYSTEM_DEPS=true
shift
;;
Debug | Release | Profiling | Fuzzing | debug | release | profiling | fuzzing)
if [[ -n "${BUILD_TYPE_INPUT}" ]]; then
echo "Error: Build type specified multiple times" >&2
exit 1
fi
BUILD_TYPE_INPUT="$1"
shift
;;
*)
echo "Usage: $0 [Debug|Release|Profiling|Fuzzing] [--coverage] [--tsan] [--no-tsan] [--asan] [--no-asan] [--with-tests] [--with-onnx] [--offline] [--system-deps]" >&2
exit 1
;;
esac
done
BUILD_TYPE_INPUT=${BUILD_TYPE_INPUT:-Release}
# ONNX Runtime is deprecated and opt-in (--with-onnx). YAMS_DISABLE_ONNX=true still
# forces it off for older scripts.
if [[ "${YAMS_DISABLE_ONNX:-}" == "true" ]]; then
ENABLE_ONNX=false
fi
BUILD_TYPE_INPUT_LOWER=$(echo "${BUILD_TYPE_INPUT}" | tr '[:upper:]' '[:lower:]')
case "${BUILD_TYPE_INPUT_LOWER}" in
debug)
BUILD_TYPE="Debug"
;;
release)
BUILD_TYPE="Release"
;;
profiling)
# Profile what ships: Release Conan dependencies and the Release Meson
# configuration (-O3 -g, NDEBUG), plus Tracy and frame pointers. No sanitizers.
BUILD_TYPE="Release"
ENABLE_PROFILING=true
;;
fuzzing)
BUILD_TYPE="Debug" # Use Debug as base for Conan/Meson
ENABLE_FUZZING=true
echo "Fuzzing build: AFL++/libFuzzer instrumentation enabled"
;;
*)
echo "Unknown build type: ${BUILD_TYPE_INPUT}. Expected Debug, Release, Profiling, or Fuzzing." >&2
exit 1
;;
esac
if [[ "${ENABLE_COVERAGE}" == "true" ]] && [[ "${BUILD_TYPE}" != "Debug" ]]; then
echo "Error: --coverage flag requires Debug build type" >&2
exit 1
fi
# Select desired C++ standard.
# Override with YAMS_CPPSTD=20/23.
# macOS defaults to C++20 to stay aligned with the repo's supported Conan host
# profiles and current CI dependency graph. Other platforms prefer C++23 when
# the compiler supports constexpr containers.
if [[ -z "${YAMS_CPPSTD:-}" ]]; then
if [[ "$(uname -s)" == "Darwin" ]]; then
CPPSTD_INPUT="20"
echo "macOS defaulting to C++20 for Conan/Meson compatibility"
else
# Test if compiler supports C++23 with constexpr containers
if echo '#include <vector>
#if __cplusplus >= 202302L && defined(__cpp_lib_constexpr_vector) && __cpp_lib_constexpr_vector >= 201907L
#define HAS_CPP23_CONTAINERS 1
#else
#define HAS_CPP23_CONTAINERS 0
#endif
#if !HAS_CPP23_CONTAINERS
#error "No C++23 constexpr containers"
#endif' | "${CXX:-c++}" -std=c++23 -fsyntax-only -x c++ - >/dev/null 2>&1; then
CPPSTD_INPUT="23"
echo "Auto-detected C++23 compiler support (constexpr containers available)"
else
CPPSTD_INPUT="20"
echo "C++23 not fully supported, using C++20"
fi
fi
else
CPPSTD_INPUT="${YAMS_CPPSTD}"
fi
case "${CPPSTD_INPUT}" in
17 | 20 | 23)
CPPSTD="${CPPSTD_INPUT}"
;;
c++17 | c++20 | c++23)
CPPSTD="${CPPSTD_INPUT#c++}"
;;
*)
echo "Unknown C++ standard: ${CPPSTD_INPUT}. Expected 17|20|23 or c++17|c++20|c++23." >&2
exit 1
;;
esac
# Value for Meson project option
MESON_CPPSTD="c++${CPPSTD}"
if [[ -z "${YAMS_ENABLE_MODULES:-}" ]]; then
ENABLE_MODULES=false
if echo 'export module test; export int foo() { return 42; }' |
"${CXX:-c++}" -std=c++20 -fmodules -fsyntax-only -x c++ - >/dev/null 2>&1; then
ENABLE_MODULES=true
echo "Auto-detected C++20 module support (Clang -fmodules)"
elif echo 'export module test; export int foo() { return 42; }' |
"${CXX:-c++}" -std=c++20 -fmodules-ts -fsyntax-only -x c++ - >/dev/null 2>&1; then
ENABLE_MODULES=true
echo "Auto-detected C++20 module support (GCC -fmodules-ts)"
else
echo "C++20 modules not fully supported by compiler, disabled"
fi
else
if [[ "${YAMS_ENABLE_MODULES}" == "true" ]]; then
ENABLE_MODULES=true
echo "C++20 modules enabled via YAMS_ENABLE_MODULES=true"
else
ENABLE_MODULES=false
echo "C++20 modules disabled via YAMS_ENABLE_MODULES=false"
fi
fi
# libc++ hardening mode (only applies when using libc++)
# Options: none, fast, extensive, debug
# Override with YAMS_LIBCXX_HARDENING=fast|extensive|debug|none
LIBCXX_HARDENING=${YAMS_LIBCXX_HARDENING:-none}
# Check if using explicit Conan profile (CI mode)
CONAN_HOST_PROFILE=${YAMS_CONAN_HOST_PROFILE:-}
CONAN_ARCH=${YAMS_CONAN_ARCH:-}
# Conan names 64-bit ARM "armv8"; callers often pass uname/pacman labels.
case "${CONAN_ARCH}" in
aarch64 | arm64) CONAN_ARCH="armv8" ;;
esac
CONAN_EXTRA_OPTIONS=${YAMS_CONAN_EXTRA_OPTIONS:-}
USE_PROFILE=false
if [[ -z "${CONAN_HOST_PROFILE}" ]] && [[ "$(uname -s)" == "Darwin" ]]; then
uname_machine=$(uname -m)
if [[ "${uname_machine}" == "x86_64" ]]; then
CONAN_HOST_PROFILE="./conan/profiles/host-macos-apple-clang-x86"
else
CONAN_HOST_PROFILE="./conan/profiles/host-macos-apple-clang"
fi
fi
if [[ -n "${CONAN_HOST_PROFILE}" ]]; then
if [[ ! -f "${CONAN_HOST_PROFILE}" ]]; then
echo "ERROR: Conan host profile not found: ${CONAN_HOST_PROFILE}" >&2
exit 1
fi
USE_PROFILE=true
echo "Using explicit Conan host profile: ${CONAN_HOST_PROFILE}"
fi
CONAN_ARGS=(-s "build_type=${BUILD_TYPE}")
if [[ "${ENABLE_OFFLINE}" == "true" ]]; then
CONAN_ARGS+=(-nr --build=never)
else
CONAN_ARGS+=(--update --build=missing)
fi
# Add common Conan options (sqlite3 with FTS5, etc.)
CONAN_ARGS+=(-o "sqlite3/*:fts5=True")
# Fuzzing builds: disable programs in zstd to avoid symlink issues in Docker
if [[ "${ENABLE_FUZZING:-false}" == "true" ]]; then
CONAN_ARGS+=(-o "zstd/*:build_programs=False")
fi
# Add extra Conan options from environment (for CI)
if [[ -n "${CONAN_EXTRA_OPTIONS}" ]]; then
# shellcheck disable=SC2086
read -ra EXTRA_OPTS <<<"${CONAN_EXTRA_OPTIONS}"
CONAN_ARGS+=("${EXTRA_OPTS[@]}")
echo "Extra Conan options: ${CONAN_EXTRA_OPTIONS}"
fi
detect_version() {
local bin="$1"
if command -v "${bin}" >/dev/null 2>&1; then
if "${bin}" -dumpfullversion >/dev/null 2>&1; then
"${bin}" -dumpfullversion
elif "${bin}" -dumpversion >/dev/null 2>&1; then
"${bin}" -dumpversion
else
"${bin}" --version | head -n1 | grep -oE '[0-9]+(\.[0-9]+)*' | head -n1
fi
fi
}
COERCE_MAJOR() {
echo "$1" | cut -d. -f1
}
COMPILER_OVERRIDE=${YAMS_COMPILER:-}
# When using explicit profile, skip auto-detection of compiler settings
if [[ "${USE_PROFILE}" == "true" ]]; then
echo "Skipping compiler auto-detection (using profile: ${CONAN_HOST_PROFILE})"
# Still set CC/CXX for Meson if not already set
if [[ -z "${CC:-}" ]] && command -v clang >/dev/null 2>&1; then
export CC="clang"
export CXX="clang++"
elif [[ -z "${CC:-}" ]] && command -v gcc >/dev/null 2>&1; then
export CC="gcc"
export CXX="g++"
fi
# Add profile to Conan args
CONAN_ARGS+=(-pr:h "${CONAN_HOST_PROFILE}" -pr:b default)
CONAN_ARGS+=(-s:h "build_type=${BUILD_TYPE}")
CONAN_ARGS+=(-s:h "compiler.cppstd=${CPPSTD}")
# Add architecture if specified
if [[ -n "${CONAN_ARCH}" ]]; then
CONAN_ARGS+=(-s:h "arch=${CONAN_ARCH}")
echo "Target architecture: ${CONAN_ARCH}"
fi
else
# Auto-detection mode (original behavior)
if [[ "${COMPILER_OVERRIDE}" == clang ]] || { [[ -z "${COMPILER_OVERRIDE}" ]] && command -v clang >/dev/null 2>&1 && command -v clang++ >/dev/null 2>&1; }; then
echo "--- Using Clang toolchain ---"
export CC="clang"
export CXX="clang++"
# macOS requires libc++, Linux can use libstdc++11
if [[ "$(uname -s)" == "Darwin" ]]; then
LIBCXX="libc++"
else
LIBCXX="libstdc++11"
fi
CLANG_VERSION=$(detect_version clang++)
CLANG_MAJOR=$(COERCE_MAJOR "${CLANG_VERSION:-0}")
if [[ -z "${CLANG_MAJOR}" || "${CLANG_MAJOR}" == 0 ]]; then
echo "Unable to detect clang version." >&2
exit 1
fi
if [[ "$(uname -s)" == "Darwin" ]]; then
# On macOS use Conan's apple-clang compiler model
CONAN_ARGS+=(
-s "compiler=apple-clang"
-s "compiler.version=${CLANG_MAJOR}"
-s "compiler.libcxx=${LIBCXX}"
-s "compiler.cppstd=${CPPSTD}"
)
else
CONAN_ARGS+=(
-s "compiler=clang"
-s "compiler.version=${CLANG_MAJOR}"
-s "compiler.libcxx=${LIBCXX}"
-s "compiler.cppstd=${CPPSTD}"
)
fi
else
echo "--- Using GCC toolchain ---"
export CC="gcc"
export CXX="g++"
if ! command -v g++ >/dev/null 2>&1; then
echo "g++ not found in PATH." >&2
exit 1
fi
GCC_VERSION=$(detect_version g++)
GCC_MAJOR=$(COERCE_MAJOR "${GCC_VERSION:-0}")
if [[ -z "${GCC_MAJOR}" || "${GCC_MAJOR}" == 0 ]]; then
echo "Unable to detect gcc version." >&2
exit 1
fi
CONAN_ARGS+=(
-s "compiler=gcc"
-s "compiler.version=${GCC_MAJOR}"
-s "compiler.libcxx=libstdc++11"
-s "compiler.cppstd=${CPPSTD}"
)
fi
fi
# Wrap CC/CXX with ccache if available for faster incremental builds.
# Skip ccache environment wrapping when YAMS_SKIP_CCACHE=1 is set —
# useful for CI environments where ccache interferes with Conan
# dependency builds (e.g. OpenBLAS ASM compilation via CMake).
if [[ "${YAMS_SKIP_CCACHE:-}" == "1" ]]; then
echo "--- Skipping ccache wrapper (YAMS_SKIP_CCACHE=1) ---"
elif command -v ccache >/dev/null 2>&1; then
echo "--- Enabling ccache for faster incremental builds ---"
# Only wrap if not already wrapped
if [[ "${CC:-}" != ccache* ]]; then
export CC="ccache ${CC:-cc}"
export CXX="ccache ${CXX:-c++}"
fi
# Show ccache stats if verbose
ccache -s 2>/dev/null | head -5 || true
fi
if [[ "${ENABLE_PROFILING:-false}" == "true" ]]; then
BUILD_DIR="build/profiling"
CONAN_SUBDIR="build-profiling"
# Conan writes the Release-based profiling toolchain under a nested build-release directory
CONAN_ALT_SUBDIR="build-release"
# Profile optimized code: -O0 zone timings do not predict release hot spots.
BUILD_TYPE_MESON_LOWER="release"
elif [[ "${ENABLE_FUZZING:-false}" == "true" ]]; then
BUILD_DIR="build/fuzzing"
CONAN_SUBDIR="build-fuzzing"
# Fuzzing uses Debug as the Conan build type, so Conan writes its toolchain under
# build-debug (same as Profiling).
CONAN_ALT_SUBDIR="build-debug"
BUILD_TYPE_MESON_LOWER="debug"
elif [[ "${BUILD_TYPE}" == "Debug" ]]; then
BUILD_DIR="build/debug"
CONAN_SUBDIR="build-debug"
BUILD_TYPE_MESON_LOWER="debug"
else
BUILD_DIR="build/${BUILD_TYPE_INPUT_LOWER}"
CONAN_SUBDIR="build-${BUILD_TYPE_INPUT_LOWER}"
BUILD_TYPE_MESON_LOWER="release"
fi
if [[ -n "${YAMS_BUILD_DIR:-}" ]]; then
BUILD_DIR="${YAMS_BUILD_DIR}"
fi
# Some Conan generators ignore --output-folder and always emit to build-<build_type>.
# Remember locations to probe for toolchains and Meson cache files.
CONAN_GENERATED_DIR="${BUILD_DIR}/build-${BUILD_TYPE_INPUT_LOWER}"
CONAN_ALT_DIR="${BUILD_DIR}/${CONAN_ALT_SUBDIR:-}"
# Detect install prefix based on platform
if [[ "$(uname -s)" == "Darwin" ]]; then
# macOS: check for Homebrew installation
if command -v brew >/dev/null 2>&1; then
BREW_PREFIX=$(brew --prefix 2>/dev/null || echo "/usr/local")
INSTALL_PREFIX="${BREW_PREFIX}"
else
INSTALL_PREFIX="/usr/local"
fi
else
# Linux: use standard prefix
INSTALL_PREFIX="/usr/local"
fi
# Allow override via environment
INSTALL_PREFIX="${YAMS_INSTALL_PREFIX:-${INSTALL_PREFIX}}"
echo "Build Type: ${BUILD_TYPE_INPUT}"
if [[ "${ENABLE_PROFILING:-false}" == "true" ]]; then
echo "Profiling: Tracy enabled"
fi
if [[ "${ENABLE_FUZZING:-false}" == "true" ]]; then
echo "Fuzzing: AFL++/libFuzzer enabled"
fi
echo "Build Dir: ${BUILD_DIR}"
echo "Install Prefix: ${INSTALL_PREFIX}"
echo "C++ Std: ${MESON_CPPSTD} (Conan: ${CPPSTD})"
echo "Offline Mode: ${ENABLE_OFFLINE}"
echo "System Deps: ${USE_SYSTEM_DEPS}"
if [[ "${LIBCXX_HARDENING}" != "none" ]]; then
echo "libc++ Hardening: ${LIBCXX_HARDENING}"
fi
if [[ "${ENABLE_RELEASE_TESTS}" == "true" ]]; then
echo "Release Tests: enabled (--with-tests)"
fi
if [[ "${USE_SYSTEM_DEPS}" != "true" ]]; then
echo "--- Exporting custom Conan recipes... ---"
# Export custom onnxruntime recipe if it exists
if [[ "${ENABLE_ONNX}" != "true" ]]; then
echo "Skipping onnxruntime recipe export (ONNX disabled)"
elif [[ -f "conan/onnxruntime/conanfile.py" ]]; then
# Avoid sporadic Conan cache race/collision errors by retrying export once.
echo "Exporting onnxruntime/1.23.0 from conan/onnxruntime/"
if ! conan export conan/onnxruntime --name=onnxruntime --version=1.23.0; then
echo "Retrying conan export onnxruntime/1.23.0..."
conan export conan/onnxruntime --name=onnxruntime --version=1.23.0
fi
fi
echo "--- Running conan install... ---"
# Add policy toolchain for legacy recipes if in Docker or CI
POLICY_TC=""
if [[ -n "${DOCKERFILE_CONF_REV:-}" ]] || [[ -n "${CI:-}" ]]; then
POLICY_TC="/tmp/yams_policy_toolchain.cmake"
echo 'cmake_policy(VERSION 3.5)' >"$POLICY_TC"
CONAN_ARGS+=(-c "tools.cmake.cmaketoolchain:user_toolchain+=${POLICY_TC}")
fi
fi
# Enable tests for Debug builds in Conan (needed for Catch2/gtest dependencies)
if [[ "${BUILD_TYPE}" == "Debug" ]] || [[ "${ENABLE_PROFILING:-false}" == "true" ]] || [[ "${ENABLE_FUZZING:-false}" == "true" ]]; then
CONAN_ARGS+=(-o "&:build_tests=True")
fi
# Optionally enable tests in Conan for Release builds.
if [[ "${BUILD_TYPE}" == "Release" ]] && [[ "${ENABLE_RELEASE_TESTS}" == "true" ]]; then
CONAN_ARGS+=(-o "&:build_tests=True")
fi
# Enable Tracy profiling for profiling builds
if [[ "${ENABLE_PROFILING:-false}" == "true" ]]; then
echo "Tracy profiling enabled"
CONAN_ARGS+=(-o "yams/*:enable_profiling=True")
CONAN_ARGS+=(-o "tracy/*:enable=True")
fi
if [[ "${ENABLE_ONNX}" != "true" ]]; then
echo "ONNX support disabled (deprecated; pass --with-onnx to build it)"
CONAN_ARGS+=(-o "yams/*:enable_onnx=False")
else
CONAN_ARGS+=(-o "yams/*:enable_onnx=True")
# --- Detect system-installed onnxruntime ---
# Override with YAMS_USE_SYSTEM_ONNX=true|false|auto (default: auto)
# When auto, we detect system onnxruntime and use it if found.
# Set YAMS_ONNX_PREFIX to override the detected prefix path.
USE_SYSTEM_ONNX="${YAMS_USE_SYSTEM_ONNX:-auto}"
SYSTEM_ONNX_PREFIX="${YAMS_ONNX_PREFIX:-}"
if [[ "${USE_SYSTEM_ONNX}" == "auto" ]]; then
# Auto-detect: check pkg-config, Homebrew, common paths
DETECTED_ONNX_PREFIX=""
# 1) pkg-config
if command -v pkg-config >/dev/null 2>&1 && pkg-config --exists onnxruntime 2>/dev/null; then
DETECTED_ONNX_PREFIX=$(pkg-config --variable=prefix onnxruntime 2>/dev/null || true)
if [[ -n "${DETECTED_ONNX_PREFIX}" && -d "${DETECTED_ONNX_PREFIX}" ]]; then
echo "System onnxruntime detected via pkg-config: ${DETECTED_ONNX_PREFIX}"
else
DETECTED_ONNX_PREFIX=""
fi
fi
# 2) Homebrew (macOS)
if [[ -z "${DETECTED_ONNX_PREFIX}" && "$(uname -s)" == "Darwin" ]]; then
if command -v brew >/dev/null 2>&1; then
BREW_ONNX_PREFIX=$(brew --prefix onnxruntime 2>/dev/null || true)
if [[ -n "${BREW_ONNX_PREFIX}" && -d "${BREW_ONNX_PREFIX}/lib" && -d "${BREW_ONNX_PREFIX}/include" ]]; then
# Verify it's actually installed (brew --prefix returns cellar path even if not installed)
if ls "${BREW_ONNX_PREFIX}"/lib/libonnxruntime* >/dev/null 2>&1; then
DETECTED_ONNX_PREFIX="${BREW_ONNX_PREFIX}"
echo "System onnxruntime detected via Homebrew: ${DETECTED_ONNX_PREFIX}"
fi
fi
fi
fi
# 3) Common system paths
if [[ -z "${DETECTED_ONNX_PREFIX}" ]]; then
for candidate_prefix in /opt/homebrew /usr/local /usr; do
if ls "${candidate_prefix}"/lib/libonnxruntime* >/dev/null 2>&1; then
# Check all known header layouts:
# Flat: include/onnxruntime_c_api.h (prebuilt archives)
# Homebrew: include/onnxruntime/onnxruntime_c_api.h (brew)
# Source: include/onnxruntime/core/session/onnxruntime_c_api.h (apt/manual)
if [[ -f "${candidate_prefix}/include/onnxruntime_c_api.h" ]] ||
[[ -f "${candidate_prefix}/include/onnxruntime/onnxruntime_c_api.h" ]] ||
[[ -f "${candidate_prefix}/include/onnxruntime/core/session/onnxruntime_c_api.h" ]]; then
DETECTED_ONNX_PREFIX="${candidate_prefix}"
echo "System onnxruntime detected at: ${DETECTED_ONNX_PREFIX}"
break
fi
fi
done
fi
if [[ -n "${DETECTED_ONNX_PREFIX}" ]]; then
USE_SYSTEM_ONNX="true"
SYSTEM_ONNX_PREFIX="${SYSTEM_ONNX_PREFIX:-${DETECTED_ONNX_PREFIX}}"
else
USE_SYSTEM_ONNX="false"
fi
fi
if [[ "${USE_SYSTEM_ONNX}" == "true" ]]; then
echo "Using system onnxruntime from: ${SYSTEM_ONNX_PREFIX}"
CONAN_ARGS+=(-o "onnxruntime/*:use_system=True")
if [[ -n "${SYSTEM_ONNX_PREFIX}" ]]; then
CONAN_ARGS+=(-o "onnxruntime/*:system_prefix=${SYSTEM_ONNX_PREFIX}")
fi
# Detect system onnxruntime version for informational purposes
if command -v pkg-config >/dev/null 2>&1 && pkg-config --exists onnxruntime 2>/dev/null; then
SYS_ONNX_VERSION=$(pkg-config --modversion onnxruntime 2>/dev/null || true)
if [[ -n "${SYS_ONNX_VERSION}" ]]; then
echo "System onnxruntime version: ${SYS_ONNX_VERSION}"
fi
fi
fi
# Auto-detect GPU for ONNX acceleration (skip when using system onnxruntime)
# Override with YAMS_ONNX_GPU=cuda|coreml|migraphx|none
if [[ "${USE_SYSTEM_ONNX}" == "true" ]]; then
echo "Skipping GPU auto-detection (using system onnxruntime)"
ONNX_GPU="none" # System build has its own GPU support baked in
else
ONNX_GPU="${YAMS_ONNX_GPU:-auto}"
if [[ "${ONNX_GPU}" == "auto" ]]; then
if [[ "$(uname -s)" == "Darwin" ]]; then
# macOS: CoreML is included in standard ONNX Runtime builds
ONNX_GPU="coreml"
echo "macOS detected: enabling CoreML GPU acceleration (Neural Engine + GPU)"
elif command -v nvidia-smi >/dev/null 2>&1 && nvidia-smi >/dev/null 2>&1; then
# NVIDIA GPU detected - now validate CUDA toolkit is actually available
# nvidia-smi presence alone doesn't guarantee CUDA runtime is usable
if command -v nvcc >/dev/null 2>&1; then
# nvcc is available (CUDA toolkit installed via package manager or CUDA SDK)
ONNX_GPU="cuda"
echo "NVIDIA GPU + CUDA toolkit (nvcc) detected: enabling CUDA acceleration"
elif [[ -f /usr/local/cuda/lib64/libcudart.so ]] ||
[[ -f /usr/local/cuda/lib/libcudart.so ]] ||
ldconfig -p 2>/dev/null | grep -q libcudart; then
# CUDA runtime library found (sufficient for inference, even without nvcc)
ONNX_GPU="cuda"
echo "NVIDIA GPU + CUDA runtime (libcudart) detected: enabling CUDA acceleration"
else
# GPU present but CUDA toolkit/runtime missing - use CPU to avoid runtime failures
ONNX_GPU="none"
echo "NVIDIA GPU detected but CUDA toolkit missing: using CPU-only ONNX Runtime"
echo " Install CUDA toolkit for GPU acceleration: https://developer.nvidia.com/cuda-downloads"
fi
elif [[ -d /opt/rocm ]] || command -v rocm-smi >/dev/null 2>&1; then
# ROCm detected - use MIGraphX (builds ONNX Runtime from source)
# Detect ROCm path
if [[ -d /opt/rocm ]]; then
ROCM_PATH="/opt/rocm"
elif command -v rocm-smi >/dev/null 2>&1; then
ROCM_PATH="$(dirname "$(dirname "$(command -v rocm-smi)")")"
fi
# Check if MIGraphX is installed (ROCm 7.x uses lib/migraphx/lib/)
if [[ -f "${ROCM_PATH}/lib/libmigraphx.so" ]] || [[ -f "${ROCM_PATH}/lib/migraphx/lib/libmigraphx.so" ]] || [[ -f "${ROCM_PATH}/lib/libmigraphx_c.so" ]]; then
ONNX_GPU="migraphx"
echo "ROCm + MIGraphX detected at ${ROCM_PATH}: enabling AMD GPU acceleration"
echo " Note: ONNX Runtime will be built from source (takes ~20-30 minutes)"
# Auto-detect GPU architecture using rocminfo (with timeout in case of permission issues)
if [[ -z "${YAMS_ROCM_GPU_TARGETS:-}" ]] && command -v rocminfo >/dev/null 2>&1; then
DETECTED_TARGETS=$(timeout 5 rocminfo 2>/dev/null | grep -oP 'gfx[0-9a-z]+' | sort -u | tr '\n' ';' | sed 's/;$//' || true)
if [[ -n "${DETECTED_TARGETS}" ]]; then
export YAMS_ROCM_GPU_TARGETS="${DETECTED_TARGETS}"
echo " Auto-detected GPU target(s): ${DETECTED_TARGETS}"
else
echo " Could not auto-detect GPU targets (rocminfo failed or no permission)"
echo " Set YAMS_ROCM_GPU_TARGETS manually if needed (e.g., gfx1100 for RX 7900)"
fi
fi
else
ONNX_GPU="none"
echo "ROCm detected at ${ROCM_PATH} but MIGraphX not found"
echo " Using CPU-only ONNX Runtime"
echo " Install MIGraphX for GPU acceleration: https://github.com/ROCm/AMDMIGraphX"
fi
else
ONNX_GPU="none"
echo "No GPU detected: using CPU-only ONNX Runtime"
fi
fi
fi # end: skip GPU detection for system onnxruntime
# Handle MIGraphX-specific options (only for non-system builds)
if [[ "${USE_SYSTEM_ONNX}" != "true" ]] && [[ "${ONNX_GPU}" == "migraphx" ]]; then
# Set ROCm path for conan (use detected path or environment override)
ROCM_PATH="${YAMS_ROCM_PATH:-${ROCM_PATH:-/opt/rocm}}"
if [[ ! -d "${ROCM_PATH}" ]]; then
echo "ERROR: ROCm not found at ${ROCM_PATH}" >&2
echo " Set YAMS_ROCM_PATH to your ROCm installation path" >&2
exit 1
fi
CONAN_ARGS+=(-o "onnxruntime/*:rocm_path=${ROCM_PATH}")
echo "ROCm path: ${ROCM_PATH}"
# Pass GPU targets if specified
if [[ -n "${YAMS_ROCM_GPU_TARGETS:-}" ]]; then
echo "ROCm GPU targets: ${YAMS_ROCM_GPU_TARGETS}"
fi
fi
if [[ "${USE_SYSTEM_ONNX}" != "true" ]] && [[ "${ONNX_GPU}" != "none" ]]; then
CONAN_ARGS+=(-o "onnxruntime/*:with_gpu=${ONNX_GPU}")
echo "ONNX GPU provider: ${ONNX_GPU}"
fi
fi
if [[ "${YAMS_DISABLE_RE2:-}" == "true" ]]; then
echo "RE2 disabled (YAMS_DISABLE_RE2=true)"
CONAN_ARGS+=(-o "yams/*:enable_re2=False")
fi
MESON_TOOLCHAIN_ARG=""
MESON_TOOLCHAIN_FILE=""
MESON_NATIVE_FILE_OVERRIDE="${YAMS_MESON_NATIVE_FILE:-}"
MESON_CROSS_FILE_OVERRIDE="${YAMS_MESON_CROSS_FILE:-}"
MESON_EXTRA_NATIVE_FILES=()
if [[ -n "${YAMS_EXTRA_MESON_NATIVE_FILES:-}" ]]; then
IFS=':' read -r -a RAW_EXTRA_NATIVE_FILES <<<"${YAMS_EXTRA_MESON_NATIVE_FILES}"
for extra_native_file in "${RAW_EXTRA_NATIVE_FILES[@]}"; do
[[ -z "${extra_native_file}" ]] && continue
if [[ ! -f "${extra_native_file}" ]]; then
echo "Error: Extra Meson native file not found: ${extra_native_file}" >&2
exit 1
fi
MESON_EXTRA_NATIVE_FILES+=("${extra_native_file}")
done
fi
if [[ -n "${MESON_NATIVE_FILE_OVERRIDE}" && -n "${MESON_CROSS_FILE_OVERRIDE}" ]]; then
echo "Error: Set only one of YAMS_MESON_NATIVE_FILE or YAMS_MESON_CROSS_FILE." >&2
exit 1
fi
if [[ -n "${MESON_NATIVE_FILE_OVERRIDE}" ]]; then
if [[ ! -f "${MESON_NATIVE_FILE_OVERRIDE}" ]]; then
echo "Error: Meson native file not found: ${MESON_NATIVE_FILE_OVERRIDE}" >&2
exit 1
fi
MESON_TOOLCHAIN_ARG="--native-file"
MESON_TOOLCHAIN_FILE="${MESON_NATIVE_FILE_OVERRIDE}"
elif [[ -n "${MESON_CROSS_FILE_OVERRIDE}" ]]; then
if [[ ! -f "${MESON_CROSS_FILE_OVERRIDE}" ]]; then
echo "Error: Meson cross file not found: ${MESON_CROSS_FILE_OVERRIDE}" >&2
exit 1
fi
MESON_TOOLCHAIN_ARG="--cross-file"
MESON_TOOLCHAIN_FILE="${MESON_CROSS_FILE_OVERRIDE}"
fi
if [[ "${USE_SYSTEM_DEPS}" == "true" ]]; then
echo "--- Skipping Conan install (system dependency mode) ---"
else
# Use runtime_deploy to copy shared libraries next to executables (mainly for Windows, no-op on Unix with RPATH)
conan install . -of "${BUILD_DIR}" "${CONAN_ARGS[@]}" --deployer=runtime_deploy --deployer-folder="${BUILD_DIR}" -c tools.deployer:symlinks=False
# Check for either native or cross file (Conan generates cross file for cross-compilation)
# Conan 2.x sometimes nests generator output under build-<type>/conan even when -of points at
# ${BUILD_DIR}. Probe primary, then CONAN_GENERATED_DIR, then the legacy root, then CONAN_ALT_DIR.
NATIVE_FILE_PRIMARY="${BUILD_DIR}/${CONAN_SUBDIR}/conan/conan_meson_native.ini"
CROSS_FILE_PRIMARY="${BUILD_DIR}/${CONAN_SUBDIR}/conan/conan_meson_cross.ini"
NATIVE_FILE_SECONDARY="${CONAN_GENERATED_DIR}/conan/conan_meson_native.ini"
CROSS_FILE_SECONDARY="${CONAN_GENERATED_DIR}/conan/conan_meson_cross.ini"
NATIVE_FILE_TERTIARY="${BUILD_DIR}/conan/conan_meson_native.ini"
CROSS_FILE_TERTIARY="${BUILD_DIR}/conan/conan_meson_cross.ini"
NATIVE_FILE_QUAT="${CONAN_ALT_DIR}/conan/conan_meson_native.ini"
CROSS_FILE_QUAT="${CONAN_ALT_DIR}/conan/conan_meson_cross.ini"
if [[ -z "${MESON_TOOLCHAIN_ARG}" && -f "${NATIVE_FILE_PRIMARY}" ]]; then
MESON_TOOLCHAIN_ARG="--native-file"
MESON_TOOLCHAIN_FILE="${NATIVE_FILE_PRIMARY}"
elif [[ -z "${MESON_TOOLCHAIN_ARG}" && -f "${CROSS_FILE_PRIMARY}" ]]; then
MESON_TOOLCHAIN_ARG="--cross-file"
MESON_TOOLCHAIN_FILE="${CROSS_FILE_PRIMARY}"
elif [[ -z "${MESON_TOOLCHAIN_ARG}" && -f "${NATIVE_FILE_SECONDARY}" ]]; then
MESON_TOOLCHAIN_ARG="--native-file"
MESON_TOOLCHAIN_FILE="${NATIVE_FILE_SECONDARY}"
elif [[ -z "${MESON_TOOLCHAIN_ARG}" && -f "${CROSS_FILE_SECONDARY}" ]]; then
MESON_TOOLCHAIN_ARG="--cross-file"
MESON_TOOLCHAIN_FILE="${CROSS_FILE_SECONDARY}"
elif [[ -z "${MESON_TOOLCHAIN_ARG}" && -f "${NATIVE_FILE_TERTIARY}" ]]; then
MESON_TOOLCHAIN_ARG="--native-file"
MESON_TOOLCHAIN_FILE="${NATIVE_FILE_TERTIARY}"
elif [[ -z "${MESON_TOOLCHAIN_ARG}" && -f "${CROSS_FILE_TERTIARY}" ]]; then
MESON_TOOLCHAIN_ARG="--cross-file"
MESON_TOOLCHAIN_FILE="${CROSS_FILE_TERTIARY}"
elif [[ -z "${MESON_TOOLCHAIN_ARG}" && -n "${CONAN_ALT_DIR}" && -f "${NATIVE_FILE_QUAT}" ]]; then
MESON_TOOLCHAIN_ARG="--native-file"
MESON_TOOLCHAIN_FILE="${NATIVE_FILE_QUAT}"
elif [[ -z "${MESON_TOOLCHAIN_ARG}" && -n "${CONAN_ALT_DIR}" && -f "${CROSS_FILE_QUAT}" ]]; then
MESON_TOOLCHAIN_ARG="--cross-file"
MESON_TOOLCHAIN_FILE="${CROSS_FILE_QUAT}"
fi
if [[ -z "${MESON_TOOLCHAIN_ARG}" ]]; then
echo "Error: Conan meson toolchain file not found" >&2
echo "Checked: ${NATIVE_FILE_PRIMARY}" >&2
echo " and: ${CROSS_FILE_PRIMARY}" >&2
echo " and: ${NATIVE_FILE_SECONDARY}" >&2
echo " and: ${CROSS_FILE_SECONDARY}" >&2
echo " and: ${NATIVE_FILE_TERTIARY}" >&2
echo " and: ${CROSS_FILE_TERTIARY}" >&2
if [[ -n "${CONAN_ALT_DIR}" ]]; then
echo " and: ${NATIVE_FILE_QUAT}" >&2
echo " and: ${CROSS_FILE_QUAT}" >&2
fi
echo "Please check the output path from 'conan install'." >&2
exit 1
fi
CONAN_TOOLCHAIN_DIR="$(cd "$(dirname "${MESON_TOOLCHAIN_FILE}")" && pwd)"
CONAN_BUILD_ENV="${CONAN_TOOLCHAIN_DIR}/conanbuild.sh"
if [[ -f "${CONAN_BUILD_ENV}" ]]; then
# Activate Conan's build-machine tools before Meson configure so cross-build
# generators resolve native utilities like protoc from the Conan graph.
# Conan-generated env scripts may reference variables (e.g. ACLOCAL_PATH)
# that are unset in minimal environments like Docker; disable nounset temporarily.
for maybe_unset_var in ACLOCAL_PATH PKG_CONFIG_PATH LD_LIBRARY_PATH DYLD_LIBRARY_PATH \
DYLD_FALLBACK_LIBRARY_PATH LIBRARY_PATH CMAKE_PREFIX_PATH MANPATH; do
if [[ -z "${!maybe_unset_var+x}" ]]; then
export "${maybe_unset_var}="
fi
done
# shellcheck disable=SC1090
set +u
source "${CONAN_BUILD_ENV}"
set -u
fi
# Export BOOST_ROOT so meson's auto-detect Boost dep finds the Conan-installed
# Boost on platforms with no system Boost (Windows). Read prefix= directly
# from Conan's boost.pc — that file's location is the same dir holding
# CONAN_BUILD_ENV.
if [[ -z "${BOOST_ROOT:-}" ]]; then
BOOST_PC="${CONAN_TOOLCHAIN_DIR:-}/boost.pc"
if [[ -f "${BOOST_PC}" ]]; then
BOOST_ROOT_CANDIDATE=$(awk -F= '/^prefix[[:space:]]*=/ { print $2; exit }' "${BOOST_PC}")
if [[ -n "${BOOST_ROOT_CANDIDATE}" && -d "${BOOST_ROOT_CANDIDATE}/include/boost" ]]; then
export BOOST_ROOT="${BOOST_ROOT_CANDIDATE}"
echo "BOOST_ROOT set to: ${BOOST_ROOT}"
fi
fi
fi
fi
YAMS_PROTOC_PATH="${YAMS_PROTOC_PATH:-}"
MESON_WRAPPER_PATH=""
ACTIVE_MESON_BIN="$(command_path_or_empty meson)"
ACTIVE_NINJA_BIN="$(command_path_or_empty ninja)"
CONAN_TOOLS_OVERRIDE=false
if [[ -f "${CONAN_BUILD_ENV:-}" ]]; then
MESON_WRAPPER_PATH="${BUILD_DIR}/mesonw"
CONAN_BUILD_ENV_ESCAPED=$(printf '%q' "${CONAN_BUILD_ENV}")
cat >"${MESON_WRAPPER_PATH}" <<EOF
#!/usr/bin/env bash
set -euo pipefail
CONAN_BUILD_ENV=${CONAN_BUILD_ENV_ESCAPED}
for maybe_unset_var in ACLOCAL_PATH PKG_CONFIG_PATH LD_LIBRARY_PATH DYLD_LIBRARY_PATH \
DYLD_FALLBACK_LIBRARY_PATH LIBRARY_PATH CMAKE_PREFIX_PATH MANPATH; do
if [[ -z "\${!maybe_unset_var+x}" ]]; then
export "\${maybe_unset_var}="
fi
done
if [[ -f "\${CONAN_BUILD_ENV}" ]]; then
set +u
# shellcheck disable=SC1090
source "\${CONAN_BUILD_ENV}"
set -u
fi
exec meson "\$@"
EOF
chmod +x "${MESON_WRAPPER_PATH}"
if [[ "${ACTIVE_MESON_BIN}" != "${INITIAL_MESON_BIN}" || "${ACTIVE_NINJA_BIN}" != "${INITIAL_NINJA_BIN}" ]]; then
CONAN_TOOLS_OVERRIDE=true
fi
fi
MESON_ARGS=(
"${BUILD_DIR}"
"--prefix" "${INSTALL_PREFIX}"
"--buildtype" "${BUILD_TYPE_MESON_LOWER}"
)
if [[ -n "${MESON_TOOLCHAIN_ARG}" ]]; then
MESON_ARGS+=("${MESON_TOOLCHAIN_ARG}" "${MESON_TOOLCHAIN_FILE}")
fi
if [[ ${#MESON_EXTRA_NATIVE_FILES[@]} -gt 0 ]]; then
for extra_native_file in "${MESON_EXTRA_NATIVE_FILES[@]}"; do
MESON_ARGS+=("--native-file" "${extra_native_file}")
done
fi
MESON_WRAP_MODE="${YAMS_MESON_WRAP_MODE:-}"
if [[ -z "${MESON_WRAP_MODE}" ]]; then
if [[ "${USE_SYSTEM_DEPS}" == "true" ]]; then
MESON_WRAP_MODE="nofallback"
elif [[ "${ENABLE_OFFLINE}" == "true" ]]; then
MESON_WRAP_MODE="nodownload"
else
MESON_WRAP_MODE="default"
fi
fi
if [[ "${MESON_WRAP_MODE}" != "default" ]]; then
MESON_ARGS+=("--wrap-mode" "${MESON_WRAP_MODE}")
fi
if [[ -n "${YAMS_PKG_CONFIG_PATH:-}" ]]; then
MESON_ARGS+=("--pkg-config-path" "${YAMS_PKG_CONFIG_PATH}")
fi
if [[ -n "${YAMS_BUILD_PKG_CONFIG_PATH:-}" ]]; then
MESON_ARGS+=("--build.pkg-config-path" "${YAMS_BUILD_PKG_CONFIG_PATH}")
fi
if [[ -n "${YAMS_CMAKE_PREFIX_PATH:-}" ]]; then
MESON_ARGS+=("--cmake-prefix-path" "${YAMS_CMAKE_PREFIX_PATH}")
fi
if [[ -n "${YAMS_BUILD_CMAKE_PREFIX_PATH:-}" ]]; then
MESON_ARGS+=("--build.cmake-prefix-path" "${YAMS_BUILD_CMAKE_PREFIX_PATH}")
fi
# Detect previous configured cpp_std to decide on reconfigure vs wipe
PREV_CPPSTD=""
INTRO_OPTS_JSON="${BUILD_DIR}/meson-info/intro-buildoptions.json"
if [[ -f "${INTRO_OPTS_JSON}" ]]; then
PREV_CPPSTD=$(awk '/"name"\s*:\s*"cpp_std"/{flag=1} flag && /"value"/{gsub(/.*"value"\s*:\s*"|".*/,"",$0); print; exit}' "${INTRO_OPTS_JSON}" || true)
fi
MESON_OPTIONS=("-Dbuild-cli=true" "-Dcpp_std=${MESON_CPPSTD}" "-Denable-modules=${ENABLE_MODULES}")
if [[ -n "${YAMS_PROTOC_PATH}" ]]; then
MESON_OPTIONS+=("-Dprotoc-path=${YAMS_PROTOC_PATH}")
fi
# Add libc++ hardening mode if specified
if [[ "${LIBCXX_HARDENING}" != "none" ]]; then
MESON_OPTIONS+=("-Dlibcxx-hardening=${LIBCXX_HARDENING}")
fi
# Handle optional feature flags for Meson (must match Conan options)
if [[ "${ENABLE_ONNX}" == "true" ]]; then
MESON_OPTIONS+=("-Denable-onnx=enabled")
else
MESON_OPTIONS+=("-Denable-onnx=disabled")
MESON_OPTIONS+=("-Dplugin-onnx=false")
fi
if [[ "${YAMS_DISABLE_RE2:-}" == "true" ]]; then
MESON_OPTIONS+=("-Denable-re2=disabled")
fi
# Release/Profiling: -O3 with NDEBUG, PIE, and -g kept so packaging can split the
# debug info into separate symbol files (scripts/split-debug-symbols.sh) instead
# of shipping it. Debug/Fuzzing keep -O0 -g with assertions.
MESON_OPTIONS+=("-Db_pie=true")
if [[ "${BUILD_TYPE_MESON_LOWER}" == "release" || "${BUILD_TYPE_MESON_LOWER}" == "debugoptimized" ]]; then
MESON_OPTIONS+=("-Db_ndebug=true" "-Ddebug=true")
else
MESON_OPTIONS+=("-Db_ndebug=false")
fi
# libSQL backend: default to libsql but fall back to sqlite on Linux when unavailable
LIBSQL_BACKEND_INPUT=${YAMS_DATABASE_BACKEND:-libsql}
LIBSQL_BACKEND_LOWER=$(echo "${LIBSQL_BACKEND_INPUT}" | tr '[:upper:]' '[:lower:]')
if [[ "${LIBSQL_BACKEND_LOWER}" != "libsql" && "${LIBSQL_BACKEND_LOWER}" != "sqlite" ]]; then
echo "Error: Unknown database backend: ${LIBSQL_BACKEND_INPUT}. Expected libsql or sqlite." >&2
exit 1
fi
if [[ "${LIBSQL_BACKEND_LOWER}" == "libsql" && "${USE_SYSTEM_DEPS}" == "true" ]]; then
LIBSQL_PKG_FOUND=false
if command -v pkg-config >/dev/null 2>&1; then
if pkg-config --exists libsql || pkg-config --exists libsql-sqlite3; then
LIBSQL_PKG_FOUND=true
fi
fi
if [[ "${LIBSQL_PKG_FOUND}" != "true" ]]; then
echo "System dependency mode: libSQL not found via pkg-config; falling back to sqlite" >&2
LIBSQL_BACKEND_LOWER="sqlite"
fi
elif [[ "${LIBSQL_BACKEND_LOWER}" == "libsql" && "$(uname -s)" != "Darwin" ]]; then
LIBSQL_PKG_FOUND=false
if command -v pkg-config >/dev/null 2>&1; then
if pkg-config --exists libsql || pkg-config --exists libsql-sqlite3; then
LIBSQL_PKG_FOUND=true
fi
fi
if [[ "${LIBSQL_PKG_FOUND}" != "true" ]]; then
if [[ -d "subprojects/libsql" ]]; then
echo "libSQL subproject detected at subprojects/libsql; keeping libsql backend"
elif ! command -v cargo >/dev/null 2>&1; then
echo "libSQL not detected on Linux (pkg-config/cargo missing); falling back to sqlite" >&2
LIBSQL_BACKEND_LOWER="sqlite"
fi
fi
fi
if [[ "${LIBSQL_BACKEND_LOWER}" == "libsql" ]]; then
echo "Database backend: libsql"
if [[ -f "subprojects/libsql.wrap" && ! -d "subprojects/libsql" ]]; then
if [[ "${USE_SYSTEM_DEPS}" == "true" ]]; then
echo "libSQL not found via system dependencies; falling back to sqlite" >&2
LIBSQL_BACKEND_LOWER="sqlite"
elif [[ "${ENABLE_OFFLINE}" == "true" ]]; then
echo "Offline mode: not fetching libSQL subproject; falling back to sqlite" >&2
LIBSQL_BACKEND_LOWER="sqlite"
elif command -v meson >/dev/null 2>&1; then
echo "Fetching libSQL subproject (meson wrap)..."
if ! meson subprojects download libsql; then
echo "Failed to fetch libSQL subproject; falling back to sqlite" >&2
LIBSQL_BACKEND_LOWER="sqlite"
fi
else
echo "Meson not found; cannot fetch libSQL subproject. Falling back to sqlite" >&2
LIBSQL_BACKEND_LOWER="sqlite"
fi
fi
else
echo "Database backend: sqlite"
fi
MESON_OPTIONS+=("-Ddatabase-backend=${LIBSQL_BACKEND_LOWER}")
# Mobile bindings are off by default; allow CI/users to opt-in.
if [[ "${YAMS_ENABLE_MOBILE_BINDINGS:-}" == "true" ]]; then
MESON_OPTIONS+=("-Denable-mobile-bindings=true")
echo "Mobile bindings enabled (YAMS_ENABLE_MOBILE_BINDINGS=true)"
elif [[ "${YAMS_ENABLE_MOBILE_BINDINGS:-}" == "false" ]]; then
MESON_OPTIONS+=("-Denable-mobile-bindings=false")
echo "Mobile bindings disabled (YAMS_ENABLE_MOBILE_BINDINGS=false)"
fi
# ThreadSanitizer: default enabled for Debug builds, can be overridden with --tsan/--no-tsan
if [[ -z "${ENABLE_TSAN}" ]]; then
# Profiling stays uninstrumented so Tracy timings are not TSAN-distorted (opt in with --tsan).
# Fuzzing stays uninstrumented too: the fuzz harnesses link -fsanitize=fuzzer,address,undefined
# and ASan cannot share a process with TSan.
if [[ "${ENABLE_PROFILING:-false}" == "true" ]] || [[ "${ENABLE_FUZZING:-false}" == "true" ]]; then
ENABLE_TSAN=false
elif [[ "${BUILD_TYPE}" == "Debug" ]]; then
ENABLE_TSAN=true
else
ENABLE_TSAN=false
fi
fi
# AddressSanitizer: default disabled; enable with --asan or ENABLE_ASAN=true
if [[ -z "${ENABLE_ASAN}" ]]; then
ENABLE_ASAN=false
fi
# If ASAN was explicitly requested and TSAN wasn't explicitly chosen, prefer ASAN for this build.
if [[ "${ENABLE_ASAN}" == "true" ]] && [[ "${ASAN_EXPLICIT}" == "true" ]] && [[ "${TSAN_EXPLICIT}" == "false" ]]; then
ENABLE_TSAN=false
fi
if [[ "${ENABLE_TSAN}" == "true" ]] && [[ "${ENABLE_ASAN}" == "true" ]]; then
echo "Error: TSAN and ASAN cannot be enabled in the same build directory." >&2
echo "Use separate invocations, e.g.:" >&2
echo " ./setup.sh Debug --tsan --no-asan" >&2
echo " ./setup.sh Debug --asan --no-tsan" >&2
exit 1
fi
# Fuzz harnesses link -fsanitize=fuzzer,address,undefined, so TSan cannot share the binary.
if [[ "${ENABLE_FUZZING:-false}" == "true" ]] && [[ "${ENABLE_TSAN}" == "true" ]]; then
echo "Error: Fuzzing builds link ASan/UBSan via the fuzz harnesses and cannot use TSAN." >&2
echo "Drop --tsan, or configure a separate directory without -Dbuild-fuzzers=true." >&2
exit 1
fi
if [[ "${BUILD_TYPE}" == "Debug" ]] || [[ "${ENABLE_PROFILING:-false}" == "true" ]] || [[ "${ENABLE_FUZZING:-false}" == "true" ]]; then
MESON_OPTIONS+=(
"-Dbuild-tests=true"
"-Denable-bench-tests=true"
"-Denable-vector-tests=true"
"-Dbuild-benchmarks=true"
)
echo "Vector/embedding tests and benchmarks enabled for ${BUILD_TYPE} build"
fi
# Optionally enable tests in Release builds for benchmark executables.
if [[ "${BUILD_TYPE}" == "Release" ]] && [[ "${ENABLE_RELEASE_TESTS}" == "true" ]]; then
MESON_OPTIONS+=(
"-Dbuild-tests=true"
)
echo "Tests enabled for Release build (benchmarks under tests/benchmarks will be built)"
fi
# Sanitizers are selected through Meson's b_sanitize; enable-tsan/enable-asan only
# mirror that choice for third_party/symspell.
MESON_OPTIONS+=("-Denable-tsan=${ENABLE_TSAN}")
MESON_OPTIONS+=("-Denable-asan=${ENABLE_ASAN}")
if [[ "${ENABLE_TSAN}" == "true" ]]; then
MESON_OPTIONS+=("-Db_sanitize=thread")
cc_tool="${CC##* }"
cxx_tool="${CXX##* }"
if [[ "${cc_tool}" == clang* ]] || [[ "${cxx_tool}" == clang++* ]]; then
MESON_OPTIONS+=("-Db_lundef=false")
echo "ThreadSanitizer on Clang: forcing b_lundef=false to avoid Meson link warnings"
fi
echo "ThreadSanitizer enabled (race detection)"
elif [[ "${ENABLE_ASAN}" == "true" ]]; then
MESON_OPTIONS+=("-Db_sanitize=address,undefined")
echo "AddressSanitizer enabled (memory safety checks)"
else
# Ensure any previous sanitizer setting doesn't persist across reconfigure.
# Users can override via YAMS_EXTRA_MESON_FLAGS. Only reset explicit sanitizer
# state when reconfiguring an existing builddir; clean builds do not need it.
if [[ -f "${INTRO_OPTS_JSON}" ]]; then
MESON_OPTIONS+=("-Db_sanitize=none")
fi
fi
if [[ "${ENABLE_PROFILING:-false}" == "true" ]]; then
MESON_OPTIONS+=(
"-Denable-profiling=true"
)
echo "Tracy profiling enabled for Meson build"
fi
if [[ "${ENABLE_FUZZING:-false}" == "true" ]]; then
MESON_OPTIONS+=(
"-Dbuild-fuzzers=true"
# Sanitizer and SanitizerCoverage symbols in shared libraries resolve against the
# fuzzer executable, so shared objects must not be linked with --no-undefined.
"-Db_lundef=false"
)
echo "Fuzzing targets enabled for Meson build"
fi
if [[ "${ENABLE_COVERAGE}" == "true" ]]; then
MESON_OPTIONS+=("-Db_coverage=true")
echo "Coverage instrumentation enabled"
fi
# Add extra Meson flags from environment (for CI)
if [[ -n "${YAMS_EXTRA_MESON_FLAGS:-}" ]]; then
# shellcheck disable=SC2086
read -ra EXTRA_FLAGS <<<"${YAMS_EXTRA_MESON_FLAGS}"
MESON_OPTIONS+=("${EXTRA_FLAGS[@]}")
echo "Extra Meson flags: ${YAMS_EXTRA_MESON_FLAGS}"
fi
# Auto-enable zyp plugin if Zig 0.15.2+ is installed
if command -v zig >/dev/null 2>&1; then
ZIG_VERSION=$(zig version 2>/dev/null || echo "0.0.0")
# Parse major.minor.patch
ZIG_MAJOR=$(echo "${ZIG_VERSION}" | cut -d. -f1)
ZIG_MINOR=$(echo "${ZIG_VERSION}" | cut -d. -f2)
ZIG_PATCH=$(echo "${ZIG_VERSION}" | cut -d. -f3 | cut -d- -f1) # Handle -dev suffix
# Check if >= 0.15.2
if [[ "${ZIG_MAJOR:-0}" -gt 0 ]] ||
{ [[ "${ZIG_MAJOR:-0}" -eq 0 ]] && [[ "${ZIG_MINOR:-0}" -gt 15 ]]; } ||
{ [[ "${ZIG_MAJOR:-0}" -eq 0 ]] && [[ "${ZIG_MINOR:-0}" -eq 15 ]] && [[ "${ZIG_PATCH:-0}" -ge 2 ]]; }; then
echo "Zig ${ZIG_VERSION} detected, enabling zyp PDF plugin"
MESON_OPTIONS+=("-Dplugin-zyp=true")
else
echo "Zig ${ZIG_VERSION} found but requires 0.15.2+ for zyp plugin"
fi
fi
# Glint NL entity extractor plugin (GLiNER-based) requires ONNX Runtime.
if [[ "${ENABLE_ONNX}" == "true" ]]; then
echo "Enabling Glint NL entity extractor plugin"
MESON_OPTIONS+=("-Dplugin-glint=true")
fi
echo "--- Running meson setup... ---"
if [[ -n "${PREV_CPPSTD}" ]]; then
if [[ "${PREV_CPPSTD}" != "${MESON_CPPSTD}" ]]; then
echo "cpp_std changed (${PREV_CPPSTD} -> ${MESON_CPPSTD}); wiping build directory configuration..."
meson setup "${MESON_ARGS[@]}" "${MESON_OPTIONS[@]}" --wipe
else
meson setup "${MESON_ARGS[@]}" "${MESON_OPTIONS[@]}" --reconfigure
fi
else
meson setup "${MESON_ARGS[@]}" "${MESON_OPTIONS[@]}"
fi
echo
echo "--- Setup complete! ---"
if [[ "${CONAN_TOOLS_OVERRIDE}" == "true" ]]; then
echo "Conan provided Meson/Ninja for this build directory."
echo "To compile, run: ${MESON_WRAPPER_PATH} compile -C ${BUILD_DIR}"
echo "To install, run: ${MESON_WRAPPER_PATH} install -C ${BUILD_DIR}"
elif [[ -n "${MESON_WRAPPER_PATH}" ]]; then
echo "To compile, run: meson compile -C ${BUILD_DIR}"
echo "To install, run: meson install -C ${BUILD_DIR}"
echo "Fallback wrapper (replays Conan build env if needed): ${MESON_WRAPPER_PATH}"
else
echo "To compile, run: meson compile -C ${BUILD_DIR}"
echo "To install, run: meson install -C ${BUILD_DIR}"
fi
Reference in a new issue View git blame Copy permalink