trevon/yams
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 2
yams/docs/changelogs/v0.2.md
Trevon 956c43b80b
 v0.6.12 experiemental
2025-09-11 11:28:42 -04:00

15 KiB
Raw Permalink Blame History

Changelog Archive: v0.2.x Series

[v0.2.9] - 2025-08-14

Enhanced Deduplication Reporting

  • New stats --dedup command for block-level analysis
  • Shows deduplication ratio, space savings, most duplicated blocks
  • Validated 32% deduplication on real codebases CI version bump
  • Fixed CMake export error with spdlog dependency
  • Resolved CI failures on macOS x64 and Linux x86

[v0.2.8] - 2025-08-14

Added

  • Repair Command: New comprehensive storage maintenance command

    • yams repair --orphans: Clean orphaned metadata entries (removed 28,101 orphaned entries in testing)
    • yams repair --chunks: Remove orphaned chunk files (freed 38MB from 23,988 orphaned chunks)
    • yams repair --mime: Fix missing MIME types in documents
    • yams repair --optimize: Vacuum and optimize database (reduced 314MB to 272MB)
    • yams repair --all: Run all repair operations
    • Dry-run support with --dry-run flag for safe preview
    • Force mode with --force to skip confirmations

  • Delete Command Enhancements:

    • Added --directory option for recursive directory deletion
    • Improved metadata cleanup when deleting documents
    • Added progress indicator for large deletion operations
    • Properly removes both manifest and chunk files

  • Config Command Enhancements:

    • Added compression tuning support via configuration
    • config get - Retrieve configuration values
    • config set - Set configuration values (placeholder)
    • config list - List all configuration settings
    • config validate - Validate configuration file
    • config export - Export configuration in TOML or JSON format

Fixed

  • Stats Command:

    • Fixed object file counting (was showing 0, now correctly counts chunks)
    • Fixed unique document calculation to match total when no duplicates exist
    • Added orphaned chunk detection and reporting
    • Fixed deduplication stats to only show when duplicates exist
    • Properly filters orphaned metadata entries from counts (28,101 orphaned found)
    • Added chunk health reporting with progress indicator
    • Fixed verbose mode not being properly tracked

  • Storage Path Issues:

    • Fixed duplicate "storage" in path construction bug throughout codebase
    • Fixed content_store_impl.cpp manifest and object paths
    • Fixed same issue in delete_command.cpp and repair_command.cpp

  • Chunk Management Crisis:

    • Delete command wasn't properly removing chunks when ref_count reached 0
    • Implemented proper chunk cleanup when documents are deleted
    • Added reference counting verification in repair command
    • Fixed hash format mismatch between database (64 chars) and filesystem (62 chars + 2 char directory)

  • Database Issues:

    • Fixed view vs table issue with unreferenced_blocks (it's a VIEW not a table)
    • Properly clean block_references table where ref_count = 0
    • Added database optimization to reclaim space from deleted entries
    • Database reduced from 314MB to 272MB after VACUUM

Changed

  • Stats Output: Improved formatting and information display
    • Shows storage overhead calculation
    • Displays warnings for orphaned chunks and metadata
    • Only shows deduplication info when relevant
    • Added database inconsistency warnings
    • Better progress indicators for long operations

[v0.2.7] - 2025-08-14

Fixed

  • CI version bump
    • Conan flag fix in CI

[v0.2.6] - 2025-08-14

Fixed

  • CI version bump
    • Conan flag fix in CI

[v0.2.5] - 2025-08-14

Fixed

  • CI version bump
    • Updated git pages site
    • Build system improvements

[v0.2.4]

Fixed

  • MCP Server Signal Handling

    • Server now responds properly to Ctrl+C (SIGINT) signals
    • Replaced std::signal with sigaction to allow interrupting blocking I/O
    • Added non-blocking stdin polling with 100ms timeout in StdioTransport
    • Server checks external shutdown flag in main loop
    • Handles EINTR errors and clears stdin error state properly
    • Added clear startup messages and usage instructions to stderr

  • Dockerfile

    • Dockerfile should build correctly

  • Cross-Compilation Support

    • Added explicit Conan profiles for x86_64 and ARM64 architectures
    • CI/CD workflows now properly handle cross-compilation with dual profiles
    • Fixed zstd assembly symbol errors when cross-compiling on Apple Silicon
    • Added zstd/*:build_programs=False option to avoid build issues
    • Workflows detect native vs cross-compilation scenarios automatically

Added

  • MCP Server Documentation
    • Comprehensive MCP usage guide at docs/user_guide/mcp.md (I personally use the CLI)

Technical Details

  • Signal Handling: Uses sigaction with sa_flags=0 to interrupt blocking system calls
  • Non-blocking I/O: Implements poll() on Unix/macOS for stdin availability checking
  • Cross-Compilation: Uses Conan's dual profile system (-pr:b for build, -pr:h for host)

[v0.2.3] - 2025-08-14

Fixed

  • CI version bump
    • README updates (removed install instructions with curl, download binary)
    • Build system improvements for docker files and release

[v0.2.2] - 2025-08-14

Fixed

  • Docker Build C++20 Configuration
    • Added explicit C++20 standard setting in Conan profile
    • Fixed "Current cppstd (gnu17) is lower than required C++ standard (20)" error
    • Ensures Docker builds use correct C++ standard matching project requirements
  • Release Notes Formatting
    • Fixed Conan profile output contaminating GitHub release descriptions
    • Redirected debug output to stderr to keep release notes clean
    • Release pages now display properly formatted markdown
  • macOS Cross-Compilation
    • Fixed Boost build failures when cross-compiling x86_64 on ARM64 macOS
    • Disabled Boost.Locale and Boost.Stacktrace.Backtrace components that fail in cross-compilation
    • Enables successful builds on Apple Silicon machines with Rosetta 2

Changed

  • Docker Build Process
    • Switched from CMake presets to direct CMake commands with Conan toolchain
    • Added ninja-build to Docker dependencies for faster builds
    • More robust build process that doesn't rely on dynamically generated presets

[v0.2.1] - 2025-08-13

Fixed

  • Build System Standardization: All workflows now use Conan exclusively
    • Fixed CMake export errors for spdlog and other dependencies
    • Release workflow migrated from plain CMake to Conan-based builds
    • Ensures consistent dependency management across CI and release
    • Resolves "target requires target that is not in any export set" errors
  • Docker Build Issues
    • Fixed Conan 2.0 command syntax (conan profile show instead of conan profile show default)
    • Fixed CMake preset issues by using direct CMake commands with Conan toolchain
    • Added ninja-build to Docker dependencies for faster builds
    • Improved version handling: uses git tags for releases, dev-<sha> for development builds
    • Fixed undefined variable warnings in Dockerfile labels
    • Added proper build argument handling for YAMS_VERSION and GITHUB_SHA
  • Release Page Formatting
    • Fixed Conan profile output contaminating release notes
    • Redirected debug output to stderr to keep release notes clean

Changed

  • MIME Type Detection Default Behavior
    • MIME types are now detected automatically during add operations
    • repair-mime command remains available for fixing existing documents
    • Detection uses FileTypeDetector with both signature and extension methods
  • Docker Versioning Strategy
    • Tagged releases use semantic version from git tag
    • Development builds use dev-<short-sha> format
    • Provides clear distinction between releases and development builds

Technical Details

  • Testing Interface: MCP server now exposes test methods via conditional compilation
    • #ifdef YAMS_TESTING enables test-specific public methods
    • Replaces hacky #define private public approach
    • Maintains encapsulation while allowing proper unit testing
  • Build Workflows: Unified Conan usage with proper profiles and caching
    • Conan profile detection with C++20 enforcement
    • Package caching for faster builds
    • Consistent preset usage (conan-release) across all platforms

[v0.2.0] - 2025-08-13

Added

  • LLM Ergonomics Enhancements
    • Added --paths-only flag to search command for LLM-friendly output (one file path per line)
    • Enhanced codebase indexing workflows for AI agent integration
    • Improved documentation in PROMPT-eng.md with LLM-optimized YAMS patterns
    • Support for using YAMS as "memory layer for low context" in AI workflows
    • Magic number patterns for #include, import, function, class, shebang lines, etc.
    • Automatic programming language detection during recursive directory operations
  • Grep Command: Full-featured regex search within indexed file contents
    • Standard grep options: -A/--after, -B/--before, -C/--context for context lines
    • Pattern matching options: -i (case-insensitive), -w (whole word), -v (invert)
    • Output options: -n (line numbers), -H (with filename), -c (count), -l (files with matches)
    • Color highlighting support with --color option
    • Supports searching all indexed files or specific paths
  • Knowledge Graph Integration in Get Command
    • Added --graph flag to display related documents
    • Added --depth option (1-5) to control graph traversal depth
    • Shows documents in same directory and with similar extensions
    • Displays document relationships and metadata
  • Line-Level Search with Context
    • Enhanced search command with -n/--line-numbers flag
    • Added context options: -A/--after, -B/--before, -C/--context
    • Highlights matching lines with color
    • Shows surrounding context lines for better understanding
  • C++17 Compatibility Mode for broader system support
    • New YAMS_CXX17_MODE CMake option for legacy compiler compatibility
    • Support for GCC 9+, Clang 9+, older MSVC versions
    • Automatic fallbacks: boost::span for std::span, fmt for std::format
    • Compatible with Ubuntu 20.04 LTS, CentOS 8, older macOS versions
    • Comprehensive compatibility documentation and build matrix
    • Runtime feature detection with graceful degradation
  • Enhanced Build System Flexibility
    • Dual-mode compilation: full C++20 for modern systems, C++17 for legacy
    • Automatic dependency management with Conan and FetchContent fallbacks
    • Compiler-specific optimizations and warning configurations
    • Performance-oriented feature selection based on available capabilities
  • MCP Server Feature Parity Enhancements (~85% CLI-MCP parity achieved)
    • Enhanced search_documents tool with LLM ergonomics optimizations
      • Added paths_only parameter for LLM-friendly file path output (one per line)
      • Added line context parameters: line_numbers, after_context, before_context, context
      • Added color parameter for syntax highlighting control (always, never, auto)
    • New grep_documents tool with full CLI grep parity
      • Complete regex pattern matching across indexed document contents
      • Context options: -A/--after, -B/--before, -C/--context for surrounding lines
      • Pattern options: ignore_case, word, invert for flexible matching
      • Output modes: line_numbers, with_filename, count, files_with_matches, files_without_match
      • Color highlighting and max_count limit support
    • Knowledge graph integration in retrieve_document tool
      • Added graph parameter to include related documents in response
      • Added depth parameter (1-5) for graph traversal control
      • Added include_content parameter for full document content in graph responses
      • Finds documents in same directory, similar extensions, and metadata relationships
    • New update_metadata tool for document metadata management
      • Update metadata by document hash or name with automatic resolution
      • Support for multiple key=value metadata pairs in single operation
      • Verbose output option for detailed update information
    • Enhanced list_documents tool with comprehensive filtering and sorting
      • File type filtering: type, mime, extension, binary, text parameters
      • Time-based filtering: created_after/before, modified_after/before, indexed_after/before
      • Recent documents filter: recent parameter for N most recent files
      • Sorting control: sort_by (name, size, created, modified, indexed) and sort_order (asc, desc)
    • Enhanced get_stats tool with file type breakdown analysis
      • Added file_types parameter for detailed file type distribution
      • Shows file type counts, sizes, top extensions per type, and top MIME types

Enhanced

  • MCP Tool Documentation comprehensively updated with new schemas and examples
    • Added complete JSON schemas for all enhanced tools with parameter descriptions
    • Added practical usage examples for each tool capability
    • Updated integration documentation for Claude Desktop configuration
  • YAMS-first workflow guidance: default to indexing and searching code via YAMS pre-watch; avoid external grep/find/rg for repository queries.
  • CLI docs enhanced with pre-watch indexing workflow and YAMS-only search/grep examples.

Technical Details

  • File type classification system with MIME type analysis using getFileTypeFromMime() and isBinaryMimeType() helper methods
  • Knowledge graph traversal leverages document relationships through metadata repository queries
  • LLM ergonomics optimizations reduce context usage while maintaining full functionality
  • Tool schema definitions follow JSON-RPC standards for seamless MCP integration

Fixed

  • Indexing Tests: Updated for new API with proper factory function declarations
  • Namespace Issues: Fixed forward declarations in document_indexer.h
  • CLI11 Conflicts: Resolved -h flag conflict in grep command by using only --no-filename
  • Test Suite Stabilization
    • Fixed SEGV crash in vector_database_tests (DocumentChunkerTest bounds checking)
    • Fixed AddressSanitizer container overflow in detection_tests
    • Added thread safety to FixtureManager test infrastructure with proper mutex protection
    • Added thread safety to FileTypeDetector for libmagic operations (not thread-safe by default)
    • Fixed compilation errors in multiple test files (header includes, error codes, GMock dependencies)
    • Fixed EXPECT_NO_THROW macro usage in detection tests
  • Recursive Add Pattern Parsing
    • Fixed CLI11 comma-separated pattern handling in --include and --exclude options
    • Now properly splits patterns like "*.cpp,*.h,*.md" into individual patterns
    • Added proper whitespace trimming and empty pattern filtering
  • CI/CD Pipeline Improvements
    • Fixed Conan options quoting for proper test builds ("yams/*:build_tests=True")
    • Added explicit YAMS_BUILD_TESTS=ON to CMake configuration
    • Added individual test target builds before running ctest
    • Implemented graceful test failure handling (CI continues, reports detailed results)
    • Added ASAN relaxation options for CI environment compatibility
    • Fixed YAML syntax error in release.yml (Python script indentation)