yams/tests/CATCH2_PATTERNS.md

Catch2 Test Patterns for YAMS

Version: 1.0
Updated: 2025-10-25
Based on: 5 migrated Catch2 tests (daemon, metadata, services)

This document captures established patterns for writing Catch2 tests in YAMS, extracted from successfully migrated tests.

---

Table of Contents

1. Basic Structure
2. Schema Validation Pattern
3. Isolated Test Harnesses
4. BDD-Style Organization
5. Async Operations
6. Tag Conventions
7. Migration Guidelines

---

Basic Structure

Minimal Test File

#include <catch2/catch_test_macros.hpp>
#include <catch2/catch_session.hpp>

using namespace yams;

TEST_CASE("Component behavior description", "[module][component]") {
    // Setup
    auto component = createTestComponent();
    
    SECTION("specific behavior case") {
        // Arrange
        auto input = createInput();
        
        // Act
        auto result = component.process(input);
        
        // Assert
        REQUIRE(result.has_value());
        REQUIRE(result->expected_field == expected_value);
    }
}

Main Runner (for standalone test files)

#define CATCH_CONFIG_MAIN
#include <catch2/catch_test_macros.hpp>
#include <catch2/catch_session.hpp>

// Tests follow...

---

Schema Validation Pattern

Used in: retrieval_schema_validation_test.cpp, search_metadata_interface_test.cpp, symbol_metadata_schema_test.cpp

Pattern: Validate API Parameters Match Database Schema

#include <catch2/catch_test_macros.hpp>
#include <yams/metadata/database.h>
#include <yams/metadata/metadata_repository.h>

namespace {

class SchemaValidator {
public:
    SchemaValidator() {
        // Create temp database
        auto tempDir = std::filesystem::temp_directory_path() / "yams_test_schema";
        std::filesystem::create_directories(tempDir);
        auto ts = std::chrono::steady_clock::now().time_since_epoch().count();
        dbPath_ = tempDir / (std::string("schema_") + std::to_string(ts) + ".db");
        
        // Initialize with migrations
        pool_ = std::make_shared<ConnectionPool>(dbPath_.string());
        auto initResult = pool_->initialize();
        REQUIRE(initResult.has_value());
        
        pool_->withConnection([](Database& db) -> Result<void> {
            MigrationManager mm(db);
            auto initResult = mm.initialize();
            if (!initResult) return initResult.error();
            
            mm.registerMigrations(YamsMetadataMigrations::getAllMigrations());
            return mm.migrate();
        });
        
        // Cache column info
        pool_->withConnection([this](Database& db) -> Result<void> {
            table_columns_ = getColumnsFromDB(db, "target_table");
            return Result<void>{};
        });
    }
    
    ~SchemaValidator() {
        repo_.reset();
        pool_.reset();
        std::filesystem::remove(dbPath_);
    }
    
    std::set<std::string> getColumnsFromDB(Database& db, const std::string& table) {
        std::set<std::string> columns;
        auto result = db.executeQuery("PRAGMA table_info(" + table + ")");
        if (result && result->hasRows()) {
            while (result->hasRow()) {
                columns.insert(result->getString(1)); // column name
                result->step();
            }
        }
        return columns;
    }
    
    bool hasColumn(const std::string& table, const std::string& column) {
        return table_columns_.count(column) > 0;
    }

private:
    std::filesystem::path dbPath_;
    std::shared_ptr<ConnectionPool> pool_;
    std::unique_ptr<MetadataRepository> repo_;
    std::set<std::string> table_columns_;
};

} // namespace

TEST_CASE("Service parameters map to schema", "[metadata][schema][validation]") {
    SchemaValidator validator;
    
    SECTION("field X maps to documents.column_y") {
        REQUIRE(validator.hasColumn("documents", "column_y"));
    }
    
    SECTION("metadata filter uses valid columns") {
        // Example: validate that MetadataFilter::path_pattern uses "path" column
        REQUIRE(validator.hasColumn("documents", "path"));
    }
}

Benefits:

• Catches schema drift at compile/test time
• Prevents runtime errors from mismatched field names
• Documents schema dependencies in tests

---

Isolated Test Harnesses

Pattern: DaemonHarness (Integration Tests)

Used in: daemon_socket_client_test.cpp

#include "test_daemon_harness.h"
#include <catch2/catch_test_macros.hpp>

TEST_CASE("Daemon integration test", "[daemon][integration]") {
    DaemonHarness harness; // RAII - creates temp socket/db/cas
    
    SECTION("operation requires running daemon") {
        REQUIRE(harness.start()); // Blocks until Ready state
        
        auto client = createClient(harness.socketPath());
        auto result = yams::cli::run_sync(client.connect(), 3s);
        
        REQUIRE(result.has_value());
        // harness.stop() called automatically on destruction
    }
}

DaemonHarness Features:

• Creates isolated temp directory per test
• Manages daemon lifecycle (start/stop)
• Provides socket path, db path, CAS path
• Cleans up automatically (RAII)

Pattern: IsolatedStorage (Unit Tests)

Proposed (not yet implemented, but follows established pattern):

class IsolatedStorage {
public:
    IsolatedStorage() {
        temp_dir_ = std::filesystem::temp_directory_path() / 
                    ("yams_test_" + generateUniqueId());
        std::filesystem::create_directories(temp_dir_);
        
        db_ = std::make_unique<Database>(temp_dir_ / "test.db");
        cas_ = std::make_unique<CAS>(temp_dir_ / "cas");
        repo_ = std::make_unique<MetadataRepository>(*pool_);
    }
    
    ~IsolatedStorage() {
        repo_.reset();
        cas_.reset();
        db_.reset();
        std::filesystem::remove_all(temp_dir_);
    }
    
    Database& db() { return *db_; }
    CAS& cas() { return *cas_; }
    MetadataRepository& repo() { return *repo_; }

private:
    std::filesystem::path temp_dir_;
    std::unique_ptr<Database> db_;
    std::unique_ptr<CAS> cas_;
    std::unique_ptr<MetadataRepository> repo_;
};

TEST_CASE("Storage operation", "[storage][unit]") {
    IsolatedStorage storage;
    
    SECTION("operation") {
        auto result = storage.repo().insertDocument(...);
        REQUIRE(result.has_value());
    }
}

---

BDD-Style Organization

Pattern: Behavior-Driven Structure with SECTION

Used in: All 5 migrated tests

TEST_CASE("Component::method behavior", "[component][method]") {
    // Shared setup
    auto component = createComponent();
    
    SECTION("succeeds with valid input") {
        auto result = component.method(validInput);
        REQUIRE(result.has_value());
        REQUIRE(result->value == expected);
    }
    
    SECTION("fails with invalid input") {
        auto result = component.method(invalidInput);
        REQUIRE(!result.has_value());
        REQUIRE(result.error().code == ErrorCode::kInvalidInput);
    }
    
    SECTION("handles edge case X") {
        auto result = component.method(edgeCaseInput);
        REQUIRE(result.has_value());
        // Edge case specific assertions
    }
}

Benefits:

• Readable test names (clear intent)
• Shared setup code (runs once per SECTION)
• Easy to add new test cases
• Better test failure messages

Nested SECTIONS for Complex Scenarios

TEST_CASE("Complex component behavior", "[component]") {
    auto component = createComponent();
    
    SECTION("when configured with option A") {
        component.configure(OptionA);
        
        SECTION("processes input X correctly") {
            REQUIRE(component.process(X).has_value());
        }
        
        SECTION("rejects input Y") {
            REQUIRE(!component.process(Y).has_value());
        }
    }
    
    SECTION("when configured with option B") {
        component.configure(OptionB);
        
        SECTION("processes input X differently") {
            auto result = component.process(X);
            REQUIRE(result.has_value());
            REQUIRE(result->differentBehavior == true);
        }
    }
}

---

Async Operations

Pattern: run_sync for Awaitable Operations

Used in: daemon_socket_client_test.cpp

#include "test_async_helpers.h"
#include <catch2/catch_test_macros.hpp>

TEST_CASE("Async operation", "[async]") {
    SECTION("completes successfully") {
        auto client = createClient();
        
        // Convert awaitable to sync for testing
        auto result = yams::cli::run_sync(
            client.asyncOperation(),  // Returns awaitable<Result<T>>
            5s                        // Timeout
        );
        
        REQUIRE(result.has_value());
        REQUIRE(result->field == expected);
    }
    
    SECTION("times out appropriately") {
        auto client = createSlowClient();
        
        auto result = yams::cli::run_sync(
            client.slowOperation(),
            100ms  // Short timeout
        );
        
        // May timeout or complete - test both cases
        if (!result.has_value()) {
            // Timeout expected
            REQUIRE(result.error().code == ErrorCode::kTimeout);
        }
    }
}

---

Tag Conventions

Based on existing tests:

Module Tags (Required)

• [daemon] - Daemon-related tests
• [metadata] - Metadata/database tests
• [services] - Service layer tests
• [search] - Search engine tests
• [storage] - Storage (DB/CAS) tests
• [plugins] - Plugin tests
• [mcp] - MCP protocol tests

Component Tags (Recommended)

• [socket] - Socket communication
• [schema] - Schema validation
• [client] - Client implementation
• [harness] - Test harness utilities

Test Type Tags (Recommended)

• [unit] - Unit tests
• [integration] - Integration tests
• [validation] - Validation/schema tests

Usage Examples

TEST_CASE("Daemon client connects", "[daemon][socket][integration]") { ... }
TEST_CASE("Metadata schema valid", "[metadata][schema][validation]") { ... }
TEST_CASE("Storage transaction", "[storage][unit]") { ... }

Running tests by tag:

# Run all daemon tests
./test_executable "[daemon]"

# Run integration tests only
./test_executable "[integration]"

# Run daemon integration tests
./test_executable "[daemon][integration]"

# Exclude slow tests
./test_executable "~[slow]"

---

Migration Guidelines

GTest → Catch2 Conversion

1. Basic Assertion Mapping

GTest	Catch2	Notes
ASSERT_TRUE(x)	REQUIRE(x)	Aborts test on failure
EXPECT_TRUE(x)	CHECK(x)	Continues test on failure
ASSERT_EQ(a, b)	REQUIRE(a == b)	Better diagnostics in Catch2
ASSERT_NE(a, b)	REQUIRE(a != b)
ASSERT_LT(a, b)	REQUIRE(a < b)
ASSERT_THAT(x, matcher)	REQUIRE_THAT(x, matcher)	Catch2 has built-in matchers

2. Test Structure Conversion

GTest:

class ComponentTest : public ::testing::Test {
protected:
    void SetUp() override {
        component_ = std::make_unique<Component>();
    }
    
    void TearDown() override {
        component_.reset();
    }
    
    std::unique_ptr<Component> component_;
};

TEST_F(ComponentTest, ProcessesValidInput) {
    auto result = component_->process(validInput);
    ASSERT_TRUE(result.ok());
    EXPECT_EQ(result.value(), expected);
}

TEST_F(ComponentTest, RejectsInvalidInput) {
    auto result = component_->process(invalidInput);
    ASSERT_FALSE(result.ok());
}

Catch2:

TEST_CASE("Component behavior", "[component]") {
    // Setup runs once, shared by all SECTIONS
    auto component = std::make_unique<Component>();
    
    SECTION("processes valid input") {
        auto result = component->process(validInput);
        REQUIRE(result.ok());
        REQUIRE(result.value() == expected);
    }
    
    SECTION("rejects invalid input") {
        auto result = component->process(invalidInput);
        REQUIRE(!result.ok());
    }
    
    // TearDown automatic via RAII
}

3. Parameterized Tests

GTest:

class ParamTest : public ::testing::TestWithParam<int> {};

TEST_P(ParamTest, HandlesValue) {
    int value = GetParam();
    EXPECT_TRUE(validate(value));
}

INSTANTIATE_TEST_SUITE_P(Values, ParamTest,
    ::testing::Values(1, 2, 3, 4, 5));

Catch2:

#include <catch2/generators/catch_generators.hpp>

TEST_CASE("Parameterized test", "[param]") {
    auto value = GENERATE(1, 2, 3, 4, 5);
    REQUIRE(validate(value));
}

// Or with named generators
TEST_CASE("Range test", "[param]") {
    auto value = GENERATE(range(1, 6)); // 1..5
    REQUIRE(validate(value));
}

4. Migration Checklist

• Update includes: #include <catch2/catch_test_macros.hpp>
• Convert TEST_F → TEST_CASE with SECTIONs
• Convert ASSERT_* → REQUIRE()
• Convert EXPECT_* → CHECK()
• Add appropriate tags [module][component][type]
• Use BDD-style test names (behavior description)
• Replace fixtures with RAII or harness classes
• Update parameterized tests to use GENERATE
• Remove GTest-specific setup (TEST_MAIN, etc.)
• Add #define CATCH_CONFIG_MAIN if standalone

---

Anti-Patterns to Avoid

❌ Don't: Share mutable state across TEST_CASEs

static Database* globalDb = nullptr; // Bad!

TEST_CASE("Test 1") {
    globalDb = new Database(...);
    // ...
}

TEST_CASE("Test 2") {
    // Depends on Test 1 having run!
    globalDb->query(...);
}

✅ Do: Use isolated harnesses or SECTION-scoped state

TEST_CASE("Database tests") {
    IsolatedStorage storage; // Fresh per TEST_CASE
    
    SECTION("Test 1") {
        storage.db().query(...);
    }
    
    SECTION("Test 2") {
        storage.db().query(...); // Independent!
    }
}

❌ Don't: Write overly long TEST_CASEs

TEST_CASE("Everything about component") {
    // 500 lines of tests...
}

✅ Do: Split into focused TEST_CASEs with SECTIONs

TEST_CASE("Component initialization", "[component][init]") {
    SECTION("with default config") { ... }
    SECTION("with custom config") { ... }
}

TEST_CASE("Component processing", "[component][process]") {
    SECTION("valid input") { ... }
    SECTION("invalid input") { ... }
}

---

References

• Catch2 Documentation
• Catch2 Tutorial
• Existing YAMS Catch2 tests (examples):
  • tests/integration/daemon/daemon_socket_client_test.cpp - Daemon harness, async
  • tests/unit/metadata/search_metadata_interface_test.cpp - Schema validation
  • tests/unit/app/services/retrieval_schema_validation_test.cpp - Service validation

---

Document maintained by: YAMS Testing Infrastructure
Questions/Updates: Create issue in PBI-068