strawberry/include/strawberry/learn/shadow.hpp
trvon f76db466eb
Some checks failed
Build and Test / build (debug, clang) (push) Has been cancelled
Build and Test / build (debug, gcc) (push) Has been cancelled
Build and Test / build (release, clang) (push) Has been cancelled
Build and Test / build (release, gcc) (push) Has been cancelled
Build and Test / macos (push) Has been cancelled
Build and Test / docker (push) Has been cancelled
Stage 4: keep the live rarity model across daemon restarts
The shadow scorer's rarity model started empty at every daemon start, so
each restart cost an hour of warm-up with no decisions. The daemon now
saves it to model_dir/rarity.state (checksummed, atomic write, hourly and
at stop) and restores it at start. A restored state younger than the
7-day half-life skips the warm-up; decay catches up on the first event.

A file made with other rarity settings, a bad checksum, or a negative or
non-finite count is refused with a reason (status model.rarity_state,
doctor model.rarity_state) and the model starts empty. The file must be
owned by the daemon user and not group/world writable, like model files.

Also fixes status model.warmup_windows, which was never filled in.

Tests: R1/R2 in test_learn (round trip, decay clock, refusals), S8 in
test_shadow (warm-up skipped only when recent), daemon restart restores
the saved state, doctor warning; new fuzz_rarity_state target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 23:25:01 -06:00

124 lines
4.9 KiB
C++

#ifndef STRAWBERRY_LEARN_SHADOW_HPP
#define STRAWBERRY_LEARN_SHADOW_HPP
// Shadow mode for the inspection model (Stage 4, increment 3).
//
// The daemon feeds every telemetry record it writes into a live replay
// engine (same windows and rarity as `strawberry replay`). When a window
// closes, the shadow model scores it and the scorer decides which extra
// inspection it WOULD request:
// - the model's value v (probability the window turns out bad);
// - candidate actions with an expected benefit v x reach(action) minus
// a CPU cost x lambda; the best one if its benefit is positive and the
// hour's budget allows, else none;
// - with probability `explore`, a uniformly random other action instead
// (seeded): recorded with its propensity, so off-policy evaluation can
// judge actions the model would rarely pick.
// Each decision becomes an inspection_action telemetry record. Shadow
// decisions are logged only. A separately promoted model may hand bounded
// actions to the daemon, under an independent estimated-cost budget. Acting
// only adds inspections; it never controls blocking or rule-triggered scans.
//
// Decisions come when a window closes (about window_ns + the reorder
// horizon after its first exec). The live rarity model is saved by the
// daemon (rarity_state / restore_rarity) and restored at start; without a
// usable saved state it starts empty and the first warmup_ns are learned,
// not decided. A restored state skips the warm-up unless it is older than
// the rarity half-life (decay then catches up on the first event).
// Single-threaded (the telemetry writer thread); stats() is thread-safe.
#include "strawberry/learn/features.hpp"
#include "strawberry/learn/mlp.hpp"
#include "strawberry/learn/inspection.hpp"
#include "strawberry/learn/replay.hpp"
#include <array>
#include <atomic>
#include <memory>
#include <mutex>
#include <random>
#include <string>
#include <vector>
namespace strawberry::learn {
struct ShadowStats {
bool active = false;
uint64_t model_generation = 0;
uint64_t windows = 0;
uint64_t warmup_windows = 0;
uint64_t decisions = 0;
uint64_t explored = 0;
std::array<uint64_t, kActions> by_action{};
double would_spend_seconds_this_hour = 0;
uint64_t budget_refusals = 0;
// The promoted (acting) model.
bool acting = false;
uint64_t promoted_generation = 0;
uint64_t act_decisions = 0;
std::array<uint64_t, kActions> act_by_action{};
double act_spend_seconds_this_hour = 0;
uint64_t act_budget_refusals = 0;
uint64_t action_queue_drops = 0;
uint64_t decision_record_drops = 0;
// Saved live rarity state.
bool rarity_restored = false;
bool warmup_skipped = false;
int64_t rarity_restored_saved_ns = 0; // when the restored state was saved
};
class ShadowScorer {
public:
explicit ShadowScorer(ShadowOptions options = {});
~ShadowScorer();
ShadowScorer(const ShadowScorer&) = delete;
ShadowScorer& operator=(const ShadowScorer&) = delete;
/// nullptr stops decisions (windows are still tracked, so the rarity
/// model keeps learning).
void set_model(std::shared_ptr<const Model> model);
/// The promoted model: its decisions (limited to kExecutableActions,
/// with their own hourly budget) are logged with shadow=false and
/// handed out by take_actions() for the daemon to execute.
void set_promoted(std::shared_ptr<const Model> model);
std::vector<InspectionDecision> take_actions();
void feed(store::RecordType type, std::string_view payload);
std::vector<InspectionDecision> take_decisions();
ShadowStats stats() const;
/// Restore a state written by rarity_state(); call before feed().
/// On failure the model stays empty and `error` says why.
bool restore_rarity(std::string_view bytes, int64_t now_ns, std::string& error) noexcept;
/// The live rarity model as a state file (RarityModel::encode_state).
std::string rarity_state(int64_t now_ns) const;
private:
static constexpr std::size_t kMaxPendingActions = 64;
static constexpr std::size_t kMaxPendingDecisions = 8192;
struct Budget {
int64_t hour_start_ns = 0;
double spent = 0;
};
void on_window(const ReplayWindow& w, const WindowSummary& s);
InspectionDecision decide(const Model& model, const ShadowOptions& options, Budget& budget,
const ReplayWindow& w, const WindowSummary& s);
ShadowOptions options_;
ShadowOptions act_options_;
std::shared_ptr<const Model> model_;
std::shared_ptr<const Model> promoted_;
Budget shadow_budget_;
Budget act_budget_;
std::vector<InspectionDecision> actions_;
std::unique_ptr<Replayer> replayer_;
std::vector<InspectionDecision> pending_;
std::mt19937_64 rng_;
uint64_t warmup_seen_ = 0;
mutable std::mutex stats_mutex_;
ShadowStats stats_;
};
} // namespace strawberry::learn
#endif // STRAWBERRY_LEARN_SHADOW_HPP