Struct Engine 

pub struct Engine {
    pub cfg: SessionConfig,
    pub prior: PatchGrammarPrior,
    pub standardizer: Option<Arc<Standardizer>>,
    pub log: ObservationLog,
    pub posterior: Option<Arc<TastePosterior>>,
    pub session: usize,
    pub pool: Vec<Candidate>,
    pub lineage: Vec<LineageEvent>,
    pub generation: usize,
    pub style_names: Vec<String>,
    pub events: Vec<ImplicitEvent>,
    pub forecasts: Vec<Forecast>,
    /* private fields */
}

The session engine.

Fields

cfg: SessionConfig

Configuration.

prior: PatchGrammarPrior

The patch prior.

standardizer: Option<Arc<Standardizer>>

Standardizer fit on the first pool fill; persisted with the profile.

log: ObservationLog

The observation log (source of truth).

posterior: Option<Arc<TastePosterior>>

The current posterior, if fit (label-aligned).

session: usize

Current session index.

pool: Vec<Candidate>

The candidate pool.

lineage: Vec<LineageEvent>

Evolution/edit history.

generation: usize

Generation counter (one per refinement call).

style_names: Vec<String>

User-given style names (index = aligned style index; empty = unnamed).

events: Vec<ImplicitEvent>

Implicit preference events (logged, not yet modeled).

forecasts: Vec<Forecast>

Out-of-sample duel forecasts, scored before each answer was known.

Implementations

impl Engine

pub fn new(prior: PatchGrammarPrior, cfg: SessionConfig) -> Self

Create an engine over the given prior.

pub fn set_memo(&mut self, memo: RenderMemo)

Replace the featurization memo every featurize in this engine consults — fill, insert, restore, and the refinement surrogate.

Shared rather than owned so a frontend can pre-load one and read back what the engine learned. Refinement captures it by clone at Engine::refine_one time, so swapping it mid-generation is not something to do.

pub fn memo(&self) -> &RenderMemo

The featurization memo.

pub fn key_of(&self, id: u64) -> Option<&str>

Content address of candidate id’s featurization.

pub fn render_of(&mut self, id: u64) -> Option<Arc<Audition>>

The audition buffer of candidate id, materializing it if RenderPolicy::Lazy deferred it.

None for an unknown id, for RenderPolicy::None, or for a term that no longer renders — a restored bank outlives the DSP that made it, and a caller that cannot distinguish “not yet” from “never” will wait forever. This is the only honest source of that answer.

Bit-identical to the buffer the candidate’s features were measured on (auracle_features::render_playback).

Shared rather than copied: the pool, the memo and the caller all hold the same ~565 KB allocation through an Arc, so a repeat request is a refcount bump. Callers that must own samples clone the inner value at their own call site, where the cost is visible.

pub fn find(&self, id: u64) -> Option<usize>

Pool index of a candidate id.

pub fn begin_session(&mut self) -> usize

Start a new session (its own τ latent). Returns its index.

pub fn fill_pool<R: Rng>(&mut self, rng: &mut R)

Fill the pool with vetted prior draws (up to pool_size). Fits the standardizer on the first successful fill.

pub fn fill_pool_step<R: Rng>(&mut self, rng: &mut R, max_new: usize) -> usize

Add up to max_new vetted candidates (bounded by max_draws attempts). Returns how many were added — the incremental unit that lets a frontend post progress between batches. Standardization runs once the pool first reaches pool_size (or on any later addition).

This is the serial fold of the indexed draw stream (crate::farm): index i is consumed whatever its outcome, and dedupe / vetting decide only whether it lands. The farm path (Engine::fill_draw + Engine::absorb_prior) is the same fold with the render moved off-engine, so the two produce the same pool from the same fill_seed — and so does any chunking of max_new, because the cursor lives in the engine rather than in a loop variable.

pub fn ensure_fill_seed<R: Rng>(&mut self, rng: &mut R) -> u64

Base seed of this engine’s pool-draw stream, taking one from rng if the stream has not started yet.

Exactly one u64 is drawn from the caller’s RNG per engine, on the first fill. That is deliberate: the serial and farm paths consume the same amount of the caller’s stream, so everything downstream of the fill that shares that RNG (duel selection, MCMC) stays aligned between them.

pub fn fill_seed(&self) -> Option<u64>

Base seed of the pool-draw stream, if it has started.

pub fn set_fill_seed(&mut self, seed: u64)

Pin the pool-draw stream to an explicit base seed. Only meaningful before the first draw; a fill in progress keeps the seed it started on.

pub fn draw_cursor(&self) -> u64

Next index of the draw stream the fold will consume.

pub fn draw_at(&self, index: u64) -> Option<PatchTree>

The term at index of this engine’s draw stream — a pure function of (fill_seed, index) and the prior, costing microseconds and no render.

This is what makes a lost farm job re-issuable with no retained state: the job is its index.

pub fn fill_draw(&mut self, n: usize) -> Vec<Draw>

Hand out up to n unrendered draws for off-engine featurization.

Returns fewer than n — or nothing — when the pool has as much work outstanding as it can still use, or the max_draws budget is spent. An empty return is not by itself a stop signal: it may simply mean every slot the pool can still fill is already in flight. The caller stops when the pool reaches its target, or when an empty return coincides with nothing outstanding.

Requires a started stream (Engine::ensure_fill_seed or Engine::set_fill_seed); yields nothing otherwise.

pub fn absorb_prior( &mut self, index: u64, pre: Option<PreFeaturized>, ) -> Option<u64>

Fold one off-engine result into the pool.

index must be Engine::draw_cursor — results are absorbed in index order, and that ordering is the entire determinism argument: the pool at index i is a pure function of indices < i, so it cannot depend on how many renders were running. Anything else is refused (returns None without consuming), because silently absorbing out of order would produce a pool no width reproduces.

pre is None for a draw the farm rejected — a vet failure, a compile failure, or a result that failed to survive transport. The index is consumed either way, exactly as a failed draw burns an attempt in the serial loop.

Returns the new candidate id, or None when the draw did not land (rejected, duplicate, or the pool was already full).

pub fn standardize_now(&mut self)

Give every pool member a φ_std now, fitting a standardizer from the current pool if none exists yet — so a partially filled pool is already duel-able.

Engine::fill_pool_step only fits once the pool reaches pool_size, and that single condition is what forces a frontend to sit out the entire fill before it can ask its first question: Engine::next_duel_full skips candidates whose phi_std is empty, so a half-filled pool contains no legal pair at all. This is the escape hatch a progressive boot needs — it costs no renders, only the mean/variance of what has already been drawn.

It never replaces an existing standardizer. θ is only meaningful relative to the standardization its φ were measured under, so an imported profile’s geometry has to survive a boot that tops the pool up (Engine::import_profile). Re-fitting is Engine::restandardize_if_untaught’s job, and it is only safe before a posterior exists.

pub fn restandardize_if_untaught(&mut self)

Re-fit the standardizer over the finished pool — a no-op the moment a posterior exists.

A progressive boot fits a provisional standardizer over the first handful of draws (Engine::standardize_now) so the user can start voting; the completed pool is a better reference population, and re-expressing φ on it is lossless because the log stores raw values (refit_standardizer’s rationale). But once θ has been fit, its coordinates are denominated in the standardizer that was live at fit time, and moving the scale under a live posterior would silently rescale every utility in the app. So this refuses in exactly that case: the next Engine::fit_posterior re-fits both together, in the order that keeps them consistent.

pub fn fit_posterior<R: Rng>(&mut self, rng: &mut R)

Fit (or re-fit) the taste posterior from the observation log. The stored posterior is label-aligned (safe for per-style summaries) and its importance weights are reset to uniform.

pub fn style_shares(&self) -> &[StyleShareRecord]

Style shares recorded at each fit, oldest first. See StyleShareRecord.

pub fn posterior_ess(&self) -> Option<f64>

Effective sample size of the current posterior’s importance weights — how much of the draw set still carries information after the observations folded in since the last full fit. None before the first fit.

pub fn needs_refit(&self) -> bool

True when the cheap between-fit updates have run out of road and a full MCMC refit is worth its seconds: the weights have collapsed (ESS below half the draws) at least once since the last fit, or the log has evidence no posterior has seen. A frontend can drive refits off this instead of a fixed vote count.

pub fn utility_of(&self, phi_std: &[f64]) -> f64

Posterior-mean mixture utility of a standardized φ (0 with no posterior).

pub fn violates_locks( prev: &Trace, next: &Trace, locked: &HashSet<String>, ) -> bool

Did the step from prev to next touch any locked address? “Touch” = change its value, delete it, or create it (structure moves that would rewrite a locked module’s path are rejected too — locked means don’t touch).

Both directions are checked, and that is not pedantry. Scanning only prev lets a birth at a locked address through while rejecting the death that would undo it. The constraint region is then asymmetric — x → x′ allowed, x′ → x rejected — which breaks detailed balance and makes the Metropolis-within-Gibbs argument for locking being exact simply false. The chain would drift into locked structure it can never leave.

What this does and does not guarantee. locked is a set of exact address strings, typically snapshotted from the UI. Every address in it is frozen, in both directions, and that is exact. It is not the same as freezing a module: a structural move that grows a brand-new address inside a locked module — one that was in neither trace when the set was taken, so it cannot be in the set — is not caught. That case is symmetric (unmatched by construction in both directions), so it costs nothing in detailed balance; it just means “locked” is a guarantee about addresses, not about subtrees.

pub fn predict_duel(&self, a: usize, b: usize) -> Option<f64>

Posterior probability that pool member a beats b in a duel (None before the first fit).

pub fn log_event(&mut self, kind: &str, id: u64, value: f64)

Log an implicit preference event (promote, play time, …). Logged only — not yet part of the likelihood.

pub fn log_event_detail( &mut self, kind: &str, id: u64, value: f64, detail: &str, phi_before: Vec<f64>, phi_after: Vec<f64>, )

The same, carrying the editor’s detail and (for a transition) the raw φ on both sides of it. See ImplicitEvent for why the detail is an opaque string and why none of this reaches the likelihood.

pub fn set_style_name(&mut self, k: usize, name: &str)

Name (or rename; empty clears) an aligned style index.

pub fn refine<R: Rng>(&mut self, rng: &mut R)

Taste-guided refinement: run fugue-evo typed MH on the Boltzmann target from each of the top seeds, and add improved, vetted, novel candidates to the pool (evicting the worst if full). Each injection is recorded as a lineage event.

pub fn refine_begin(&mut self) -> Vec<u64>

Open a generation and return the parent ids it will refine from, best first. Empty if there is nothing to refine toward yet (no posterior), in which case the generation counter is not advanced.

This exists so a caller can drive refinement one seed at a time and report progress between seeds. A whole generation is tens of seconds of render-bound work — running it as one opaque call is what made the app look hung.

pub fn refine_seed<R: Rng>( &mut self, rng: &mut R, parent_id: u64, ) -> Option<u64>

Refine from one seed of the open generation. Returns the injected child id, or None if the walk was rejected or landed on a patch the pool already holds.

pub fn refine_from<R: Rng>( &mut self, rng: &mut R, seed_id: u64, locked: &[String], ) -> Option<u64>

Locked refinement from one explicit seed candidate: evolve everything except the locked addresses. Returns the injected child id.

pub fn commit_edit( &mut self, original_id: Option<u64>, tree: PatchTree, outcome: EditOutcome, ) -> Option<u64>

Commit a hand-edited tree as a new candidate. If original_id is given, a lineage event links them; outcome says what the player reported about the pair, and only a told outcome writes an observation.

pub fn ranked(&self) -> Vec<(usize, f64, f64)>

Pool indices ranked by posterior-mean mixture utility (descending); with no posterior, arbitrary order with zero scores.

pub fn next_duel<R: Rng>(&mut self, rng: &mut R) -> Option<(usize, usize)>

Choose the next duel by expected information gain about θ (BALD), traded off against how pleasant the duel is to answer and penalized for repetition, then sampled from a softmax rather than argmaxed.

Returns pool indices (a, b); None if fewer than two candidates are standardized. See Engine::next_duel_full for the annotated form.

Why not dueling Thompson sampling

The obvious acquisition here — draw two posterior samples, duel each one’s champion — is a real algorithm, correctly implemented, and the wrong objective. DTS is best-arm identification: it converges on finding the single top patch. What this system needs from a duel is information about θ, because θ is what reshapes the proposal distribution and paints the taste map. Those goals diverge sharply. The Fisher information in one Bradley–Terry duel is

I(θ) = p(1−p) · Δ Δᵀ ,   Δ = φ_a − φ_b ,   p = σ(θ·Δ)

which scales with p(1−p) and with ‖Δ‖². DTS maximizes the first (champions tie at p ≈ 0.5) while actively minimizing the second: two champions of the same concentrating posterior are two high-utility patches, which in a 48-member pool means two similar patches. It systematically picks the least informative near-tie available. And once the draw set concentrates, both champions become the same index and the user is shown top-1 vs top-2 over and over.

BALD scores the mutual information between the outcome and θ, I = H(E_s[p_s]) − E_s[H(p_s)] — high exactly when the posterior disagrees with itself about who wins, which is the definition of a question worth asking.

Measured against DTS on the synthetic user (10 paired seeds, 72 duels): pool-ranking correlation +0.101 ± 0.058, predictive excess −0.040 ± 0.017 nats. Measured against uniformly random pairing: no difference outside noise on any metric. See Acquisition for the full table and for why Bald is still the default.

pub fn next_duel_full<R: Rng>(&mut self, rng: &mut R) -> Option<DuelChoice>

Engine::next_duel with the reasoning attached: which rule chose the pair, its expected information gain in nats, and whether it is one of the uniformly-random check duels that calibration is scored on.

pub fn record_duel(&mut self, a: usize, b: usize, chose_a: bool)

Record a duel outcome between two pool members (by pool index).

The out-of-sample forecast is scored here, before the observation is appended — the model has to commit before it is told the answer, which is what makes Engine::calibration prequential rather than a in-sample self-assessment.

pub fn record_keep(&mut self, idx: usize, kept: bool)

Record a keep/kill decision on a pool member (by pool index).

pub fn record_stars(&mut self, idx: usize, rating: u8)

Record a star rating on a pool member (by pool index).

pub fn calibration(&self) -> Calibration

Prequential calibration over every duel forecast so far.

pub fn explain(&self, id: u64) -> Option<Explanation>

Exact per-feature decomposition of a candidate’s utility under the lens that claims it (B9 — see Explanation).

pub fn explain_phi(&self, phi_raw: &[f64]) -> Option<Explanation>

The same decomposition for a φ that is not a pool member — the workbench, which is a patch under the player’s hands and not a candidate until they commit it.

This is what makes the readout above the rack honest. The WHY line used to be fetched once, for the candidate that was loaded, and then went on describing it through any number of edits: it named features of a patch the player had already edited away. The bench re-featurizes on every edit anyway, so the true decomposition is a dot product away — there was never a cost reason for the stale one.

Takes raw φ and standardizes here, because raw is what the featurizer produces and what the log stores; θ is denominated in the standardizer, so the transform is not optional.

pub fn display_names(&self) -> HashMap<u64, String>

Musical display names for the whole pool, unique across it. Keyed by candidate id; a user-given name always wins.

User and preset names are claimed first and through the same registry as generated ones. Substituting them afterwards, as this once did, let a preset called Glass Pad and a generated Glass Pad both survive into the bank: the preset occupied the name without ever competing for it.

pub fn set_name(&mut self, id: u64, name: &str)

Name (or rename; empty clears) a candidate.

pub fn pin_cap(&self) -> usize

How many patches may be pinned at once: a quarter of the pool.

The pool is the model’s working set, not storage — duel pairing is uniform over it and refinement seeds from the top of ranked() — so pins are spent capacity, and the only wholly wasted duel is one where both sides are pinned. At a quarter of the pool that is ~6% of pairs, with three quarters of the pool still free to churn; at half it is 25%. A quarter buys the user far more than they lose.

The cap also keeps “everything is pinned” unreachable, which matters because that state has no honest report: it surfaces as Engine::insert_candidate returning None, which every caller already renders as “no proposal beat its parent” — a statement about the search that would then be a lie about storage.

pub fn pinned_count(&self) -> usize

How many pool members are currently pinned.

pub fn set_pinned(&mut self, id: u64, pinned: bool) -> bool

Pin or unpin a patch against eviction. Returns false when the id is unknown, or when pinning would exceed Engine::pin_cap — callers are expected to say which, rather than letting the control fail silently.

Records no observation: a pin says what the user wants to keep, not what they think of it. See Candidate::pinned.

pub fn insert_preset(&mut self, tree: PatchTree, name: &str) -> Option<u64>

Insert a named preset into the pool (protected from immediate eviction pressure only by its utility, like any candidate). Returns the new id.

pub fn export_profile(&self) -> Profile

Export the portable profile (log + standardizer, which only mean anything together).

pub fn export_state(&self) -> SessionState

Export the full session (profile + bank + lineage) for persistence. Renders and features are intentionally omitted — trees re-featurize deterministically on import.

pub fn import_state(&mut self, state: SessionState) -> usize

Restore a saved session, replacing pool, log, standardizer, lineage, and id allocation. Each bank tree is re-featurized (and re-rendered when keep_renders); entries that no longer vet are dropped. Returns how many bank entries were restored.

pub fn import_state_deferred(&mut self, state: SessionState) -> Vec<BankEntry>

Restore a saved session without rendering the bank: everything Engine::import_state does except the per-entry featurize, returning the bank entries for off-engine work, in bank order.

Restore is the returning user’s boot and today it is worse than a cold one — a full bank of serial re-renders behind a bar that cannot move, because nothing lands until all of it finishes. This is the seam that lets the farm do it: each entry comes back through Engine::absorb_bank_entry and Engine::finish_restore closes the restore, and the three together are exactly import_state.

Profile-then-clear ordering is preserved from import_state: Engine::import_profile may re-fit a standardizer over the current pool, so clearing before it would change the scale a restore lands on.

pub fn repair_report(&self) -> (usize, usize, usize)

How many saved terms, log cells and whole observations the last Engine::import_state_deferred had to repair. All three are zero for a session written by a build that has this gate.

Reported rather than logged because the frontend is the only thing that can tell the player their profile was mended, and a silent repair of the evidence a model is fitted on is exactly the kind of quiet the rest of this app was built to stop.

pub fn absorb_bank_entry(&mut self, entry: BankEntry, pre: PreFeaturized)

Reinstate one restored bank entry with its saved identity, from a featurization performed off-engine.

Bypasses the pool-size and novelty checks, as import_state’s push does: a bank is a bank, not a candidate competition. entry supplies the identity (id, origin, name) and the term; pre supplies φ.

pub fn finish_restore(&mut self) -> usize

Close a deferred restore once every entry that was going to land has. Returns the restored bank size.

The standardizer normally comes from the profile; a session saved before the first fit completes has none — fit one from the restored bank so φ isn’t left raw. Idempotent, and safe on an empty pool.

pub fn import_profile(&mut self, profile: Profile)

Import a profile: replaces the log and re-establishes a standardizer for it.

A profile written before raw-φ logging carries standardized vectors, which are only interpretable through the standardizer that shipped with them — so that pairing is exactly what makes the migration possible (crate::migrate): invert the transform, convert the coordinates whose units changed, and the log becomes raw evidence again. Its standardizer is then obsolete by construction (it has the wrong dimension for the current feature set) and a fresh one is fit from the migrated data. A same-schema profile keeps its standardizer, so imported θ geometry stays valid until the next fit refreshes it.

impl Engine

pub fn taste_map(&self) -> TasteMap

Build the taste map over the pool plus recent observation history.