t2k.aiLearning guides

Closed-loop decision learning

Teach the system without letting it quietly teach itself.

T2K turns each consequential decision into a governed episode: what the system knew, what it recommended, who authorized it, what ran, what happened, and why a policy change was or was not promoted.

Canonical architecture

The ontology defines more than state.

It defines the meaning of state variables, permissible actions, measurements, rewards, constraints, authority, and terminal conditions.

KnowledgeState= Compile(Ontology, AcceptedClaims, Provenance, Time, Context)Recommendation= Reason(PolicyVersion, KnowledgeState, Objective, Policies)RewardAssessment= Evaluate(Observations, Baseline, RewardSpec, Attribution)NextPolicy= GovernedPromote(Learn(Episodes), Evaluation, HumanReview)
01Claims
02State snapshot
03Reasoning run
04Authorization
05Execution receipt
06Observation
07Reward vector
08Policy candidate
09Promotion
01

Outcome is not reward.

Reward is a versioned interpretation of outcomes under an objective and guardrails.

02

Override is not truth.

A human override is useful feedback, but it can reflect preference, missing context, or error.

03

Correlation is not credit.

An outcome after an action is not automatically attributable to that action.

04

Learning is not mutation.

Policy, objective, reward, and ontology updates create reviewed versions with rollback.

Hands-on path

Close one real episode end to end.

Begin with a repeated decision that has a measurable result and a reversible action. TransferOS owner-dependency reduction is a useful example.

  1. 01

    Freeze state

    Create an immutable Decision Context from accepted claims, the resolved pack set, objectives, policies, alternatives, uncertainty, and authority requirements.

  2. 02

    Bind policy

    Create the Decision Context after baseline deployment. It freezes the active policy descriptor, and the episode must honor that version even after a later deployment.

  3. 03

    Record reasoning

    Store input and output hashes, model and prompt versions, tool versions, latency, cost, trace, and action probability when off-policy evaluation will need it.

  4. 04

    Authorize and execute

    Keep recommendation, authorization, and execution separate. External effects require an idempotent attempt and a successful execution receipt.

  5. 05

    Observe

    Record sourced measurements after execution. An observation is not automatically an accepted business fact and is not automatically caused by the action.

  6. 06

    Assess reward

    Evaluate the observation vector against direction, weight, baseline, window, attribution method, and guardrails. Do not hide a failed guardrail inside one scalar.

  7. 07

    Close episode

    Close only after the authorized decision is bound, required measurements are complete, no execution attempts remain unresolved, and every guardrail violation has an independent waiver.

  8. 08

    Challenge and promote

    Build from training episodes, evaluate against a disjoint holdout, require action coverage, inspect intervals and safety, then let a different human promote the immutable version.

TransferOS example

Reduce owner dependency without damaging service quality.

State
Owner hours, approval concentration, process coverage, customer concentration.
Actions
Document, delegate, cross-train, change approval limits, or hold.
Reward
Operating independence, continuity risk, cost, disruption, and service guardrails.
Windows
30-day adoption check, 90-day performance check, terminal transition review.
import { T2kClient } from "@t2kai/core";

// The agent API is a structured decision problem, not a prompt.
const episode = await t2k.createDecisionEpisode(graphKey, {
  episodeKey: "episode:owner-dependency:2026-07",
  // This context was created after baseline deployment and froze that binding.
  decisionContextKey: "decision:owner-dependency:2026-07"
});

await t2k.recordReasoningRun(episode.id, {
  runKey: "run:owner-dependency:001",
  modelRef: "agent:operations-reviewer",
  promptVersion: "owner-dependency-v3",
  codeVersion: gitSha,
  inputHash: context.contextHash,
  outputHash: recommendationHash
});

await t2k.recordEpisodeObservation(episode.id, {
  measureRef: "operations.owner_approval_share",
  observedValue: 0.42,
  baselineValue: 0.71,
  observationWindow: "90d",
  sourceRefs: ["workflow://approvals/2026-q3"],
  provenance: { method: "approval_log_aggregation" },
  observedAt: new Date().toISOString()
});

const assessment = await t2k.assessEpisodeReward(episode.id, {
  assessmentKey: "assessment:owner-dependency:90d"
});

if (assessment.lifecycleStatus === "guardrail_violation") {
  throw new Error("Independent waiver or remediation required before closure");
}

await t2k.closeDecisionEpisode(episode.id, "Required review is complete");

const evaluation = await evaluator.evaluateLearningCandidate(candidate.id, {
  evaluationKey: "replay:owner-dependency:1.1.0",
  evaluationType: "historical_replay",
  holdoutEpisodeIds
});

if (evaluation.lifecycleStatus !== "passed") {
  throw new Error("Do not promote an unsupported challenger");
}

await promoter.promoteLearningCandidate(candidate.id, {
  reviewRationale: "Held-out replay passed; intervals and safety reviewed.",
  deploy: true
});

Executable evidence

Not a diagram. A 24-episode acceptance run.

Harborlight Field Service is fully synthetic. Its public clean-room fixture validates the exact schema, compiles the pack, and computes a disjoint held-out replay. Studio's acceptance suite extends the same evidence through services, Postgres, promotion, and rollback.

Verified golden demo

Replay has support, evidence stays disjoint, and rollback is real.

Episodes
4 training + 20 held out; every episode is authorized, executed, observed, rewarded, and closed.
Coverage
50% challenger and 70% baseline action coverage over the holdout.
Evidence
Estimated improvement 0.55; paired 95% lower bound 0.393.
Control
Four human roles, deploy 1.1.0, then restore the exact 1.0.0 parent.
# Create and run a local decision project
npx create-t2k@latest my-decision-loop
cd my-decision-loop
npm start

# Verify the same scaffold from source
git clone https://github.com/sigaihealth/t2k-core.git
cd t2k-core
npm ci
npm run example:scaffold
Inspect the public fixture

Choose the weakest sufficient learner

Do not call every feedback loop RL.

The learning mode should match the decision topology, reversibility, evidence volume, and risk. Most SMB teams should begin with supervised feedback.

Supervised feedback

Start here for consequential SMB decisions.

Humans review outcomes and propose policy improvements.

Contextual bandit

Repeated, bounded, reversible one-step choices.

Log action propensity and evaluate challengers offline first.

Sequential RL

Only when actions change future states and delayed reward is measurable.

Requires terminal conditions, strict authority, and an exploration budget.

Optimization

Scheduling, allocation, routing, and constrained planning.

The solver can change while objectives and constraints remain governed.

01

Replay is computed.

Historical verdicts, metrics, and safety results come from the reference evaluator, not the caller.

02

Coverage is a gate.

A candidate cannot pass when the logged behavior does not support the actions it selects.

03

Low N stays visible.

A configured technical pass below 20 episodes remains directional evidence, not statistical generalization.

04

Duties stay separate.

The candidate proposer, evaluator, and promoter must be different signed-in humans.

Production rule

No consequential online exploration by default.

Online exploration is accepted only when authority is explicit, actions are reversible, maximum exploration probability is bounded, guardrails are measurable, and rollback is operational.