# AI Teachers | Build Your Agentic OS Original advanced learning course · Updated 2026-10-25 Free written course and local-only code lab. You must not add real customer records or credentials to the demonstration. ## The outcome Nine-layer original course covering operational architecture, role separation, model routing, budgeting, independent verification, monitoring, earned trust, prompt-injection defense and a measured 30-day launch. ## Scenario A fictional product team's weekly customer-feedback digest and documentation-fix process. No live accounts, customers, messages or deployments are touched in the starter exercise. ## Layer 1: Define the loop before buying tools Start with one business result that can be measured independently of the model's opinion. Our running example is a fictional software team's weekly customer-feedback digest. We are not trying to replace a company with a prompt; we are building one low-risk process with inputs, an owner, an evidence trail and a useful output. ### Steps 1. Choose one repeatable responsibility (for example: collect product feedback and propose documentation fixes), and write down the current human process. 2. Specify the trigger, approved sources, intended recipient, desired output and action that stays manual. 3. Choose three success metrics: evidence coverage, reviewer time and accepted recommendation rate. 4. Define a failure condition and when a human must stop the loop. ### Copyable practice prompt You are helping design an AI-assisted operations workflow, not operating it yet. For the fictional weekly customer-feedback digest, draft a loop map: input → analysis → proposed action → independent check → human decision → feedback for next run. Define three measurable quality metrics and two explicit reasons to stop. Keep all publishing and customer contact human-approved. ### Deliverable A one-page loop map and baseline comparison to the manual process. ### Independent checkpoint Can another person tell whether the result was useful without asking the AI to grade itself? ### Caution A business improving 'while you sleep' is an aspiration, not evidence that unattended actions are safe. ### Role paths Founders: map a marketing or customer-support loop. Builders: map a narrow issue-to-PR preparation loop. My result: Evidence checked: Issues and next experiment: --- ## Layer 2: Write rules the system can actually enforce A constitution is only useful when critical rules are implemented in code, policy or platform permissions. Use the included constitution.json to specify allowed task types, maximum estimated spend, output requirements and external-action policy. A free-form prompt can describe your values; it cannot enforce an operating-system permission boundary by itself. ### Steps 1. Open constitution.json from the free code kit and inspect the approved task types and daily budget. 2. Write three machine-checkable rules: no external actions, limit the cost of one run, and reject unsourced claims. 3. Write three editorial preferences separately: concise tone, clear headings and friendly wording. 4. Run the local demo with the default rules, then reduce its daily cap and observe the refusal. 5. Document who is allowed to change the policy file and how changes are reviewed. ### Copyable practice prompt Turn these operating principles into two columns: (A) rules that need enforced technical controls and (B) writing preferences that can remain prompts. Principles: never publish without approval; cite every source; keep spend below our daily limit; write clearly; never reveal secrets; avoid duplicate requests. For each technical rule, give a test that would prove enforcement. ### Deliverable Policy file, rule-to-test map and an explicit exception process. ### Independent checkpoint Does a budget- or permissions-violating run fail even if the planner asks nicely? ### Caution A model can misinterpret a rule. Run critical checks outside the model. ### Role paths Founders: define spending and approval authority. Builders: implement deny-by-default permissions. My result: Evidence checked: Issues and next experiment: --- ## Layer 3: Separate planning, doing, reviewing and approval Avoid letting the same agent invent work, claim success and approve the release. Use distinct components: planner drafts a constrained plan, worker prepares an artifact, independent verifier tests evidence, and policy gate decides whether a human can review it. For a real deployment, isolation requires separate credentials and runtime boundaries. ### Steps 1. Draw four lanes: plan, execute, verify, authorize. 2. Give the planner read-only access and the worker only a sandbox or draft destination. 3. Define the verifier's input as the original sources plus output artifact, not the worker's success claim. 4. Keep the authorizing gate deterministic and the human approver identified by role. 5. Run the simulated agentic_os.py program; inspect its stage-by-stage results. ### Copyable practice prompt Draft a four-role RACI matrix for a fictional feedback-digest agent. Planner may propose work; worker may prepare a draft; verifier independently checks sources and completeness; policy gate checks budget and permissions; human approves external publication. List credentials, allowed actions, blocked actions, and how to test each boundary. ### Deliverable An architecture drawing and enforceable permissions table. ### Independent checkpoint Can the planner write to production or can the worker declare its own output verified? The answer must be no. ### Caution Our starter separates stages in functions for learning. It is not a hardened security sandbox. ### Role paths Founders: assign real accountability. Builders: design separate identities and token scopes. My result: Evidence checked: Issues and next experiment: --- ## Layer 4: Route work to the right model and control cost Use expensive reasoning only where it improves independently measured outcomes. Anthropic now lists Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5 at materially different token prices. Routing should use empirical quality tests and task risk, not price alone. The academy's interactive estimator is illustrative, not a provider invoice. ### Steps 1. Estimate input tokens, output tokens, attempts and weekly workload for a sample task. 2. Compare a Fable-only scenario to an evaluated mixed-model scenario in the cost estimator. 3. Try a simple rule: use the least costly tested model meeting your quality threshold. 4. Add hard call-count, token and daily-spend guards in the application layer before making API requests. 5. Read the current provider documentation for caching and task budgets; label advisory limits clearly. ### Copyable practice prompt Design a model-routing policy for three task types: low-risk classification, documented drafting, and novel architecture review. Recommend evaluation criteria rather than blindly assigning a specific model. Include token estimates, cache strategy, fallback, and conditions where the task should stop for human help. ### Deliverable Routing decision table, expense estimate and fallback plan. ### Independent checkpoint Can you show cost per accepted result, not just cost per generated token? ### Caution Claude API task_budget is advisory, not a billing ceiling; the demo's spend cap applies only to its own estimated, simulated activity. ### Role paths Founders: compare cost per approved brief. Builders: add metering before vendor calls and reconcile against provider billing. My result: Evidence checked: Issues and next experiment: --- ## Layer 5: Make 'done' a verifiable state Treat the artifact, sources, tests and review record as evidence. Never trust a completion claim by itself. Our demo produces a draft from a set of fictional source records. A separate verifier checks evidence coverage and an approvals rule. In a real software task, checks could include unit tests, diff review and a smoke test; in marketing, source validation and human approval. ### Steps 1. Run a normal fixture and locate the proposed output in the local run ledger. 2. Compare every claim to its cited source and check that no owner/date was invented. 3. Run the incomplete or malicious fixture; inspect the independent failure reason. 4. Test one intentionally incorrect statement in your own fictional input and ensure it is flagged. 5. Record accepted, rejected and needs-review states distinctly. ### Copyable practice prompt Create an independent verification rubric for a weekly feedback digest: source coverage, no unsupported claims, clear action owner, human approval, safe handling of missing data. Provide five test cases, a pass/fail decision for each and the evidence an auditor must inspect. Do not treat the generator's self-assessment as a test result. ### Deliverable Review rubric, passing example and failure example. ### Independent checkpoint Can the worker's claim of success be contradicted and rejected by an independent check? ### Caution A second language model is not automatically independent; prefer objective tool outputs, immutable evidence and human audit where appropriate. ### Role paths Founders: audit a proposed customer communication. Builders: use tests and externally inspected artifacts. My result: Evidence checked: Issues and next experiment: --- ## Layer 6: Add a heartbeat without creating an endless loop Review whether previously accepted work is still valid while keeping bounded checks, expiry and clear ownership. Some outputs expire quickly. A weekly status brief is stale next week; a launched feature may break after a dependency update. Monitoring should recheck defined conditions on a schedule, not continuously consume tokens or repeat consequential actions. ### Steps 1. Pick what has to remain true for your output to stay valid. 2. Define a reasonable check interval and a maximum number of retries. 3. Separate detecting an issue from fixing it; alerts and proposed patches require human decisions. 4. Run agentic_os.py monitor against the local evidence ledger. 5. Document how the system marks outcomes stale and when a responsible person is notified. ### Copyable practice prompt Design a bounded monitoring plan for a fictional support-knowledge-base update. List checks that can be done cheaply without AI, when to escalate, the maximum checks per day, when to stop, and the owner of each alert. Avoid any unchecked self-triggering loop. ### Deliverable A scheduled verification plan and exception escalation tree. ### Independent checkpoint Can monitoring stop safely when a source or budget is unavailable? ### Caution The included monitor command rechecks saved demonstration evidence; it does not contact live websites, production systems or customers. ### Role paths Founders: define business process alerts. Builders: design cron or event triggers only after safe dry runs. My result: Evidence checked: Issues and next experiment: --- ## Layer 7: Earn autonomy from audited performance Increase delegation only after repeatable, human-reviewed results, with immediate escalation when quality declines. The kit demonstrates an illustrative promotion threshold of at least 20 independently reviewed runs with at least 95% passing. This is a teaching policy, not a universal safety standard; high-impact work may require stricter thresholds and formal governance. ### Steps 1. Run several simulated tasks and note that self-verification alone cannot raise the trust level. 2. Use the audit command to record human review outcomes and names in the local training ledger. 3. Introduce a failed audit and confirm that eligibility to use less supervision is removed. 4. Create levels such as draft-only, supervised-tool-use and human-reviewed execution. 5. Define tasks that always require approval, regardless of past performance. ### Copyable practice prompt Design a reversible autonomy ladder for a weekly operations assistant. Include draft-only, supervised, and narrowly delegated modes. Require independent audits, define a quality threshold, revoke eligibility on any critical error, and keep client contact, spending and production changes human-approved. ### Deliverable Trust ladder, signed-off audit policy and revocation test. ### Independent checkpoint Does a failed human audit trigger review instead of letting the agent continue on past reputation? ### Caution Do not treat sample review counts as actuarial guarantees. Protect real audit logs from tampering. ### Role paths Founders: define who approves expanded authority. Builders: store append-only, access-controlled audit events. My result: Evidence checked: Issues and next experiment: --- ## Layer 8: Defend the boundary between data and instructions Treat retrieved documents, tickets, emails and logs as untrusted inputs that must not issue new commands. Prompt injection occurs when untrusted content attempts to change system behavior. The training fixture includes a harmless example of an instruction hidden inside a support note. The sample filters it as evidence, not an instruction; real systems need deeper controls. ### Steps 1. Open the malicious demonstration fixture and identify the instruction that does not belong to the support evidence. 2. Run the fixture and check that it is rejected without executing commands or sending data. 3. Write a least-privilege secrets plan: no API keys in prompts, logs or committed files. 4. Document an incident runbook for credential exposure, cost spike, repeated failures and attempted prompt injection. 5. Create a pause procedure that can shut down job triggers and revoke credentials. ### Copyable practice prompt Threat-model a customer-feedback agent that reads outside submissions and drafts an internal report. Include prompt injection, poisoned sources, secret leakage, excessive tool permissions, repeated job triggers and misleading completion reports. Recommend defense-in-depth controls and two safe test fixtures, with owner and stop condition. ### Deliverable Threat model, defensive test results and incident response runbook. ### Independent checkpoint Do untrusted source notes remain data, even when they contain a direct command? ### Caution Pattern detection in the demo is intentionally simple and cannot guarantee full prompt-injection protection. ### Role paths Founders: practice incident escalation. Builders: enforce access controls outside the model and limit tool execution. My result: Evidence checked: Issues and next experiment: --- ## Layer 9: Run a 30-day pilot and test the economics Scale verified outcomes, not the number of agents, tokens or impressive demos. The 30-day plan introduces one task at a time, tracks cost per accepted output, keeps detailed failure notes and expands only with evidence. ROI calculations subtract setup, review, correction, vendor and maintenance costs; no savings are guaranteed. ### Steps 1. Days 1–7: design the loop, choose a low-risk fixture, set policy and baseline manual effort. 2. Days 8–14: run under manual review, log failures and improve evidence quality. 3. Days 15–21: test a second input class, verify costs and deliberately try failure cases. 4. Days 22–30: hold a go/no-go review; document trusted tasks, open risks and follow-up ownership. 5. Compare measured time, error rates and provider costs against the original process. ### Copyable practice prompt Evaluate whether our fictional AI feedback workflow is ready to progress after 30 days. I will provide runs, human-review times, failure counts and estimated API spend. Calculate accepted-result rate, cost per accepted result, review burden and unresolved risks. Give a go/hold/stop decision with evidence and a clearly assigned human owner. Do not assume productivity gains not supported by the data. ### Deliverable 30-day scorecard, go/hold/stop decision and an accountable operating owner. ### Independent checkpoint Would you still operate the system if it produced fewer but higher-quality results? ### Caution The economics of agentic labor differ by task, quality, compliance, tool setup and human oversight. ### Role paths Founders: calculate contribution after cost and rework. Builders: document model switching, schema versioning and rollback. My result: Evidence checked: Issues and next experiment: --- ## Lab files https://aiteachers.cc/downloads/ai-teachers-agentic-os-lab.zip ## 30-day rollout - Week 1: baseline and boundaries - Week 2: reviewed pilots - Week 3: adversarial evaluations and cost reconciliation - Week 4: human go / hold / stop decision ## Independent references - Claude models and Fable 5.1: https://platform.claude.com/docs/en/models/overview - Anthropic token pricing (check before use): https://platform.claude.com/docs/en/about-claude/pricing - Task budgets (advisory, API only): https://platform.claude.com/docs/en/build-with-claude/task-budgets - Prompt caching: https://platform.claude.com/docs/en/build-with-claude/prompt-caching - Y Combinator — Tom Blomfield talk: https://www.ycombinator.com/people/tom-blomfield - Existing AI Teachers agent course: https://aiteachers.cc/academy/build-your-ai-agent-team/ - Agent to Outcome starter kit: https://aiteachers.cc/agent-to-outcome/kit/ This document is an original AI Teachers course, not a reproduction of a paid publication or a production-hardened system.