We don't just assert it — we ground it in research and demonstrate it by running the whole system on our own development, on open models we host. Here's both.
Octos's own roadmap, open questions, and decisions live inside the engine — governed by the same runtime we sell. Every architectural call has evidence behind it. Every prediction comes back to be scored. The system proposes its own refinements for us to approve.
This is the most demanding test we could give it: we are its most demanding user, and every rough edge surfaces here before it reaches a customer. Running the company-brain vertical on our own development is also the prototype for the June 20 pilot.
A real concern fired in Octos's own development — surfaced by the same system we're building.
These aren't vibes — they're verified research. The failures Octos addresses are measurable, citable, and accelerating.
"Why Do Multi-Agent LLM Systems Fail?" (Berkeley/Databricks, 2025) analyzed 1600+ execution traces across 7 frameworks and found 14 distinct failure modes grouped into system design, inter-agent misalignment, and task verification. The failures are structural, not individual — better models alone don't fix them.
On NoLiMa (a benchmark that removes literal-match crutches), 10 of 12 tested models drop below 50% of their short-context baseline by 32K tokens. GPT-4o: 99.3→69.7%. Claude 3.5 Sonnet: 87.6→29.8%. Liu et al. "Lost in the Middle" (2023) showed retrieval follows a U-shaped curve — worst when relevant info is in the middle.
MIT's 2025 analysis of 300 enterprise AI deployments (Project NANDA) found 95% of generative-AI pilots deliver no measurable P&L impact — the gap is integration and process, not model quality. Benchmark wins don't survive contact with real work.
EU AI Act Article 12 (in force August 2, 2026): "High-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system." NIST AI RMF and ISO/IEC 42001 A.6.2.8 also require event-log and decision-provenance capabilities.
Not a claim about the engine — an object it produced. A prediction Octos made about its own development: recorded with a machine-checkable test, watched on a schedule, closed with a verdict. Copied verbatim from the graph.
ls entities/channel-adapters/*.yaml — at least one with state=connected and subtype=telegram.0 18 * * *), deadline 2026-05-18.Honest scope: every prediction is registered like this — with a measurement and a deadline — the moment it's made. The graph today holds 276 registered predictions and hundreds of decisions, but only a handful have run all the way to a verdict so far — the record grows faster than the grading does. We'd rather show you one real closed loop than a headline number we can't stand behind.
Every decision carries its evidence, alternatives, and a timestamped record. Audit trails are structural, not bolted on.
The engine and your graph run on your own infrastructure. Self-host the model too and nothing leaves your perimeter at inference; on a hosted model, only the turn reaches the provider you picked.
Automatic event logging is built in — not retrofittable. Art. 12 compliance by design, not by patch.