How do you know an AI system will refuse the thing it must always refuse? Today the answer is testing: ask it a thousand ways and hope the thousand-and-first isn't different. The policy in this talk has 55 variables a caller controls: that's 36 quadrillion situations! It also has a test suite of 33 unit tests, all green.

This talk is about deciding the question instead of sampling it. A safety rule like "never do X without checking Y" becomes a logic circuit, and a circuit becomes arithmetic: equations whose only solutions are the situations the rule allows. Asking whether the rule can be broken turns into asking whether that system has any solution. Live: one rule proved unbreakable, with a short record of why that you can check yourself without trusting the program that found it; and two that break, each repaired and re-proved on the spot. This proves the rail, not the model.

Jobs for TechLeads