5dfc4a9fad
Rebuild the repo so its spine is a real, reproducible demonstration of
licensing an actual model capability, not payload-agnostic crypto on
stand-in blobs. The clean-room Ed25519 + AES-256-GCM primitives stay as
the fast mechanism layer; the real thing is now the headline.
New demo/ walkthrough (steps 1-7), each a standalone script printing
machine-checked evidence:
1 download Qwen2.5-0.5B-Instruct from Hugging Face (gitignored cache)
2 base scores 0.000 on an invented tool-call protocol (capability C)
3 train a PEFT LoRA on C, base frozen (SHA-256 byte-identical proof)
4 base + LoRA scores 0.925 on a held-out set with unseen arguments
5 seal the adapter as an AES-256-GCM unit under an Ed25519 leaf cert
6 valid licence decrypts-at-load and runs C at 0.925
7 access-gate then crypto-erase: original and exfiltrated copy both
permanently undecryptable, base alone back to 0.000
Reference run on an RTX 4090 captured the observed numbers now in the
README. keystore.py gains export_state/load_state so the authority (and
crypto-erasure) persists across the separate demo commands. A single
run_demo.sh drives steps 1-7; run_all.sh + pytest remain the fast
crypto-only mechanism tests.
Ships code only: base weights, HF cache, trained adapter, wrapping keys
and every sealed unit are gitignored and never committed. README rewritten
to lead with the demo and the observed numbers, with honest bounds
(in-memory adapter during a live licence needs a hardware enclave) and a
capability-tree scale-up as future work.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
92 lines
3.1 KiB
Python
92 lines
3.1 KiB
Python
"""Capability C: the fictional SIGIL control-directive protocol.
|
|
|
|
This is the "new capability" the base model lacks and a LoRA teaches. It is
|
|
invented from scratch for this repository so that novelty is guaranteed by
|
|
construction: no pre-trained model has seen the SIGIL frame or its opcodes,
|
|
and success is checkable by a deterministic oracle rather than by a judge.
|
|
|
|
The task
|
|
--------
|
|
Given a natural-language home-automation request, emit exactly one directive:
|
|
|
|
<<SIGIL>> route=<ROUTE> verb=<VERB> arg=<ARG> <</SIGIL>>
|
|
|
|
* ROUTE is an invented channel code for the device domain
|
|
(lights -> LUM, door -> PORT, thermostat -> THRM, alarm -> WARD,
|
|
speaker -> RESON, blinds -> VEIL).
|
|
* VERB is an invented opcode for the action
|
|
(on -> IGNITE, off -> DOUSE, query -> SCRY, set -> BIND, cancel -> BANISH).
|
|
* ARG is the single copied slot, extracted verbatim from the request: the
|
|
location for IGNITE/DOUSE/SCRY/BANISH, or the numeric value for BIND. This
|
|
is exactly the argument-extraction a real tool-call protocol performs.
|
|
|
|
None of these tokens or the frame exist in the wild. A base instruct model,
|
|
prompted with the request but never shown the mapping, cannot produce the
|
|
correct directive. The mapping and the copy are exactly what the adapter
|
|
carries.
|
|
|
|
The oracle below is strict: it parses the first well-formed SIGIL frame from a
|
|
model's output and requires an exact match on all three of route, verb and arg.
|
|
No partial credit, no fuzzy matching.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import re
|
|
from dataclasses import dataclass
|
|
from typing import Optional
|
|
|
|
# Invented, closed vocabularies. Documented here so the novelty is auditable.
|
|
ROUTES = {
|
|
"lights": "LUM",
|
|
"door": "PORT",
|
|
"thermostat": "THRM",
|
|
"alarm": "WARD",
|
|
"speaker": "RESON",
|
|
"blinds": "VEIL",
|
|
}
|
|
VERBS = {
|
|
"on": "IGNITE",
|
|
"off": "DOUSE",
|
|
"query": "SCRY",
|
|
"set": "BIND",
|
|
"cancel": "BANISH",
|
|
}
|
|
|
|
FRAME_OPEN = "<<SIGIL>>"
|
|
FRAME_CLOSE = "<</SIGIL>>"
|
|
|
|
# Strict parser for the first well-formed frame in a block of text.
|
|
_DIRECTIVE_RE = re.compile(
|
|
r"<<SIGIL>>\s*route=([A-Z]+)\s+verb=([A-Z]+)\s+arg=([A-Za-z0-9]+)\s*<</SIGIL>>"
|
|
)
|
|
|
|
|
|
@dataclass(frozen=True)
|
|
class Directive:
|
|
route: str
|
|
verb: str
|
|
arg: str
|
|
|
|
def render(self) -> str:
|
|
"""The canonical target string a correct model must emit."""
|
|
return f"{FRAME_OPEN} route={self.route} verb={self.verb} arg={self.arg} {FRAME_CLOSE}"
|
|
|
|
|
|
def parse_directive(text: str) -> Optional[Directive]:
|
|
"""Extract the first well-formed SIGIL directive from ``text``.
|
|
|
|
Returns ``None`` if no frame is present or it is malformed. This is what
|
|
makes base-model accuracy near zero: without the protocol, the model does
|
|
not emit the frame at all, or emits English instead of an opcode.
|
|
"""
|
|
match = _DIRECTIVE_RE.search(text)
|
|
if match is None:
|
|
return None
|
|
return Directive(route=match.group(1), verb=match.group(2), arg=match.group(3))
|
|
|
|
|
|
def score_one(output_text: str, gold: Directive) -> bool:
|
|
"""True iff the model's output contains the exact expected directive."""
|
|
got = parse_directive(output_text)
|
|
return got is not None and got == gold
|