codecalc/providers.py: PROVIDER_INTERFACE_VERSION, a Protocol, a frozen
descriptor, a ProviderRegistry with interface-major validation, and
tests/_provider_conformance.py). This extends the same, proven pattern to the
other three extension types and adds the cross-cutting trust/discovery layer the
ticket’s acceptance criteria require.
The original ticket parked three of four types on “an SDK designed before its
second consumer is guessed API.” This design answers that directly: every new
interface ships with a built-in reference implementation and a reference
third-party extension — the second consumer that validates the shape, not a
guess.
Threat model — what this protects against, and what it does NOT
A gemini-3.1-pro architecture review (2026-08-19) flagged the load-bearing point: an extension is Python that runs IN-PROCESS, so a manifest and a permission allowlist cannot confine it — once imported it canimport os/socket or
monkeypatch internals no matter what it “declared.” Pretending otherwise would
be the exact dishonesty this codebase’s disclosure discipline exists to avoid.
So the scope is stated plainly and the SDK does not overclaim:
- Extensions are operator-installed and trusted-by-installation — the same
trust posture as a
pipdependency, apytestplugin, or a Django app. The operator chooses to install one; installing it IS the trust decision. - The untrusted thing is the executed payload (the user’s code), and it is confined by the execution providers (gVisor / AppContainer / rlimits) — a different, already-built layer. Extensions do not widen that boundary: a language pack contributes an argv template, but the payload still runs inside the provider sandbox; a renderer/verifier never executes payload code.
- The extension trust model is therefore provenance + operator policy + audit + a kill switch, not runtime confinement of extension code: integrity-digest pinning, an explicit permission declaration the operator policy must approve, an auditable record of what loaded, and the ability to disable third-party extensions entirely. This is real, useful control over what an operator runs; it is not, and is documented not to be, a sandbox against a hostile extension author. Confining hostile extension code (WASM / out-of-process RPC) is explicitly out of scope for v1.0.0 and noted as the future boundary if untrusted-author extensions are ever a goal.
Shared layer — codecalc/extensions.py
One place for what is common to all extension kinds, so a kind’s own module
(providers/language_packs/renderers/verifiers) stays about its domain.
Identity, trust, policy (the acceptance criteria that aren’t per-type)
- No impersonation.
extension_idfor athird_partyextension may not use the reservedbuiltin:prefix, nor collide with a registered builtin id. Registration rejects both withextension_identity_conflict. Built-ins register withorigin="builtin"and abuiltin:-prefixed id. - Explicit, policy-controlled permissions. An
ExtensionPolicy(built from env) holds an allowlist of permitted capability strings andallow_third_party: bool. Registration of a third-party extension whosedeclared_permissionsexceed the allowlist failsextension_permission_denied. Nothing is granted implicitly. - Disable third-party entirely.
CODECALC_DISABLE_THIRD_PARTY_EXTENSIONS=1setsallow_third_party=False; anythird_partyregistration then failsextension_disabled. Built-ins are unaffected. - Integrity + trust model. Enforced at the admission point:
register(ext, payload=...)recomputes the payload sha256 when the manifest pinsintegrity, refuses on mismatch, and ALSO refuses a pinned manifest with no payload to check against (an unverifiable pin must not pass as verified) —extension_integrity_failurein both cases. A built-in (no pin) needs no payload. The documented distribution trust model reuses the release story: third-party extensions SHOULD be distributed with a sigstore attestation (same mechanism as codecalc’s own artifacts) and pinned by digest; the loader verifies the declared digest and the operator pins which origins/publishers are trusted. (Signature verification against a keyring is a documented, opt-in extension point; digest-pinning is enforced now.) - Version compatibility. The ENFORCED gate is exactly what
ProviderRegistryalready does: the extension’sinterface_versionMAJOR must equal the kind’s*_INTERFACE_VERSIONmajor, elseextension_interface_mismatch.compatible_codecalc/compatible_contractare recorded and surfaced in discovery as the extension’s declared support, and checked only by MAJOR (a compatible-major compare), NOT by solving a semver RANGE — pip/npm-style range resolution is a deliberate non-goal for v1.0.0 (interface-major matching already guarantees API compatibility). - Isolation + stable error codes. Every registry operation that calls into
an extension wraps it: an exception becomes an
ExtensionOperationFailure(.code = f"extension_{op}_failed") carrying the extension id, never propagating a raw crash. Shared codes live here:unsupported_capability,unknown_extension,extension_interface_mismatch,extension_disabled,extension_integrity_failure,extension_permission_denied,extension_identity_conflict,extension_operation_failed. - Capability discovery.
describe_extensions()returns, across all kinds, each extension’s\{kind, extension_id, name, version, origin, health, supported_operations\}— the machine-readable capability document.
ExtensionRegistry is a small generic base (register, get, descriptors,
guard) that the three NEW per-kind registries subclass. The shipped
ProviderRegistry predates this base and is deliberately left untouched (its own
identity/version checks keep working); unifying it onto the shared base is
optional future cleanup, not part of this change — the two implementations agree
in behaviour but are, for now, maintained separately.
The three new kind interfaces (each mirrors providers.py)
codecalc/language_packs.py — LANGUAGE_PACK_INTERFACE_VERSION = "1.0.0"
A LanguagePack describes how a set of languages is discovered, compiled,
invoked, and how its diagnostics normalize. Built-in BuiltinLanguagePack wraps
the existing registry.LANGUAGES/ALIASES/EXTENSIONS. Reference third-party:
examples/extensions/lolcode_pack/ — one toy language, proving discovery →
run → normalized diagnostics through the same registry the server uses.
codecalc/renderers.py — RENDERER_INTERFACE_VERSION = "1.0.0"
A Renderer turns a result envelope (or a typed payload) into a representation
(table, chart spec, notebook, report), declaring the formats it can emit and
whether it can_render a given result — never mutating the result. Built-in:
TextRenderer (the current stdout/verdict view) + a MarkdownTableRenderer.
Reference third-party: examples/extensions/csv_renderer/.
codecalc/verifiers.py — VERIFIER_INTERFACE_VERSION = "1.0.0"
A Verifier inspects a claim and submits evidence; grades.py alone maps
evidence → grade. Four safeguards, from the gemini review, keep a verifier from
assigning trust even so:
Evidenceis a frozen dataclass (claim_kind,verifier_id,outcome ∈ \{supports, refutes, inconclusive\},detail,cost) — not a mutable dict, so the record the grader reads cannot be reshaped after the fact.- The registry passes DEEP COPIES of
claim/contextintoverify, so a verifier that mutates its arguments in place cannot poison the data the grader or a later verifier relies on. - Declared-claim scoping: the registry DROPS any evidence whose
claim_kindis not in the verifier’s ownclaims()— a verifier cannot speak to a claim it never declared. - Advisory weighting: evidence is an INPUT, never a verdict.
grades.pykeeps its versioned rules; a singlesupportsdoes not lift a grade. Built-in evidence and third-party evidence are weighed by policy — third-party verifier evidence corroborates, it is not sufficient alone — so a lying/colluding third-party verifier cannot unilaterally certify a fraudulent claim.
examples/extensions/parity_verifier/.
RenderOutput: format, media_type, body) and every descriptor.
Core operations stay synchronous to match the shipped provider Protocol (whose
execute is sync); an async variant, if an I/O-bound verifier needs one, is a
future interface-version addition, not a v1.0.0 shape change.
Conformance suites (one per kind, reusable)
tests/_language_pack_conformance.py, _renderer_conformance.py,
_verifier_conformance.py, each exposing run_*_conformance(impl, check) and
asserting: manifest JSON-serializable + identity/version rules; health reports
ready: bool; discovery returns the declared shape; the core operation works on
a known input; optional ops either work or raise UnsupportedCapability;
verifiers additionally must return evidence whose schema carries no grade
field (a verifier that tries to grade fails conformance). tests/test_*s.py run
each built-in and each reference extension through the suite, wired into
ci-python.yml beside the existing gates.
Docs
docs/extensions/README.md— the four kinds, the manifest, the trust model (identity, permissions, integrity/signing, the disable switch), and the compatibility / deprecation / security-response policies.docs/contract/language-pack-v1.md,renderer-v1.md,verifier-v1.md— the published per-kind interface specs, versioned likeprovider-v1.md.
Acceptance-criteria coverage
Each numbered criterion maps to: machine-readable manifest (ExtensionManifest + describe_extensions); policy-can’t-exceed-request (permission allowlist); missing enforcement explicit (capabilities dict + unsupported_capability); requested-vs-enforced (providers already; manifests carry declared vs the conformance-verified supported_operations); default-deny + policy (disable switch + allowlist); reference extensions for all four; evidence-not-grades (verifier contract); identity/no-impersonation (reserved prefix); integrity + trust model (integrity digest + documented signing); discovery (describe_extensions); disable-third-party (env switch); conformance per kind.Build phases
- Shared
extensions.py(manifest, policy, registry base, error codes, discovery) as NEW code the three new kinds use;ProviderRegistryis left as-is (unifying it is optional future cleanup) + tests. - Language packs: interface + BuiltinLanguagePack + conformance + reference ext.
- Renderers: interface + Text/MarkdownTable + conformance + reference ext.
- Verifiers: interface + built-ins (evidence-only) + conformance + reference ext.
- Docs + CI wiring +
describe_extensionssurfaced (doctor / an MCP capability).