Study 002 · Model vs Model · pre-registered

Does medical specialization actually help on real foot photos?

Clinician review open

MedGemma is Google's Gemma 3 specialized for medicine — same decoder architecture, retrained on medical images and text. So we ran a clean head-to-head on a real, privacy-controlled foot-photo series: base Gemma 3 27B vs MedGemma 27B. Same prompt, same box, same photos — the only variable is the weights. Protocol pre-registered and hash-frozen before any results; graded by deterministic gates (no AI judging an AI); every exchange hash-chained.

The photos never left the box. An n=1 real foot-photo series reviewed 100% locally on owned hardware (2× RTX PRO 6000). Only the deterministic receipts — aggregate numbers, agreement, and the tamper-evident ledger — are published here. No images, no per-photo observations. Private by design, defendable by receipt.

The published prior (Google's own benchmarks)

Google reports MedGemma beating base Gemma 3 across medical imaging — but none of those benchmarks are diabetic foot-wound photos. The closest transfer is the dermatology (skin) pre-training. This study is the first check on that out-of-distribution domain.

Benchmark	Domain	Gemma 3 27B	MedGemma 27B
US-DermMCQA	dermatology / skin	66.9	71.7
EyePACS	diabetic retinopathy	20.3	75.3
MIMIC-CXR (F1)	chest X-ray	71.7	90.0

Source: Sellergren et al., MedGemma Technical Report, arXiv:2507.05201 (2025).

Deterministic scorecard — 35 photos, identical config

Metric	Gemma 3 27B (base)	MedGemma 27B (specialized)
Structured output valid, clean first pass	35/35	35/35
Repairs / hard-fails	0 / 0	0 / 0
Tiers — urgent / contact / watch / routine	35 / 0 / 0 / 0	34 / 0 / 0 / 1
No-diagnosis / no-dosing screen	0 flags	0 flags
Latency (wall-time proxy)	~21 s/img	~21 s/img

Per-field inter-model agreement (Cohen's κ) — how often the two models agree on what they see:

Observation	agree	κ
maceration	71%	0.45
drainage / fluid	57%	0.34
discoloration	94%	0.31
redness	57%	0.07
swelling	23%	0.02
pressure / friction	0%	0.00

Tamper-evident ledger: 70 links, GENESIS-rooted, verified; flip one byte and the chain breaks at that link. Protocol, prior, scorecard, and ledger are published below for replay.

What the receipts say — honestly

1. Reliability is a tie. On this explicit closed-schema prompt, base Gemma 3 held the structure just as well as MedGemma — both 100% valid, zero repairs.

2. Base Gemma 3 blankets everything "urgent" (35/35); MedGemma discriminates — it pulled out the one clearly mild photo the base model over-escalated.

3. The two models genuinely see differently — near-zero κ on redness, swelling, and pressure. The specialization changes the observation, not just the wording. This held after removing every confound.

The deterministic gates prove behavior — reliable, in-lane, consistent. They cannot say whether MedGemma's finer triage or Gemma 3's blanket caution is clinically correct. There is no ground truth on the box. That verdict belongs to a clinician — which is why this study is marked open.

Tier-2 — independent clinical review (open)

The clinical verdict is deferred to a blinded A/B adjudication by a board-certified podiatrist / wound-care clinician — the two models' observations shown side by side with identities hidden. That review is open and unsigned. We never claim a clinical sign-off we don't have.

Read & reproduce

Pre-registered protocol (hash-frozen): protocol.md · Published prior: prior-evidence.md · Deterministic scorecard: scorecard.json · Hash-chained ledger: ledger.jsonl.

Not medical advice. A public, PHI-free evaluation — the foot photos and per-photo observations stay on the owner's box and were never published.