OntoProbe

An ontology-grounded probe set engine for mechanistic interpretability.

Multilingual · Structured · Released under CC BY 4.0

Dataset

A two-dimensional matrix of 16 vocabulary topics × 16 grammar functions, classified across CEFR/JLPT/TOPIK levels and released in parallel English, Japanese, and Korean ontologies.

All entries are systematically classified along the 16×16 topic-function matrix. Companion tables of relations, exercises, and questions accompany the core dataset.

Engine

Behind the dataset is a generator: an LLM-backed pipeline that synthesizes probe sentences on demand, constrained by the ontology's 2D matrix and level grading. Custom probes can be produced for any cell of the matrix in any supported language.

Engine release planned alongside the resource paper. Until then, the released static dataset covers the published probe sentences.

Used in

• paper 12 — Interpretable Heads, Redundant Output: Multi-Layer MLP Causality and Steerable Grammar in Causal Language Models (2019–2024)

  Cynn, 2026. Cross-LLM grammar mechanism analysis using a 75-sentence subset of OntoProbe-EN (15 functions × 5 examples, 5.5% of the full dataset).

About

OntoProbe is built and maintained by River Lab. The ontology design and engine are original work; CEFR alignment philosophy is influenced by the Cambridge English Profile project. The dataset will be released under CC BY 4.0 and the engine code under Apache 2.0.

We invite the community to inform us of comparable prior work; the current literature scope as of 2026 returns no matching combination of structural dimensions (multilingual 2D ontology matrix + level grading + generator engine).