BQEB: A Reproducibility-First Architecture for Modular Energy AI Benchmarking

Abstract

Reproducibility in applied energy AI research depends on more than a public dataset or a published benchmark: it depends on whether the software producing a benchmark’s numbers can be independently re-run, whether artifacts are versioned and checksummed, and whether governance practices are documented well enough that a discrepancy can be found rather than concealed. This paper presents the BQEB Reproducibility Architecture, a modular research-artifact framework for energy AI research, now formally released as an independent, citable artifact (DOI: 10.5281/zenodo.22291930). The architecture comprises four layers: a Research Artifact Layer connecting manuscripts, datasets, models, and experiments; a Reproducibility Layer built on deterministic execution, checksummed artifacts, and an evidence registry; a Benchmark Evolution Layer supporting extensions and comparative studies; and a Scientific Lineage Layer maintaining citation tracking, version management, and DOI separation across related artifacts. BQEB ForecastBench (DOI: 10.5281/zenodo.21735978), previously published standalone, is the first validated benchmark implementation developed within the BQEB Reproducibility Architecture—built using the framework, not identical to it. Within-environment deterministic execution has been verified through 15 real executions (3 seeds × 5 repeats, independently re-confirmed twice); checksummed-artifact verification has been executed against this program’s own engineering files, with real tamper-detection confirmed. Cross-environment reproducibility, statistical significance of model comparisons, and extensibility beyond this architecture’s first module remain explicitly disclosed as pending or future work, not claimed as validated. Where internal audit identified an unresolved issue—a numerical discrepancy between two published versions of the same benchmark result—we document it as a governance case rather than resolving it here. The contribution is the architecture itself: how a research program’s artifacts can be structured, checksummed, and tracked for independent verification, at the evidence tier this paper actually supports.