Blyx

Calibration

Probability metrics library · 0.1.1

Check agreement between predicted probabilities and observed outcomes.

Install with Conda

Create an environment containing Calibration 0.1.1, its Blyx source and a runnable example. Run the second command with Blyx 0.2.1, Rust and a host linker installed.

conda create -y --prefix ./blyx-env --override-channels -c https://rahulchaube1.github.io/blyx-packages blyx-calibration=0.1.1=0
blyxc native-run ./blyx-env/share/blyx/calibration/main.blyx

To use this package in your application, copy share/blyx/calibration/vendor/calibration from the environment into your project's vendor directory.

Install the full collection with Homebrew or Scoop

Import into a Blyx project

Install Blyx 0.2.1, Rust 1.85 or newer and a host linker. From the directory containing your entry file, clone this tagged release.

git clone --depth 1 --branch v0.1.1 https://github.com/Rahulchaube1/blyx-calibration.git vendor/calibration

Save this as main.blyx beside vendor and run blyxc native-run main.blyx.

use vendor::calibration::calibration;

fn main() {
    let labels = [0, 0, 1, 1];
    let scores = [0.25, 0.25, 0.75, 0.75];
    println!(cal_brier(labels, scores));
    println!(cal_ece(labels, scores, 2));
}

Expected output

0.0625
0.25

API and input requirements

Use equal-length, nonempty vectors of 0/1 labels and probabilities. Supply between one and 100 bins. A probability of 1 belongs in the last bin. The bin count matters when comparing expected calibration error.

APIUse
cal_brier(labels, scores)Compute mean squared probability error.
cal_bin(labels, scores, index, bins)Return a reliability bin’s count, forecast total and outcome total.
cal_ece(labels, scores, bins)Summarize bin gaps weighted by observation count.

Develop and contribute

Open the tagged source to read the implementation and examples. Include the smallest reproducible Blyx program, compiler version and expected result when reporting an issue.