Blyx

Conformal

Uncertainty framework · 0.1.0

Calibrate a regression interval from held-out errors and measure coverage on a separate test set.

Install Conformal in your editor

Extension RahulChaube.blyx-conformal · 0.1.0

The extension includes a runnable Blyx example, function completions and the framework source. For the modular project, extract the starter, open its folder and run blyxc native-run main.blyx with the native toolchain installed.

Create an empty .blyx file, type blyx-conformal-demo and accept the completion. Choose Blyx: Run File to run the example.

For an offline installation, choose Extensions: Install from VSIX for Blyx Language first, then for Conformal.

Download the companion Blyx Language VSIX

Install with Conda

Create an environment containing Conformal 0.1.0, its Blyx source and a runnable example. Run the second command with Blyx 0.2.1, Rust and a host linker installed.

conda create -y --prefix ./blyx-env --override-channels -c https://rahulchaube1.github.io/blyx-packages blyx-conformal=0.1.0=0
blyxc native-run ./blyx-env/share/blyx/conformal/main.blyx

To use this package in your application, copy share/blyx/conformal/vendor/conformal from the environment into your project's vendor directory.

Install the full collection with Homebrew or Scoop

Import into a Blyx project

Install Blyx 0.2.1, Rust 1.85 or newer and a host linker. From the directory containing your entry file, clone this tagged release.

git clone --depth 1 --branch v0.1.0 https://github.com/Rahulchaube1/blyx-conformal.git vendor/conformal

Save this as main.blyx beside vendor and run blyxc native-run main.blyx.

use vendor::conformal::conformal;
fn main() {
    let calibration = cf_calibrate([10, 12, 7, 11], [10, 10, 10, 10], 3, 5);
    let interval = cf_interval(20.0, at(calibration, 0));
    println!(at(calibration, 0));
    println!(at(interval, 0));
    println!(at(interval, 1));
}

Expected output

2
18
22

API and input requirements

Use examples held out from model training. Each interval is prediction ± a finite calibration radius. Coverage is marginal under exchangeability, and can differ by subgroup or after a distribution shift. A 90% request needs at least nine calibration examples for a finite interval.

APIUse
cf_calibrate(actual, predicted, numerator, denominator)Select an observed absolute residual using the finite-sample adjusted rank.
cf_radius(scores, numerator, denominator)Calibrate from previously calculated nonnegative scores.
cf_interval(prediction, radius), cf_coverage(actual, predicted, radius)Return the two endpoints, or measure the fraction of a separate test set covered.

Develop and contribute

Open the tagged source to read the implementation and examples. Include the smallest reproducible Blyx program, compiler version and expected result when reporting an issue.