Blyx

Dipper

CSV ingestion framework · 0.1.0

Parse quoted CSV records, preserve embedded line breaks, and validate a row width before consuming a file.

Import a two-column notes file

The importer preserves a literal quote in the first note and an embedded newline in the second. Give it a UTF-8 CSV file to check its width before reading the note column.

Download the standalone example

Extract the archive and run dipper-demo.exe on Windows or ./dipper-demo on Linux and macOS. Each archive includes the Blyx source and tests.

Build from source

With Blyx 0.2.1, Rust and a host linker installed, run these commands to edit and rebuild the example.

git clone --depth 1 --branch v0.1.0 https://github.com/Rahulchaube1/blyx-dipper.git
cd blyx-dipper
blyxc native-run project.blyx

Why this framework exists

Splitting each line on commas changes both field contents and record counts when a field contains a quoted comma or newline. RFC 4180 describes the quoting and doubled-quote rules that the Dipper parser uses. RFC 4180, Common Format and MIME Type for CSV Files

Install Dipper in your editor

Extension RahulChaube.blyx-dipper · 0.1.0

The extension includes a runnable Blyx example, function completions and the framework source. For the modular project, extract the starter, open its folder and run blyxc native-run main.blyx with the native toolchain installed.

Create an empty .blyx file, type blyx-dipper-demo and accept the completion. Choose Blyx: Run File to run the example.

For an offline installation, choose Extensions: Install from VSIX for Blyx Language first, then for Dipper.

Download the companion Blyx Language VSIX

Install with Conda

Create an environment containing Dipper 0.1.0, its Blyx source and a runnable example. Run the second command with Blyx 0.2.1, Rust and a host linker installed.

conda create -y --prefix ./blyx-env --override-channels -c https://rahulchaube1.github.io/blyx-packages blyx-dipper=0.1.0=0
blyxc native-run ./blyx-env/share/blyx/dipper/main.blyx

To use this package in your application, copy share/blyx/dipper/vendor/dipper from the environment into your project's vendor directory.

Install the full collection with Homebrew or Scoop

Import into a Blyx project

Install Blyx 0.2.1, Rust 1.85 or newer and a host linker. From the directory containing your entry file, clone this tagged release.

git clone --depth 1 --branch v0.1.0 https://github.com/Rahulchaube1/blyx-dipper.git vendor/dipper

Save this as main.blyx beside vendor and run blyxc native-run main.blyx.

use vendor::dipper::dipper;

fn main() {
    // Tokens for a quoted field containing a comma followed by a second field.
    let shape = di_shape([2, 0, 1, 0, 2, 1, 0, 3]);
    println!(at(shape, 0));
    println!(at(shape, 1));
    println!(len(di_shape([2, 0])));
}

Expected output

1
2
0

API and input requirements

Parse a UTF-8 string in memory. Delimiters are commas. Records accept CRLF, LF or CR; embedded quoted newlines preserve their original bytes. Spaces are data. A malformed quote discards partial results. Empty input contains zero rows; a blank line holds one empty cell. di_bad_row applies a caller-supplied column count.

APIUse
di_parse(input), di_ok(records)Import csv. Parse into F-prefixed field events and R end-of-row events. A lone E-prefixed event describes a syntax error.
di_cell(records, row, column)Return one string for a present cell, including an empty cell; an absent cell returns an empty list.
di_bad_row(records, width)Return the first zero-based row of the wrong width, −1 for success or −2 for a parse or argument error.
di_transition(state, token), di_shape(tokens)Walk the portable quote state machine. A valid stream yields [rows, fields].

Develop and contribute

Open the tagged source to read the implementation and examples. Include the smallest reproducible Blyx program, compiler version and expected result when reporting an issue.