Data Science: Logistic Regression with SmartCore.

Join Medium with my referral link - applied.math.coding

This story is part of my Data Science series.

In our previous story (see here) we have looked at the theory behind logistic regression and implemented a very rudimentary version in Rust. For real productive problems this implementation though still is missing many details. For this reason, I want to quickly show you in this story how one can use the well-written crate SmartCore for this task.

The data set will be the same, that is weathercsv, and the preprocessing has been down by using pandas in order to remove some null values and create suitable dummy variables from all categorical variables. Moreover, continuous variables have been left as they are but normalized to the interval [0, 1].

If you have any sort of similar data, you can simply use these instead to create your own model.

Our task will be to predict conditional expectations of the binary outcome that indicates the next day to rain or not to rain.

The dependencies of the project look like so:

[dependencies]
smartcore = "0.2.0"
ndarray = "0.15.0"

For convenience, we will use the same data-loading technique as in the last story:

use ndarray::{concatenate, Array2, ArrayBase, Axis, Dim, OwnedRepr, ViewRepr};
use std::{
    fs::File,
    io::{BufRead, BufReader},
};

fn load_data() -> ArrayBase<OwnedRepr<f64>, Dim<[usize; 2]>> {
    let f =
        File::open("data/weather/weather-processed.csv")
      .expect("problem with opening the file");
    let reader = BufReader::new(f);
    let data: Vec<Vec<f64>> = reader
        .lines()
        .skip(1)
        .map(|line| {
            line.unwrap()
                .split(',')
                .map(|s| {
                    s.parse::<f64>()
                        .expect(&format!("cannot parse to f64: {}", s))
                })
                .collect()
        })
        .collect();
    let shape = (data.len(), data[0].len());
    Array2::from_shape_vec(shape, data.into_iter().flatten()…