GitHub

Binary classification

Binary classification of hand-written digits in Julia.

This page comes from a single Julia file: class01.jl.

You can access the source code for such Julia documentation using the 'Edit on GitHub' link in the top right. You can view the corresponding notebook in nbviewer here: class01.ipynb, or open it in binder here: class01.ipynb.

Setup

Add the Julia packages used in this demo. Change false to true in the following code block if you are using any of the following packages for the first time.

if false
    import Pkg
    Pkg.add([
        "InteractiveUtils"
        "LaTeXStrings"
        "LinearAlgebra"
        "MIRTjim"
        "MLDatasets"
        "Plots"
        "Random"
        "StatsBase"
    ])
end

Tell Julia to use the following packages. Run Pkg.add() in the preceding code block first, if needed.

using InteractiveUtils: versioninfo
using LaTeXStrings # nice plot labels
using LinearAlgebra: dot
using MIRTjim: jim, prompt
using MLDatasets: MNIST
using Plots: default, gui, savefig
using Plots: histogram, histogram!, plot
using Plots: RGB, cgrad
using Plots.PlotMeasures: px
using Random: seed!, randperm
using StatsBase: mean
default(); default(markersize=5, markerstrokecolor=:auto, label="",
 tickfontsize=14, labelfontsize=18, legendfontsize=18, titlefontsize=18)

The following line is helpful when running this file as a script; this way it will prompt user to hit a key after each figure is displayed.

isinteractive() ? jim(:prompt, true) : prompt(:draw);

Load data

Read the MNIST data for some handwritten digits. This code will automatically download the data from web if needed and put it in a folder like: ~/.julia/datadeps/MNIST/.

if !@isdefined(data)
    digitn = (0, 1) # which digits to use
    isinteractive() || (ENV["DATADEPS_ALWAYS_ACCEPT"] = true) # avoid prompt
    dataset = MNIST(Float32, :train)
    nrep = 100 # how many of each digit
    # function to extract the 1st `nrep` examples of digit n:
    data = n -> dataset.features[:,:,findall(==(n), dataset.targets)[1:nrep]]
    data = cat(dims=4, data.(digitn)...)
    labels = vcat([fill(d, nrep) for d in digitn]...) # to check later
    nx, ny, nrep, ndigit = size(data)
    data = data[:,2:ny,:,:] # make images non-square to force debug
    ny = size(data,2)
    data = reshape(data, nx, ny, :)
    seed!(0)
    tmp = randperm(nrep * ndigit)
    data = data[:,:,tmp]
    labels = labels[tmp]
    size(data) # (nx, ny, nrep*ndigit)
end
(28, 27, 200)

Look at "unlabeled" image data

pd = jim(data, "Data"; size=(600,300), tickfontsize=8,)
Example block output

Extract training data

data0 = data[:,:,labels .== 0]
data1 = data[:,:,labels .== 1];

pd0 = jim(data0[:,:,1:36]; nrow=6, colorbar=nothing, size=(400,400))
pd1 = jim(data1[:,:,1:36]; nrow=6, colorbar=nothing, size=(400,400))
# savefig(pd0, "class01-0.pdf")
# savefig(pd1, "class01-1.pdf")
Example block output

red-black-blue colorbar:

RGB255(args...) = RGB((args ./ 255)...)
color = cgrad([RGB255(230, 80, 65), :black, RGB255(23, 120, 232)]);

Weights

Compute sample average of each training class and define classifier weights as differences of the means.

μ0 = mean(data0, dims=3)
μ1 = mean(data1, dims=3)
w = μ1 - μ0; # hand-crafted weights

images of means and weights

siz = (540,400)
args = (xaxis = false, yaxis = false) # book
p0 = jim(μ0; clim=(0,1), size=siz, cticks=[0,1], args...)
p1 = jim(μ1; clim=(0,1), size=siz, cticks=[0,1], args...)
pw = jim(w; color, clim=(-1,1).*0.8, size=siz, cticks=(-1:1)*0.75, args...)
pm = plot( p0, p1, pw;
  size = (1400, 350),
  layout = (1,3),
  rightmargin = 20px,
)
# savefig(p0, "class01-0.pdf")
# savefig(p1, "class01-1.pdf")
# savefig(pw, "class01-w.pdf")
# savefig(pm, "class01-mean.pdf")
Example block output

Inner products

Examine performance of simple linear classifier. (Should be done with test data, not training data...)

i0 = [dot(w, x) for x in eachslice(data0, dims=3)]
i1 = [dot(w, x) for x in eachslice(data1, dims=3)];

bins = -80:20
ph = plot(
 xaxis = (L"⟨\mathbf{\mathit{v}},\mathbf{\mathit{x}}⟩", (-80, 20), -80:20:20),
 yaxis = ("", (0, 25), 0:10:20),
 size = (600, 250), bottommargin = 20px,
)
histogram!(i0; bins, color=:red , label="0")
histogram!(i1; bins, color=:blue, label="1")

# savefig(ph, "class01-hist.pdf")
Example block output
prompt()

Reproducibility

This page was generated with the following version of Julia:

using InteractiveUtils: versioninfo
io = IOBuffer(); versioninfo(io); split(String(take!(io)), '\n')
11-element Vector{SubString{String}}:
 "Julia Version 1.11.1"
 "Commit 8f5b7ca12ad (2024-10-16 10:53 UTC)"
 "Build Info:"
 "  Official https://julialang.org/ release"
 "Platform Info:"
 "  OS: Linux (x86_64-linux-gnu)"
 "  CPU: 4 × AMD EPYC 7763 64-Core Processor"
 "  WORD_SIZE: 64"
 "  LLVM: libLLVM-16.0.6 (ORCJIT, znver3)"
 "Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores)"
 ""

And with the following package versions

import Pkg; Pkg.status()
Status `~/work/book-la-demo/book-la-demo/docs/Project.toml`
  [6e4b80f9] BenchmarkTools v1.5.0
  [aaaa29a8] Clustering v0.15.7
  [35d6a980] ColorSchemes v3.27.1
⌅ [3da002f7] ColorTypes v0.11.5
⌃ [c3611d14] ColorVectorSpace v0.10.0
  [717857b8] DSP v0.7.10
  [72c85766] Demos v0.1.0 `~/work/book-la-demo/book-la-demo`
  [e30172f5] Documenter v1.7.0
  [4f61f5a4] FFTViews v0.3.2
  [7a1cc6ca] FFTW v1.8.0
  [587475ba] Flux v0.14.25
  [a09fc81d] ImageCore v0.10.4
  [71a99df6] ImagePhantoms v0.8.1
  [b964fa9f] LaTeXStrings v1.4.0
  [7031d0ef] LazyGrids v1.0.0
  [599c1a8e] LinearMapsAA v0.12.0
  [98b081ad] Literate v2.20.1
  [7035ae7a] MIRT v0.18.2
  [170b2178] MIRTjim v0.25.0
  [eb30cadb] MLDatasets v0.7.18
  [efe261a4] NFFT v0.13.5
  [6ef6ca0d] NMF v1.0.3
  [15e1cf62] NPZ v0.4.3
  [0b1bfda6] OneHotArrays v0.2.5
  [429524aa] Optim v1.10.0
  [91a5bcdd] Plots v1.40.9
  [f27b6e38] Polynomials v4.0.11
  [2913bbd2] StatsBase v0.34.3
  [d6d074c3] VideoIO v1.1.0
  [b77e0a4c] InteractiveUtils v1.11.0
  [37e2e46d] LinearAlgebra v1.11.0
  [44cfe95a] Pkg v1.11.0
  [9a3f8284] Random v1.11.0
Info Packages marked with ⌃ and ⌅ have new versions available. Those with ⌃ may be upgradable, but those with ⌅ are restricted by compatibility constraints from upgrading. To see why use `status --outdated`

This page was generated using Literate.jl.