ML From Scratch

A small machine learning library built from scratch in Python and NumPy with no PyTorch or TensorFlow. Built to actually understand how these machine learning models work under the hood: forward/backward passes, gradient descent, and loss functions written by hand.

Core

• neuralnetwork.py — A neural network framework: DenseLayer, ReLULayer, SoftmaxLayer, SigmoidLayer, loss functions CrossEntropyLoss, BCELoss, MSELoss and momentum-based gradient descent. Logistic and linear regression are both implemented as special cases of this (a single dense layer, with and without a sigmoid repectfully).
• decisiontree.py — A decision tree classifier built from Gini impurity and information gain, with recursive best split search with a RandomForestClassifier also.

Examples

Examples using this machine learning library:

File	What it does
examples/linearregression.py	Predicts house prices from a feature set (square footage, bedrooms, lot size, etc) using a single-layer linear regression neural network
examples/logisticregression.py	Binary classification on a small 2D toy dataset using a dense layer + sigmoid
examples/mnistmodel.py	Trains a slightly more complex neural network on MNIST digit classification, with data augmentation (random rotation/shift/zoom) for better training data
examples/mnistvisualiser.py	An interactive pygame GUI where you can draw a digit by hand and watch the trained MNIST model predict it live
examples/diabetesmodel.py	Trains a decision tree on the Pima Indians Diabetes dataset to predict diagnosis

Why

Most of my other work uses existing ML libraries, this project is the opposite: reimplementing the fundamentals (backprop, gradient descent, tree splitting) from scratch to understand what is actually happening.