Minitorch is a way of me learning about autograd and creating a pytorch like library.

1. Imports like Pytorch,
2. Implements basic functions like backpropagation, gradient descent, neurons, linear layers
3. Extensible with suport of custom modules like Python
4. You can save the model weights and load the earlier model weights too

1. Vectorization
2. Implementing CNNs, Transformers

I am working on theory of Jacobian Matrix and it's use for computing derivatives in a vector. So, will do this in V2.

1. Install the necessary packages using

pip install -r requirements.txt

2. Run the individual demo using

python demos/00-autograd.py

Use this if only you are planning to update the package. do not use this to run the demo.

1. Install all the necessary requirements using;

pip install -r requirements.txt

2. Build the minitorch package for local installation.

python -m build

3. Install the minitorch package locally

pip install .

1. Demos can be found in /demos, you can run individual of them by;

python -m ./demos/example-file-name>gt;.py

2. If you want to create your own model, please look at documentation below.

Note that since, the minitorch library doesn't use vectorization things are not efficient hence, you cannot train the model on big data even like on MNIST. It becomes too slow. Try implementing some simpler stuffs.

• This is similar to Pytorch Tensor.

from minitorch import Value


a = Value(0.0)

a.grad    # gradient of a

a.backward()    # backward propagate the value

You can create a complex arithmetic operation and do backward on top to see demo of backprop. See demos/00-autograd.py.

You can import

• Linear
• Neuron
• Module
• RNN
• CrossEntropy
• MSE
• NLL

from minitorch.nn and create your own model.

Examples are at demos/01-or-gate.py, demos/02-custom-model.py.

Following activations;

• Sigmoid
• Softmax
• ReLU
• LogSoftmax
• TanH

can be imported from minitorch.activations.

And you can also load optimizer, for now gradient descent only.

See example at; 05-simple-polynomial.py

Upload from GitHub