Description: In this assignment, you are going to write a python program to implement logistics
regression for digit recognition.
The image size is 28 x 28. Here is one of the training example.
Here are the tasks for this assignment
• Add ones to the first row to the data matrix
• Make the indicator variable associated with class labels
[[0. 1. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 1. 0. 1.]
[0. 0. 0. … 0. 0. 0.]]
• Initialize a random weight matrix whose size is the image dimension * num of classes
[[1. 1. 1. … 1. 1. 1.]
[0. 0. 0. … 0. 0. 0.]
[0. 0. 0. … 0. 0. 0.]
…
• Next, you need write a train method performing the following steps
▪ Write a softmax function to compute
where W is the weight matrix, T is the matrix transpose, and X is the data matrix
▪ Compute the gradient which is X * (P(y | z) – indicator)
▪ Perform gradient descent
o W = W – learning_rate * gradient
Prob, W, X, indicator and Gradient are matrices
def train():
Loop for max_epoch
Prob <- softmax(W, X)
Gradient <- X * (Prob – indicator)
W <- W – learning_rate * Gradient
return W
• Save the W to a pickle file
To Run:
python TrainLogit.py
Submission:
You need to ZIP up the entire folder and submit the ZIP file to blackboard
** If the file is not a ZIP file, it will be 10 points deduction.
Sample Solution
