Workflow Overview
graph TB
subgraph Pytorch Introduction
node4( Linear Regression)
click node4 "#linear-regression"
end
subgraph tried but got nothing to say about this.
end
subgraph declaring data
end
subgraph wrapping up the data in variables
end
subgraph Variable
end
subgraph Tensor
end
subgraph variable with values
end
subgraph One in and one out
end
subgraph our model
end
subgraph Construct our loss function and an Optimizer. The call to model.parameters()
end
subgraph in the SGD constructor will contain the learnable parameters of the two
end
subgraph nn.Linear modules which are members of the model.
end
subgraph Training loop
end
subgraph Forward pass: Compute predicted y by passing x to the model
end
subgraph Compute and print loss
end
subgraph Zero gradients, perform a backward pass, and update the weights.
end
subgraph After training
node4( Linear Regression) --> node23( Below Pytorch)
click node23 "#below-pytorch"
end
subgraph Rank 1 view of the second row of a
end
subgraph Rank 2 view of the second row of a
end
subgraph Prints "[5 6 7 8] (4,)"
end
subgraph Prints "[[5 6 7 8]] (1, 4)"
end
subgraph We can make the same distinction when accessing columns of an array:
end
subgraph Prints "[ 2 6 10] (3,)"
end
subgraph Prints "[[ 2]
end
subgraph Compute sum of all elements; prints "10"
end
subgraph Compute sum of each column; prints "[4 6]"
end
subgraph v has shape (3,)
end
Pytorch Introduction
A starter tutorial on pytorch. Aims at introducing pytorch through examples namely by implementing
1) Linear Regression
2) Logistic Regression
3) Simple Neural Network
Before going into implementations let’s understand the basic unit of pytorch i.e a Tensor.
A tensor is a simple n dimensional array similar to numpy but can run on GPU’s. Hence computation can be spedup.
Another important feature that pytorch provides is autograd. i.e automatic differentiation wrt variables, this helps us obtain the gradient during the backpropagation step in any neural network
Now that we know the basic inners of pytorch. Let’s try and implement the most basic ML algo.
Linear Regression
import torch
# tried but got nothing to say about this.#declaring data
x = torch.Tensor([[1.0],[2.0],[4.0],[5.0]])
y = torch.Tensor([[2.0],[4.0],[8.0],[10.0]])The autograd feature we mentioned previously cannot be directly used with pytorch tensors. We need to wrap these tensors in Variables to use the feature.
In the computational graph we are going to build, this variable object is going to be one of the nodes.
If w is a Variable then w.data is a Tensor, and w.grad is another Variable holding the gradient of x with respect to some scalar value (usually the loss function).
from torch.autograd import Variable
# wrapping up the data in variables
x = Variable(x)
y = Variable(y)print(type(x)) # Variable
print(type(x.data)) # Tensor # variable with values
print(x)import torch
from torch.autograd import Variable
import torch.nn.functional as Fx_data = Variable(torch.Tensor([[1.0], [2.0], [3.0], [4.0]]))
y_data = Variable(torch.Tensor([[0.0], [0.0], [1.0], [1.0]]))class Model(torch.nn.module):
import torch
from torch.autograd import Variable
x_data = Variable(torch.Tensor([[1.0], [2.0], [3.0]]))
y_data = Variable(torch.Tensor([[2.0], [4.0], [6.0]]))
class Model(torch.nn.Module):
def __init__(self):
"""
In the constructor we instantiate two nn.Linear module
"""
super(Model, self).__init__()
self.linear = torch.nn.Linear(1, 1) # One in and one out
def forward(self, x):
"""
In the forward function we accept a Variable of input data and we must return
a Variable of output data. We can use Modules defined in the constructor as
well as arbitrary operators on Variables.
"""
y_pred = self.linear(x)
return y_pred# our model
model = Model()
# Construct our loss function and an Optimizer. The call to model.parameters()
# in the SGD constructor will contain the learnable parameters of the two
# nn.Linear modules which are members of the model.
criterion = torch.nn.MSELoss(size_average=False)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
# Training loop
for epoch in range(500):
# Forward pass: Compute predicted y by passing x to the model
y_pred = model(x_data)
# Compute and print loss
loss = criterion(y_pred, y_data)
print(epoch, loss.data[0])
# Zero gradients, perform a backward pass, and update the weights.
optimizer.zero_grad()
loss.backward()
optimizer.step()model(hour_var).data[0]# After training
hour_var = Variable(torch.Tensor([[4.0]]))
print("predict (after training)", 4, model.forward(hour_var).data[0][0])Below Pytorch
import numpy as npa = np.array([[1,2], [3, 4], [5, 6]])
aprint a[[0,1], [1]]import numpy as np
a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])row_r1 = a[1, :] # Rank 1 view of the second row of a
row_r2 = a[1:3, :] # Rank 2 view of the second row of a
print(row_r1, row_r1.shape) # Prints "[5 6 7 8] (4,)"
print(row_r2, row_r2.shape) # Prints "[[5 6 7 8]] (1, 4)"# We can make the same distinction when accessing columns of an array:
col_r1 = a[:, 1]
col_r2 = a[:, 1:2]
print(col_r1, col_r1.shape) # Prints "[ 2 6 10] (3,)"
print(col_r2, col_r2.shape) # Prints "[[ 2]a = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])b = [1,0,1,2]aa[np.arange(4),b] += 10a[np.arange(4),b]bool_idx = (a>10)bool_idxa[bool_idx]print a[a>10]a.dtypex = np.array([[1,2],[3,4]])
y = np.array([[5,6],[7,8]])
v = np.array([9,10])
w = np.array([11, 12])print(np.dot(v, w))print(np.dot(x, v))print(np.dot(x, y))import numpy as np
x = np.array([[1,2],[3,4]])
print(np.sum(x)) # Compute sum of all elements; prints "10"
print(np.sum(x, axis=0)) # Compute sum of each column; prints "[4 6]"
print(np.sum(x, axis=1))xnp.sum(x,axis = 0)x = np.array([[1,2],[1,3]])print x.Tx = np.array([[1,2,3], [4,5,6]])
v = np.array([1,2,3]) # v has shape (3,)
w = np.array([4,5])x+vx.Tx.T+wprint w
print np.reshape(w,(2,1))
print xprint(x + np.reshape(w, (2, 1)))