Opening the paper…
What will be the result of applying ReLU to the following values?:
-2.7, 3.9, -1.0, 4.2, 6.4, -7.3
What will be the result of applying ReLU to the following values?:\ \-2.7, 3.9, -1.0, 4.2, 6.4, -7.3 Suppose you run gradient descent for linear regression for 500 iterations with a learning rate 0.02. You observe that the training loss (sum of squared loss) is increasing after every iteration. What may be the reason? What changes would you make to the set-up for the gradient descent to converge to a solution? Figure from the original question paper