Q: 3
An ML engineer has trained a neural network by using stochastic gradient descent (SGD). The neural
network performs poorly on the test set. The values for training loss and validation loss remain high
and show an oscillating pattern. The values decrease for a few epochs and then increase for a few
epochs before repeating the same cycle.
What should the ML engineer do to improve the training process?
Options
Discussion
D . The high and oscillating loss is usually a sign that the learning rate is too aggressive, causing SGD to overshoot minima. Trap here is C, but increasing it would make the bounce worse. If someone disagrees let me know.
D is the way I'd go. When both training and validation loss are up and down like that, usually means the learning rate’s just too high and SGD keeps overshooting. Lowering it helps things converge more smoothly. Not 100% but this lines up with what I've seen.
Probably D, the loss jumping up and down usually means learning rate is too high.
Wish AWS would stop recycling these SGD questions, D
Option C
Nah, I don’t think it’s C since high, bouncing losses point to learning rate being too big. D.
Looks like C could help if you want things to converge faster. Sometimes increasing the learning rate fixes slow training, but here the losses are oscillating. I think I'd still try C before anything else though, since D feels a bit too safe for speed. Disagree?
That bouncing loss pattern usually means the learning rate's too high, so D is the fix. Lowering it can help SGD settle down. Pretty sure this is what they're looking for but happy if someone disagrees.
Oscillating high loss on both train and val sets usually points to a learning rate that's too big, so D makes sense here. Saw similar Qs in exam reports, dropping the rate helps stabilize SGD. Not 100% but this fits the scenario.
D imo. Had something like this in a mock before-oscillating loss is classic too-high learning rate. Dropping it helps SGD settle better, less bouncing around. Pretty sure this is the fix, but open if someone had a different result.
Be respectful. No spam.
