-
Prefer F.relu
prefer F.relu(tensor) instead of torch.relu(tensor) or tensor.relu()
prefer nn.ReLU() when composing -
Using explicit layers are preferred over lazy layers, as it forces calculation, and catches miscalculation and therefore mismatched expectations. using layers for prototyping is fine.