LibMTL.weighting.GradNorm

class GradNorm

Bases: LibMTL.weighting.abstract_weighting.AbsWeighting

Gradient Normalization (GradNorm).

This method is proposed in GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks (ICML 2018) and implemented by us.

Parameters:

alpha (float, default=1.5) – The strength of the restoring force which pulls tasks back to a common training rate.

init_param()

Define and initialize some trainable parameters required by specific weighting methods.

backward(losses, **kwargs)

Parameters:

• losses (list) – A list of losses of each task.

• kwargs (dict) – A dictionary of hyperparameters of weighting methods.