Introduction¶
LibMTL
is an open-source library built on PyTorch for Multi-Task Learning (MTL). This library has the following three characteristics.
Unified:
LibMTL
provides a unified code base to implement and a consistent evaluation procedure including data processing, metric objectives, and hyper-parameters on several representative MTL benchmark datasets, which allows quantitative, fair, and consistent comparisons between different MTL algorithms.Comprehensive:
LibMTL
supports 84 MTL models combined by 7 architectures and 12 loss weighting strategies. Meanwhile,LibMTL
provides a fair comparison on 3 computer vision datasets.Extensible:
LibMTL
follows the modular design principles, which allows users to flexibly and conveniently add customized components or make personalized modifications. Therefore, users can easily and fast develop novel loss weighting strategies and architectures or apply the existing MTL algorithms to new application scenarios with the support ofLibMTL
.
Supported Algorithms¶
LibMTL
currently supports the following algorithms:
12 loss weighting strategies.
Weighting Strategy | Venues | Comments |
---|---|---|
Equal Weighting (EW) | - | Implemented by us |
Gradient Normalization (GradNorm) | ICML 2018 | Implemented by us |
Uncertainty Weights (UW) | CVPR 2018 | Implemented by us |
MGDA | NeurIPS 2018 | Referenced from official PyTorch implementation |
Dynamic Weight Average (DWA) | CVPR 2019 | Referenced from official PyTorch implementation |
Geometric Loss Strategy (GLS) | CVPR 2019 workshop | Implemented by us |
Projecting Conflicting Gradient (PCGrad) | NeurIPS 2020 | Implemented by us |
Gradient sign Dropout (GradDrop) | NeurIPS 2020 | Implemented by us |
Impartial Multi-Task Learning (IMTL) | ICLR 2021 | Implemented by us |
Gradient Vaccine (GradVac) | ICLR 2021 Spotlight | Implemented by us |
Conflict-Averse Gradient descent (CAGrad) | NeurIPS 2021 | Referenced from official PyTorch implementation |
Random Loss Weighting (RLW) | arXiv | Implemented by us |
7 architectures.
Architecture | Venues | Comments |
---|---|---|
Hard Parameter Sharing (HPS) | ICML 1993 | Implemented by us |
Cross-stitch Networks (Cross_stitch) | CVPR 2016 | Implemented by us |
Multi-gate Mixture-of-Experts (MMoE) | KDD 2018 | Implemented by us |
Multi-Task Attention Network (MTAN) | CVPR 2019 | Referenced from official PyTorch implementation |
Customized Gate Control (CGC) | ACM RecSys 2020 Best Paper | Implemented by us |
Progressive Layered Extraction (PLE) | ACM RecSys 2020 Best Paper | Implemented by us |
DSelect-k | NeurIPS 2021 | Referenced from official TensorFlow implementation |
84 combinations of different architectures and loss weighting strategies.
Citation¶
If you find LibMTL
useful for your research or development, please cite the following:
@article{LibMTL,
title={LibMTL: A Python Library for Multi-Task Learning},
author={Baijiong Lin and Yu Zhang},
journal={arXiv preprint arXiv:2203.14338},
year={2022}
}
Contributors¶
LibMTL
is developed and maintained by Baijiong Lin and Yu Zhang.
Contact Us¶
If you have any question or suggestion, please feel free to contact us by raising an issue or sending an email to bj.lin.email@gmail.com
.
Acknowledgements¶
We would like to thank the authors that release the public repositories (listed alphabetically): CAGrad, dselect_k_moe, MultiObjectiveOptimization, and mtan.