End-to-End Multi-Task Learning with Attention

Shikun Liu, Edward Johns, Andrew J. Davison
3/28/2018
cs.CV

Official code: lorenmt/mtan (737 stars, MIT, PyTorch, last commit 2022-02-07).

Abstract

We propose a novel multi-task learning architecture, which allows learning of task-specific feature-level attention. Our design, the Multi-Task Attention Network (MTAN), consists of a single shared network containing a global feature pool, together with a soft-attention module for each task. These modules allow for learning of task-specific features from the global features, whilst simultaneously allowing for features to be shared across different tasks. The architecture can be trained end-to-end and can be built upon any feed-forward neural network, is simple to implement, and is parameter efficient. We evaluate our approach on a variety of datasets, across both image-to-image predictions and image classification tasks. We show that our architecture is state-of-the-art in multi-task learning compared to existing methods, and is also less sensitive to various weighting schemes in the multi-task loss function. Code is available at https://github.com/lorenmt/mtan.

View on arXivView PDF

Code Implementations(1)

lorenmt/mtanOfficial100%
PyTorchMIT

The implementation of "End-to-End Multi-Task Learning with Attention" [CVPR 2019].

737118PythonJan 6, 20194 years agoMIT
attention-modelmulti-task-learningpytorch

Similar Papers

MTLSI-Net: A Linear Semantic Interaction Network for Parameter-Efficient Multi-Task Dense Prediction

87% match2026

YOLOR-Based Multi-Task Learning

86% match1 repo2.0k2023

MTLSegFormer: Multi-task Learning with Transformers for Semantic Segmentation in Precision Agriculture

86% match2023

Intra-task Mutual Attention based Vision Transformer for Few-Shot Learning

85% match2024

ChangeMinds: Multi-task Framework for Detecting and Describing Changes in Remote Sensing

85% match1 repo82024

3D-Aware Multi-Task Learning with Cross-View Correlations for Dense Scene Understanding

85% match2025

Cite this paper

@article{liu2018endtoend,
  title  = {End-to-End Multi-Task Learning with Attention},
  author = {Shikun Liu and Edward Johns and Andrew J. Davison},
  year   = {2018},
  eprint = {1803.10704},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1803.10704}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1803.10704.svg)](https://codeofpaper.com/paper/1803.10704)

Discussion