YOLOR-Based Multi-Task Learning

Hung-Shuo Chang, Chien-Yao Wang, Richard Robert Wang, Gene Chou, Hong-Yuan Mark Liao
9/29/2023
cs.CV

No official code found. Best implementation: WongKinYiu/yolor (2,004 stars, GPL-3.0, PyTorch, last commit 2024-11-03).

Abstract

Multi-task learning (MTL) aims to learn multiple tasks using a single model and jointly improve all of them assuming generalization and shared semantics. Reducing conflicts between tasks during joint learning is difficult and generally requires careful network design and extremely large models. We propose building on You Only Learn One Representation (YOLOR), a network architecture specifically designed for multitasking. YOLOR leverages both explicit and implicit knowledge, from data observations and learned latents, respectively, to improve a shared representation while minimizing the number of training parameters. However, YOLOR and its follow-up, YOLOv7, only trained two tasks at once. In this paper, we jointly train object detection, instance segmentation, semantic segmentation, and image captioning. We analyze tradeoffs and attempt to maximize sharing of semantic information. Through our architecture and training strategies, we find that our method achieves competitive performance on all tasks while maintaining a low parameter count and without any pre-training. We will release code soon.

View on arXivView PDF

Code Implementations(1)

WongKinYiu/yolorCommunity100%
PyTorchGPL-3.0

implementation of paper - You Only Learn One Representation: Unified Network for Multiple Tasks (https://arxiv.org/abs/2105.04206)

2,004506Shell, PythonApr 12, 20211 years agoGPL-3.0
darknetexplicitimplicitpytorchrepresentation+6 more

Similar Papers

MTLSegFormer: Multi-task Learning with Transformers for Semantic Segmentation in Precision Agriculture

87% match2023

AHMAD: Adaptive Hybrid Multi-task Vision Learning with Assisted Distillation for Keypoint Detection

87% match2026

Mitigating Task Interference in Multi-Task Learning via Explicit Task Routing with Non-Learnable Primitives

86% match2026

Data exploitation: multi-task learning of object detection and semantic segmentation on partially annotated data

86% match1 repo82023

CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representations

86% match2026

Saliency-Regularized Deep Multi-Task Learning

85% match1 repo122022

Cite this paper

@article{chang2023yolorbased,
  title  = {YOLOR-Based Multi-Task Learning},
  author = {Hung-Shuo Chang and Chien-Yao Wang and Richard Robert Wang and Gene Chou and Hong-Yuan Mark Liao},
  year   = {2023},
  eprint = {2309.16921},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.16921v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.16921.svg)](https://codeofpaper.com/paper/2309.16921)

Discussion