Task Graph Maximum Likelihood Estimation for Procedural Activity Understanding in Egocentric Videos

Luigi Seminara, Giovanni Maria Farinella, Antonino Furnari
2/25/2025
cs.CV

Official code: fpv-iplab/Differentiable-Task-Graph-Learning (25 stars, last commit 2025-01-09).

Abstract

We introduce a gradient-based approach for learning task graphs from procedural activities, improving over hand-crafted methods. Our method directly optimizes edge weights via maximum likelihood, enabling integration into neural architectures. We validate our approach on CaptainCook4D, EgoPER, and EgoProceL, achieving +14.5%, +10.2%, and +13.6% F1-score improvements. Our feature-based approach for predicting task graphs from textual/video embeddings demonstrates emerging video understanding abilities. We also achieved top performance on the procedure understanding benchmark on Ego-Exo4D and significantly improved online mistake detection (+19.8% on Assembly101-O, +6.4% on EPIC-Tent-O). Code: https://github.com/fpv-iplab/Differentiable-Task-Graph-Learning.

View on arXivView PDF

Code Implementations(1)

fpv-iplab/Differentiable-Task-Graph-LearningOfficial100%

Code for the paper "Differentiable Task Graph Learning: Procedural Activity Representation and Online Mistake Detection from Egocentric Videos" [NeurIPS (spotlight), 2024]

253PythonJun 3, 20241 years agoNOASSERTION
egocentric-videosmaximum-likelihoodmistake-detectionneurips-2024spotlight+2 more

Similar Papers

Differentiable Task Graph Learning: Procedural Activity Representation and Online Mistake Detection from Egocentric Videos

93% match1 repo252024

Mosaic of Modalities: A Comprehensive Benchmark for Multimodal Graph Learning

81% match2024

Taskflow: A Lightweight Parallel and Heterogeneous Task Graph Computing System

81% match1 repo12.2k2020

Revisit the Imbalance Optimization in Multi-task Learning: An Experimental Analysis

80% match2025

CORE-MTL: Rethinking Gradient Balancing via Causal Orthogonal Representations

80% match2026

TBGC: Task-level Backbone-Oriented Gradient Clip for Multi-Task Foundation Model Learning

80% match2023

Cite this paper

@article{seminara2025task,
  title  = {Task Graph Maximum Likelihood Estimation for Procedural Activity Understanding in Egocentric Videos},
  author = {Luigi Seminara and Giovanni Maria Farinella and Antonino Furnari},
  year   = {2025},
  eprint = {2502.17753},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2502.17753v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2502.17753.svg)](https://codeofpaper.com/paper/2502.17753)

Discussion