Tracking Multiple Deformable Objects in Egocentric Videos

Mingzhen Huang, Xiaoxing Li, Jun Hu, Honghong Peng, Siwei Lyu
2/14/2026

Abstract

Most existing multiple object tracking (MOT) methods that solely rely on appearance features struggle in tracking highly deformable objects. Other MOT methods that use motion clues to associate identities across frames have difficulty handling egocentric videos effectively or efficiently. In this work, we present DogThruGlasses, a large-scale deformable multi-object tracking dataset, with 150 videos and 73K annotated frames, which is collected exclusively by smart glasses. We also propose DETracker, a new MOT method that jointly detects and tracks deformable objects in egocentric videos. DETracker uses three novel modules, namely the motion disentanglement network (MDN), the patch association network (PAN) and the patch memory network (PMN), to explicitly tackle severe ego motion and track fast morphing target objects. DETracker is end-to-end trainable and achieves near real-time speed, which outperforms existing state-of-the-art method on DogThruGlasses and YouTube-Hand.

DOISemantic Scholar

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Cite this paper

@article{huang2026tracking,
  title  = {Tracking Multiple Deformable Objects in Egocentric Videos},
  author = {Mingzhen Huang and Xiaoxing Li and Jun Hu and Honghong Peng and Siwei Lyu},
  year   = {2026},
  doi    = {10.1109/CVPR52729.2023.00147},
  url    = {https://doi.org/10.1109/CVPR52729.2023.00147},
  journal = {CVPR 2023 2023}
}

Discussion