3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network

Qinyu Chen, Zuowen Wang, Shih-Chii Liu, Chang Gao
8/22/2023
cs.CVcs.AIcs.LG

Official code: qinche106/cb-convlstm-eyetracking (26 stars, MIT, last commit 2024-01-26).

Abstract

This paper presents a sparse Change-Based Convolutional Long Short-Term Memory (CB-ConvLSTM) model for event-based eye tracking, key for next-generation wearable healthcare technology such as AR/VR headsets. We leverage the benefits of retina-inspired event cameras, namely their low-latency response and sparse output event stream, over traditional frame-based cameras. Our CB-ConvLSTM architecture efficiently extracts spatio-temporal features for pupil tracking from the event stream, outperforming conventional CNN structures. Utilizing a delta-encoded recurrent path enhancing activation sparsity, CB-ConvLSTM reduces arithmetic operations by approximately 4.7$\times$ without losing accuracy when tested on a \texttt{v2e}-generated event dataset of labeled pupils. This increase in efficiency makes it ideal for real-time eye tracking in resource-constrained devices. The project code and dataset are openly available at \url{https://github.com/qinche106/cb-convlstm-eyetracking}.

View on arXivView PDF

Code Implementations(1)

qinche106/cb-convlstm-eyetrackingOfficial100%
MIT

Event-based Eye Tracking using ConvLSTM networks

266PythonJun 23, 20232 years agoMIT

Similar Papers

FAPNet: An Effective Frequency Adaptive Point-based Eye Tracker

90% match2024

Retina : Low-Power Eye Tracking with Event Camera and Spiking Hardware

88% match1 repo552023

EyeGraph: Modularity-aware Spatio Temporal Graph Clustering for Continuous Event-based Eye Tracking

87% match1 repo42026

Co-designing a Sub-millisecond Latency Event-based Eye Tracking System with Submanifold Sparse CNN

87% match1 repo162024

MambaPupil: Bidirectional Selective Recurrent model for Event-based Eye tracking

86% match2024

EETnet: a CNN for Gaze Detection and Tracking for Smart-Eyewear

86% match2025

Cite this paper

@article{chen2023et,
  title  = {3ET: Efficient Event-based Eye Tracking using a Change-Based ConvLSTM Network},
  author = {Qinyu Chen and Zuowen Wang and Shih-Chii Liu and Chang Gao},
  year   = {2023},
  eprint = {2308.11771},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2308.11771v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2308.11771.svg)](https://codeofpaper.com/paper/2308.11771)

Discussion