Parallel Algorithms Align with Neural Execution

Valerie Engelmayer, Dobrik Georgiev, Petar Veličković
7/8/2023
cs.LG

No confident code implementation found for this paper.

Abstract

Neural algorithmic reasoners are parallel processors. Teaching them sequential algorithms contradicts this nature, rendering a significant share of their computations redundant. Parallel algorithms however may exploit their full computational power, therefore requiring fewer layers to be executed. This drastically reduces training times, as we observe when comparing parallel implementations of searching, sorting and finding strongly connected components to their sequential counterparts on the CLRS framework. Additionally, parallel versions achieve (often strongly) superior predictive performance.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

DeepPCR: Parallelizing Sequential Operations in Neural Networks

84% match2026

Unifying Optimization and Dynamics to Parallelize Sequential Computation: A Guide to Parallel Newton Methods for Breaking Sequential Bottlenecks

83% match2026

Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning

82% match2025

PaSE: Parallelization Strategies for Efficient DNN Training

80% match2024

OptEx: Expediting First-Order Optimization with Approximately Parallelized Iterations

80% match1 repo82024

Positional Attention: Expressivity and Learnability of Algorithmic Computation

79% match1 repo142024

Cite this paper

@article{engelmayer2023parallel,
  title  = {Parallel Algorithms Align with Neural Execution},
  author = {Valerie Engelmayer and Dobrik Georgiev and Petar Veličković},
  year   = {2023},
  eprint = {2307.04049},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2307.04049v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2307.04049.svg)](https://codeofpaper.com/paper/2307.04049)

Discussion