Software-based Automatic Differentiation is Flawed

Daniel Johnson, Trevor Maxfield, Yongxu Jin, Ronald Fedkiw
5/5/2023
cs.LG

No confident code implementation found for this paper.

Abstract

Various software efforts embrace the idea that object oriented programming enables a convenient implementation of the chain rule, facilitating so-called automatic differentiation via backpropagation. Such frameworks have no mechanism for simplifying the expressions (obtained via the chain rule) before evaluating them. As we illustrate below, the resulting errors tend to be unbounded.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Understanding Automatic Differentiation Pitfalls

83% match2023

A Step-by-step Introduction to the Implementation of Automatic Differentiation

78% match2024

A Benchmark of Selected Algorithmic Differentiation Tools on Some Problems in Computer Vision and Machine Learning

75% match1 repo32018

Programming by Backprop: LLMs Acquire Reusable Algorithmic Abstractions During Code Training

72% match2025

Combinatory Adjoints and Differentiation

71% match2022

Matrix Calculus (for Machine Learning and Beyond)

71% match1 repo5942025

Cite this paper

@article{johnson2023softwarebased,
  title  = {Software-based Automatic Differentiation is Flawed},
  author = {Daniel Johnson and Trevor Maxfield and Yongxu Jin and Ronald Fedkiw},
  year   = {2023},
  eprint = {2305.03863},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2305.03863v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2305.03863.svg)](https://codeofpaper.com/paper/2305.03863)

Discussion