Institutions | About Us | Help | Gaeilge
rian logo


Mark
Go Back
Reverse-Mode AD in a Functional Framework: Lambda the Ultimate Backpropagator
Pearlmutter, Barak A.; Siskind, Jeffrey Mark
We show that reverse-mode AD (Automatic Differentiation)—a generalized gradient-calculation operator—can be incorporated as a first-class function in an augmented lambda calculus, and therefore into a functional-programming language. Closure is achieved, in that the new operator can be applied to any expression in the augmented language, yielding an expression in that language. This requires the resolution of two major technical issues: (a) how to transform nested lambda expressions, including those with free-variable references, and (b) how to support self application of the AD machinery. AD transformations preserve certain complexity properties, among them that the reverse phase of the reverse-mode AD transformation of a function have the same temporal complexity as the original untransformed function. First-class unrestricted AD operators increase the expressive power available to the numeric programmer, and may have significant practical implications for the construction of numeric software that is robust, modular, concise, correct, and efficient.
Keyword(s): Hamilton Institute; Computer Science; closures; derivatives; forward-mode AD; higher-order AD; higher-order functional languages; Jacobian; program transformation; reflection
Publication Date:
2008
Type: Journal article
Peer-Reviewed: Yes
Institution: Maynooth University
Funder(s): Science Foundation Ireland
Citation(s): Pearlmutter, Barak A. and Siskind, Jeffrey Mark (2008) Reverse-Mode AD in a Functional Framework: Lambda the Ultimate Backpropagator. ACM Transactions on Programming Languages and Systems, 30 (2). 7.1-7.36. ISSN 0164-0925
Publisher(s): Association for Computing Machinery (ACM)
File Format(s): application/pdf
Related Link(s): http://eprints.maynoothuniversity.ie/3504/1/BP_Reverse-Mode.pdf
First Indexed: 2014-09-20 05:03:52 Last Updated: 2018-11-30 06:14:49