Parallel computation in the Stan probabilistic programming language

Rok Češnovar (Author), Patricio Bulić (Mentor), Erik Štrumbelj (Co-mentor)

Abstract

The field of Bayesian statistics has experienced a major boom in recent years with tools based on Hamiltonian Monte Carlo. These tools are computationally demanding, as they have to calculate a large number of gradients, most often with a large amount of data and a large number of parameters. Among the more popular tools for Bayesian inference, Stan is considered the most tailored to statisticians, as it has a very expressive and flexible language. Prior to the 2.19 release, Stan did not have support for parallel computing on graphics processing units (GPU). In this work, we present how we developed GPU support for Stan. We have achieved this by upgrading the Stan Math library so that it allows gradients to be calculated on the GPU using OpenCL. The implementation we made is extendable and easy to maintain, which is crucial when implementing support for a mature tool like Stan. In addition to the Math library, we modified the Stan-to-C++ transpiler to efficiently use the new support in the Stan Math library, and added a scheduling algorithm that ensures the fastest device is selected. With the upgrades in the CmdStan and by creating CmdStanR, we brought the presented work closer to a wider range of users. The work shows that both Stan Math functions as well as Stan models run significantly faster using a GPU, with speedups up to 60. With these new features Stan is now on par with the other currently widely used tools for Bayesian inference, where users have to focus more on the computational aspects of their implementation.

Keywords

graphics processing unit;OpenCL;automatic differentiation;Bayesian inference;computer science;doctoral dissertations;

Data

Language:	English
Year of publishing:	2022
Typology:	2.08 - Doctoral Dissertation
Organization:	UL FRI - Faculty of Computer and Information Science
Publisher:	[R. Češnovar]
UDC:	004.43(043.3)
COBISS:	108349187
Views:	143
Downloads:	25
Average score:	0 (0 votes)
Metadata:

Other data

Secondary language:	Slovenian
Secondary title:	Vzporedno računanje v verjetnostnem programskem jeziku Stan
Secondary abstract:	Področje Bayesove statistike je v zadnjih letih doživelo velik razcvet z orodji, ki temeljijo na algoritmu Hamiltonskega Monte Carla. Ta orodja so računsko zahtevna, saj morajo med delovanjem izračunati veliko število gradientov, največkrat pri veliko podatkih in velikem številu parametrov. Med bolj popularnimi orodji za Bayesovo sklepanje je Stan, ki velja za statistikom najbolj prijazno, saj ima zelo ekspresiven in fleksibilen jezik. Stan pred različico 2.19 ni imel podpore za vzporedno računanje na grafičnih procesnih enotah (GPE). V tem delu predstavimo razvoj le-te. Uspšeno implementacijo podpore za GPE smo dosegli z nadgradnjo matematične knjižnice Stan Math tako, da omogoča računanje gradientov na GPE s pomočjo ogrodja OpenCL-a. Implementirana rešitev je razširljiva ter omogoča enostavno vzdrževanje, kar je ključnega pomena pri implementaciji podpore za zrelo orodje, kot je Stan. Poleg knjižnice smo razvili novo funkcionalnost za translator iz jezika Stan v C++, da je zmožen smiselno uporabiti novo podporo v knjižnici Stan Math, algoritem za avtomatsko izbiro pa poskrbi, da izbere najhitrejšo napravo brez posredovanja uporabnika. Z nadgradnjami v vmesniku CmdStan in razvojem novega vmesnika CmdStanR pa smo podporo za GPE približali široki množici uporabnikov. Rezultati kažejo, da se tako novo razvite funkcije kot tudi modeli, ki jih uporabljajo, na GPE izvajajo občutno hitreje, tudi do 60-krat. S tem delom je Stan po zmogljivosti dohitel ostala trenutno široko uporabljena orodja za Bayesovo statistiko, kjer morajo uporabniki sami poskrbeti za učinkovito izvajanje.
Secondary keywords:	grafične procesne enote;OpenCL;avtomatsko odvajanje;Bayesovo sklepanje;doktorske disertacije;Programski jeziki;Računalništvo;Univerzitetna in visokošolska dela;
Type (COBISS):	Doctoral dissertation
Study programme:	1000474
Embargo end date (OpenAIRE):	1970-01-01
Thesis comment:	Univ. v Ljubljani, Fak. za računalništvo in informatiko
Pages:	XX, 162 str.
ID:	15268244