The WebIO Jupyter extension was not detected. See the WebIO Jupyter integration documentation for more information.
Julia vs Stan performance comparison
Caveats
This page compares the performance of Julia’s and Stan’s log density and log density gradient computations for the implemented posteriors. Several caveats apply:
- The
posteriordbStan implementations were never meant to represent “perfect and best-performant” practices. - The StanBlocks.jl implementations are not line-by-line translations of the Stan implementations. Sometimes small optimizations were applied, to make the implementation fall more in line with common Julia practices, or to make the code more friendly for Julia’s AD packages, e.g. by avoiding mutation.
- Stan often automatically drops constant terms (unless configured differently), see https://mc-stan.org/docs/reference-manual/statements.html#log-probability-increment-vs.-distribution-statement, thus avoiding superfluous (for its purposes) computation, while the StanBlocks.jl implementations do not.
- Stan implements a lot of custom AD rules, while StanBlocks.jl does not at all, and Enzyme.jl does rarely (if ever?). I suspect that adding AD rules for
_glm_type functions would further improve Julia’s performance. - The StanBlocks.jl “sampling” statements try to be clever about avoiding repeated computations. While I am not sure whether Stan applies the same optimizations, in principle it could do that without extra work by the user.
- While preliminary benchmark runs included “all” Julia AD packages, all of them are almost always much slower than Enzyme.jl for the implemented posteriors, which on top of that performance advantage also supports more Julia language features than some of the other AD packages. As such, I am only comparing Enzyme and Stan. Enzyme outperforming every other AD package for these posteriors/loss functions does of course not mean that it will necessarily do as well for other applications.
- Enzyme’s development is progressing quite quickly. While it currently sometimes crashes Julia, or it sometimes errors while trying to compute a gradient, in general Enzyme’s performance and reliability are continuously and quickly improving.
- Stan’s benchmark is done from Julia via
BridgeStan.jl. While I think that any performance penalty should be extremely small, I am not 100% sure. BridgeStan uses the-O3compiler flag by default, but no additional ones. - All benchmarks are happening with a single thread on my local machine.
- There are probably more caveats!
Warning
In general, doing performance comparisons is quite tricky, for more reasons than just the ones mentioned above. The below plot and tables should most definitely NOT be interpreted as “A is X-times faster than B”.
Visualization
Warning
In general, doing performance comparisons is quite tricky, for more reasons than just the ones mentioned above. The below plot and tables should most definitely NOT be interpreted as “A is X-times faster than B”.
The below plot shows the relative primitive runtime (x-axis, Julia vs Stan, left: Julia is faster) and the relative gradient runtime (y-axis, Julia+X vs Stan, bottom: Julia is faster) for the posteriordb models for which the overview table has a value less than 1e-8 in the median relative ulpdf error column. The color of the points represents the Julia AD framework used, which currently includes Enzyme.jl and Mooncake.jl. Hovering over the data points will show the posterior name, its dimension, the allocations required by Julia during the primitive and gradient run and a short explanation, e.g. mesquite-logmesquite_logvash (D=7, #allocs=0->70) - Julia's primitive is ~4.5 times faster, but Enzyme's gradient is ~16.0 times slower. Any time spent on garbage collection has been subtracted from the measured wall times. All mean runtime estimates were run until the estimated standard error of the mean was smaller than roughly .5% of the estimated mean. Due to this, I have removed the credible intervals and standard errors from the plot and table.
┌ Warning: attempting to remove probably stale pidfile
│ path = "/home/niko/.jlassetregistry.lock"
└ @ Pidfile ~/.julia/packages/Pidfile/DDu3M/src/Pidfile.jl:260Tabular data
The below table shows information about the implemented posteriors. Will elaborate on the exact meaning of columns.
| posterior name | implementations | dimension | median relative ulpdf error | relative mean primitive Julia runtime | relative mean primitive Stan runtime | relative mean Enzyme runtime | relative mean Mooncake runtime | relative mean Stan gradient runtime | primitive Julia allocations | Enzyme allocations | Mooncake allocations | median lpdf difference | median Enzyme relative gradient error | median Mooncake relative gradient error | Bridgestan | Enzyme | Mooncake |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GLMM_data-GLMM1_model | 237 | 1.2e-16 | 1.0 | 3.9 | 1.0 | 3.7 | 2.2 | 0 | 0 | 18 | -2.3 | 1.2e-17 | 1.2e-17 | 2.6.1 | 0.13.30 | 0.4.86 | |
| GLM_Binomial_data-GLM_Binomial_model | 3 | 1.4e-16 | 1.0 | 1.5 | 1.2 | 3.1 | 1.0 | 0 | 0 | 47 | -14.0 | 3.3e-16 | 3.8e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| GLM_Poisson_Data-GLM_Poisson_model | 4 | 3.6e-16 | 1.0 | 2.3 | 1.0 | 6.6 | 1.5 | 0 | 0 | 47 | 0.0 | 4.3e-16 | 4.0e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| M0_data-M0_model | 2 | 1.7e-16 | 1.0 | 1.2 | 1.1 | 2.0 | 1.0 | 0 | 0 | 5 | 0.0 | 3.0e-15 | 8.9e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| Mb_data-Mb_model | 3 | 0.0 | 1.0 | 2.6 | 2.3 | 3.2 | 1.0 | 3 | 32 | 5770 | 0.0 | 6.6e-15 | 4.8e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| Mh_data-Mh_model | 388 | 4.8e-16 | 1.0 | 1.7 | 1.0 | 1.6 | 1.2 | 1 | 2 | 14 | -160.0 | 1.2e-15 | 7.4e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| Mt_data-Mt_model | 4 | 1.5e-16 | 1.0 | 1.9 | 3.9 | 3.9 | 1.0 | 1 | 26 | 4297 | 0.0 | 1.9e-15 | 2.6e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| Mtbh_data-Mtbh_model | 154 | 1.8e-16 | 1.0 | 2.4 | 1.8 | 2.5 | 1.0 | 6 | 38 | 2718 | -2.3 | 6.1e-16 | 3.4e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| Mth_data-Mth_model | 394 | 1.7e-16 | 1.0 | 2.2 | 2.0 | 2.9 | 1.0 | 3 | 30 | 7015 | 0.0 | 1.9e-15 | 6.6e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| Rate_1_data-Rate_1_model | 1 | 9.2e-17 | 1.0 | 2.3 | 1.0 | 2.3 | 2.1 | 0 | 0 | 0 | 5.5 | 1.5e-16 | 2.8e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| Rate_2_data-Rate_2_model | 2 | 8.7e-17 | 1.0 | 2.0 | 1.0 | 2.6 | 1.7 | 0 | 0 | 0 | 10.0 | 2.1e-16 | 2.7e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| Rate_3_data-Rate_3_model | 1 | 0.0 | 1.0 | 1.7 | 1.0 | 2.6 | 1.6 | 0 | 0 | 0 | 10.0 | 2.0e-16 | 2.0e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| Rate_4_data-Rate_4_model | 2 | 8.7e-17 | 1.0 | 2.5 | 1.0 | 2.5 | 2.2 | 0 | 0 | 0 | 2.7 | 1.6e-16 | 1.9e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| Rate_5_data-Rate_5_model | 1 | 0.0 | 1.0 | 2.0 | 1.0 | 3.1 | 2.1 | 0 | 0 | 0 | 0.0 | 1.8e-16 | 2.4e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| Survey_data-Survey_model | 1 | 5.4e-16 | 1.0 | 1.1 | 2.8 | 6.8 | 1.0 | 1 | 40 | 20865 | 1.4e-14 | 7.4e-14 | 5.4e-14 | 2.6.1 | 0.13.30 | 0.4.86 | |
| arK-arK | 7 | 1.4e-16 | 1.0 | 5.7 | 1.0 | 5.6 | 4.6 | 0 | 0 | 15 | -15.0 | 1.4e-15 | 1.5e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| arma-arma11 | 4 | 2.3e-16 | 1.0 | 3.8 | 1.0 | 5.4 | 5.5 | 0 | 0 | 5 | -4.6 | 1.5e-16 | 2.6e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| bball_drive_event_0-hmm_drive_0 | 6 | 1.2e-16 | 1.0 | 2.4 | 3.8 | 2.8 | 1.0 | 9 | 40 | 77 | -2.3 | 4.0e-12 | 2.9e-12 | 2.6.1 | 0.13.30 | 0.4.86 | |
| bball_drive_event_1-hmm_drive_1 | 6 | 9.1e-16 | 1.0 | 2.6 | 2.7 | 2.5 | 1.0 | 11 | 44 | 84 | 760.0 | 3.2e-12 | 3.8e-12 | 2.6.1 | 0.13.30 | 0.4.86 | |
| bones_data-bones_model | 13 | 1.3e-16 | 1.0 | 2.6 | 1.1 | 1.6 | 1.0 | 3 | 6 | 25 | -47.0 | 1.4e-16 | 1.4e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| diamonds-diamonds | 26 | 1.2e-15 | 1.0 | 1.5 | 1.6 | 5.0 | 1.0 | 2 | 4 | 43 | 4600.0 | 2.1e-15 | 1.9e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| dogs-dogs | 3 | 4.2e-16 | 1.0 | 5.1 | 1.0 | 1.6 | 3.4 | 0 | 0 | 5 | -14.0 | 1.8e-16 | 1.8e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| dogs-dogs_hierarchical | 2 | 1.1e-15 | 1.0 | 1.7 | 2.1 | 1.9 | 1.0 | 0 | 0 | 23 | 2.3e-13 | 1.2e-15 | 1.2e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| dogs-dogs_log | 2 | 5.0e-16 | 1.0 | 2.5 | 1.0 | 1.2 | 1.9 | 0 | 0 | 5 | -9.2 | 3.7e-15 | 1.8e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| dugongs_data-dugongs_model | 4 | 1.7e-16 | 1.0 | 1.7 | 1.0 | 2.6 | 1.1 | 0 | 0 | 5 | -22.0 | 1.7e-16 | 2.9e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| earnings-earn_height | 3 | 6.6e-16 | 1.0 | 9.3 | 1.0 | 16.0 | 7.4 | 0 | 0 | 25 | 0.0 | 8.2e-16 | 7.6e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| earnings-log10earn_height | 3 | 8.1e-16 | 1.0 | 9.5 | 1.0 | 12.0 | 5.7 | 0 | 0 | 25 | 0.0 | 1.0e-15 | 1.0e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| earnings-logearn_height | 3 | 8.1e-16 | 1.0 | 9.2 | 1.0 | 12.0 | 5.6 | 0 | 0 | 20 | 0.0 | 1.0e-15 | 9.9e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| earnings-logearn_height_male | 4 | 7.6e-16 | 1.0 | 2.8 | 1.0 | 20.0 | 6.7 | 0 | 0 | 29 | 0.0 | 9.7e-16 | 1.1e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| earnings-logearn_interaction | 5 | 1.0e-15 | 1.0 | 3.2 | 1.0 | 17.0 | 6.9 | 0 | 0 | 29 | -1.2e-10 | 1.0e-15 | 9.0e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| earnings-logearn_interaction_z | 5 | 7.6e-16 | 1.0 | 3.2 | 1.0 | 19.0 | 7.6 | 0 | 0 | 24 | 0.0 | 1.0e-15 | 8.1e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| earnings-logearn_logheight_male | 4 | 6.9e-16 | 1.0 | 3.0 | 1.0 | 19.0 | 6.6 | 0 | 0 | 29 | -1.8e-12 | 8.9e-16 | 9.5e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| eight_schools-eight_schools_centered | 10 | 2.1e-16 | 1.0 | 1.9 | 1.0 | 9.8 | 1.6 | 0 | 0 | 47 | -23.0 | 1.8e-16 | 1.9e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| eight_schools-eight_schools_noncentered | 10 | 0.0 | 1.0 | 2.4 | 1.0 | 9.9 | 2.0 | 0 | 0 | 44 | -23.0 | 3.5e-17 | 3.6e-17 | 2.6.1 | 0.13.30 | 0.4.86 | |
| election88-election88_full | 90 | 9.1e-17 | 1.0 | 1.9 | 1.0 | 3.7 | 1.8 | 10 | 10 | 123 | -23.0 | 1.1e-16 | 1.4e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| garch-garch11 | 4 | 3.2e-16 | 1.0 | 3.4 | 1.0 | 1.9 | 2.6 | 0 | 0 | 0 | 0.0 | 5.0e-16 | 5.3e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| gp_pois_regr-gp_pois_regr | 13 | 1.8e-16 | 1.0 | 1.5 | 1.8 | 5.5 | 1.0 | 6 | 10 | 88 | -21.0 | 2.3e-16 | 2.9e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| gp_pois_regr-gp_regr | 3 | 2.0e-16 | 1.0 | 1.9 | 1.6 | 4.4 | 1.0 | 9 | 15 | 62 | -31.0 | 2.0e-16 | 2.1e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| hmm_example-hmm_example | 4 | 3.2e-16 | 1.0 | 2.8 | 2.2 | 1.0 | 2.2 | 7 | 23 | 28 | 94.0 | 1.8e-13 | 2.0e-13 | 2.6.1 | 0.13.30 | 0.4.86 | |
| hmm_gaussian_simulated-hmm_gaussian | 14 | 6.9e-16 | 1.0 | 2.3 | 1.9 | 1.8 | 1.0 | 8 | 17 | 67 | 460.0 | 1.8e-12 | 2.1e-12 | 2.6.1 | 0.13.30 | 0.4.86 | |
| irt_2pl-irt_2pl | 144 | 3.4e-16 | 1.0 | 1.5 | 1.2 | 1.9 | 1.0 | 21 | 34 | 501 | 15.0 | 8.2e-16 | 1.4e-12 | 2.6.1 | 0.13.30 | 0.4.86 | |
| kidiq-kidscore_interaction | 5 | 4.0e-16 | 1.0 | 3.4 | 1.0 | 17.0 | 5.9 | 0 | 0 | 29 | -0.92 | 6.2e-16 | 5.8e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| kidiq-kidscore_momhs | 3 | 5.3e-16 | 1.0 | 9.0 | 1.0 | 13.0 | 6.3 | 0 | 0 | 20 | -0.92 | 5.0e-16 | 5.9e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| kidiq-kidscore_momhsiq | 4 | 3.8e-16 | 1.0 | 2.9 | 1.0 | 20.0 | 5.8 | 0 | 0 | 29 | -0.92 | 5.7e-16 | 5.3e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| kidiq-kidscore_momiq | 3 | 4.1e-16 | 1.0 | 9.2 | 1.0 | 17.0 | 5.1 | 0 | 0 | 20 | -0.92 | 5.5e-16 | 6.3e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| kidiq_with_mom_work-kidscore_interaction_c | 5 | 4.3e-16 | 1.0 | 3.4 | 1.0 | 16.0 | 6.3 | 0 | 0 | 24 | 0.0 | 4.5e-16 | 5.8e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| kidiq_with_mom_work-kidscore_interaction_c2 | 5 | 4.8e-16 | 1.0 | 3.5 | 1.0 | 16.0 | 6.4 | 0 | 0 | 24 | 0.0 | 3.8e-16 | 4.3e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| kidiq_with_mom_work-kidscore_interaction_z | 5 | 4.4e-16 | 1.0 | 3.2 | 1.0 | 16.0 | 6.1 | 0 | 0 | 24 | 0.0 | 4.4e-16 | 4.3e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| kidiq_with_mom_work-kidscore_mom_work | 5 | 4.0e-16 | 1.0 | 3.2 | 1.0 | 16.0 | 6.1 | 0 | 0 | 24 | 0.0 | 6.0e-16 | 4.6e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| kilpisjarvi_mod-kilpisjarvi | 3 | 2.0e-16 | 1.0 | 7.0 | 1.0 | 9.7 | 4.1 | 0 | 0 | 20 | -1.2 | 2.0e-16 | 2.3e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| loss_curves-losscurve_sislob | 15 | 1.8e-16 | 1.0 | 1.6 | 6.5 | 4.2 | 1.0 | 3 | 48 | 68 | 16.0 | 1.9e-16 | 1.8e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| low_dim_gauss_mix-low_dim_gauss_mix | 5 | 5.1e-16 | 1.0 | 2.2 | 1.0 | 1.6 | 1.4 | 4 | 9 | 38 | 920.0 | 3.9e-16 | 2.8e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| low_dim_gauss_mix_collapse-low_dim_gauss_mix_collapse | 5 | 5.2e-16 | 1.0 | 2.1 | 1.0 | 1.8 | 1.4 | 1 | 2 | 14 | 920.0 | 4.6e-16 | 3.0e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| lsat_data-lsat_model | 1006 | 1.5e-16 | 1.0 | 1.4 | 1.3 | 2.9 | 1.0 | 15 | 74 | 195 | -28.0 | 1.1e-15 | 1.2e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| mcycle_gp-accel_gp | 66 | 3.3e-16 | 1.0 | 2.5 | 1.0 | 3.7 | 1.2 | 12 | 24 | 716 | 180.0 | 3.8e-16 | 3.7e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| mcycle_splines-accel_splines | 82 | 6.2e-16 | 1.0 | 2.5 | 1.0 | 3.6 | 1.3 | 11 | 22 | 102 | 190.0 | 6.2e-16 | 6.3e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| mesquite-logmesquite | 8 | 1.5e-16 | 1.0 | 3.6 | 14.0 | 3.2 | 1.0 | 0 | 84 | 29 | 0.0 | 2.0e-16 | 2.1e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| mesquite-logmesquite_logva | 5 | 1.8e-16 | 1.0 | 2.9 | 1.0 | 13.0 | 4.3 | 0 | 0 | 24 | 0.0 | 2.3e-16 | 2.4e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| mesquite-logmesquite_logvas | 8 | 1.3e-16 | 1.0 | 3.5 | 14.0 | 2.6 | 1.0 | 0 | 84 | 24 | 0.0 | 2.4e-16 | 2.2e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| mesquite-logmesquite_logvash | 7 | 1.8e-16 | 1.0 | 3.6 | 14.0 | 3.5 | 1.0 | 0 | 70 | 29 | 0.0 | 2.3e-16 | 2.3e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| mesquite-logmesquite_logvolume | 3 | 1.6e-16 | 1.0 | 7.3 | 1.0 | 13.0 | 3.7 | 0 | 0 | 25 | 0.0 | 2.2e-16 | 2.1e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| mesquite-mesquite | 8 | 1.5e-16 | 1.0 | 3.5 | 11.0 | 3.3 | 1.0 | 0 | 70 | 29 | 0.0 | 1.8e-16 | 1.7e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| nes1972-nes | 10 | 6.4e-16 | 1.0 | 3.8 | 6.1 | 2.4 | 1.0 | 0 | 88 | 29 | 0.0 | 1.3e-15 | 1.1e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| nes1976-nes | 10 | 6.1e-16 | 1.0 | 3.8 | 6.6 | 2.2 | 1.0 | 0 | 88 | 29 | 0.0 | 9.8e-16 | 1.2e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| nes1980-nes | 10 | 4.4e-16 | 1.0 | 3.9 | 6.2 | 2.1 | 1.0 | 0 | 88 | 29 | 0.0 | 8.6e-16 | 8.1e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| nes1984-nes | 10 | 6.0e-16 | 1.0 | 3.8 | 6.1 | 2.2 | 1.0 | 0 | 88 | 29 | 0.0 | 1.1e-15 | 1.1e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| nes1988-nes | 10 | 5.1e-16 | 1.0 | 3.9 | 6.0 | 2.1 | 1.0 | 0 | 88 | 29 | 4.5e-13 | 1.1e-15 | 1.0e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| nes1992-nes | 10 | 6.4e-16 | 1.0 | 3.9 | 6.2 | 2.1 | 1.0 | 0 | 88 | 29 | -4.5e-13 | 1.1e-15 | 1.1e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| nes1996-nes | 10 | 6.0e-16 | 1.0 | 3.8 | 6.1 | 2.1 | 1.0 | 0 | 88 | 29 | 0.0 | 1.0e-15 | 1.0e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| nes2000-nes | 10 | 3.0e-16 | 1.0 | 4.0 | 6.1 | 2.1 | 1.0 | 0 | 56 | 29 | 0.0 | 6.7e-16 | 6.7e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| nes_logit_data-nes_logit_model | 2 | 0.0 | 1.0 | 1.0 | 2.2 | 4.5 | 1.0 | 1 | 2 | 29 | 0.0 | 5.8e-15 | 6.7e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| normal_2-normal_mixture | 3 | 1.0e-15 | 1.0 | 2.7 | 1.0 | 1.9 | 1.8 | 0 | 0 | 0 | 910.0 | 1.4e-16 | 1.2e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| normal_5-normal_mixture_k | 14 | 7.9e-16 | 1.0 | 1.8 | 3.1 | 2.8 | 1.0 | 2 | 10 | 35761 | 1600.0 | 8.1e-16 | 8.1e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| ovarian-logistic_regression_rhs | 3075 | 1.5e-13 | 1.0 | 1.3 | 2.0 | 4.1 | 1.0 | 3 | 23 | 58 | -1700.0 | 8.0e-11 | 8.7e-11 | 2.6.1 | 0.13.30 | 0.4.86 | |
| pilots-pilots | 18 | 0.0 | 1.0 | 2.3 | 1.0 | 7.3 | 1.2 | 2 | 4 | 67 | 0.0 | 1.7e-16 | 1.6e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| prideprejudice_chapter-ldaK5 | 7714 | 5.5e-15 | 1.0 | 1.3 | 2.8 | 3.2 | 1.0 | 3 | 24 | 658743 | -3.5e-10 | 5.2e-16 | 5.4e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| prideprejudice_paragraph-ldaK5 | 15570 | 3.5e-15 | 1.0 | 1.4 | 2.8 | 2.7 | 1.0 | 4 | 32 | 694097 | 2.9e-11 | 3.0e-16 | 3.1e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| prostate-logistic_regression_rhs | 11935 | 4.4e-15 | 1.0 | 1.5 | 1.6 | 2.9 | 1.0 | 5 | 27 | 62 | -6800.0 | 5.6e-15 | 5.4e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_county_intercept | 388 | 4.5e-15 | 1.0 | 4.1 | 1.0 | 11.0 | 12.0 | 0 | 0 | 10 | 11000.0 | 6.6e-15 | 1.2e-13 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_hierarchical_intercept_centered | 391 | 3.9e-15 | 1.0 | 4.6 | 1.0 | 12.0 | 13.0 | 0 | 0 | 28 | 12000.0 | 8.7e-15 | 1.1e-13 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_hierarchical_intercept_noncentered | 391 | 3.2e-15 | 1.0 | 4.6 | 1.0 | 9.5 | 11.0 | 1 | 2 | 29 | 12000.0 | 5.3e-15 | 9.7e-14 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_partially_pooled_centered | 389 | 3.6e-15 | 1.0 | 3.6 | 1.0 | 12.0 | 11.0 | 0 | 0 | 18 | 12000.0 | 8.5e-15 | 1.6e-13 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_partially_pooled_noncentered | 389 | 3.5e-15 | 1.0 | 3.7 | 1.0 | 9.6 | 11.0 | 1 | 2 | 19 | 12000.0 | 6.9e-15 | 1.2e-13 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_pooled | 3 | 1.4e-14 | 1.0 | 33.0 | 1.0 | 12.0 | 20.0 | 0 | 0 | 20 | 12000.0 | 1.7e-14 | 1.9e-14 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_variable_intercept_centered | 390 | 3.6e-15 | 1.0 | 4.2 | 1.0 | 11.0 | 12.0 | 0 | 0 | 13 | 12000.0 | 7.9e-15 | 1.1e-13 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_variable_intercept_noncentered | 390 | 4.0e-15 | 1.0 | 4.4 | 1.0 | 8.1 | 11.0 | 1 | 2 | 14 | 12000.0 | 6.3e-15 | 9.7e-14 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_variable_intercept_slope_centered | 777 | 3.7e-15 | 1.0 | 4.0 | 1.0 | 11.0 | 11.0 | 0 | 0 | 26 | 12000.0 | 6.1e-15 | 9.8e-14 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_variable_intercept_slope_noncentered | 777 | 5.1e-15 | 1.0 | 4.2 | 1.0 | 6.8 | 8.6 | 2 | 4 | 28 | 12000.0 | 5.5e-15 | 8.2e-14 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_variable_slope_centered | 390 | 1.4e-14 | 1.0 | 4.3 | 1.0 | 11.0 | 11.0 | 0 | 0 | 13 | 12000.0 | 1.1e-14 | 1.0e-13 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_all-radon_variable_slope_noncentered | 390 | 1.6e-14 | 1.0 | 4.5 | 1.0 | 7.7 | 10.0 | 1 | 2 | 14 | 12000.0 | 8.1e-15 | 7.8e-14 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_county_intercept | 87 | 6.9e-16 | 1.0 | 4.0 | 1.0 | 9.8 | 10.0 | 0 | 0 | 10 | 650.0 | 1.3e-15 | 1.0e-14 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_hierarchical_intercept_centered | 90 | 7.3e-16 | 1.0 | 4.6 | 1.0 | 11.0 | 11.0 | 0 | 0 | 28 | 840.0 | 8.3e-16 | 8.7e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_hierarchical_intercept_noncentered | 90 | 6.4e-16 | 1.0 | 4.7 | 1.0 | 9.1 | 9.6 | 1 | 2 | 29 | 840.0 | 9.2e-16 | 9.0e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_partially_pooled_centered | 88 | 8.4e-16 | 1.0 | 3.5 | 1.0 | 12.0 | 9.7 | 0 | 0 | 18 | 840.0 | 1.0e-15 | 7.1e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_partially_pooled_noncentered | 88 | 8.3e-16 | 1.0 | 3.6 | 1.0 | 8.8 | 8.7 | 1 | 2 | 19 | 840.0 | 7.8e-16 | 8.4e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_pooled | 3 | 1.2e-15 | 1.0 | 28.0 | 1.0 | 11.0 | 16.0 | 0 | 0 | 20 | 840.0 | 1.5e-15 | 1.3e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_variable_intercept_centered | 89 | 7.1e-16 | 1.0 | 4.1 | 1.0 | 10.0 | 11.0 | 0 | 0 | 13 | 840.0 | 7.2e-16 | 8.3e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_variable_intercept_noncentered | 89 | 8.0e-16 | 1.0 | 4.4 | 1.0 | 7.4 | 9.7 | 1 | 2 | 14 | 840.0 | 1.0e-15 | 8.2e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_variable_intercept_slope_centered | 175 | 8.3e-16 | 1.0 | 3.9 | 1.0 | 11.0 | 9.3 | 0 | 0 | 26 | 840.0 | 7.4e-16 | 8.3e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_variable_intercept_slope_noncentered | 175 | 8.6e-16 | 1.0 | 4.3 | 1.0 | 6.5 | 7.5 | 2 | 4 | 28 | 840.0 | 8.1e-16 | 8.0e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_variable_slope_centered | 89 | 1.1e-15 | 1.0 | 4.2 | 1.0 | 10.0 | 10.0 | 0 | 0 | 13 | 840.0 | 9.7e-16 | 6.5e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mn-radon_variable_slope_noncentered | 89 | 1.1e-15 | 1.0 | 4.5 | 1.0 | 7.0 | 9.2 | 1 | 2 | 14 | 840.0 | 9.0e-16 | 7.0e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| radon_mod-radon_county | 389 | 0.0 | 1.0 | 6.5 | 1.0 | 7.4 | 2.3 | 0 | 0 | 33 | 0.0 | 6.8e-16 | 7.8e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| rats_data-rats_model | 65 | 9.7e-17 | 1.0 | 3.4 | 1.0 | 9.8 | 6.8 | 0 | 0 | 31 | -9.2 | 4.5e-16 | 2.7e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| sblrc-blr | 6 | 2.6e-16 | 1.0 | 4.1 | 1.0 | 7.7 | 2.0 | 1 | 2 | 40 | 97.0 | 2.3e-16 | 2.2e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| sblri-blr | 6 | 2.3e-16 | 1.0 | 4.4 | 1.0 | 7.8 | 2.0 | 1 | 2 | 40 | 97.0 | 2.5e-16 | 2.0e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| science_irt-grsm_latent_reg_irt | 408 | 1.1e-16 | 1.0 | 1.7 | 1.5 | 2.6 | 1.0 | 10980 | 30202 | 49500 | 15.0 | 1.5e-16 | 1.4e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| seeds_data-seeds_centered_model | 26 | 1.5e-16 | 1.0 | 1.6 | 1.0 | 3.9 | 1.0 | 1 | 2 | 62 | 0.0 | 2.1e-16 | 2.3e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| seeds_data-seeds_model | 26 | 1.4e-16 | 1.0 | 1.3 | 1.1 | 4.2 | 1.0 | 0 | 0 | 58 | -35.0 | 2.3e-16 | 2.2e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| seeds_data-seeds_stanified_model | 26 | 1.4e-16 | 1.0 | 1.4 | 1.1 | 4.1 | 1.0 | 0 | 0 | 58 | 0.0 | 2.5e-16 | 2.2e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| sesame_data-sesame_one_pred_a | 3 | 7.4e-16 | 1.0 | 9.4 | 1.0 | 12.0 | 5.9 | 0 | 0 | 20 | 0.0 | 1.7e-15 | 1.3e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| surgical_data-surgical_model | 14 | 1.5e-16 | 1.0 | 2.1 | 1.0 | 2.7 | 1.2 | 0 | 0 | 53 | -14.0 | 1.2e-16 | 1.4e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| three_docs1200-ldaK2 | 7 | 4.4e-16 | 1.0 | 1.5 | 4.8 | 4.8 | 1.0 | 2 | 34 | 24103 | 0.0 | 2.9e-16 | 3.0e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| three_men1-ldaK2 | 502 | 1.7e-15 | 1.0 | 1.4 | 3.0 | 4.4 | 1.0 | 2 | 34 | 100137 | 2.2e-11 | 5.6e-16 | 5.8e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| three_men2-ldaK2 | 510 | 1.6e-15 | 1.0 | 1.4 | 3.0 | 4.4 | 1.0 | 2 | 34 | 100281 | -7.3e-12 | 4.1e-16 | 4.0e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| three_men3-ldaK2 | 505 | 1.7e-15 | 1.0 | 1.4 | 3.1 | 4.5 | 1.0 | 2 | 34 | 100191 | -7.3e-12 | 5.0e-16 | 4.6e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| traffic_accident_nyc-bym2_offset_only | 3845 | 1.1e-15 | 1.0 | 2.6 | 1.0 | 5.6 | 1.3 | 6 | 8 | 79 | -0.65 | 1.4e-15 | 1.4e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| wells_data-wells_daae_c_model | 6 | 0.0 | 1.0 | 1.0 | 2.0 | 2.7 | 1.0 | 2 | 4 | 31 | 0.0 | 9.1e-16 | 9.3e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| wells_data-wells_dae_c_model | 5 | 1.5e-16 | 1.0 | 1.0 | 2.1 | 2.6 | 1.0 | 2 | 4 | 31 | 0.0 | 9.3e-16 | 9.9e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| wells_data-wells_dae_inter_model | 7 | 1.5e-16 | 1.0 | 1.0 | 2.0 | 2.6 | 1.0 | 2 | 4 | 31 | 0.0 | 9.8e-16 | 1.1e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| wells_data-wells_dae_model | 4 | 0.0 | 1.0 | 1.0 | 2.0 | 2.4 | 1.0 | 2 | 4 | 31 | 0.0 | 7.2e-16 | 8.5e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| wells_data-wells_dist | 2 | 2.6e-13 | 1.0 | 1.7 | 1.2 | 1.5 | 1.0 | 0 | 0 | 18 | -4.6e-8 | 4.2e-11 | 3.6e-11 | 2.6.1 | 0.13.30 | 0.4.86 | |
| wells_data-wells_dist100_model | 2 | 0.0 | 1.0 | 1.0 | 2.0 | 2.5 | 1.0 | 2 | 4 | 31 | 0.0 | 9.9e-16 | 8.8e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| wells_data-wells_dist100ars_model | 3 | 0.0 | 1.0 | 1.0 | 2.0 | 2.5 | 1.0 | 2 | 4 | 31 | 0.0 | 8.9e-16 | 7.8e-16 | 2.6.1 | 0.13.30 | 0.4.86 | |
| wells_data-wells_interaction_c_model | 4 | 1.5e-16 | 1.0 | 1.0 | 2.0 | 2.6 | 1.0 | 2 | 4 | 31 | 0.0 | 1.1e-15 | 1.0e-15 | 2.6.1 | 0.13.30 | 0.4.86 | |
| wells_data-wells_interaction_model | 4 | 0.0 | 1.0 | 1.0 | 2.0 | 2.5 | 1.0 | 2 | 4 | 31 | 0.0 | 8.6e-16 | 8.7e-16 | 2.6.1 | 0.13.30 | 0.4.86 |