The Great Cross-Cloud Ruby Benchmark

Comparing iteration times across 12 machine types on several public clouds.

This site was borne of my continuous frustration at Speedshop of clients running on old generation AWS instances, when I knew "free speed" was on the table if they just upgraded to a newer generation. However, I could never quantify the speed boost. This page simply runs ruby-bench, a series of mostly CPU-bound benchmarks maintained by Ruby core and the YJIT developers. We run across multiple instances to reduce the effect of noisy neighbors.

For Rails apps, the most relevant benchmark is likely shipit. Keep in mind that this benchmark likely still overstates the performance difference because it runs the database in-memory, which makes the benchmark more CPU-bound than a real-world app with a real database.

All benchmarks were run on 00022483 with YJIT enabled on Ruby 4.0.1. View the code for this page here.

aws: c8a c7a c8g c8i c7g c7i c6g c6i | azure: fals-v6 dpls-v6 dals-v6 | fargate: fargate select all | deselect all
0%
300%+ (% slower than fastest)
Benchmark aws-c8a azure-fals-v6 aws-c7a aws-c8g azure-dpls-v6 aws-c8i azure-dals-v6 aws-c7g aws-c7i fargate-fargate aws-c6g aws-c6i
ms ±% vs best ms ±% vs best ms ±% vs best ms ±% vs best ms ±% vs best ms ±% vs best ms ±% vs best ms ±% vs best ms ±% vs best ms ±% vs best ms ±% vs best ms ±% vs best
all 7636 10773 +41.1% 11333 +48.4% 12804 +67.7% 15551 +103.7% 16264 +113.0% 17922 +134.7% 18073 +136.7% 21252 +178.3% 21548 +182.2% 21807 +185.6% 23223 +204.1%
activerecord 157 ±0.5 235 ±0.1 +49.7% 240 ±0.5 +52.9% 243 ±0.2 +54.8% 267 ±0.3 +70.1% 328 ±0.9 +108.9% 382 ±0.3 +143.3% 318 ±0.6 +102.5% 430 ±2.5 +173.9% 427 ±2.9 +172.0% 457 ±0.6 +191.1% 535 ±1.1 +240.8%
chunky-png 453 ±0.7 693 ±0.4 +53.0% 697 ±1.0 +53.9% 737 ±0.3 +62.7% 849 ±0.2 +87.4% 896 ±0.4 +97.8% 1162 ±0.1 +156.5% 1068 ±0.5 +135.8% 1130 ±1.4 +149.4% 1181 ±7.2 +160.7% 1415 ±0.5 +212.4% 1669 ±0.2 +268.4%
erubi-rails 761 ±0.6 1027 ±0.5 +35.0% 1058 ±1.4 +39.0% 983 ±0.2 +29.2% 1831 ±0.5 +140.6% 1613 ±0.8 +112.0% 2028 ±0.5 +166.5% 1852 ±1.2 +143.4% 2014 ±1.4 +164.7% 2131 ±2.7 +180.0% 2259 ±0.4 +196.8% 2560 ±0.5 +236.4%
hexapdf 1478 ±0.4 2008 ±0.4 +35.9% 2037 ±0.6 +37.8% 2052 ±0.6 +38.8% 2337 ±0.4 +58.1% 3022 ±0.3 +104.5% 3221 ±0.6 +117.9% 2859 ±1.7 +93.4% 3942 ±2.1 +166.7% 3867 ±5.0 +161.6% 3755 ±0.5 +154.1% 4536 ±0.5 +206.9%
liquid-c 34 ±0.2 47 ±0.4 +38.2% 48 ±0.5 +41.2% 52 ±0.5 +52.9% 65 ±0.4 +91.2% 75 ±0.5 +120.6% 92 ±0.5 +170.6% 76 ±0.8 +123.5% 102 ±1.3 +200.0% 100 ±0.9 +194.1% 106 ±0.7 +211.8% 120 ±0.5 +252.9%
liquid-compile 34 ±0.4 41 ±0.3 +20.6% 43 ±0.4 +26.5% 43 ±0.3 +26.5% 51 ±0.4 +50.0% 65 ±0.3 +91.2% 71 ±0.3 +108.8% 60 ±0.5 +76.5% 89 ±0.8 +161.8% 87 ±1.8 +155.9% 86 ±0.4 +152.9% 104 ±0.3 +205.9%
liquid-render 89 ±0.2 122 ±0.3 +37.1% 128 ±0.5 +43.8% 135 ±0.3 +51.7% 162 ±0.5 +82.0% 185 ±0.4 +107.9% 216 ±1.0 +142.7% 189 ±1.5 +112.4% 246 ±1.7 +176.4% 238 ±1.6 +167.4% 261 ±0.7 +193.3% 290 ±0.4 +225.8%
lobsters 632 ±0.7 897 ±0.6 +41.9% 967 ±1.8 +53.0% 1214 ±1.5 +92.1% 1386 ±1.1 +119.3% 1346 ±0.5 +113.0% 1442 ±0.8 +128.2% 1588 ±5.5 +151.3% 1834 ±4.6 +190.2% 1963 ±6.4 +210.6% 1815 ±0.9 +187.2% 1676 ±0.7 +165.2%
mail 78 ±0.2 113 ±0.3 +44.9% 114 ±0.4 +46.2% 125 ±0.8 +60.3% 143 ±0.4 +83.3% 155 ±0.5 +98.7% 184 ±0.3 +135.9% 170 ±1.4 +117.9% 208 ±1.7 +166.7% 224 ±2.3 +187.2% 223 ±0.3 +185.9% 234 ±0.2 +200.0%
psych-load 1191 ±0.5 1760 ±0.8 +47.8% 1863 ±1.6 +56.4% 1971 ±0.4 +65.5% 2067 ±0.8 +73.6% 2336 ±0.5 +96.1% 3111 ±0.6 +161.2% 2554 ±1.4 +114.4% 3358 ±2.2 +181.9% 3133 ±5.3 +163.1% 3593 ±0.6 +201.7% 3939 ±0.3 +230.7%
railsbench 1316 ±0.5 1906 ±0.3 +44.8% 2058 ±0.7 +56.4% 2762 ±5.6 +109.9% 3159 ±0.5 +140.0% 3171 ±0.3 +141.0% 2956 ±0.3 +124.6% 3787 ±7.5 +187.8% 3954 ±2.1 +200.5% 4194 ±6.0 +218.7% 3858 ±0.5 +193.2% 3717 ±0.3 +182.4%
rubocop 111 ±0.6 161 ±0.1 +45.0% 173 ±1.2 +55.9% 187 ±0.3 +68.5% 226 ±0.3 +103.6% 236 ±0.8 +112.6% 256 ±0.5 +130.6% 249 ±2.2 +124.3% 297 ±2.6 +167.6% 294 ±2.3 +164.9% 320 ±0.7 +188.3% 319 ±0.6 +187.4%
ruby-lsp 98 ±0.3 132 ±0.1 +34.7% 138 ±0.6 +40.8% 143 ±0.2 +45.9% 182 ±0.2 +85.7% 201 ±0.4 +105.1% 223 ±0.3 +127.6% 203 ±0.6 +107.1% 257 ±0.9 +162.2% 253 ±3.2 +158.2% 272 ±0.2 +177.6% 308 ±0.3 +214.3%
sequel 29 ±0.1 44 ±0.1 +51.7% 45 ±0.1 +55.2% 50 ±0.2 +72.4% 60 ±0.2 +106.9% 74 ±0.7 +155.2% 106 ±1.0 +265.5% 73 ±0.7 +151.7% 98 ±0.9 +237.9% 106 ±0.7 +265.5% 108 ±0.7 +272.4% 126 ±0.4 +334.5%
shipit 1175 ±1.0 1587 ±0.7 +35.1% 1724 ±0.9 +46.7% 2107 ±1.9 +79.3% 2766 ±1.0 +135.4% 2561 ±0.8 +118.0% 2472 ±1.2 +110.4% 3027 ±5.2 +157.6% 3293 ±3.7 +180.3% 3350 ±3.3 +185.1% 3279 ±0.9 +179.1% 3090 ±0.6 +163.0%
Instance all activerecord chunky-png erubi-rails hexapdf liquid-c liquid-compile liquid-render lobsters mail psych-load railsbench rubocop ruby-lsp sequel shipit
aws-c8a 7636 157 453 761 1478 34 34 89 632 78 1191 1316 111 98 29 1175
azure-fals-v6 10773 +41.1% 235 +49.7% 693 +53.0% 1027 +35.0% 2008 +35.9% 47 +38.2% 41 +20.6% 122 +37.1% 897 +41.9% 113 +44.9% 1760 +47.8% 1906 +44.8% 161 +45.0% 132 +34.7% 44 +51.7% 1587 +35.1%
aws-c7a 11333 +48.4% 240 +52.9% 697 +53.9% 1058 +39.0% 2037 +37.8% 48 +41.2% 43 +26.5% 128 +43.8% 967 +53.0% 114 +46.2% 1863 +56.4% 2058 +56.4% 173 +55.9% 138 +40.8% 45 +55.2% 1724 +46.7%
aws-c8g 12804 +67.7% 243 +54.8% 737 +62.7% 983 +29.2% 2052 +38.8% 52 +52.9% 43 +26.5% 135 +51.7% 1214 +92.1% 125 +60.3% 1971 +65.5% 2762 +109.9% 187 +68.5% 143 +45.9% 50 +72.4% 2107 +79.3%
azure-dpls-v6 15551 +103.7% 267 +70.1% 849 +87.4% 1831 +140.6% 2337 +58.1% 65 +91.2% 51 +50.0% 162 +82.0% 1386 +119.3% 143 +83.3% 2067 +73.6% 3159 +140.0% 226 +103.6% 182 +85.7% 60 +106.9% 2766 +135.4%
aws-c8i 16264 +113.0% 328 +108.9% 896 +97.8% 1613 +112.0% 3022 +104.5% 75 +120.6% 65 +91.2% 185 +107.9% 1346 +113.0% 155 +98.7% 2336 +96.1% 3171 +141.0% 236 +112.6% 201 +105.1% 74 +155.2% 2561 +118.0%
azure-dals-v6 17922 +134.7% 382 +143.3% 1162 +156.5% 2028 +166.5% 3221 +117.9% 92 +170.6% 71 +108.8% 216 +142.7% 1442 +128.2% 184 +135.9% 3111 +161.2% 2956 +124.6% 256 +130.6% 223 +127.6% 106 +265.5% 2472 +110.4%
aws-c7g 18073 +136.7% 318 +102.5% 1068 +135.8% 1852 +143.4% 2859 +93.4% 76 +123.5% 60 +76.5% 189 +112.4% 1588 +151.3% 170 +117.9% 2554 +114.4% 3787 +187.8% 249 +124.3% 203 +107.1% 73 +151.7% 3027 +157.6%
aws-c7i 21252 +178.3% 430 +173.9% 1130 +149.4% 2014 +164.7% 3942 +166.7% 102 +200.0% 89 +161.8% 246 +176.4% 1834 +190.2% 208 +166.7% 3358 +181.9% 3954 +200.5% 297 +167.6% 257 +162.2% 98 +237.9% 3293 +180.3%
fargate-fargate 21548 +182.2% 427 +172.0% 1181 +160.7% 2131 +180.0% 3867 +161.6% 100 +194.1% 87 +155.9% 238 +167.4% 1963 +210.6% 224 +187.2% 3133 +163.1% 4194 +218.7% 294 +164.9% 253 +158.2% 106 +265.5% 3350 +185.1%
aws-c6g 21807 +185.6% 457 +191.1% 1415 +212.4% 2259 +196.8% 3755 +154.1% 106 +211.8% 86 +152.9% 261 +193.3% 1815 +187.2% 223 +185.9% 3593 +201.7% 3858 +193.2% 320 +188.3% 272 +177.6% 108 +272.4% 3279 +179.1%
aws-c6i 23223 +204.1% 535 +240.8% 1669 +268.4% 2560 +236.4% 4536 +206.9% 120 +252.9% 104 +205.9% 290 +225.8% 1676 +165.2% 234 +200.0% 3939 +230.7% 3717 +182.4% 319 +187.4% 308 +214.3% 126 +334.5% 3090 +163.0%