Scaleway: Bare metal SSD ARM servers
In this technical retrospective (2015), @wut42 shares practical architectural lessons learned from running, scaling, or migrating production software under lean constraints.
Core Principles for Independent Builders:
Defaulting to simpler architectures (such as monolithic frameworks, SQLite, or managed VPS) prevents early engineering bankruptcy.
Evaluating true compute and egress costs reveals that high cloud bills can often be slashed by 80%+ through judicious re-platforming.
Code you write at 2 AM as a solo engineer must be easy to debug six months later without complex microservice dependencies.
Community Debate Themes:
HN engineers debated the tradeoffs between cloud convenience and raw server economics, database indexing strategies, and resisting premature architectural complexity.
CATEGORY Frugal Engineering →
HISTORICAL ERA Cloud & Mobile Era (2014–2019)
COMMUNITY SCORE ★ 326 points
TOTAL DISCUSSIONS 166 comments
The CPU performance is a joke though.
c1 from instantcloud.io:
``` ubuntu@c1-10-1-33-39:~$ openssl speed rsa Doing 512 bit private rsa's for 10s: 11119 512 bit private RSA's in 10.00s Doing 512 bit public rsa's for 10s: 111478 512 bit public RSA's in 10.00s Doing 1024 bit private rsa's for 10s: 1950 1024 bit private RSA's in 10.00s Doing 1024 bit public rsa's for 10s: 34638 1024 bit public RSA's in 10.00s Doing 2048 bit private rsa's for 10s: 291 2048 bit private RSA's in 10.01s Doing 2048 bit public rsa's for 10s: 9393 2048 bit public RSA's in 10.00s Doing 4096 bit private rsa's for 10s: 40 4096 bit private RSA's in 10.16s Doing 4096 bit public rsa's for 10s: 2402 4096 bit public RSA's in 10.01s OpenSSL 1.0.1f 6 Jan 2014 built on: Thu Oct 16 16:07:24 UTC 2014 options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) blowfish(ptr) compiler: cc -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DGHASH_ASM sign verify sign/s verify/s rsa 512 bits 0.000899s 0.000090s 1111.9 11147.8 rsa 1024 bits 0.005128s 0.000289s 195.0 3463.8 rsa 2048 bits 0.034399s 0.001065s 29.1 939.3 rsa 4096 bits 0.254000s 0.004167s 3.9 240.0
``` local odroid u3:
``` thresh@odroid:~$ openssl speed rsa Doing 512 bit private rsa's for 10s: 19334 512 bit private RSA's in 9.99s Doing 512 bit public rsa's for 10s: 204957 512 bit public RSA's in 9.97s Doing 1024 bit private rsa's for 10s: 3490 1024 bit private RSA's in 10.00s Doing 1024 bit public rsa's for 10s: 62453 1024 bit public RSA's in 10.00s Doing 2048 bit private rsa's for 10s: 516 2048 bit private RSA's in 10.00s Doing 2048 bit public rsa's for 10s: 17051 2048 bit public RSA's in 10.00s Doing 4096 bit private rsa's for 10s: 72 4096 bit private RSA's in 10.09s Doing 4096 bit public rsa's for 10s: 4434 4096 bit public RSA's in 10.00s OpenSSL 1.0.1f 6 Jan 2014 built on: Fri Jun 20 19:00:28 UTC 2014 options:bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) aes(partial) blowfish(ptr) compiler: cc -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wa,--noexecstack -Wall -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DGHASH_ASM sign verify sign/s verify/s rsa 512 bits 0.000517s 0.000049s 1935.3 20557.4 rsa 1024 bits 0.002865s 0.000160s 349.0 6245.3 rsa 2048 bits 0.019380s 0.000586s 51.6 1705.1 rsa 4096 bits 0.140139s 0.002255s 7.1 443.4
``` digitalocean cheapest vps:
``` thresh@thre ~ $ openssl speed rsa Doing 512 bit private rsa's for 10s: 119305 512 bit private RSA's in 9.81s Doing 512 bit public rsa's for 10s: 1450435 512 bit public RSA's in 9.85s Doing 1024 bit private rsa's for 10s: 35347 1024 bit private RSA's in 9.85s Doing 1024 bit public rsa's for 10s: 541458 1024 bit public RSA's in 9.86s Doing 2048 bit private rsa's for 10s: 5259 2048 bit private RSA's in 9.84s Doing 2048 bit public rsa's for 10s: 161094 2048 bit public RSA's in 9.88s Doing 4096 bit private rsa's for 10s: 627 4096 bit private RSA's in 9.81s Doing 4096 bit public rsa's for 10s: 41353 4096 bit public RSA's in 9.75s OpenSSL 1.0.1e 11 Feb 2013 built on: Thu Mar 19 18:31:36 UTC 2015 options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial) blowfish(idx) compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM sign verify sign/s verify/s rsa 512 bits 0.000082s 0.000007s 12161.6 147252.3 rsa 1024 bits 0.000279s 0.000018s 3588.5 54914.6 rsa 2048 bits 0.001871s 0.000061s 534.5 16305.1 rsa 4096 bits 0.015646s 0.000236s 63.9 4241.3
``` amazon t1.micro:
``` [ec2-user@ip-10-71-147-165 ~]$ openssl speed rsa Doing 512 bit private rsa's for 10s: 85447 512 bit private RSA's in 9.48s Doing 512 bit public rsa's for 10s: 123882 512 bit public RSA's in 9.96s Doing 1024 bit private rsa's for 10s: 7720 1024 bit private RSA's in 9.98s Doing 1024 bit public rsa's for 10s: 187876 1024 bit public RSA's in 10.04s Doing 2048 bit private rsa's for 10s: 1268 2048 bit private RSA's in 9.99s Doing 2048 bit public rsa's for 10s: 67337 2048 bit public RSA's in 10.07s Doing 4096 bit private rsa's for 10s: 36 4096 bit private RSA's in 7.89s Doing 4096 bit public rsa's for 10s: 25180 4096 bit public RSA's in 9.85s OpenSSL 1.0.1g-fips 7 Apr 2014 built on: Wed Apr 9 21:51:31 UTC 2014 options:bn(64,64) md2(int) rc4(16x,int) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) compiler: gcc -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DL_ENDIAN -DTERMIO -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wa,--noexecstack -DPURIFY -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM sign verify sign/s verify/s rsa 512 bits 0.000111s 0.000080s 9013.4 12438.0 rsa 1024 bits 0.001293s 0.000053s 773.5 18712.7 rsa 2048 bits 0.007879s 0.000150s 126.9 6686.9 rsa 4096 bits 0.219167s 0.000391s 4.6 2556.3 ```
I tried their free trial when it was still branded as Online.net Labs. What follows is my experience, which is good as anecdote only.
I use MariaDB-Galera in a project of mine to be able to run it in a parallel fashion. To determine if the servers were suitable for my use, I tried to install MariaDB-Galera, only to find out:
- MariaDB doesn't officially support ARM yet;
- The Debian and Ubuntu repos don't have the MariaDB-Galera edition available;
- All repos I could find with the Galera edition didn't have it compiled for armhf, including the official repo, by virtue of the first point.
- Compiling it on the server resulted in other problems: besides the usual compile errors that can be more or less sorted out by hand, and the dependency hell of the resulting deb packages (remember, no armhf packages available) I figured it would be unacceptable (in terms of ease-of-maintenance) to have to recompile MariaDB every time a new version with security fixes came out. This is something I can live with when toying with Raspberry Pis and similar things, but not on production stuff.
I eventually gave up, declaring the servers unsuitable for my use. End of anecdote.
IMHO they are charging too much for the product, at a time when the ARM ecosystem is not yet ready for server use (the architecture and OS support are fine, the user-land software, not so much). The servers are bare metal (it's awesome being able to use as much CPU as you want without breaching AUPs or hitting artificial caps), but the networked storage, while definitely better than HDDs, may not fulfill the needs of those with higher I/O requirements (I admit I haven't tested, also by virtue of Galera not being available). Lastly, the prices first show without VAT, which means that if you're in the EU and without a VAT ID, it'll turn out to be even more expensive. I'd consider something like €7/month (perhaps with less storage) to be a more acceptable price.
It seems they're using xnbd to provide the storage for the box. On Debian, the box is running a xnbd-client, which provides the rootfs:
``` xnbd-client --blocksize 4096 --retry=900 10.1.32.197 4096 /dev/nbd0
root@scw-6d2eff:~# df -h Filesystem Size Used Avail Use% Mounted on /dev/nbd0 46G 507M 44G 2% /
``` I did not check, but I hope there's some filtering/authentication for that process.
Like EC2 and others, the box has an internal ip allocated and an external ip via nat (or something else).
I was able to download a file from OVH with ~500 Mbit/s throughput when writing to the filesystem. Writing to /dev/null it increases to ~650 Mbit/s.
Running an untuned nginx, it can do ~300 req/s with ~20% cpu util.
Seeing as the storage is still virtual, I'm not really sure why this would be so much better than a virtual machine. I like the idea though.
I like the basic, easy design. One suggestion though (take this with a grain of salt though, since I am a backend programmer and not a design specialist). I didn't see the 20 euro preauthorization message the first time the new card dialog came up, and instead I closed it and went to the billing faq to find out what that was about when I saw Visa asking me to approve a 20 euro charge. I saw the message the second time around, but perhaps making that message a little more prominent would be helpful, especially for simpletons like me to tend to miss things.
I have used online.net labs bare metal arm cloud servers and scaleway seems to offer the exact same. Can anyone explain the difference or is this just a re-releasing?
fwiw, i am from the east coast us, and have used both online.net labs and ovh dedicated cloud from ks and sys, and I have found that online.net has a lower latency and higher throughput to me.
also fwiw, i have not had issues with finding arm packages for developing in node, golang, or mono