Shared Storage (Ceph)

While Docker Swarm is great for keeping containers running (and restarting those that fail), it does nothing for persistent storage. This means if you actually want your containers to keep any data persistent across restarts (hint: you do!), you need to provide shared storage to every docker node.


This is a companion discussion topic for the original entry at https://geek-cookbook.funkypenguin.co.nz/docker-swarm/shared-storage-ceph/

Thank you for the great guide!

Is this configuration expandable? Is it possible to add nodes and change redundancy configuration on the live cluster?

And what was the problem with GlusterFS? There’s an anchor to explanation, but it doesn’t exist. Perhaps, something has changed since 2019?

Sort of. Ceph is happy to extend storage space by adding OSD, but changing the redundancy configuration is usually not as easy. You could (for example) change your failure domain from host to rack, for example, but once you’ve created a pool of 3 replicas, you’re stuck with it. Any other configuration will require a new pool, and a long and painful (ask me how I know!) migration…

1 Like

Kia ora! Do you have any experience with utilising Docker Volume plugins/drivers that connect to Ceph RBDs rather than using a CephFS mounts that are then bind mounted to the containers? Potentially its unnecessary faff, but I wondered whether it would provide better performance and reliability, particularly with database applications (particularly those that run on sqlite).

I’m currently running GlusterFS and it is awful (so much so that I don’t run most database workloads off the shared storage), so just wanted to check whether also moving away from bind mounts (as I used today with Gluster) would also be helpful.