Hardware for AutoPirate

Comments Blog
1

Hey cooks;

I’m interested in running AutoPirate on a dedicated machine, but would love some feedback on what kind of hardware is needed for good performance (I’m wary of the number of containers!)

2

Hey BC :slight_smile:

I’m running my stack currently on 3 x Atomic hosts, each with 2GB RAM and 2VCPUs. The underlying hardware is Dell R720 hosts (~5 years old):

Current stacks running:

[root@ds1 ~]# docker stack ls
NAME            SERVICES
autopirate      16
portainer       2
huginn          3
traefik         2
nextcloud       4
duplicity       1
docker-cleanup  1
miniflux        1
kanboard        2
registry        1
[root@ds1 ~]#

Current loading:

[davidy@ds1 ~]$ uptime
 12:15:12 up 2 days,  2:17,  1 user,  load average: 0.93, 1.02, 1.02
[davidy@ds1 ~]$

[davidy@ds2 ~]$ uptime
 12:14:58 up 1 day, 22:59,  1 user,  load average: 1.08, 1.04, 1.27
[davidy@ds2 ~]$

[davidy@ds3 ~]$ uptime
 12:14:57 up 1 day, 22:54,  1 user,  load average: 0.01, 0.03, 0.05
[davidy@ds3 ~]$

I do note the occasional docker “context deadline” fault when deploying a new, heavy stack (i.e., AutoPirate), which I imagine is due to the amount of load placed on the dockerd daemons during deployment, but after 1-2 retries, the stacks deploy.

However, if you’re only speccing a single machine, I’d suggest that you’d be fine with 8GB RAM and 4CPUs at a minimum. Obviously the more the better though :wink:

D

3

Just to illustrate the above, I’m still not sure whether this behaviour is due to the (relatively) old version of Docker 1.13 in Atomic, or my underpowered hardware, but running the deploy a second time usually fixes it:

[root@ds1 portainer]# docker stack deploy portainer -c portainer.yml
Creating network portainer_internal
Creating service portainer_app
Creating service portainer_proxy
Error response from daemon: rpc error: code = 4 desc = context deadline exceeded

[root@ds1 portainer]# docker stack deploy portainer -c portainer.yml
Creating service portainer_proxy
Updating service portainer_app (id: tthx1b042rcbbeg7xvr035x6k)
[root@ds1 portainer]#
4

For what my 2c is worth; I read on the 'net this can occur when communication with the registry times out when creating a service. I see you’re also running a registry on your hosts; the load during service creation may be too much to for the registry to respond to image requests timely. Maybe try seeing what happens if you shift the registry to another machine.(Just for the sake of technical curiosity! :slightly_smiling_face: )

5

I think you might be right - I had noticed that when my registry stack is not running, the error doesn’t occur. (But then, it can take longer to stand up a swarm since my nodes have to pull their images off the 'net, over a puny 10Mbps connection)

Out of interest @ggilley, this is why I added the registry stack - I got tired of waiting 20 min for refreshed GitLab images to be downloaded to each node every time I restarted the stack! :wink:

D

6

Yep, I think my particular “context deadlink exceeded” messages are related to the registry.

With the registry stack running:

<snip>
Creating service autopirate_plexpy
Creating service autopirate_nzbhydra
Creating service autopirate_ombi
Error response from daemon: rpc error: code = 4 desc = context deadline exceeded
[root@ds3 autopirate]#

Without the registry stack running:

[root@ds3 autopirate]# docker stack deploy autopirate -c /var/data/config/autopirate/autopirate.yml
Updating service autopirate_sonarr (id: c13uw5fztocn559nr3wq85ylw)
Creating service autopirate_nzbhydra_proxy
Updating service autopirate_headphones_proxy (id: ebgq7ployppy258heb43r9jfv)
Updating service autopirate_lazylibrarian (id: szadf0vi24iopl79l2k98zhh9)
Updating service autopirate_nzbhydra (id: rbus4kowq8mypyum4hfodi8hi)
Updating service autopirate_plexpy (id: vsrkmylgahd6mi3lwflrlyxnf)
Creating service autopirate_mylar_proxy
Creating service autopirate_ombi
Updating service autopirate_plexpy_proxy (id: mff38q8sxt8etsvhh25l716ei)
Creating service autopirate_mylar
Creating service autopirate_radarr_proxy
Updating service autopirate_radarr (id: v0wjq2fjyc41jbbhsz3kbnb3n)
Updating service autopirate_lazylibrarian_proxy (id: q9pzncvrzubgwuzsy08up5x6m)
Updating service autopirate_ombi_proxy (id: dj6wyi8t00zr4nfm11cs7h6qe)
Creating service autopirate_headphones
Updating service autopirate_sonarr_proxy (id: m9ykglf397jxroe73t8byrm4e)
[root@ds3 autopirate]#

Maybe I’ll leave registry turned off for now - it’s of negligible value if (a) we’re using small, alpine-based containers, and (b) the stack is relatively stable.

D

7

As an update, for a christmas gift to myself I picked up a 2nd hand Xeon based machine with 16GB. Let’s see how AutoPirate handles :slight_smile: