DRBD on RHEL/CentOS 6 fails to sync with UnknownMandatoryTag error

We use DRBD as part of Linux HA designs. I was recently tasked with setting up DRBD 8.4 for a new customer installation. I chose a complicated, 64-character random string as my shared secret, and configured both nodes.

This is a companion discussion topic for the original entry at https://www.funkypenguin.co.nz/note/drbd-fails-to-sync-with-unknownmandatorytag/