“Too many redirects” when using WordPress as container

WordPress in Docker can cause quite some headache. Especially the reverse proxy, which we need to pass through traffic from host to container. I struggled with a "Too Many Redirects" error, which can come from some different sources.