So after getting my proxy going, I now have kind of a weird setup where my web server and turn server are hosted on the same backend, but the turn server isn't using the proxy. Like so:
NAT (443) => Proxy => Backend server => Web service
NAT (coturn port) => Backend server => Turn service
And for some reason it doesn't work. But if I revert to the previous configuration without the proxy, it works.