Not the OP, but my current solution involves a small instance in AWS with a wireguard server in docker. This is configured with a few peers. One peer is a container on my home server that can access my jellyfin deployment. This container is also running socat to redirect the traffic to jellyfin. Then my phone and laptop are the other peers and I have a DNS record pointed to the IP of the wireguard peer on the server, if that makes sense.

I've been using this image pretty painlessly. The only hiccup I had with setup was ensuring persistent keep alive was configured on the peer forwarding traffic to jellyfin.