I‘ve been their webhosting for 10 years and besides some minor errors (website didnt work for 30 mins here and there) I‘m very happy.
I informed myself about their cloud programme with help of a sales rep and although it is quite expensive compared to hetzner, it seems very usable.
I dont think you need to worry about them for a vps. Its not like they would rob you or your data will be lost. They might turn out to be slightly better or worse than hetzner long term which would both be fine in my book.
I dont know about caddy but nginx proxy manager does this very well. Also, if running in docker, you can expose the port that runs ssh for forgejo und a different port than the host machine‘s (eg 2222). In that case you just put the remote in with the port and call it a day.