It doesn’t have to be the main GPU. I’m not even sure it would be possible to pass through integrated graphics. But if all you need is HDMI output, you can use the absolute cheapest GPU you can find (assuming there’s an open PCIe slot). PCIe pass-through does require CPU support (Intel VT-d or AMD-Vi) on the host and may need to be enabled in the BIOS/UEFI. I have an NVIDIA Telsa card passed through to a VM on my Proxmox server, but I’m only using it for compute; my card doesn’t even have a video output.
Have you shopped eBay for used switches?