It would have to be at least a bit larger than the maximum size of the VHD file created on it.
There are several 32 GB usb drives available now at sensible prices and I'm sure a 64 GB one what be long in appearing.
Incidentally I quite like the idea of having the system on one of those micro SD cards like you see in some smart phones. You'll probably need to have the micro sd card enclosed in a standard SD adapter which you can boot from as a USB device.
Those tiny SDHC cards are available in 32GB for around 35 EUR these days so not too expensive.