Only tablets running Windows RT are restricted and need to download apps from the store. If you get a tablet that runs full Windows then, yes, it runs typical Windows software that can be installed the traditional way. I have a Dell Venue 8 Pro tablet that is running the full version of Win 8.1.1 and can install software via USB. The only caveat is that it requires an inexpensive special USB cable called an OTG cable to adapt USB devices to the micro USB used on the tablet.
It isn't that Windows RT wouldn't let you connect USB devices, it's that typical Windows software won't run on that version of Windows. The Windows store is one of the few sources for RT compatible software.