Seems to me that it is an issue with regards to metrics and routing. Navigate to your cellular wireless internet connection and then go to the TCP/IPv4 properties. From here adjust the metric and set it to 1. This way Windows should prefer to use your tethered internet connection for internet access instead of trying to use your router. The picture below should provide more details.
Originally Posted by CarvedDuck
I am currently using Windows 7 however please understand that these steps apply to Windows 8 as well.
Hope This Helps,