Friend has Windows 8.1 32-bit, fully updated and when we were playing with installing another language we got this toolbar. I did everything I could in the Advanced settings under Language in the Control Panel and I am unable to remove it. Is it a bug? How do we get rid of this?
Maybe try this, does not seem to work properly here, but maybe you have more luck.
Control Panel > Language > Advanced Settings > Tick the box for "use the desktop language bar when it's available" > Options. In this dialogue box choose Language Bar Tab > Language bar > Hidden. Apply, OK etc.
Like your friend I find the Language/different keyboard(UK/US) icon appearing at odd intervals.
I've been through all the options in the Control Panel, Location Settings, Language, Keyboard settings.
Just can't get rid of it permanently. Only a re-start seems to work for a while anyway.