Taskbar icons - Duplicates and warning flags

I have problems on my Windows 8.1 taskbar. Some of my icons have small symbols added to them and one of my programs, Internet Explorer, has TWO icons on the taskbar, one normal and one with an extra symbol overlaid on it. I'd like to know what these extra symbols mean and what I need to do to get rid of them.

This is definitely a case where a picture is worth a thousand words so I've done a screen cap of my task bar.

The icons with the extra symbols are (from left to right): Firefox, and the first Internet Explorer. (Ignore the tiny "1" on the Skype icon; I know that was just someone trying to contact me.) I can't make out what the extra bit on the Firefox and first IE icons is supposed to be; it's too small for me to make out. It could be a tiny "1" or maybe a graphic of a keyhole or something else. In any case, I'm trying to figure out what this signifies and what I need to do about it.

I also want to get rid of the duplicate IE icon and could do that easily enough with a left-click and then an "unpin from taskbar" but will getting rid of that extra bit of graphic on the first IE icon get rid of the duplicate automatically? I'm really not sure what to do here....

I'd appreciate some advice from the experts here