I am finding that when I try moving an image file from one directory to another in Windows 8.1, I am getting another copy of the same image created in the destination folder which has a new number in that file name. For example: if I have IMG_1234.jpg in one folder and I move the same image from another folder into one where that image is already ( too many images to move to remember) I get the same image, but it would now be named IMG_2345.jpg. Rather than tell me that that file already exists and ask if I would like to overwrite it.
I am finding this to be both confusing and frustrating at the same time, so I would be grateful if anyone can tell me why this is happening.

Many Thanks in advance!