Edit the shortcut properties (right click the icon, then right click the File Explorer shortcut, and choose Properties) to change the Target parameter to the directory you desire. You can get the directory easily by navigating to it normally in explorer, then copying it from the address bar.
Personally, I was not able to edit the Target parameter of the default File Explorer shortcut that is pinned to the Start Bar.
So you can create a new shortcut that points to explorer.exe followed by the directory that you want it to open to, such as "explorer.exe c:\games" (without the quotes.) or by adding the desired directory to the Target parameter of the new shortcut.
You can also create a shortcut to the folder you want opened by navigating to that folder, right clicking, and choosing Send To >> Desktop (Create Shortcut).
Then in either case, you can rename the shortcut and change its icon to be anything you want including the explorer.exe icon, and pin it to the Task Bar or Start menu.
