I think the reason why Mail doesn't disappear from Task Manager is because of its background task.
Try this,
Start, write lock, switch into Settings and open Lock Screen.
On Lock Screen apps click on Mail and chose don't display any badge here.
Now, open Mail and close it with Alt+F4...