When you "remove" a message it isn't actually erased from the system, simply, it's marked with the flag "/deleted" or moved to a trash folder.
IMAP4 clients simply doesn't shows the mails flagged as "/deleted".
E-mail client has to trigger the order "expunge" in order to remove the e-mail effectively. So the behaviour you described is normal.
As a sysadmin, of course, you have some "tricks" in order to do these kind of stuff that rest of people can't
:
https://wiki.dovecot.org/Tools/Doveadm/ExpungeIn relation with point 2, I don't know if there's some config option on your e-mail client but you can play with the imap4 flags manually:
https://wiki.dovecot.org/Tools/Doveadm/FlagsCheers!