Please note that Gmail for web, if you use that too, also saves everyone you've replied to as contacts.
So maybe it's not AquaMail being "annoying", but Gmail / web.
You can tell by examining which *group* those contacts get created in.
AquaMail creates them in a separate group called, well, "AquaMail".
Gmail does not use a separate group.