On what screen(s)?
If it's the compose screen and auto-complete (drop-down when entering recipients) you can tap the prompt and deny the permission. Then do it once more and the "grant permission" window will have a checkbox to "stop asking".
Other will keep prompting, but they only do so after a "user action", e.g. tapping a sender's image when viewing a message.
( the only exception is settings screens where prompts are always visible -- we want to avoid the more typical users not being able to grant permissions and not knowing why a particular feature doesn't work, e.g. contact images in message lists... but hopefully you're not spending most of your time looking at the app's settings screens... )