iCloud mails not delivered if (Send Message-ID) is enabled

[bencom]

I was wondering why my recipients were not getting my mails sent from my @icloud.com account using Aquamail.  But test sents with browser webmail and Gmail works.

Narrowed it down to this setting (Options and folders/Composing and sending/Send Message-Id).  If enabled, iCloud mails won't get delivered.  It will appear in Sent folder in Aquamail and iCloud webmail, no errors anywhere, everything seems fine from my end but recipients don't received it. 

What is this setting for?  What is the default?  I'm quite sure I didn't change it.

(Ver 1.6.2.9)

[StR]

I have never dealt with iCloud.com, so, I am not aware of its idiosyncrasies.
This option in Aquamail allows to include Message-ID generated by the client. Typically, this is what is done by most (if not all) e-mail clients that I know. In general, a unique Message-ID should be generated is by either the mail client, and if it doesn't exist, it is generated by the first mail-submission agent (server).

However, some servers may overwrite the Message-ID generated by the client, and some can bark, as in your case. I am guessing that's the reason why Kostya created this option to disable generation of Message-ID, leaving it to the first mail server. And it is enabled by default, because that's (IMHO) the preferred behavior.

For further reading: https://en.wikipedia.org/wiki/Message-ID

[bencom]

I'm beginning to suspect iCloud SMTP server doesn't like the Message-id created by Aquamail.

my testing shows emails sent from:
- default Gmail app, gets a Message-id (ending with  .....email.android.com)
- Aquamail with that setting enabled, gets a Message-id (ending with  .....icloud.com)

So since Gmail's Message-id works with iCloud, does it means a issue with Aquamail's Message-id? 

Perhaps iCloud sees Aquamail's Message-id and mistaken it as its own and wrongly assumes the mail has been delivered?

[StR]

I'm beginning to suspect iCloud SMTP server doesn't like the Message-id created by Aquamail.

That might be the case.

my testing shows emails sent from:
- default Gmail app, gets a Message-id (ending with  .....email.android.com)
- Aquamail with that setting enabled, gets a Message-id (ending with  .....icloud.com)

I didn't look at what Gmail app is doing, but indeed, Aquamail generates the Message-id which domain-name portion coincides with that for the "From" address for the account or the identity used in Aquamail (if used). From your example it looks like Gmail doesn't do that.

I personally think that the Aquamail behavior is better. I don't like Message-IDs that end with generic parts like @mail or @java. As a matter of fact, on the server I administer, I have some spam filters that use some non-domain-specific Message-IDs as a part of identifying SPAM.

So since Gmail's Message-id works with iCloud, does it means a issue with Aquamail's Message-id? 

Perhaps iCloud sees Aquamail's Message-id and mistaken it as its own and wrongly assumes the mail has been delivered?

That should not be the case, but I don't know what iCloud engineers think.

I'd wait to hear from Kostya, Aquamail developer. I would expect him to provide the insights on this issue, including the historical reference on why the option to disable Message-ID was created in the first place. That might shed more light on your situation.

[bencom]

You may have a point there on spam filters, Aquamail's default to Message-id ending in ....icloud.com may have tripped some Apple filters.  I noticed if Aquamail Message-id setting is disabled, their servers generated a Message-id ending in ......me.com.  Perhaps ...icloud.com is somehow restricted.

A word of caution for all not to put too much faith in any email system...

The way this issue manifest is quite dangerous, creating possible legal/financial jeopardy for both sender and recipient with both claiming they are correct - sender insisting the mail was sent vs recipient adamant about not receiving it.  Almost got myself into a sticky situation with the authorities over this. 

[StR]

Is your e-mail address (From:) used for these messages @me.com or @icloud.com?
From what you explained, - it must be @icloud.com

I see that Apple instructions for the e-mail servers are using .me.com servers (for both SMTP and IMAP): https://support.apple.com/en-us/HT202304

So, what you are saying sounds consistent. But it is stupid on the part of Apple: the fact that sending messages with From: user@icloud.com while having a message-ID ending with @icloud.com yields a problem is dumb!

(And having the username entered differently [with and without @icloud.com] for IMAP and SMTP server is ... [choose your own adjective]  )

[Kostya Vasilyev]

Yep. Seen something like this before, with Apple iCloud account, not sure if "all" are affected (if they were, they'd be many more reports).

At some point (a few years back) it was not liking the User-Agent, maybe that's another thing to try.

I was seeing their SMTP server accept the emails and drop them on the floor. There'd be no errors (on submit), no bounces (returned message), they'd just disappear.

---

I've just made a change to generate ... @me.com for @icloud.com addresses.

I'm not at a point where I can post a "dev" build yet* but it will be there.

*] because of the MobiSystems thing, we need to work a few things out before forum builds can resume (but they will).