AquaMail Forum

English - Android => Bug reports => Topic started by: sthalik on November 29, 2015, 06:04:54 pm

Title: text/plain email HTML markup rendered as HTML
Post by: sthalik on November 29, 2015, 06:04:54 pm
HTML markup of a text/plain message rendered as HTML. This happens when viewing a sent message. There's no text/html part at all.

All addresses, including message-id snipped due to forum policy.

Message with headers:


To: [...]
From: [snip]
Subject: html css
Message-ID: [snip]
Date: Sun, 29 Nov 2015 13:47:44 +0100
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101
 Thunderbird/38.4.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit

<!DOCTYPE html>
<html>
<head>
<style>
ul {
     list-style-type: none;
     //margin-left: 0px;
     padding-left: 0px;
}
</style>
</head>
<body>

<p>Example of unordered lists:</p>
<ul>
   <li>Coffee</li>
   <li>Tea</li>
   <li>Coca Cola</li>
</ul>

<ul>
   <li>Coffee</li>
   <li>Tea</li>
   <li>Coca Cola</li>
</ul>

</body>
</html>
Title: Re: text/plain email HTML markup rendered as HTML
Post by: Kostya Vasilyev on November 29, 2015, 07:34:34 pm
The message content here is clearly HTML, but the header says it's plain text.

Was this message sent with AquaMail (hope not)?
Title: Re: text/plain email HTML markup rendered as HTML
Post by: Kostya Vasilyev on November 29, 2015, 07:37:21 pm
Ah, sorry I misread your message.

Yes, AquaMail checks text/plain parts to see if they're actually text/html.

This happens quite often, mostly "email marketing campaign" messages (and I don't enjoy reading about how "your app can't handle HTML, how come, etc.").

Had there been any text before the DOCTYPE, the logic wouldn't have triggered.
Title: Re: text/plain email HTML markup rendered as HTML
Post by: sthalik on December 02, 2015, 01:52:38 pm
I think the messages you mention have both text/plain and text/html MIME parts. It should be enough to prefer text/html. In case they do, and the text/plain part is in fact HTML, the current heuristic could be used to prefer the text/html part.

There's no behavior like this in Thunderbird and probably other mail clients, given how it treats the standards. Would you offer an option to disable the heuristic? I think it should be default to treat text/plain for what it is, but at the very least an option to do so is better than the current behavior.
Title: Re: text/plain email HTML markup rendered as HTML
Post by: Kostya Vasilyev on December 02, 2015, 06:59:47 pm
Re: mixed text/html and text/plan

This logic only kicks in if a message only has a text/plain part.

Re: There's no behavior like this in Thunderbird and probably other mail clients

And?

I'm not the developer of Thunderbird or "probably other mail clients".

However, I did use to receive support emails about how "this app is broken and can't render HTML" -- and so I added this logic.

Don't know if the developers of Thunderbird or K9 Mail or Outlook ever received messages like that.

Re: setting

Don't think so.
Title: Re: text/plain email HTML markup rendered as HTML
Post by: sthalik on December 02, 2015, 07:15:58 pm
> Re: There's no behavior like this in Thunderbird and probably other mail clients
> And?

If sending HTML as text/plain with no other parts was prevalent, other clients would follow suit. I'd give much stronger benefit of the doubt to the standardized behavior. This might just as well be one or two organizations that can't get their headers straight. Hardly worth misinterpreting everyone else's messages.
Title: Re: text/plain email HTML markup rendered as HTML
Post by: Kostya Vasilyev on December 02, 2015, 07:27:15 pm
It's not prevalent, but it does happen.

"Standards"?

People sending e-mail messages often don't know what those are, or they just shit on them.

I'm talking mostly about all kinds of automated emails -- promos, notifications, newsletters -- despite there being various libraries for constructing messages not to mention that RFC's are easy to find.

But at the end of this chain are users who are just sooo confident that "there is something wrong with your app", so I made it so that it doesn't look that way.

Of course now there is your case, and you don't just think there is something wrong with the app -- you *know it*, confirmed by the app's developer!

---

You know how at Miss Universe and so on, the contestants give their little speeches about "how to make the world a better place"?

Unfortunately, I'm too old and ugly (not to mention - male) to enter, or I'd use it to "promote well structured email messages everywhere".

Oh well.
Title: Re: text/plain email HTML markup rendered as HTML
Post by: StR on December 02, 2015, 08:19:23 pm
Quote from: Kostya Vasilyev, Aqua Mail on December 02, 2015, 07:27:15 pm
...

You know how at Miss Universe and so on, the contestants give their little speeches about "how to make the world a better place"?

Unfortunately, I'm too old and ugly (not to mention - male) to enter, or I'd use it to "promote well structured email messages everywhere".

Oh well.

Kostya, you are so kind.
I would just lynch all those pseudo-developers whose degenerate code generates broken mail messages that violate RFCs. (Violation of RFCs is a violent behavior and must be punished as such!)  ;D
... "And world peace!"  8)
Title: Re: text/plain email HTML markup rendered as HTML
Post by: sthalik on December 02, 2015, 08:53:57 pm
Apparently it doesn't take you more than a random bug report to come off as rude, and rant about nonsense. I'm not going to discuss how you misrepresented my position, and turned it into some absurdity.

I'm done here, carry on.
Title: Re: text/plain email HTML markup rendered as HTML
Post by: Kostya Vasilyev on December 02, 2015, 11:45:58 pm
Don't know why you took my response as "rude" or "nonsense".

---

There are email messages that are not compliant to "standards" no matter how much you (and I) dislike this.

Specifically, some messages may have HTML content whose mime type is set as "text/plain" in the Content-Type header.

And I have had actual emails from users who, when coming across such messages, rendered as text/plain (so they show actual HTML source code) -- immediately assumed that "AquaMail is broken and can't even do something as basic as rendering HTML".

I did not mean you. Clearly you came across the opposite case, where you do not want this "detection".

---

Two.

This used to happen fairly regularly until I added the "detect and treat as HTML".

---

Three.

As I wrote above, this happens with automatically generated messages -- various promotional / notification / informational messages coming from web sites.

The people -- web developers -- responsible for such messages are clueless about proper MIME message structure.

They manage to screw up over and over and over again (there are a lot of web sites out there which send email), despite RFCs being easy to find, and despite various libraries that help with sending email.

One can call those people idiots, can call them something else, but it doesn't change anything.

My users would receive those improperly structured messages and they "knew" that "it's was horrible bug in AquaMail".

Since I can't find every developer responsible for improperly structured MIME messages (and then fix their code for them) --

-- I had to add a workaround in my app, which I can change much more easily.

---

And I will not be changing it back, or adding a setting:

The world is not a perfect place, those clueless people are out there, and their web sites send those garbage messages -- which I have to handle in order to avoid user complaints.
Title: Re: text/plain email HTML markup rendered as HTML
Post by: StR on December 02, 2015, 11:51:50 pm
@sthalik:
I understand your frustration: you discovered and reported the behavior that deviates from the standards and/or common logic, but it is not labeled as a bug.
(Been there, done that, and not only with Aquamail or other software, but with other ("everyday") things in life.)

But I don't think Kostya's response is rude or nonsense. It might not be the most pampering response (as prescribed e.g. by Dale Carnegie), but the way I read his response is: "Yes, that's deliberate, to compensate for the real-life situations [even if those violate RFCs], and sorry but I have reasons why I am going to keep it that way."  It's like the [in]famous "It's not a bug, it's a feature".

If this "feature" were affecting me, most likely I'd be reporting it as you did. And I would be frustrated in a similar way. 
But at the same time, I see why Kostya made his decision for this feature. There is a much larger portion of any [general purpose] software users who are ignorant about the standards or what is "right and wrong". Even worse, many of them are unwilling to listen or to try to understand. And, of course, they are quick to throw a tantrum like a 3-year-old. As a person who singlehandedly has to balance responses to all different users while providing great and timely support and developing new features, Kostya is sometimes forced to make decisions that minimize the inflow of those complaints. It is not a software-developer's decision, it's more of a business decision.

{philosophy}
Yes, it feels like the ignorance and breaking the rules get awarded in those cases. And it is not fair. But life is not fair. And, unfortunately more frequently then I wished, loud ignorant people get more than they deserve. (For that, just turn on your TV news on almost any day.)
{\philosophy}

As for Aquamail, - there is no ideal software that fulfills needs and desires of all and every user, but Aquamail got the closest to that. And Kostya has managed to choose the right balance for his app, listening to various sides, and making his own decisions. I might disagree with some of those decisions individually, but it is the entire combination of those decisions that allowed him to bring this app to this point, and to sustain it at this level.

You may feel mortally offended by this small "refusal", and I cannot do anything about that. But as a happy user of Aquamail (the software I am most fond of), I would encourage you to look at a bigger picture of what this app and his sole creator have to offer.

(This message was written before Kostya's more recent response that just got posted.)
Title: Re: text/plain email HTML markup rendered as HTML
Post by: sthalik on December 03, 2015, 01:14:30 am
Hey guys,

I'd like to clarify some things I've said in the past few messages.

@StR

My point was misrepresented - nowhere did I ask for supporting relevant SMTP/IMAP/MIME to the letter unconditionally. There are many extremely common "abuses" of, say, SMTP - rejecting by the EHLO hostname, for instance. In today's deluge of spam it's perfectly reasonable to reject the EHLO hostname if it doesn't exist or doesn't point to the sender's connection. The issue here is a real-world use-case of pasting HTML as a message's body, where the sender can fully expect it to be treated as such.

The standards are means to an end, yes, hence my reference to other MUAs like Thunderbird. It seems like neither TB, GMail nor Blue Mail on Android have this behavior either. In case of these sloppy mailings, I'd expect way more users to have problems with them, as this is pretty unheard-of behavior. GMail web interface is reasonably popular yet it renders the message as text. The botched mailings thus can be anything but successful... I don't think it's reasonable to expect Aqua Mail to render them with this heuristic if the messages don't get rendered this way in other, way more popular clients, either. A pop-up could show up asking to reparse as HTML in the same place as "view external images" bar shows up. Since the decision has been made though it's moot for me to provide further suggestions.

As for "nonsense" I referred to the dragged-out analogy about "Miss Universe"-sort-of idealism. As for rudeness I've referred to the use of such absurd analogies, misrepresenting my statements, as well as general impression of the tone of the few preceding messages. I don't wish to dwell on these points, merely to explain further my original statements.

> You may feel mortally offended by this small "refusal", and I cannot do anything about that. But as a happy user of Aquamail (the software I am most fond of), I would encourage you to look at a bigger picture of what this app and his sole creator have to offer.

Absolutely this is not the case with the refusal itself. I don't put in all the stuff users ask for either. Ridiculing strawman arguments won't help any though, and won't many anyone happy.

After my own two years (!) of usage the software is really decent. Best of luck.
Title: Re: text/plain email HTML markup rendered as HTML
Post by: StR on December 03, 2015, 02:01:10 am
https://www.youtube.com/watch?v=3st-Hai1y54