AquaMail Forum

English - Android => How do I... => Topic started by: Davey126 on January 31, 2016, 09:56:25 pm

Title: Gmail - Deleted Folder Sync
Post by: Davey126 on January 31, 2016, 09:56:25 pm
I apologize if this has already been discussed. I did search the forums and did a lot of reading/testing before posting.

First the preliminaries: Aquamail is brilliant! It has made managing (not just reading) email on android both possible and enjoyable. The following inquiry in no way represents dissatisfaction with the app. It is simply a question/request that would help me - and possibly benefit others.

Is is possible to set the Trash folder in AquaMail to sync with Gmail's '[Deleted]' folder on demand vs automatically? This can be done with the Spam and Archive folders but not Deleted, Drafts or Sent.

As currently configured the automatic Trash <-> Deleted sync is the most time consuming/expensive part of a normal sync operation. I have far more deleted items than those in the Inbox or other auto-synced folders. By default Gmail retains deleted items for 30 days. I can manually prune the folder by that's a largely manual operation as there is no native setting or automated way to accomplish this.

I do want to retain the Trash <-> Deleted relationship in AquaMail as there are times I need to reference something that has been recently deleted from a different client. I am more than happy to do a manual refresh to retrieve these items but don't want to maintain separate folders on the server and AM sides.

Thanks in advance for your thoughts.

p.s. Setting the "Message to keep" parameter on the Trash folder appears to be a minimum value (which makes sense for its intended purpose).
Title: Re: Gmail - Deleted Folder Sync
Post by: Kostya Vasilyev on February 02, 2016, 10:01:27 pm

1 - "Special" folders (Sent / Deleted / Drafts) are always "synced" (for IMAP / Exchange accounts)

2 - Moving to Deleted ([Gmail]/Trash) is the only proper way to delete messages in Gmail (I mean to remove all labels, and also from All Mail)

Now you can try to bring up the account's Options and Folders (long press) -> Folders -> Find [Gmail]/Trash, tap it and choose "Do not sync".

Then the app will use its own local folder for Deleted, and message won't be moved anywhere on the server (when you move there in the app) -- until you long press this folder and "delete all" at which point those messages will be deleted from Inbox or whatever.

This however is a highly unusual way to set up Gmail, and I can't promise that there won't be any unexpected side effects.
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 03, 2016, 01:47:48 am
Thank you for the extensive response. Before posting I had found/experimented with the "Do Not Sync" option, noted the behaviors that you describe and decided the benefits did not outweigh the disadvantages.

I spent (perhaps too much) time observing detailed sync operations over the past few days and concluded that syncing my busy 'trash' folder is not that expensive after all. The initial sync takes some time/resource (of course) but even with a 100+ items coming/going daily incremental syncs happen quickly with relatively little cpu. Apparently Aquamail is smart about the remaining 3,000 or so items in the deleted folder that have not changed and spends little/no time considering them.

Not sure who gets the efficiency nod on this one; since Google isn't in the room it goes to Kostya and Aquamail!

This is a case were dialog is good, a solution already in place and no further action needed. Thanks again for producing such a nice client along with time you devote to servicing the forums and user Q&A.
Title: Re: Gmail - Deleted Folder Sync
Post by: Kostya Vasilyev on February 03, 2016, 01:55:23 am
Re: Apparently Aquamail is smart about the remaining 3,000 or so items in the deleted folder that have not changed and spends little/no time considering them

Yes, AquaMail only loads a limited number of recent messages per folder. The relevant setting is called "messages to sync".

It would be crazy to download all 12,000 messages in my Inbox and 14,000 in another folder -- and I know there are people who keep many more messages in their accounts.

Re: Not sure who gets the efficiency nod on this one; since Google isn't in the room it goes to Kostya and Aquamail!

It's AquaMail (I do know how this works).

Google could make it even better, by making a certain something work more like it does with certain other IMAP servers (e.g. Dovecot or Cyrus)... but it doesn't.
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 03, 2016, 04:56:41 am
I have a related question that hopefully will complete the sync picture and perhaps help others who discover this thread.

At present I have "messages to sync" and "messages to cache" set to '30 days' and '250 messages' under Account options. This works very well for my use case on Gmail. Fast response (quick sync), small database, fully populated conversations within the 30 day sync horizon. Perfect.

I recently encountered the same settings under Settings -> Mail, receiving but found the values to be "25 messages" and "250 messages", respectfully. It is likely I set these values sometime in the past and forgot about them.

How is this apparent conflict handled by AquaMail? In particular "Messages to sync" which (in my case) is expressed as calendar days in one location and message count in another. I could set them to identical values and walk away ... but curiosity has set in.

I did spend a bit of time reviewing previous forum posts dealing with sync. Sorry if I missed an obvious answer.

Brief response is fine. Just want to make sure I do not need to make further tweaks or have set myself up for weird behavior. I will try not to extend this thread further...nothing more annoying than a series of follow-up questions! :)
Title: Re: Gmail - Deleted Folder Sync
Post by: Kostya Vasilyev on February 04, 2016, 01:07:46 am
Account level settings have higher priority, so you're getting "30 days worth of messages fully up to date + 250 more messages cached".

( when using a date range "messages to sync", the "messages to cache" means "in addition to..." because I can't tell how many messages "30 days" is exactly )
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 04, 2016, 04:53:57 am
Thanks. That's what I thought might be happening w/sync based on observation. Good to know about the behavior of cache in this situation. Works well for my purposes. Happy camper :)
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 06, 2016, 12:05:25 am
Account level settings have higher priority, so you're getting "30 days worth of messages fully up to date + 250 more messages cached".

( when using a date range "messages to sync", the "messages to cache" means "in addition to..." because I can't tell how many messages "30 days" is exactly )
A quick one-time observation that may not be relevant or repeatable. Posting here for reference in case it happens again (to me or someone else).

Using Gmail with account sync setting as described above. Left other sync settings (under settings -> mail receiving) at their default values. Encountered a situation were the contents of my inbox in Aquamail did not match that of the web client. Messages in question arrived within past 24 hours. Fiddled with various settings to no avail. On a whim bumped 'messages to sync' to 5000 under 'normal' settings (not account). After the next sync everything was back to normal.

I have not seen this before nor have I been able to replicate it in the few hours since it was observed. Will continue monitoring and report back if new information becomes available.

Note: Not a conversation combining issue; accounted for that.
Title: Re: Gmail - Deleted Folder Sync
Post by: Kostya Vasilyev on February 06, 2016, 12:08:45 am
Hmm, you're still using "Last 30 days" for "Messages to sync", correct?

This actually asks the serves to search for messages by this date range and loads those.

If this happens again, please capture the issue in the app's debug log if you can (the link is in my signature, below).

Just enable logging, mark a single message with a star on / off (to force a change in the account) and then "Refresh".
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 06, 2016, 12:18:47 am
Hmm, you're still using "Last 30 days" for "Messages to sync", correct?

This actually asks the serves to search for messages by this date range and loads those.

If this happens again, please capture the issue in the app's debug log if you can (the link is in my signature, below).

Just enable logging, mark a single message with a star on / off (to force a change in the account) and then "Refresh".
Correct (30 days on account options). I will try to capture a log if it reoccurs. Shouldn't be too hard as the issue is not fleeting; requires an action to rectify so there is time to enable debugging.
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 12, 2016, 10:07:01 pm
A couple follow-ups:

1) I have not been able to duplicate the one-time problem discussed in the three posts (dated Feb 06) immediately preceding this one. I view the issue as unrepeatable and therefore closed.

2) I would like to resurrect the original request as a low-priority/nice-to-have/only-if-easy-to-implement enhancement:

Is is possible to set the Trash folder in AquaMail to sync with Gmail's '[Deleted]' folder on demand vs automatically? This can be done with the Spam and Archive folders but not Deleted, Drafts or Sent.

Rational: for most users Spam and Trash represent different variations in the same general category: unwanted items that *may* need to be accessed on occasion. Spam has the option to sync on demand; Trash always Syncs. It seems silly to maintain 3000+ unwanted emails on the local device when they are rarely needed/accessed. The collection makes up 80% of my mail database. Gmail has made it difficult to prune these items in bulk on the server side so users are bound by the default (unconfigurable) 30 day expiration interval. Culling the trash folder from AquaMail leads to very long sync times (many minutes for a few hundred items) which I understand and do not view as a 'bug' as this is not the way things are normally done.

It is important to acknowledge that AquaMail efficiently handles the normal syncing of 'deleted' items as discussed in previous posts within this thread. It's not about raw performance. More of a logical, consistency thing that might lead to a smaller local database for some.
Title: Re: Gmail - Deleted Folder Sync
Post by: StR on February 12, 2016, 10:20:43 pm
I agree with Davey126 and support that idea!
( I am not affected much, as my Gmail usage is minimum.)
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 12, 2016, 11:12:01 pm
Thanks for the suggestion but it is not that easy. I delete dozens of messages every day with no intent of ever accessing them again. Otherwise they wouldn't have been deleted, either remaining in the inbox or moved to a pending folder.

Then comes the unexpected need to view/act on the content again. Off to the deleted folder for a search ...

I really do not want to engage in a discussion of 'best practice'. I'm quite familiar with the capabilities of Gmail and IMAP/POP clients. I also appreciate AquaMail and the thoughtful decisions that have been made around how it operates. Like all enhancement requests the decision to implement comes down to what the developer chooses to do based on user feedback.

Your post and mind contribute to that equation.
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 12, 2016, 11:19:42 pm
And justification for such a claim? Could be I have overlooked a peril. Please enlighten the community so we all understand the danger.

Edit: Previous post (which I failed to quote) that I was responding to was subsequently amended so this follow-up does not make complete sense in the flow of the conversation. Leaving original wording intact.
Title: Re: Gmail - Deleted Folder Sync
Post by: Kostya Vasilyev on February 12, 2016, 11:26:02 pm
Quote
It is important to acknowledge that AquaMail efficiently handles the normal syncing of 'deleted' items as discussed in previous posts within this thread. It's not about raw performance. More of a logical, consistency thing that might lead to a smaller local database for some.

Give what you wrote above (quoted) -- is this a real issue for you, are you seeing any negative effects of Trash "syncing"?

How large is the app's database, anyway, on your phone? It's listed in the app's About window.
Title: Re: Gmail - Deleted Folder Sync
Post by: StR on February 13, 2016, 01:57:34 am
Is is possible to set the Trash folder in AquaMail to sync with Gmail's '[Deleted]' folder on demand vs automatically? This can be done with the Spam and Archive folders but not Deleted, Drafts or Sent.
...

You are using Gmail. Your need can be satisfied in a smart way, simpler than your request (btw: it's silly indeed to store 3000+ in a trash folder on your local device, and to ask AquaMail to adapt to this silly behavior -- the word "silly" is yours :) )

- Create a folder in desktop Gmail, call it "Store" (or whatever you wish)
- In AquaMail, sync your Gmail account using IMAP mode
- In particular, sync your "Store" folder as plain
- When you want to store a message, to keep it for a future use, move this message to your "Store" folder (using move in AquaMail)
- that's it !

With that:
- you can keep your messages for an unlimited period of time, on your server and your local device
- you'll get a quick sync (because you'll use Aquamail's settings by default of # of sync and cache messages)
- you'll use Trash for what it is designed for: Trash is for trash (not to keep unwanted items that *may* need to be accessed on occasion)


Should you have further questions on how you implement this on your device, please let us know.

@Paris Geek:
I might be misreading your suggestion. It is also possible that I am misunderstanding what @Davey126 is suggesting (but this is less likely). But I am not seeing how your solution solves the situation with Gmail.
To work properly, Gmail requires the messages you are deleting to be moved to Trash.
Say, I am doing that on one IMAP-based client.
Now, if I am not going to look at those messages (ever), - why should Aquamail sync Trash (upon every regularly scheduled sync)?

So, similarly to how there is "Spam, do not sync" option, there should be "Trash, do no sync".


@Kostya, - I understand your question whether this has any significant effect... I cannot say, - since I have very few messages in Gmail account synced in Aquamail, and I seldom delete them. (My Trash is empty at the moment.)
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 13, 2016, 02:13:33 am
My feedback: I consider your request as not needed (cf. my suggestion you're rejecting) anf dangerous for other users:
One day, a user would unselect by inadvertance this option, then his Deleted folder will not get synced, and problems would appear. He'll put 1 star on Google play, and say that Aquamail is broken. Unfortunately this is real life.
As with any operational setting there is a risk for user misunderstanding/misconfiguration and resulting dissatisfaction. However, the point you raise is valid and needs to be weighted into any decision to proceed or not.

[not trying to make a big deal out of this request...it's just the logical process one goes through when deciding which path to take...even if you don't think of it that way]
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 13, 2016, 02:38:52 am
... is this a real issue for you, are you seeing any negative effects of Trash "syncing"?
Nope - other than the long initial sync on new/rebuilt devices (which I do a lot of but obviously not common). In routine use there is an observable but not significant increase in sync time as the deleted folder grows. It does drive frequent, short bursts of network activity but this is not personally bothersome as I enjoy robust mobile and wifi coverage in my daily travels across two countries.

What is noticeable:
- extended sync time if I attempt to cull the deleted folder from the AquaMail client. This would not happen if the sync wasn't automatic as the server would never know what was deleted locally. However, I can not hold this up as justification for change because I voluntarily initiate the cull and know the consequences of doing so (long sync). BTW - same thing happens initiating a 'purge [deleted] folders' from the account menu.
- effectively no cache beyond the sync horizon as the 250 messages I have configured are gobbled up by deleted items. This not a big problem given the speed and availability of network connectivity; I simply drag in another 30 days of messages as needed.

As previously noted it is virtually impossible to initiate a bulk clean-up from the Gmail's clients (app/browser) as 'permanently delete' is only offered for single messages. Google really, really, really wants to keep your trash for 30 days. Probably so they can read it ...
 
How large is the app's database, anyway, on your phone? It's listed in the app's About window.
A very tiddy 38MB, 80% of which is deleted items unless I have done a recent sync of a large folder. But that size could be much larger if the deleted content (message bodies) was sizable. Most of my deleted emails are tiny...a few dozen KB.

Closing comment: Recall the original request was characterized as "low-priority/nice-to-have/only-if-easy-to-implement". I feel this is gotten blown out of proportion. That said, there is justification and logic behind the request ... not just some random fluff that I thought of in the last 10 minutes.
Title: Re: Gmail - Deleted Folder Sync
Post by: mikeone on February 13, 2016, 02:37:38 pm
My feedback: I consider your request as not needed (cf. my suggestion you're rejecting) and dangerous for other users:
One day, a user would unselect by inadvertance this option, then his Deleted folder will not get synced, and problems would appear. He'll put 1 star on Google play, and say that Aquamail is broken. Unfortunately this is real life.

And justification for such a claim? Could be I have overlooked a peril. Please enlighten the community so we all understand the danger.

Edit: Previous post (which I failed to quote) that I was responding to was subsequently amended so this follow-up does not make complete sense in the flow of the conversation. Leaving original wording intact.
@Davey126
In my opinion "... the danger" could be as follows:
The user deleted a message (removed from the inbox or from any other folder,  moved to the deleted folder). Afterwards he need the content of this deleted message on another client. Unfortunately this message will be only available in AquaMail, but not on any other device (mobile or pc or even in the webmail account) ... since it was not synced back by AquaMail to the servers deleted folder.
This situation could be very annoying.
Title: Re: Gmail - Deleted Folder Sync
Post by: mikeone on February 13, 2016, 02:47:18 pm
Quote
Now, if I am not going to look at those messages (ever), - why should Aquamail sync Trash (upon every regularly scheduled sync)?
@StR
In this specific case - if you are not going to look at those deleted messages (ever) - why not using AquaMail's action "Delete now" (remove deleted messages permanently from server)? That's how I'm dealing with those useless messages.
Title: Re: Gmail - Deleted Folder Sync
Post by: StR on February 13, 2016, 04:21:17 pm
Quote
Now, if I am not going to look at those messages (ever), - why should Aquamail sync Trash (upon every regularly scheduled sync)?
@StR
In this specific case - if you are not going to look at those deleted messages (ever) - why not using AquaMail's action "Delete now" (remove deleted messages permanently from server)? That's how I'm dealing with those useless messages.

That's how I am doing that but not in Gmail.

To work properly, Gmail requires the messages you are deleting to be moved to Trash.
Title: Re: Gmail - Deleted Folder Sync
Post by: mikeone on February 13, 2016, 05:41:22 pm
@StR
...mmh, for me it's working as described, even with my Gmail account (in AquaMail)
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 13, 2016, 08:56:07 pm
First thanks to @Paris Geek, @mikeone and @SrT for the active discussion; it is appreciated. As most will agree constructive dialog usually yields the best outcomes provide emotions are kept in check (note to self) and sacred cows remain in pasture.

re: "I consider your request as not needed (cf. my suggestion you're rejecting) and dangerous for other users:
One day, a user would unselect by inadvertance this option, then his Deleted folder will not get synced, and problems would appear. He'll put 1 star on Google play, and say that Aquamail is broken. Unfortunately this is real life."


re: In my opinion "... the danger" could be as follows:
The user deleted a message (removed from the inbox or from any other folder,  moved to the deleted folder). Afterwards he need the content of this deleted message on another client. Unfortunately this message will be only available in AquaMail, but not on any other device (mobile or pc or even in the webmail account) ... since it was not synced back by AquaMail to the servers deleted folder.
This situation could be very annoying.


I had considered this but felt most would understand their choices (especially if overtly changed from default values) and recognize what had happen just like with any other selective sync. In that regard I don't see operations on the deleted folder being any different than spam but also realize everyone might not see it that way.

re: In this specific case - if you are not going to look at those deleted messages (ever) - why not using AquaMail's action "Delete now" (remove deleted messages permanently from server)? That's how I'm dealing with those useless messages.

I do use this and can confirm it works for Gmail. However, given high mail volumes it is not efficient to decide between a 'normal' delete and 'immediate' delete while processing each message. Peforming this action in bulk from the deleted folder in AquaMail leads to long sync times (seconds to minutes) on fast connections - likely due to limitations in the protocol which mandate each message be handled individually.

I am looking into a way to more efficiently cull messages on the server side using a combination of AquaMail and the GMail mobile interface which retains 'delete forever' capability on multiple messages. Current challenge is multi-select in Gmail as each item must be ticked individually. The GMail app supports multi-select but, like most clients, downloads messages rather than acting directly on the server version.

There seems to be a lot of energy around this request so I will withdraw it (again). Bit of a head-scratcher as the same selective sync choice exists for spam. Obviously misread the tea-leaves. :) :)
Title: Re: Gmail - Deleted Folder Sync
Post by: mikeone on February 13, 2016, 09:32:46 pm
Quote
In Paris, I've got unlimited mobile date (50 GB with 4G+ speed),
@Paris Geek :
Wow, you drive my a bit envious
[stuck on 1.5 GB with 4G+]

Quote
Where are u living? Do you have an issue with Wi-Fi or mobile data bandwidth? And which device are you using?
As far as I've learned Davey126 is traveling daily between two countries:
https://www.aqua-mail.com/forum/index.php?topic=4381.msg24510#msg24510
Title: Re: Gmail - Deleted Folder Sync
Post by: Davey126 on February 13, 2016, 09:45:59 pm
@Davey

Where are u living? Do you have an issue with Wi-Fi or mobile data bandwidth? And which device are you using?

In Paris, I've got unlimited mobile date (50 GB with 4G+ speed), and I'm using Note 5 (Android 5.1) and Nexus 6, + some emulated tablets on Genymotion/Linux and Genymotion/Windows.
Northeast USA. Bi-directional mobile data (LTE) hovers in the low 20 Mbps range (a bit slower in Canada); wifi around double that down; 5 Mbps up. Those are extraordinary speeds for the US which lags most of the world. Devices include Moto X and a variety of Fire tablets (great hardware) with custom roms.

The extended sync times I am reporting are largely unrelated to network speed. It is an issue with bulk operations which (I believe) are performed one at a time and involve a lot of overhead. For normal operations AquaMail performs quite well oven completing refreshes involving dozens of items in under a second according to in-app toasts.

As AquaMail is multi-threaded long syncs do not impact productivity. Culling a few hundred items from the deleted folder sets the sync icon spinning but I can immediately go to another folder and peform operations with little/no lag while the sync completes ... even on lower end devices.

Now you may be saying to yourself "everything works - what is this guys problem??". I never claimed a productivity hit largely due to the excellent design of AquaMail. It's an efficiency/optimization issue. Probably gets noticed due to my warped background in control systems where every cycle and byte counts. Obviously a very first world concern.

Again - I appreciate your interest. Probably time to let this one go and focus on more productive ventures.

Edit: removed extraneous detail


Title: Re: Gmail - Deleted Folder Sync
Post by: Kostya Vasilyev on February 13, 2016, 10:02:25 pm
Quote
The extended sync times I am reporting are largely unrelated to network speed. It is an issue with bulk operations which (I believe) are performed one at a time and involve a lot of overhead.

Yes, messages are deleted / purged one by one, so you're seeing Gmail's latency (as LTE itself has nice low latency compared to even HSDPA).