Attachments are cached on "large built-in storage" (not the "real" memory card).
You can choose the "real" memory card as the place where they get moved / copied when you long press or choose Save.
However, choose as you might, it won't work on Android 4.4, because of decisions made by Google, as I explained above.
It's true that hiding messages keeps their text -- however, something has to stay behind to that the app doesn't download same messages over and over again. All mail apps have to do something like this, it's just basic logic.
Cached attachments "owned" by hidden messages will be deleted automatically (every night at about 3am).