If you are viewing the smart folder, the bottom icon does the same as the top one - refresh all. If, however, you are viewing one account, the bottom refresh icon refreshes that account only.
Just tested it with mine, and that seems to be what happens. I have push turned on with one account out of three, but this does not change anything (or so it seems).