Making possible to cancel a tag or folder from a multiple selection

I would like to be able to remove a selected label or folder from a multiple selection/filtering, without starting over a new selection. Currently there is no way to do so, so I have to remember the full selection sequence is I do include a label by mistake (or am I missing something?).

That’s currently not possible. If you change the filter (label, folder or any filter from the right hand panel) the selection will be reset.