The best way I have found to AID and Visit the Tavern of Players is to first AID those who have Aided you as recorded in the Event History (shows a week of events).
But, if I see a player not Aiding me for a very long time (a Month or so) I may choose to stop Aiding and maybe delete them from my friend's list to make room for someone who will AID me.
I understand that many people do not or cannot AID every day for various reasons or are new to the game, so I try to cut them some slack. You can select to AID and visit the Tavern of everyone in your Guild and/or Friends regardless of whether they reciprocate or not but I choose to take note of the ones that are not Aiding me over long periods of time (about a month). I do not keep track of those who visit my Tavern because I think it is a little over the top and tedious.