Finally found the passage I had been looking for...here:
**
...But, to sum it all up: remember that the door has been thrown open. Do not become a greater coward than the children, but just as they say, "I won't play any longer," when the thing does not please them, so do you also, when things seem to you to have reached that stage, merely say, "I won't play any longer," and take your departure; but if you stay, stop lamenting.
..............................
How long, then, should we obey such commands? As long as it is beneficial, and that means, as long as I preserve what is becoming and consistent. Further, some men are unduly crabbed and have too sharp tongues and say, "I cannot dine at this fellow's house, where I have to put up with his telling every day how we fought in Moesia: 'I have told you, brother, how I climbed up to the crest of the hill; well now, I begin to be besieged again.'" But another says, "I would rather dine and hear him babble all he pleases."
And it is for you to compare these estimates; only do nothing as one burdened, or afflicted, or thinking that he is in a wretched plight; for no one forces you to do this. Has some one made a smoke in the house? If he has made a moderate amount of smoke I shall stay; if too much, I go outside. For one ought to remember always and hold fast to this, that the door stands open.
But some one says, "Do not dwell in Nicopolis." I agree not to dwell there. "Nor in Rome." I agree not to dwell in Rome, either. "Dwell in Gyara." But to dwell in Gyara seems to me to be like a great quantity of smoke in the house. I leave for a place where no one will prevent me from dwelling; for that dwelling-place stands open to every man.