Was I right to end this "emotional affair?"?

Yes, you were right to end it. Flee from all manner of temptation. And if you feel in your heart that there is something not right there, then pay attention to that. The spirit speaks to you that way.