What to do, what to do

Today was very productive in a: "you're dead fucking wrong sort of way." I told my guy that I've been eyeing for a while exactly what I've been thinking about him.
And get this, he says he felt the exact same way!
Which is weird because he never let on. But he said the same about me. And you know I feel so much better getting it all out there and leaving us to do whatever we decide. I'm glad that it's not just me, having these thoughts and then trying to battle with myself whether or not to act on them, suppress them, or what you know? I feel better that it's out in the open, at least THAT part is "out there" and done.
I, however, am not so sure if I have the balls to go through with the physical part.
(1) being my DH, how could I? And
(2) I'm a little self-conscience about my body since I've had my kids.
So what's a lady to do? All I know is that since our conversation I'm all jittery, happy, and tingly! Feels good, but weird, and happy but in a very sad sort of way.
I'm becoming the men and women I despise on television and talk-shows, soaps, real-life, etc. A cheat. Cause honestly I've already cheated in a sense, I let on, I lurd him and of course what man, with nothing to lose would turn down a ready and willing woman? Not this guy certainly!
Parts of me really really wants to go through with this, and the other part wants to just want it and that's all. Wonder, but never actually really act. Because acting turns me into a cheater, a threat to my marriage, my family, and my life as I know it.
I keep hinting at my husband that I'm too lonely sometimes to be considered a married woman, he of course doesn't take the hints, but he's so sweet about the whole thing. Wants to know what he can do to help me if anything, etc. etc. But in all honesty I don't think this irkling I have will go away until it's been satisfied in some way.
I don't know what it is about him, because in the appearance catergory he's not my type at all, I'm not attracted to him for this reason, but for reasons that lie far deeper than something as vain as beauty. I'm so enamoured with his personality, his intellect, he speech, his way of being. It all just really turns me on. And I've never had this happen before, never have I by just having a mere conversation with a person on many different levels made me want to "jump his bones" directly afterwards. But I do, I do. I think the sex would be great! I can almost feel it you know, it's weird.
I KNOW! It's weird.
But I'm scared...scared that I will actually end up doing this, end up liking it, possibly even loving it. And that just scares me more, the thought of being semi-addicted to some other man's dick, is really scary. Not to mention getting caught, or caught-up, catching feelings. So many what-ifs.
I don't know what to expect from him. And that's the weird part about it, he tells me about all his "sexcapades" and how he is towards woman and I see the way he works them over and thinks towards them: and I STILL want it? Is that not crazy? Is that not insane? I don't want to be dogged, even if I do just want a "fuck-buddy" per say. I'm so confused. I think this is far more complicated than just simply fucking here and there and moving on. It takes planning and percision, and a lack of a conscious. And I'm not sure If have all those traits. I don't want to turn my working environment, that's been pretty mellow, into something totally weird and crazy, but I do want to explore these thoughts I've been having about him.
I talked to my best-friend about it yesterday and she said something that really stuck out to me: "After sex everything is different, either in a good way or a bad way, and it's not certain that you'll both feel the same way about it"