He wasn't intimidated, it was more everybody he cared about tended to leave one way or another. Sometimes because he drove them away. Furthermore, he still felt he didn't deserve good things in life, or that the usual Winchester bad luck would rear its head sometime or another. Which it did in a fashion with the Darksoul possessions. That was just too much like crap from his world for his preference.

"Trust me? If you had my kind of luck? You'd worry."

He then added about Jayne, "Yeah thanks."