Discourses of the Prophet Joseph Smith, pg. 226; “O ye Twelve and all Saints! profit by this important Key—that in all your trials, troubles, temptations, afflictions, bonds, imprisonments and death, see to it, that you do not betray heaven; that you do not betray Jesus Christ; that you do not betray the brethren; that you do not betray the revelations of God, whether in the Bible, Book of Mormon, or Doctrine and Covenants, or any other that ever was or ever will be given and revealed unto man in this world or that which is to come. Yea, in all your kicking and flounderings, see to it that you do not this thing, lest innocent blood be found upon your skirts, and you go down to hell. All other sins are not to be compared to sinning against the Holy Ghost, and proving a traitor to the brethren. (HC 3:385.)”
Psalm 1:1-2; “Blessed is the man that walketh not in the counsel of the ungodly, nor standeth in the way of sinners, nor sitteth in the seat of the scornful. 2 But his delight is in the law of the LORD; and in his law doth he meditate day and night.”
Nice to see that heaven and Jesus Christ is equated to “the brethren”. Now you can see why people are afraid to leave.