You say it is a "one-church town," but you also speak of a person not wanting to invite someone to his own congregation. Is the town truly a "one-church" town, or did you mean that it has only one church of a particular brand or variety? Are there any other nearby towns or churches besides your own? If your town has churches from other Christian traditions or denominations than your own, you might wish to get acquainted with those. You will never find a perfect church under any label. But you might well find a group of believers who preach the gospel, love Jesus and care for each other, and that is certainly not a bad place to begin!
I hope you are not trapped in the mentality which supposes that your own church group offers the only possible option. Sadly, many unhappy believers who were taught that fallacy continue to languish in places where Jesus is not proclaimed and exalted, surrounded by spiritual bondage, devoid of godly encouragement -- thinking all the while that they are in a "true church" which they dare not leave without risking their salvation. There might be some good reason for continuing in such a place, but that is not it.
For more on the true church, click here.