It is more difficult to hold the attention of many people as compared to one or two, because two parallel conversations may start. Further, the one or two people you meet regularly are likely to be closer to you and you would feel less judged by them. However, this can be managed by social skills training. In this you work on your own thoughts and beliefs regarding your social skills, whether you are too harsh on yourself, and along with working on the social skills, learning how to begin a topic, continue the conversation, use your voice and pauses, reading facial expressions and body language, and so on. With command over both these aspects, your skills will improve and people will listen to you.