"You have no right to free speech on my blog!"
True. Bloggers have no obligation to host disagreement, no obligation to endure trolls, no duty to permit "Yes, but..." arguments. Many (e.g. Ania Sarkeesian) justifiably don't allow comments on their blogs or youtube channels because they have been subjected to organized campaigns of hate and threats of violence.
However, silencing comments is not always about harassers claiming to want "free speech". As often as not, silencing comments is done by writers who are unwilling to be fact checked. They make up things as they go, state their opinions as "fact", and then claim "offense" when someone actually takes the time to do the research and cite a source (e.g. an anti-vaxxer deleting a comment which factually reports that Andrew Wakefield is not a doctor).
If you are so full of yourself that you believe you are unquestionable and do not want your opinions challenged, then feel free to build yourself an echo chamber. You could even pretend to allow comments. Create your own sock puppet accounts to give the illusion of readership and popularity. It might even make you feel good about yourself.
Being fact checked is how you learn, and people willing to learn are not afraid of it. Comments are a community, and it shows what sort of readership you have. When you have no comments, you have no readers. It is not just the fact that no one can post, but also the fact that no one would want to read someone who is afraid of feedback or correction. Chances are, if you can not handle valid criticism, your opinions are likely not worth reading. And people shold stop reading you.