Because of sin government is necessary and because of sin government is ugly. Without government, the world would be even uglier than it is, but government does do its own part to add to the ugly.
Romans 13:1-2
Let every soul be subject unto the higher powers. For there is no power but of God: the powers that be are ordained of God. Whosoever therefore resisteth the power, resisteth the ordinance of God: and they that resist shall receive to themselves damnation.
It is good that powers exist. Image bearers were made to live in covenantal realities. People are molecular, not atomistic. We were made to live in relation to others, beginning first with parents and siblings, and then extending to church, neighbors, cities, counties, states, and nations. We organize ourselves into representative associations like marriage, churches, HOAs, school districts, and counties. These associations require covenants. Covenants require representatives. These things are not in themselves ugly realities, these are the beauty of God's design.
But because of sin, the reasons for these associations is often related to sin, e.g. law enforcement. And what Chesterton is pointing out is that the law enforcers are themselves lawbreakers. Those in charge of punishing sin are themselves sinners.