The nature of the principles of universal grammar limits what can count as a natural language grammar. There is no language in which a suitably translated version of the sentence “Sock burger insofar loggerheads” counts as grammatical. Yet there have existed moralities in which cannibalism, ritual mutilation, slavery, and rape count as morally permissible. If the putative moral capacity can produce moralities that allow things that strike our judgment as monstrously immoral–if it doesn’t really rule anything out–it can’t account for the normativity of our judgments and the linguistic analogy fails.
My purely amateurish take is that native morality seems to exist and that it takes a form that is far more basic than what Will is describing here.
For example I propose this as an incorrect moral system:
It is morally right to cause great harm to Adam to prevent minor harm to Ben AND it is morally right to cause great harm to Ben to prevent minor harm to Chris AND it is morally right to cause great harm to Chris to prevent minor harm to Adam.
This system is incorrect because there is no way to assign moral weight to the harms on Adam, Ben and Chris such that this moral ordering makes sense.
In the language of mathematics our native moralism forces a partial ordering on the world of moral entities. Further a – perhaps the – relation in our partial ordering is harm.
For example, suppose we replaced “harm” in the scenario with “laughter.” Now the system is no longer morally incorrect.
Admittedly this is not well worked out but my sense is that the “moral entities” are key and that a partial ordering exists over them is important.