Its origin is, however, likely to date to the Middle Ages when the knowledge of Greek was declining among monks charged with copying manuscripts where it sometimes appears in the margin as Graecum est; non legitur -- it is Greek, [therefore] it cannot be read.
What's interesting is that the same phrase appears in many languages around the world albeit with reference to a different confusing language. For example, the German's say Das kommt mir Spanisch vor and so clearly find Spanish more confusing than Greek. Here is an interesting diagram showing how languages are related via their versions of this phrase.
Interestingly Chinese seems to be the most confusing language there is -- given it's entirely different character set that doesn't really surprise me!