There are a couple of reasons why people say The Passion of the Christ is anti-semitic:
1) Passion Plays have long been used to fuel anti-Semitic sentiment which directly lead to actual violence against Jews, going back to the Middle Ages. Vatican II actually set forth certain rules about the content of Passion Plays, namely that they can no longer expressly blame Jews for the death of Christ. And funny enough, the Catholic church that Gibson spent millions of dollars building does not recognize Vatican II OR the Pope. So for him to claim ignorance of the historical context of the Passion Play is in all likelihood an out and out lie, particularly considering that he grew up listening to his own father say things like Vatican II was a "Masonic plot backed by the Jews" and that the Holocaust obviously didn't happen because it would be impossible to dispose of that many bodies.
Hutton Gibson - Wikipedia, the free encyclopedia
2) The Gospels are actually pretty sketchy when it comes to details of Christ's crucifixion. There certainly aren't enough to fill a whole play or two hour movie. Gibson used the writings of a crazy-ass 18th century nun to flesh out the screenplay -- St. Anne Catherine Emmerich, whose diaries say things such as "Jews in our country and elsewhere strangled Christian children and used their blood for all sorts of suspicious and diabolical practices."
Anne Catherine Emmerich - Wikipedia, the free encyclopedia
He also diverted from what actually is in the Bible in significant ways, such as portraying Pontius Pilate as a pussy who was completely browbeat by the evil Jewish priests, when in actuality he was a fucking sadistic murderer who crucified thousands of Jews. So basically when he argues that "it's in the Bible, dude, what can I say?" he is again out and out lying.
This is also beside the point. The Bible also speaks approvingly of slavery. Does that make it okay because it's in the Bible? These two things are not mutually exclusive. The movie can portray what's written in the Bible and still be anti-semitic at the same time.
3) The last disturbing thing about this movie is the reaction to it in the Muslim world. Everyone knows it's a big no-no to portray a Prophet. We know this because there have been actual riots. Movies such as The Prince of Egypt are banned because they portray the prophet Moses. According to Islam, Jesus was not the son of God but a prophet, and depicting him in any way *should* be a grave insult to Muslims. But for some reason, the Muslims fucking LOVED THIS MOVIE. And you'd see newspaper articles that quote Muslims coming out of the theater and saying things like "This shows the world the true face of the Jews." They could clearly see the anti-semitism in it, and they were willing to ignore the tenets of their own religion in order to gleefully enjoy the way it bashed Jews.
So in conclusion, was I surprised to hear the things that Gibson said during his DUI arrest? No fucking way. This guy has been an anti-Semitic piece of shit for a long time. Also, Keith is right: it's not up to him to decide when the world is "over it."