The transcript of the Pope's speech at Westminster Hall can be found on the BBC website here. He was arguing for the need for a society to continue to have a role for expressions of faith within public dialogue - that the relegation of religion to a purely private sphere is intolerant, and weakens society.
Obviously, he did not name names. However, within secularism, Stephen Gould (in "Rocks of Ages") argued precisely that the "magisterium" of faith should be private, in contrast to science and reason-based knowledge, which should be the basis of public discourse. Inevitably, he did little in the way of epistemology to justify this position. Richard Dawkins goes further in his abhorrence of faith and religion - lumping all religions together with superstitions and anti-science, declaring them to be mental viruses whose influence is entirely parasitic upon society. There are other oft-quoted writers who argue that religion is required for good people to do bad things.
You have to be fairly dishonest (or ignorant) about history to cast out the beneficial influence of some parts of Christianity - though it would be fair to point out that Benedict didn't mention the Crusades or the Inquisition. Whilst I have some sympathy with the content of his speech, and the need for religious conscience to have an impact on public discourse, I would disagree with him when it comes to his suggestion that this discussion will in part be between secular authorities and the "Holy See" as he refers to it. I would suggest that Benedict is carrying out his own form of revisionism in claiming the benefits of dialogue of conscience as being derived in part from the Catholic Church.
The Catholic Church frankly does not have a good track record in this regard. Christian conscience, in biblical terms, finds expression primarily in the individual voice reacting to secular powers. The Catholic Church operates as an alternative power (as, for that matter, do many religious organisations - cults, established churches, theocracies, even Calvin's government in Geneva), and it's the defence of that power and the attempt to reinforce that power that has led and continues to lead to abuses.
I believe people must be allowed freedom of conscience - this is a freedom that people over centuries have sacrificed their lives for, and is probably one of the most important marks of a civilised society. But I don't believe this means that institutions (including religious institutions) have the freedom to operate as alternatives to society. It was the sense that Catholicism was supposedly answerable to higher values that, in my opinion, allowed such things as the Crusades, the Inquisition, the abuses that Luther was reacting against at the time of the Reformation, and the abuse of cared-for children. There should be freedom of conscience, and a place for dialogue upon that basis, yes - but between the society and individuals, not between secular powers and religious powers.