Note that this 1895 book (first edition, 1891) by an admirer of Douglass includes Douglass' harshest remarks on religion:
Holland, Frederic May. Frederick Douglass: The Colored Orator, revised edition. New York; London; Toronto: Funk & Wagnalls Company, 1895. (American Reformers, edited by Carlos Martyn) See Chapter XIII: Marshal and Recorder.
I don't have any further information about Frederic May Holland (1836-1908), except that he also authored books on Robert Browning, the Stoics, the French Revolution, the revolutions of 1688 and 1776, and the history of freedom and liberty.