Accepting pardons as an acknowledgement of guilt?

 I keep seeing this argument, that accepting a pardon means acknowledging guilt.  I keep seeing others arguing that it does not.

What does the Department of Justice and the court system have to say about that?

Let's start with the United States District Court for the District of Columbia, in reference to a J-6 defendant.  The court states, in USA vs Dova Winegheart that:

The defendant would first have to accept the pardon, which necessitates a confession of guilt.

The court cites Burdick v. United States, 236 U.S. 79, 94, (1915). A few quotes from the case:

There are substantial differences between legislative immunity and a pardon; the latter carries an imputation of guilt and acceptance of a confession of it, while the former is noncommittal, and tantamount to silence of the witness.

In Burdick the court cites Wilson, in which is stated

confession of guilt implied in the acceptance of a pardon may be rejected, preferring to be the victim of the law rather than its acknowledged transgressor, preferring death even to such certain infamy. This, at least theoretically, is a right, and a right is often best tested in its extreme. "It may be supposed," the Court said in United States v. Wilson,

Burdick contended that:

 It is hence contended by Burdick that the pardon is illegal for the absence of specification, not reciting the offenses upon which it is intended to operate -- worthless therefore as immunity. To support the contention, cases are cited. It is asserted besides that the pardon is void as being outside of the power of the President under the Constitution of the United States because it was issued before accusation or conviction or admission of an offense. This, it is insisted, is precluded by the constitutional provision which gives power only "to grant reprieves and pardons for offenses against the United States," and it is argued, in effect, that not in the imagination or purpose of executive magistracy can an "offense against the United States" be established, but only by the confession of the offending individual or the judgment of the judicial tribunals. 

Justice McKenna, writing the opinion of the court for Burdick then summarized (using the language of Wilson):

This brings us to the differences between legislative immunity and a pardon. They are substantial. The latter carries an imputation of guilt; acceptance a confession of it. The former has no such imputation or confession. It is tantamount to the silence of the witness. It is noncommittal. It is the unobtrusive act of the law given protection against a sinister use of his testimony, not like a pardon, requiring him to confess his guilt in order to avoid a conviction of it.

I'm not a lawyer and there may be future cases that overturned this, but for what it's worth, here's some information.