Writer and economic commentator Stephen Moore defended a bigoted comment he made about former President Barack Obama during a recent appearance on PBS’ Firing Line With Margaret Hoover.

In 2016, Moore made a “joke” about the Obamas leaving the White House after Donald Trump was elected into office.

“By the way, did you see, there’s that great cartoon going along?” he asked while giving a speech on health care. “A New Your Times headline: ‘First Thing Donald Trump Does As President Is Kick a Black Family Out of Public Housing.’ And it has Obama leaving the White House. I mean, I just love that one. Just a great one.”

In a preview clip for the show, Hoover asked Moore, who is Trump’s nominee for the Federal Reserve Board, about the seemingly insensitive statement.

“So, you know, that, that is a joke that I always made about, you know, Obama lives in, you know the president lives in public housing,” he awkwardly clarified. “But I didn’t mean it like a Black person did.”

Moore said the comment was one he’d heard among his African-American colleagues.

“When I was working with a lot of women in families who were involved in the education voucher program here in D.C., and people would say—these were Black who would say—‘Why does Barack Obama get to send his kids to any school that he wants to and we can’t?’ And they’d say, ‘He lives in public housing,’ and it was just kind of a joke, and that was referring,” he explained.

Moore, 59, conceded that he “shouldn’t have said it” after Hoover made reference to the “optics” of “a conservative white man talking about kicking a black man out of public housing have a particular resonance, especially in this moment.”

The 59-year-old received backlash for other offensive comments, including derogatory remarks he wrote about women in his National Review columns in the 2000s.

“Again, you can, you go back 30 years, you’re going to be able to find clips over and over and over again about me,” he said in conclusion. “I have a long paper trail. I mean there’s no question about it. And I say things that are kind of jokes that if people want to pick them apart, then I probably won’t, you know, get on the Federal Reserve Board.”