B. Smith’s husband, Dan Gasby, has responded to backlash he received after revealing that he moved his girlfriend into his shared home with Smith as the restaurateur battles Alzheimer’s disease.

During an appearance on Today, Gasby explained that he and Smith discussed the future of their marriage before her illness worsened.

“She and I talked about what she wanted. She said if she were incapacitated, she wanted me to care for her but also have my own life,” he said. “I am doing what my wife asked me to do: Living my life. I can still care for B. and also have happiness thanks to Alex.”

Alex Lerner is Gasby’s girlfriend, who now lives with and helps him as he takes care of his ailing wife. Lerner is also White, which Gasby believes is the main motivation behind the backlash he’s received.

 “The 800-pound gorilla in this situation is she's White. I’m not supposed to be conscripted to somebody because of their race,” he said.

Dana Gasby, Gasby’s daughter from a previous relationship, also weighed in on the criticism, saying her father has the right to be happy.

“I mean, why shouldn't my dad have companionship? Why shouldn't he have someone to talk to? For the first few years, it was just me and him. And it's hard. I'm not his wife, I'm his daughter. I can't give him the support that a girlfriend, a friend, you know, a companion can give,” she said. “I am beyond happy for them [her dad and Lerner]. I'm more pissed at the people complaining about them.”