Dan Gasby
, the husband of famed restaurateur B. Smith, believes race plays a role in the backlash he received after revealing his relationship with another woman while caring for his ailing wife.

Gasby, 64, revealed in December that he and Alex Lerner, 53, who is white, were in a romantic relationship and caring for his wife who has Alzheimer’s disease.

News of the peculiar dynamic sparked outrage from B. Smith fans.

Dan and Alex sat down with Al Roker in a new interview for the TODAY show and he blamed his lover’s race for the backlash and death threats he has received.

“The 800-pound gorilla in this situation is she’s white. I’m not supposed to be conscripted to somebody because of their race,” he tells Roker.

When asked why he felt to need to go public with the relationship, Gasby said: “Because that’s what Barbara [B. Smith] asked me to do, to talk about it. It spun out of control to ‘I’m having an affair. This woman, because of who she is, is taking B’s money. We’re abusing B. B wouldn’t want this.’ These people have never even talked to B.”

Gasby also appeared on The View with Lerner in the audience to talk to Sonny Hostin, who knew both Gasby and Smith before Smith’s decline into Alzheimer’s. Hostin’s family took  care of her grandmother when her grandmother was suffering from Alzheimer’s and she has been critical of Gasby.