That is just speculation .. it may be true, but it may be that Holgate did hear something. There was a point, just before he reacted, that Firmino's moth was not visible on camera, and it could have been then that something was said. I'd like to believe that he did not say anything untoward and Holgate misheard, or even made it up, but no one can categorically state that.
TBH if people who claim racism are suddenly bombarded with criticism & claims they are doing it to just divert attention (and, worse still, a trawl of their twitter history to find something inappropriate they said when they were a kid), then it serves as a deterrent to future potential victims to report racist abuse (like Brewster). Yes, its possible some unscrupulous people will use the racism card when not warranted, but I think the lesser of two evils is to give the supposed "victims" the benefit of the doubt. That doesn't mean we accept their story without question, but it does mean that we shouldn't be accusing & abusing them, when we do not, and cannot, know the full truth.
If an investigation finds categorically that Holgate lied to try & cover himself, then fine - go to town on him, but until then hold fire.