Do you have headsets with velour earpads? Check if there's visible lint or that dandruff thing might be dust stuck into the earpads? I encounter it with my Plantronics Gamecom 388. I often leave my room door open so dust can easily enter my room or when it accidentally drop into the floor. See the photo below :(
So basically, all you need is a TAPE (if you do not have a lint roller - with the adhesive side), this one would work out too! Lint Remover doesn't work since it just spread the lint all over the material. Lint remover is different from the lint roller with the adhesive side.