Word. It's a buildup of dead skin cells. You need to exfoliate that shiet. Guys who shave a lot don't often have this problem cause they shave it off without knowing it. But for people who don't, such as myself, use St Ives Apricot Scrub. Can be found at most supermarkets like Albertsons, Safeway, etc. Skincare area. Yeah, sounds borderline homo, but try it. You only need to use it once every few days. You watch, no more white flakes on your face. Thank me later.
Oh, as was said, using less products that dry out your face will help also. So switch soaps if need be. Also, drink more water to keep your skin hydrated. That alone will help quite a bit.