As a podcast listener I can tell you that my most frequently-used…um…usage? of chapters is not to skip things, but if I want to go back later and refer to something or find something that they talked about.
Most podcasts don’t have transcripts, and trying to remember “What episodes did they talk about such-and-so?” is painfully difficult for me. Even if you have show-notes (which you 100% should have), chapters will help me get to that point in the show notes. Having show-notes with chapter references and time-stamps are the best of all options.