Yes, permanent magnets can degrade.

Permanent magnets naturally degrade, and utilizing them in this fashion could very well cause them to degrade faster. To what extent and what the service life may actually be, has yet to be determined.