I have a VLTOR A5 buffer system in my AR. I noticed these divots on the buffer along the outside. I am guessing the buffer retaining pin did it, but how?

And how do I fix it. What part do I need to replace.

https://i.imgur.com/p1d9WkR.png