A particle damper (PD) is a device that can attenuate mechanical vibrations thanks to the dissipative collisions between grains contained in a cavity attached to the vibrating structure. It has been recently suggested that, under working conditions in which the damping is optimal, the PD has a universal response in the sense that the specific dissipative properties of the grains cease to be important for the design of the device. We present evidence from simulations of PDs containing grains of different sizes, shapes and restitution coefficients, that the universal response is also valid when fragmentation of the grains occurs (generally due to intensive operation of the PD). In contrast, the welding of grains (caused by operation under high temperatures) can take the PD out of the universal response and deteriorate the attenuation. Interestingly, we observed that even at working conditions off the optimal damping, the shape of the grains remains unimportant for the response of the PD.