Soundproof/Insulate fireplace/chimney.

Ownerofawreck

Hi, I live in a solid wall 1929 semi, with a nasty neighbour. I had acoustic boards fitted to the alcove walls either side of shared chimney, as due to the one brick thick party wall the chap didn't want to risk drilling. This hasn't really had much effect (neighbour stands next to wall shouting comments), it does seem as though a lot is coming through chimney breast. When chappie was blocking up fireplace he pulled out bags of old clothing which I now think was used as soundproofing.
Is there anything I can put into the fireplace and possibly chimney to try and block this creature off? I saw reviews of Rockwool but am concerned about damp although I have vents. Many thanks for any advise.

Update: Spoke to planning department and they said that when these houses were built there would have been a vent between the two chimneys to help with air flow and protect the houses in the event of a blockage. This is probably why I can hear the people next door so clearly. Decided to sell house.