Note "Agonising calamity" doesn't work

As the title says, this note doesn't work. My herald of plague is turned on and it doesn't give me 40% chaos damage. When i turn off the herald the damage stays the same. I have this from megalomaniac, but i think it doesn't work when normally obtained either.