Can't apply Elemental Weakness as Infernalist after getting Bringer of Flame ascendancy node

Curse can be casted, but it does not apply onto anything. It didn't work at all, but I was trying different things and it started working with set 1, but still does not work with set 2 AT ALL. Removing points from Bringer of Flame does not fix the issue, but reapplying points into it breaks set 1 as well. At some point set 1 starts working again, but set 2 remains broken.