Persistent buffs disabling upon entering new zone

Character name: Miomew

Basically every time I enter a new zone, 1-2 of my persistent buffs become disabled. When I use weapon swap, they reactivate again and stay active as long as I remain in the same zone.

Teleporting back to hideout, or entering abyssal depth also causes this bug.

It is not because I lack spirit, and not because of weapon swapping itself, because once I enter a new zone and I activate the buffs, they remain active no matter how many times I weapon swap after that as long as I remain in the same zone. Additionally, my spirit comes only from my armor and necklace, so my weapons have nothing to do with it.

I am running with 4 persistent buffs, and even if I remove 1 of of my persistent buff gems so I have an excess amount of spirit, the buf still occurs.

I also discovered that the position of the gems in my skill tree affects which buffs disable. The ones positioned on top remain active, while the 2 positioned below disable.

And yes, I have enough spirit to run all 4 persistent buffs, if I didn't, I wouldn't be able to activate all 4 of them at once in the first place. Like I said, once I do a quick weapon swap, they all activate and all work normally. It's just upon entering a new zone that 2 of them disable.