Yes, hysteresis does affect modifiers. The hysteresis kicks in when the modifier is active so that, in general, the set point to deactivate the modifier is:
Original modifier value + hysteresis (for ‘is less than value’ modifiers)
Original modifier value – hysteresis (for ‘is greater than value’ modifiers).