Stored phosphorus pentoxide is going to be covered with polyphosphoric acid like a skin. If this "skin" is removed with a pair of tweezers you will again find a proper compond. "The-more-the-better"-user will often produce this kind of problem in desiccators, where a half pound of phosphorus pentoxid used as drying agent is soon without any effect, since it is separated from the scene by the "skin".
If you attempt to deactivate phosphorus pentoxide with water, you are penalised with lots of smoke, coughing, angry lab-neighbors and maybe also a broken glas container because phosphorus pentoxide reacts vigorously with water. It is better to leave the compond standing alone in contact with air and wait until it is hydrolysed by air moisture. If the powder has completely changed to a syrup, you may add riskless more water.
Batches which are completely decomposed are disposed in a separate bottle.