It looks like something wrong with the module. But before thinking in this “faulty” direction, you must prove that the W5500 board is supplied properly. This is noted here:
the power supply voltage of +3.3V, please ensure that the current is not less than 200mA, voltage is continuous and stable +3.3V
I tend to agree with this statement - PHY may consume considerable current, when switching on or off (or between modes) it may consume even more. Do you have oscilloscope to check what is going on the power lines - in particular on AVCC_3V3 (PHY is being powered by analog power rail). Start looking from the module start, because the behavior you may observe with cable removals and insertions may happen in already kind of latch-up state, happening because there was no enough current on the first PHY start.
Or even simpler way - connect to the power supply proven to provide enough continuous current @ 3V3, for example 1A, and observe the difference.
