When settings are applied RTR ?

Tell me, are the settings related to the TCP connection settings (RTR, RCR) applied immediately after writing the register or do you need to reboot the wiznet?
I assume they should be applied after opening a new socket.

No definitive information about it, but per documentation RTR/RCR are part of the basic settings and span across all the sockets. You should not need to “reboot” the chip to change basic configuration. I suspect values are being used when they need to be used (next timeout needs to be calculated).