Sending email packets - proper timing

I have successfully built up a system to send emails and texts using the w5500. I notice from time to time that as I’m sending the proper packets out, I get timeouts (command gets lost or otherwise no response from server). I notice this most often on the RCPT TO: request. So, in all my packet requests, I built in 5 tries with a 3 second timeout window. That seems to work. Every so often with the RCPT TO: request, it takes 4 tries to get a response. I’ve never seen 5 or a complete timeout yet. My question is how long is the ‘proper’ timeout window? Do I need to wait 3 seconds for an ACK/answer? Or is that too long? Any other comments on ways others have handled timeouts is welcome.