W5300 MACRAW Rx FIFO overflow condition

Hello,

I have been using W5300 in MACRAW mode for receiving Ethernet packets and it was working fine. But now I have found that during sudden inrush of data in the network the FIFO overflows. FIFO overflow is acceptable but now I have to recover from it.

I’m referring to w5300 datasheet(version 1.2.8), quoting it below for ease of reference
“In case that the free size of internal RX memory is smaller than 1528 - Default MTU(1514)+PACKET-INFO(2)+DATA packet(8)+CRC(4) - close SOCKET 0. After closing the SOCKET0, Process all received MACRAW data and then reopen the SOCKET0.”

  1. In the code snippet, packets are received, and while receiving packet after packet, the received size is reduced "recved_size = recved_size – 2 – pack_size – 4; "

From my understanding the RX FIFO size includes the dummy byte (if odd bytes are received) and hence it is never odd. But the length specified in packet info does not include this dummy byte. So shouldn’t an additional byte be deducted from the rx fifo count if packet count is odd?
recved_size = recved_size – 2 – pack_size – 4 - 1(if received odd packet);

  1. Are all the packets received after closing the FIFO valid ones?

  2. Also, could I just close and reopen the socket and skip the packet reception part if I’m not interested in processing the received packets after overflow?

Thank you!

Hi,

You seem to understand it properly about using MACRAW mode.

  1. Sn_RX_RSR is always even number regardless of whether an even or odd number of the real packet lenght.
    In case that the real packet lenght is odd, the valid received size is equal to the value of packet_info header, the dummy one byte to be read from Sn_RX_FIFOR should be ignored. In 8bit data bus access, The dummy data stroed in Sn_RX_FIFO should be read by host and ignored it.

  2. As I known, Socket memory data is still safety after closing socket.
    I will test about that again, Notify the result to you.

  3. Yes. If you don’t need any more data, Just close and reopen.
    Notice that W5300 can’t not support flushing as many as the wanted size. For flushing, You should be read to Sn_RX_FIFOR as manay as the wanted sized.

I did test to valid data in memory after closing socket.
The remained data is valid.

The below is my test sequence.

  1. Socket open with mac raw.
  2. wait unitl Sn_RX_RSR > 4096
  3. close the socket.
  4. loop until Sn_RX_RSR > 0
    4.1. read data from the memory
  1. So per my first question and your response to that, the data sheet "recved_size = recved_size – 2 – pack_size – 4; " is wrong.

Instead recved_size = recved_size – 2 – pack_size – 4 - 1(if received odd packet); should be done. Correct?

  1. In your testing, what’s 4096? I guess I’ll rephrase my question! All my questions are on the condition when an overflow happens. So, “On overflow condition, after closing the FIFO, are all the packet read from the FIFO valid?” Can you test it by letting the FIFO overflow(REMAINING SPACE IN FIFO < (ALLOCATED FIFO SIZE - 1528)) and also add packets with odd bytes?

  2. When you say W5300 doesn’t support flushing, you mean closing and reopening the socket will not clear(reset) and thus empty it’s FIFO’s? Which means the old data will be present even after closing and reopening the socket and I have to read the Sn_RX_FIFOR before starting the next reception?

I appreciate your response. Thanks!

Hi,

1> You mis-understand to my answer.
I did mean that
“Sn_RX_RSR” is the size of all received packet included even or odd packet size. Sn_RX_RSR shoud be aligned by 16bits, that is, even size. Also, all dummy data size and CRC 4byte included in it.
You did mean that
“PACKET_INFO” header( included in every the received packet) have only the real packet size excluded dummy data size and CRC size(4).

2> I just did show it for example. Don’t care the size 4096. In overflow case and closing socket, the remained data is valid in socket buffer.

3> Flushing as I said, means the socket buffer can’t be empty without reopen but can be empty with only reading Sn_FIFOR as the wanted size.
As you say, when the socket is closed and reopen, the socket buffer will be empty.

  1. The following is from page 115 in w5300 datasheet(version 1.2.8)

/* calculate the size of remained data in internal RX memory*/
recved_size = recved_size – 2 – pack_size – 4;

Let’s say two odd packets were received back to back. 101 + 101.
Now the Sn_RX_RSR would be {2(packet info) + 101(data) + 1(dummy data) + 4 (crc) = 108} * 2 = 216

From the math, recved_size = recved_size – 2 – pack_size – 4; After processing the first packet, I’ll end up with recevied_size = 216 - 2 - 101 - 4 = 109. The received count should be 108.

I would subtract (read_cnt * 2) instead of pack_size in that equation.

It doesn’t make sense to me, perhaps I’m missing something here.

  1. If nothing gets corrupted, I don’t understand why the user should close and reopen the socket?
    Quote from page 114

"Notice: In case that free buffer size of internal RX memory is smaller than the size of receiving
MAC RAW data, some parts of un-acceptable PACKET-INFO and DATA packet of the
MACRAW data can be saved in internal RX memory. This can cause the error in analyzing
PACKET-INFO (as shown in above code), and receiving correct MACRAW data. This
problem is more likely to happen when internal RX memory gets close full. This can be
solved by ignoring some loss of MACRAW data. "

Could you explain the above statement?

  1. Since closing and reopening would empty the socket buffer and make Sn_RX_RSR = 0. I’ll give that a try and see if it works.

[quote=“lr15”]1. The following is from page 115 in w5300 datasheet(version 1.2.8)

/* calculate the size of remained data in internal RX memory*/
recved_size = recved_size – 2 – pack_size – 4;

Let’s say an odd packet was received - 101 bytes. Now the Sn_RX_RSR would be
2(packet info) + 101(data) + 1(dummy data) + 4 (crc) = 108}

From the math, recved_size = recved_size – 2 – pack_size – 4; After processing the first packet, I’ll end up with recevied_size = 216 - 2 - 101 - 4 = 109. The received count should be 108.

I would subtract (read_cnt * 2) instead of pack_size in that equation.

It doesn’t make sense to me, perhaps I’m missing something here.

  1. If nothing gets corrupted, I don’t understand why the user should close and reopen the socket?
    Quote from page 114

"Notice: In case that free buffer size of internal RX memory is smaller than the size of receiving
MAC RAW data, some parts of un-acceptable PACKET-INFO and DATA packet of the
MACRAW data can be saved in internal RX memory. This can cause the error in analyzing
PACKET-INFO (as shown in above code), and receiving correct MACRAW data. This
problem is more likely to happen when internal RX memory gets close full. This can be
solved by ignoring some loss of MACRAW data. "

Could you explain the above statement?

  1. Since closing and reopening would empty the socket buffer and make Sn_RX_RSR = 0. I’ll give that a try and see if it works.[/quote]

Hi,

1> I did mistake to understand. I’m sorry about that. I just did know that your question is what is ‘recved_size’.
In the code, The caculation of recved_size is incorrect.
The code shoud be must be as the follows.

if(pack_size & 0x01)     // if pack_size is odd,
         recved_size = recved_size – 2 – (pack_size +1) – 4;
else                           // if pack_size is even.
         recved_size = recved_size - 2 - pack_size - 4;

The errata code should be modify with your help as soon as possible.

2> W5300’s MACRAW socket have some problem.
In MACRAW mode, When without processing overflows of buffers, buffers will be very barely corrupted becase the packet not to be received can be received by unknown reason.
So, We recommend the code as you know.
The code prevented from the corrupt of buffer by closing socket when buffer is almost full.

Thank very much.

Okay, I did some testing last couple days to handle the rx FIFO overflow condition in MACRAW mode.

  1. When overflow happens, just closing and reopening the socket leads to bad packet info(the length of the received packet is more than 1514). I was expecting everything to be reset and just work fine but it is not. I guess this should be easily reproducible in your set up too.

  2. Since closing and reopening failed, I decided to read the RX FIFO out as per the code in w5300 datasheet. This also gives bad packet info. Analyzing more into this, I see that the S0_RX_FIFOR shows one value before closing the socket and shows another after closing the socket. So I believe the S0_RX_FIFOR after closing the socket must be used to process the data.

Seems like some irrelevant data(perhaps from previous packet) is put in the rx fifo when overflow occurs. Even after closing the socket? Should I wait for sometime before reopening the socket/ after reopening the socket?

I’ll do more testing and post results in a few days. I’ll appreciate if you can confirm whether my testing results makes sense to you and you can reproduce it.

Thanks!

Hi,

First, I’m sorry about my poor english write. I tried to explain the phenomenon that you had experience already. But, It isn’t easy to me.
Anyway, Your experience is all right and Sn_RX_FIFOR are valid after closing the socket.

It is the best solution that fast processes all data before overflow. But this solution depends on host performance and application complexity. This solution is not recommended.

So, If you accept to discard some packet, You can prevent from overflow. In addition, When you detected to over-sized packet (PACKET_INFO>1514), you will discard the packet (For discarding, close and re-open socket).

The below codes show to prevent overflow and discard packets.

   // Assume that you can process the MACRAW data by this function.
   void process_macraw_data(uint8* mac_data, uint32 len); 

   // main example
   void main(void)
   {
      int32 remained_rsr = 0;
      int32 rsr = 0;
      int32 recv_len = 0;
      uint8 data_buf[2048];

      while(1)
      {
         switch(getSn_SSR(0))
         {
            case SOCK_MACRAW:   
               rsr = getSn_RX_RSR(0);
               if(rsr > 0) 
               {  
                  // check the received size.
                  if(getIINCHIP_MaxRX(0) - rsr < 1528) // Abnormal <-- Fro preventing overflow.
                  {
                     remained_rsr = rsr;
                     close(s);
                     while(remained_rsr > 0)           // Process all received data.
                     {
                        recv_len = recvfrom(0,data_buf,remained_rsr,0,0);
                        remained_rsr -= (recv_len+1+2+4); // length of DATA packet size - 2bytes, length of CRC - 4 bytes
                        if(recv_len & 0x01) remained_rsr = -1; // if recv_len is odd
                        if(remained_rsr < 0) 
                        {
                           // Error. It maybe receive an un-acceptable MACRAW data and should be ignored that.
                          close(s);
                           break;
                        }
                        process_macraw_data(data_buf,recv_len);
                     }
                  }
                  else                                 // Normal
                  {  
                     recv_len = recvfrom(0,data_buf,remained_rsr,0,0);
                     process_macraw_data(data_buf,recv_len);
                  }
               }
               break;
            case SOCK_CLOSED:
               socket(0,Sn_MR_MACRAW,0x1000,Sn_MR_MF); // open or re-open the MACRAW SOCKET with MAC filter flag.
               break;
         } // end switch
      } // end while
   } // end main

If overflow doesn’t occur, the over-size PACKET_INFO does not occur.
For more safe, add dection of over-sized PACKET_INFO code to recvfrom function.

uint32   recvfrom(SOCKET s, uint8 * buf, uint32 len, uint8 * addr, uint16  *port)
{
   //
   //  omitted
   //
   if ( len > 0 )
   {
      switch (IINCHIP_READ(Sn_MR(s)) & 0x07)       // check the mode of s-th SOCKET
      {                                            // -----------------------------
         case Sn_MR_UDP :                          // UDP mode 
            //
            // omitted
            //
            break;
                                                   // -----------------------
         case Sn_MR_IPRAW :                        // IPRAW mode
            // 
            // omitted
            //
            break;                                 
                                                   // -----------------------
         case Sn_MR_MACRAW :                       // MACRAW mode
            wiz_read_buf(s,(uint8*)head,2);        // extract the PACKET-INFO
            if(*((vuint16*)MR) & MR_FS)            // check FIFO swap bit
               head[0] = ((((head[0] << 8 ) & 0xFF00)) | ((head[0] >> 8)& 0x00FF));
            data_len = (uint32)head[0];            // DATA packet length
            if(data_len > 1514)  // For dection over-sized PACKET_INFO
            {
                close(s);
                setSn_CR(s,Sn_CR_OPEN);
                while(getSn_CR(s));
                return 0;
            }
            wiz_read_buf(s, buf, data_len);        // data copy.
            wiz_read_buf(s,(uint8*)crc, 4);        // extract CRC data and ignore it.

            break;
         default :
            break;
      }
      setSn_CR(s,Sn_CR_RECV);                      // recv
   }
   #ifdef __DEF_IINCHIP_DBG__
      printf("recvfrom() end ..\r\n");
   #endif
   
   return data_len;   
}

Thanks.

Okay, from what you have said I understand that packet_info bad means overflow has happened already before I could check for the condition(<1528). Now the FIFO data is definitely bad and I need to close and reopen the socket.

So, I’m going to close and reopen the socket on two instances.

  1. Overflow is about to happen - When the remaining size of the FIFO < 1528
  2. Overflow has happened - When the packet_info is bad.

There is definitely going to be a loss of data on FIFO overflow on the second condition. This is mainly because I’m not able to read the data out faster from the RX FIFO. But, loosing the already received data is bad. I think this should be in the errata even if its mentioned in the datasheet.

I appreciate your quick response midnightcow!

When overflow happens, the FIFO gets corrupted and it is identified when the packet_info is bad. Detecting corruption of the FIFO is based on packet_info. The problem is, there is no guarantee that when the overflow happens the packet_info would be always greater than 1514 bytes. It is just random bytes from the stream. So it could also be lesser than 1514 but wrong(bad).

So the user could be reading the corrupted FIFO since the packet info is less than 1514. This will eventually lead to bad packet_info and the user will detect overflow. But the previous packets read could be bad. So when working in MACRAW mode, CRC must always be used to ensure the packet is good.

Hi,
Thanks for your sugguestion. We will apply it.

Thanks again.