unstable simple web page

TCP/IP Chip W5500
1

Greetings,
I apologize for the long description and if my English is poor as it is not my native language.

I am new with W5500 and for the last month I was able to play with it using my ATmega32A by creating a very simple HTML page and view it with a browser.

My code is based on the io Library and some contributions on the internet. Everything was running fine but I noticed that some times the web page is not displayed when I refresh the page on the browser. However, when I refresh several times or when press CTL+F5 the page usually displayed correctly.

when trying to find out what is the problem, I found that sometimes W5500 miss sending part from the page and send the rest. For example, assume if my page is 500 bytes, then the browser misses the first 260 bytes and receive the rest, which of course a corrupted page. But when I click refresh two or three times it displayed completely.

After spending 3 weeks in investigation, I can confirm that the send() function receives the correct parameter which contain the required data to send, and as the page sometimes display correctly, then SPI code supposed to be working fine.

I used Windows network monitoring to capture data sent from W5500 to the browser, I found that the HTML data incomplete and have a part of it missing and the remaining part is sent.

Now I am sure it is not browser or HTML code problem. And I am sure the code is sending data correctly, but due the nature of SPI, I can not confirm if my PC received it completely.

• The problem is always with the beginning of the page. If the beginning of the page is received, then whole page will be received.
• I tried to simplify the code as much as I can, but the same behavior exists.
• I tried to send the HTML data in smaller parts (for example every part is 100 bytes) but same behavior.
• I tried to replace the W5500 with another module. The same.
• I am using ATmega32 configured with internal clock 8MHz, and SPI clock is Fosc/128
• I also tried to use external 16MHz crystal. But the same problem

Do you have thoughts may help me figuring out the problem.

Thank You

2

The first idea I got reading your situation is that you try to send data before W5500 is ready to do it (has socket properly initialized). Please post your code related to the W5500 control from the beginning (starting with preparations for socket open and ending at operation completion/socket close).

Edit: but your W5500 receives connection, not opens by itself… Anyway would be very useful to see and analyze how you sense incoming connection, and what you actually do.

3

Here is my code uploaded as zip file

  • The main() function initialize W500 and continuously execute httpServer_run() function
  • httpServer_run() function listen to the port, establish connect, receive data
  • httpServer_run() then parse the request through parse_http_request() to determine the requested method
  • parse_http_request() then handles the request according to the method using http_handler()
  • http_handler() send the response using html_send() which in turn calls NetSend()

The code is working as expected. However, randomly when I click on the home link or refresh the page it become incomplete as explained previously.

Actually, I store the HTML pages in PROGMEM as ATmega32 is very limited in RAM, but in my demonstration here I used very simple HTML page that stored in RAM.

code.zip (3.6 KB)

4

I do not see anything immediately wrong. Missing the initial piece of data is very strange in your scenario as flow is very basic and data is being written into TX buffer at once.

Can you share data from the Wireshark in Wireshark data file format (so that we can dig into the packet contents)? W5500 may fragment the data, and you may not see the first data packet (or it may be lost). Can this packet be filtered by the intermediate device? Would be very useful if you can connect PC with W5500 directly without any router and capture packets on the PC using Wireshark - to be sure you see every packet on the (local direct connect) network. However… if packet would be lost or filtered then PC would keep requesting it from W5500 because it must see sequence number and thus know about missed data! And I suspect W5500 would not clear its TX buffer until it gets ACK from PC.

As a next step of troubleshooting, try reading contents of the TX buffer starting with Sn_TX_RD:

	uint16_t ptr = 0;
	uint32_t addrsel = 0;

	ptr = getSn_TX_RD(sn);
	addrsel = ((uint32_t)ptr << 8) + (WIZCHIP_TXBUF_BLOCK(sn) << 3);
	
	uint16_t i;

	W5500_Select();

	addrsel |= (_W5500_SPI_READ_  | _W5500_SPI_VDM_OP_);
	
	//Sending Address & Control
	SPI_SndByte((addrsel & 0x00FF0000) >> 16);  //Address High Byte
	SPI_SndByte((addrsel & 0x0000FF00) >>  8);  //Address low Byte
	SPI_SndByte((addrsel & 0x000000FF) >>  0);	//Control Phase
	_delay_ms(10);
	for(i = 0; i < len; i++)    SPI_RecvByte(&tempbuf[i]);
		
	W5500_DeSelect();

     <... compare tempbuf and buf to see if they are equal...>

Note that I used function SPI_RecvByte which should be designed accordingly to read one byte from SPI. You can compare values you read to the values of buf in the loop instead of writing data into tempbuf. Put this code just before setSn_CR(sn,Sn_CR_SEND); to see that chip is having all data properly stored into the TX buffer.

You did not share contents of SPI_SndByte(), hope it is designed properly.

5

Hello Eugeny,

I am attaching the capture file. It was the first time for me to use Wireshark.
The capture include traffic when page loaded completely, and then when refreshed and incomplete. The capture file include both cases.

I do connect W5500 directly to my PC using crossover or straight cables.

Also, I tried to include the piece of code to compare tempbuf and buf but strangely my ATmega hanged and I do not why. I am attaching the SPI driver, and below is the code inserted. It hanged specifically in the part of SPI Send/Receive below"

/* -------------------------- Test Code -------------------------------------*/
ptr = 0;
addrsel = 0;

ptr = getSn_TX_RD(sn);
addrsel = ((uint32_t)ptr << 8) + (WIZCHIP_TXBUF_BLOCK(sn) << 3);

W5500_Select();

addrsel |= (_W5500_SPI_READ_  | _W5500_SPI_VDM_OP_);

uint8_t * tempbuf ={'\0', };
	
Sending Address & Control
SPI_SndByte((addrsel & 0x00FF0000) >> 16);  //Address High Byte
SPI_SndByte((addrsel & 0x0000FF00) >>  8);  //Address low Byte
SPI_SndByte((addrsel & 0x000000FF) >>  0);	//Control Phase
_delay_ms(10);
for(i = 0; i < len; i++)    tempbuf[i]=SPI_RcvByte();

W5500_DeSelect();
for(i = 0; i < len; i++)    UART_SendChar(pgm_read_byte(&buf[i]));
UART_SendString("\n\r##################################################\n\r"); 
UART_SendString((char*)tempbuf); UART_SendString("\n\r");
if (!(strcmp((char*)tempbuf,(char*)buf))) UART_SendString("Equal\n\r");  else UART_SendString("NOT Equal\n\r");

/* ----------------------- End of Test Code ---------------------------------*/

Also, please note that this time I have reduced the HTML page and stored it in PROGMEM as my MCU is limited in RAM. Note that I used SPI_SndByte(pgm_read_byte(&buf[i])); in the NetSend() function

I am attaching my files again.

Regards

code.zip (9.4 KB)

6

Can you put debug code to identify where it hangs? I suspect it may be in strcmp as you seem not to terminate tempbuf with null char.
PI_SndByte(pgm_read_byte(&buf[i])); what does it do and why did you make these changes? Our goal is to write data the same way as you originally did, and then check if TX buffer has this data written.

Will look into the logs and code later.

Edit: looked into the Wireshark logs. It is a mess. You should know that regular browser requests several resources in parallel (e.g. favicon which you can see in he log). But you are using only ONE socket. Thus for every request browser makes in parallel it gets RST, and tries later. Try another browser and you may see the difference. There was a setting in the advanced settings of Firefox to limit number of parallel (concurrent) connections browser is allowed to make… let me see… maybe “network.http.max-connections-per-server”.

Right now try simply getting the web page from the W5500 without using browser, and check if it will always return proper data:

> telnet 192.168.1.124 80
GET / HTTP1.1<cr>
<cr>

and you must get the contents onto the screen. You can output data into the file.

7

Hello Eugeny,

Thank you for bearing with me. Your last reply gave me a hint and I have changed my code to use 5 sockets and it seems things are getting improved. The core problem still not fixed but things are better.

Previously I had to refresh several times to get the complete page. Now the page loads from the first time or I have to refresh only two or three times :blush:

I noticed something strange that when the page is incomplete, it always start from a specific byte. I found it always skip first 261 bytes. Please have a look on the screen shot from my browser when the page is incomplete. It is always like this when the page is incomplete. I am also attaching two capture files , one when the page is complete and the other hen the page is incomplete.

In the incomplete capture file, in packet # 20 shows the page was not completely send to the browser. It seems that W5500 skip those first 261 bytes sometimes.

I tried to telnet W5500 192.168.1.124 80 but connection not accepted.

Please have a look on the attached files.

code.zip (18.7 KB)

8

It seems chip is really sending part of data, and packet exchange is valid. I do not have any immediate idea, therefore we must continue to investigate. The fact that you use more sockets now and it improves, shows hat there’s some flaw in the program’s logic.

  1. You are using disconnect() from the library as shown here, right? Strange but I can not find code for this routine in the WIZnet github. Can you please display what value disconnect returns (it returns a byte) every time you access the W5500? Would you please put explicit socket close() after every disconnect();
  2. you did not say if re-reading and comparison of the TX buffer contents is always successful;
  3. just before the setSn_CR(sn,Sn_CR_SEND); re-read ptrtemp = getSn_TX_RD(sn); and compare pointers to ensure that it is still the same value if(ptr != ptrtemp) <display error>. This is very low probability RD pointer has changed, but as we do not know what is going on it would be beneficial to check it just after data comparison and just before performing SEND.
9

Yes I am using disconnect from the library shown. The return of disconnect() is always 1 (SOCK_OK).
And I did put explicit socket close() after every disconnect() but no effect.

I changed the code to compare as advised in 2,3 above just before SEND, but it always not equal either the page loaded completely or not. Here is the code:

int32_t NetSend(uint8_t sn, uint8_t * buf, uint16_t len)
{
	uint16_t freesize=0;
	freesize = getSn_TxMAX(sn);
	while(1)
		{
		freesize = getSn_TX_FSR(sn);
		if(len <= freesize) break;
		}
	
	uint16_t ptr = 0;
	uint32_t addrsel = 0;

	//if(len == 0)  return;
	ptr = getSn_TX_WR(sn);
	addrsel = ((uint32_t)ptr << 8) + (WIZCHIP_TXBUF_BLOCK(sn) << 3);
	
	uint16_t i;

	//WIZCHIP_CRITICAL_ENTER();
	W5500_Select();

	addrsel |= (_W5500_SPI_WRITE_ | _W5500_SPI_VDM_OP_); //Combining Address & Control
	
	//Sending Address & Control
	SPI_SndByte((addrsel & 0x00FF0000) >> 16);  //Address High Byte
	SPI_SndByte((addrsel & 0x0000FF00) >>  8);  //Address low Byte
	SPI_SndByte((addrsel & 0x000000FF) >>  0);	//Control Phase
		
	for(i = 0; i < len; i++)    SPI_SndByte(pgm_read_byte(&buf[i]));   //SPI_SndByte(buf[i]);
		
	W5500_DeSelect();
	//WIZCHIP_CRITICAL_EXIT();
	
	ptr += len;
	setSn_TX_WR(sn,ptr);

	/* -------------------------- Test Code -------------------------------------*/
	uint16_t ptrtemp = 0;
	ptrtemp = getSn_TX_RD(sn);
	if(ptr != ptrtemp) UART_SendString("Not Equal\n\r"); else UART_SendString("Equal ====");
	/* ----------------------- End of Test Code ---------------------------------*/

	setSn_CR(sn,Sn_CR_SEND);  //Sending the data in the buffer
	
	/* wait to process the command... */
	while(getSn_CR(sn));
	
	return (int32_t)len;
}
10

Of course they will be not equal because you add len to ptr.
The point of the check is to see if original TX pointer matches final RD pointer when you perform SEND, in other words W5500 has correct byte count to send, and pointers are properly positioned in the TX buffer. You must compare original TX pointer you read at the beginning to the RX pointer before sending. You can also read final WR pointer after you written it and see if difference between final WR pointer and original WR pointer is exactly the len.

11

I changed the code as shown below

int32_t NetSend(uint8_t sn, uint8_t * buf, uint16_t len)
{
	uint16_t freesize=0;
	freesize = getSn_TxMAX(sn);
	while(1)
		{
		freesize = getSn_TX_FSR(sn);
		if(len <= freesize) break;
		}
	
	uint16_t ptr = 0; uint16_t ptr2 = 0;
	uint32_t addrsel = 0;

	//if(len == 0)  return;
	ptr = getSn_TX_WR(sn);  ptr2=ptr;
	addrsel = ((uint32_t)ptr << 8) + (WIZCHIP_TXBUF_BLOCK(sn) << 3);
	
	uint16_t i;

	//WIZCHIP_CRITICAL_ENTER();
	W5500_Select();

	addrsel |= (_W5500_SPI_WRITE_ | _W5500_SPI_VDM_OP_); //Combining Address & Control
	
	//Sending Address & Control
	SPI_SndByte((addrsel & 0x00FF0000) >> 16);  //Address High Byte
	SPI_SndByte((addrsel & 0x0000FF00) >>  8);  //Address low Byte
	SPI_SndByte((addrsel & 0x000000FF) >>  0);	//Control Phase

	for(i = 0; i < len; i++)        SPI_SndByte(pgm_read_byte(&buf[i]));   //SPI_SndByte(buf[i]);

	W5500_DeSelect();
	//WIZCHIP_CRITICAL_EXIT();
	
	ptr += len;
	setSn_TX_WR(sn,ptr);

	/* -------------------------- Test Code -------------------------------------*/
	
	uint16_t ptrtemp = 0;
	ptrtemp = getSn_TX_RD(sn);
	if(ptr2 != ptrtemp) UART_SendString("XXXX Not-Equal XXXX\n\r"); else UART_SendString("==== Equal ====\n\r");
	if(ptr-ptr2==len) UART_SendString("ok\n\r"); else UART_SendString("Not OK\n\r");
	
	/* ----------------------- End of Test Code ---------------------------------*/

	setSn_CR(sn,Sn_CR_SEND);  //Sending the data in the buffer
	
	/* wait to process the command... */
	while(getSn_CR(sn));
	sprintf(StrBuf,"len = %dl\n\r", i);
	UART_SendString(StrBuf);
	return (int32_t)len;
}

You must compare original TX pointer you read at the beginning to the RX pointer before sending.

I can see that they are always equal.

see if difference between final WR pointer and original WR pointer is exactly the len .

Yes, difference exactly = len

This makes me crazy. I believe the problem specifically when the W5500 sending it physically to cable, sometimes miss some bytes.

Could it be something related to the speed & negotiation (10/100 & Auto)?

12

The chip logic is very simple - after socket got open all pointers are initialized, you put data into the buffer, shift pointer, and issue SEND command. Chimp must use all the data between RD and WR pointers, that’s why our exercise to see if RD is the same as was before. Therefore you have proper data in the buffer properly, with properly set pointers, but chip omits first bytes for some reason. I do not believe it happens because pf PHY, there’s some issue in logic.

According to your information problem is not in SPI (protocol and communication). Problem is also not in accessing wrong registers/memory locations.

Write another program which will open socket by itself and send some data to see if problem appears also when socket is being open on the W5500 side. Note that for this program to work someone W5500 sends data to must be listening to respective port at its end.

@lawrence can you look into this case please. There may be something obvious and simple, but we do not see it.

13

Hi @mzedan ,

Could you share the hardware information?

I would like to know you use WIZnet product or your own designed module.

Thank you,

1 Like
14

Hi @mzedan ,

Could you share the hardware information?

I would like to know you use WIZnet product or your own designed module.

Thank you,

15

@lawrence @Eugeny

Greetings,

Thank you for bearing with me. My discussion with Eugeny guided me to understand more.

I have written a new program from scratch and used the io library as it is, and I think it is working as supposed to now.

I do not know where is my fault, but it seems I did something wrong when writing my own function. Before, I did not register SPI functions, but this time I registered my SPI functions. Maybe that was the problem.

I tried very simple HTML page and no problem with it. I will continue with my project to see if it is OK.

Thank you again

2 Likes
16

@mzedan

Glad to hear the news!!

And next time, I will support new issue faster !
Please leave any issues when you need others