Hello everyone,
I’m implementing a webserver using the W5500. The website that I display has some resources (12 images). I do use all sockets so far and follow the cycle in the provided source code (https://wizwiki.net/wiki/doku.php/products:w5500:driver), which is basically open->listen->establish->receive->answer->close->open->listen->…
This setting works fine, but I want to reduce the number of sockets, since I need them for other services.
When I use less then 6 sockets (I tested with chrome), some of the resources are not displayed. (I know that chrome usually uses 6 sockets…)
Question 1: Is there a way to configure the w5500 such that the browser retries its request, if all sockets are busy ?
Question 2: Is there a way to tell the browser how many sockets it is allowed to use when requesting the resources (e.g. in the HTTP header field) ? I do not want to configure my browser settings
Question 3: I also started playing with “persistent connection”, however the browser (chrome) does not send a second request, after it has send its first and received its answer. Do you have a code example, where you work with persisten connection ? I was already looking at the implementation of the w550web, but there it also uses the open->listen->establish->receive->answer->close cycle, although the http connection is set to keep-alive (might be a bug)
I only have three sockets reserved for an embedded web server. It has no problem with Chrome. However, I do use the non-blocking mode (SF_IO_NONBLOCK) for all three sockets.
How many resources (I mean images, css files, stuff that has to be loaded after the page is interpreted by the browser) do you have on your webserver? For me, the page itself can be handled with a single socket. However the 12 images that are requested after the page is received are not handled sequentially. Chrome (also checked with IE and Firefox) just fires back in parallel fashion and if I do not provide enough sockets, some of the images are not loaded.
So lets say I provide 4 sockets then
The page is transmitted
connection closes
chrome figures out that it wants to have the 12 images
Most browsers will open up to 6 concurrent connections and will not retry if a connection is rejected (e.g., because you don’t have one listening when it tries to connect).
I revised an existing W5500 library to actively manage sockets while reserving some for other purposes. You’re welcome to use it, or to scavenge it for ideas or code snippets that serve your needs.
Look in particular at sockets.cpp, in which I’ve implemented a function named manageSockets() that closes connections as needed to maintain the number of listeners you specify and leave a socket untouched if allocated to other purposes. Note that it depends on other functions recording the last time of use for each socket.
Thank You,
I read your code and in fact it tackles the problem that I have.
I understand, that the mangeSockets() functions disconnects resp. closes the sockets as soon as possible, which is very reasonable. Closing of the idle conenctions is a very good thing. I like that you track the age of the connections…
But altogether, you still have to provide at least 6 sockets for a webserver (In order to guarantee that pages with many (>6) resources are loaded completely) ? Right ?
There is no way to run a webserver with 2 sockets ?
Otherwise I’ll instantly integrate your lib
I’ve found no way to run a webserver with less than 6 sockets using the current versions of Chrome, FireFox, Brave, Opera or Microsoft Edge browsers other than limiting the number of resources used by the pages you load. Browsers open multiple sockets to improve their own performance by loading resources in parallel at the expense of imposing a greater load on servers that they view as “not my problem”. There’s no way to limit the number of concurrent sockets used or to force retries of refused connections. This is what led me to write my socket management function.
You could change your pages to reduce the opportunity for browsers to demand concurrent sockets. Embed javascripts and css in your header, and Base64-encode any jpg images and embed them in your HTML. You could also host resources that don’t strictly need to be on your system (e.g., .css & script files) on another server. If you eliminate all “linked” resources in your pages, the browser would need only one socket to load them.
Another option is to specify long cache periods using the Cache-Control pragma for linked resources in your pages if they don’t change often. You’ll still get refused connections the first few times you load your pages, as the browser must load every resource once in order to cache it. But once all are cached you’ll be fine until the specified cache period expires. The downside is that if these resources change before the cache period expires, users will have to be trained to force a reload using the relevant browser feature.
You haven’t said what your “other services” are, but clever socket sharing may allow you to do what you need despite browsers hogging 6 while loading pages. Is it possible to spread these services out in time, perhaps queuing them to run as sockets become available, or must they run on a strict schedule?
If you use my code you’ll notice that I’ve included calls to a wdtClear() function in certain loops. This function resets a watchdog timer on the ATSAMD5x family of controllers. I’ve posted my watchdog timer code at https://github.com/SapientHetero/Watchdog-Timer-for-ATSAMD51J19A if you want to use it. If not, you’ll need to comment out the wdtClear() calls or add conditional compilation directives to turn them off.
That’s how it feels, it would have been really nice though. Maybe the w5500 itself could have some “pseudo” sockets that have window size 0 and force the browser to retry until it gets a “real” socket. But this is a completely different story. I might start a new discussion on that…
That looks like a hack that could work.
I have only basic knowledge about web programming, so I was looking into http2 and pipelining (HTTP pipelining - Wikipedia), which looked very promising but did not work since most browser do not support pipelining
I have a TCP communication to poll status data from the uC. I know this takes only one socket (using a different port), but it is an unpleasant feeling to not have the control over the webserver sockets. I’m not deep into web stuff, but was just thinking what if the next update of chrome allows 8 or 10 parallel socket requests…
I know your proposed solution with java script and embedding the resources will still work, but it leaves me with the feeling that it does not scale well.
