As you already know, the UDP Header itself does not have an IP address. The IP Header has that information.
Since the W7500 chip uses the H / W TOE, this information is handled at the H / W level.
Among them, it uploads information that user often needs.
But, Destination ip address information of received UDP packet is not provided.
However, there is a way to distinguish.
On the W7500, you can block Unicast, BroadCast, and Multicast through the socket settings.
Refer to below link.(datasheet)
To do this, refer to the Sn_MR Register in the Datasheet, pages 59-61.
In other words, you only need to open two sockets and only non-blocking Unicast and Broadcast respectively.
If you do so, you can only receive unicast packets for A sockets and broadcast packets for B sockets, so you can see what kind of cast the received packet is.
I hope this helps.