Random crashes, infinite loops, and other lockups were happening to the PicoW
while under high load from multiple clients.
This seems to have been due to two issues:
* The periodic sys_check_timeouts() call from an alarm/IRQ was happening while
the core was in LWIP code. LWIP is not re-entrant or multi-core/thread safe
so this is a bad thing. Some calls may not have been locked with a manual
addition of the LWIPMutex object to hit this.
* The WiFi driver supplies packet data during an interrupt. PBUF work is
legal in an interrupt, but actually calling netif->input() from an IRQ to
queue up the Ethernet packet for processing is illegal if LWIP is already
in progress.
Rearchitect the LWIP interface to fix these problems:
* Disable interrupts during malloc/etc. to avoid the possibility of the
periodic LWIP timeout check interrupting and potentially calling user
code which did a memory operation
* Wrap all used LWIP calls to note LWIP code will be executing, instead of
manually eyeballing and adding in protection in user code.
* Remove all user code LWIPMutex blocking, the wrapping takes care of it.
* When an Ethernet packet is received by interrupt and we're in LWIP code,
just throw the packet away for now. The upper layers can handle retransmit.
(A possible optimization would be to set the packet aside and add a
housekeeping portion to the LWIP wrappers to netif->input() them when safe).
* Ignore callbacks during TCP connection teardown when the ClientContext
passed from LWIP == nullptr
10090b60a9