iperf3: Don't wait for a full block when receiving TCP data.

[kwagyeman]

Summary

Fixes to the iperf3 server so it works with the reference iperf3 client's default arguments, with parallel streams, and with old clients. Found by running iperf 3.1.3 to 3.22 against an OPENMV_N6 over Ethernet.

Don't wait for a full block when receiving TCP data. iperf3 -c <board> with default arguments (128k blocks) hangs on most runs. The TCP receive path reads one whole block per poll event with a blocking readinto(), which on MicroPython returns only once the buffer is full. The sender is not required to end the stream on a block boundary, and the reference client usually doesn't when a block is larger than the socket takes in one write: at the end of the test it abandons the block it is in the middle of writing and sends TEST_END. The server then waits forever for the rest of that block and never reads TEST_END from the control socket.

Measured during a hang:

• host ss -tin on the data connection: bytes acked = 202 × 131072 + 22456, so the stream ended mid-block (the client's own total only counts whole blocks)
• gdb on the client: test->state == TEST_END, so it has sent TEST_END and is waiting for EXCHANGE_RESULTS
• the server is inside recvninto() on the data socket

The data socket is now non-blocking when it is used to receive TCP data, and each read counts however many bytes it returns. The same applies to the client in reverse mode.

Accept clients that do not send a pacing timer. The reference client only sends pacing_timer from version 3.2 onwards. With an older client the server raised KeyError: pacing_timer and the client hung. It now defaults to 1000 ms, the value this module's own client sends.

Support parallel TCP streams in the server. The server accepted one data connection whatever the client asked for, so -P 2 never started. It now accepts as many TCP data connections as the client's parallel parameter says, polls them all, and reports bytes per stream in the results (the reference implementation numbers its streams 1, 3, 4, ...). The interval lines printed by the server remain the total of all streams. The client opens its streams back to back, so the listening socket is re-created with a backlog large enough to queue all of them before the client is asked to create them.

Keep sending in reverse mode until the client responds. At the end of a reverse mode TCP test the server sent until its socket buffer was full, then asked for the client's results and waited. A client that reads whole blocks with a blocking read (iperf 3.1.3) is left waiting for the rest of a block and never sees the request. The server now asks for the results first and keeps sending until the client responds.

Testing

OPENMV_N6 over Ethernet, iperf3 client on a Linux host, default arguments unless noted. Runs completed, before and after the first commit:

client	before	after
iperf 3.17.1	2/12	12/12
iperf 3.17.1 -l 500	5/5	5/5
iperf 3.9	2/5	3/3
iperf 3.15	3/6	3/3
iperf 3.16	1/6	3/3
iperf 3.22	2/6	3/3

Throughput with -l 500 is unchanged (42.8 Mbit/s before, 44.8 after, medians of 5).

With all commits, iperf 3.17.1 as the client unless noted:

test	result
single stream, default arguments	5/5 complete
-R	3/3 complete
-P 2	4/4 complete
-P 4, -P 4 -l 500	complete
-P 2 -R	complete
-u -b 20M, -u -R -b 20M	complete, 0% loss
iperf 3.1.3, default arguments	3/3 complete
iperf 3.1.3 -R	3/3 complete
iperf 3.1.3 -l 500, -P 2, -P 2 -R	complete
iperf 3.9, 3.15, 3.22 -R	complete

Also ran the board as the client against iperf3 -s, forward and reverse, 3 runs each, all completed (first commit only).

Not working on this board, and not addressed here:

• -P 5: it needs 6 TCP connections and the port allows 5.
• -P 3 -R: throughput collapses and send() raises EIO.

Trade-offs and Alternatives

Reads are now as large as whatever has arrived rather than one full block, so there are more Python-level reads per second with large blocks. Throughput did not drop in the runs above.

Re-creating the listening socket after the parameter exchange is the only way to size the backlog exactly, because the stream count isn't known at the first listen(). A fixed larger backlog would also work but would be a guess.

UDP and the client side still use a single stream.

[kwagyeman]

@dpgeorge - ipref is much better now.