Commit graph

1603 commits

Author SHA1 Message Date
Stefano Brivio
8af961b85b tcp, udp: Map source address to gateway for any traffic from 127.0.0.0/8
...instead of just 127.0.0.1.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-26 18:20:01 +02:00
Stefano Brivio
9663378d6d icmp: Work around possible failure on bind() due to e.g. broken SELinux policy
If we can't bind() ping sockets, the echo identifier sent out from
the socket won't be the original one seen from the tap. Binding a
ping socket doesn't require any security capability, but it might
still fail due to a broken SELinux policy, see for example:
	https://bugzilla.redhat.com/show_bug.cgi?id=1848929

Track the ICMP echo identifier as part of the epoll reference for
the socket and replace it in the reply on mismatch. We won't send
out the original identifier as sent from the guest, but still better
than missing replies.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-26 17:43:10 +02:00
Stefano Brivio
0279ec8eae tcp: Fix re-send mechanism to tap on ACK timeout
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-26 14:25:16 +02:00
Stefano Brivio
74677bddb2 tcp: Simplify ACK accounting, skip some useless operations on tap handling
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-26 14:23:34 +02:00
Stefano Brivio
39ad062100 tcp: Introduce scatter-gather IO path from socket to tap
...similarly to what was done for UDP. Quick performance test with
32KiB buffers, host to VM:

$ iperf3 -c 192.0.2.2 -N
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  8.47 GBytes  7.27 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  8.45 GBytes  7.26 Gbits/sec                  receiver

$ iperf3 -c 2a01:598:88ba:a056:271f:473a:c0d9:abc1
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  8.43 GBytes  7.24 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  8.41 GBytes  7.22 Gbits/sec                  receiver

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-26 14:20:36 +02:00
Stefano Brivio
85a820a66f tap: Don't override address observed from guest with our own notion of it
If a tap protocol handler doesn't consume the full batch of packets
in one go, we already overrode the destination address in the packet
buffer with the address which is configured at start. If we re-enter
the tap handler, we shouldn't use the address from the packet buffers
anymore to set the observed address of the guest: that's not the
address observed from the guest, it's the configured one now.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-26 14:13:21 +02:00
Stefano Brivio
86b273150a tcp, udp: Allow binding ports in init namespace to both tap and loopback
Traffic with loopback source address will be forwarded to the direct
loopback connection in the namespace, and the tap interface is used
for the rest.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-26 14:10:29 +02:00
Stefano Brivio
f4aaa471a1 doc/demo.sh: Increase tcp_rmem and tcp_wmem before running passt
...this is convenient for performance testing.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-26 11:26:16 +02:00
Stefano Brivio
16b08367a5 tap: Fill the IPv6 flow label field to represent flow association
This isn't optional: TCP streams must carry a unique, hard-to-guess,
non-zero label for each direction. Linux, probably among others,
will otherwise refuse to associate packets in a given stream to the
same connection.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-26 07:30:57 +02:00
Stefano Brivio
17765f8de0 checksum: Introduce AVX2 implementation, unify helpers
Provide an AVX2-based function using compiler intrinsics for
TCP/IP-style checksums. The load/unpack/add idea and implementation
is largely based on code from BESS (the Berkeley Extensible Software
Switch) licensed as 3-Clause BSD, with a number of modifications to
further decrease pipeline stalls and to minimise cache pollution.

This speeds up considerably data paths from sockets to tap
interfaces, decreasing overhead for checksum computation, with
16-64KiB packet buffers, from approximately 11% to 7%. The rest is
just syscalls at this point.

While at it, provide convenience targets in the Makefile for avx2,
avx2_debug, and debug targets -- these simply add target-specific
CFLAGS to the build.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-26 07:18:50 +02:00
Stefano Brivio
0be49ccd93 dhcpv6: Drop bogus option length test while checking for not-on-link IA_NA
dhcpv6_opt() already reflects consumed bytes on the remaining length,
and that we're not exceeding the message length. At this point, the
remaining length is usually zero.

While at it, drop a useless __packed__ attribute that triggers a gcc
warning.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-26 07:11:33 +02:00
Stefano Brivio
60dee2705b tcp: Don't open a new connection from tap if both SYN and ACK are set
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-21 17:48:33 +02:00
Stefano Brivio
330ea9e681 tap: Fix comment for tap_handler_pasta()
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-21 17:48:12 +02:00
Stefano Brivio
49631a38a6 tcp, udp: Split IPv4 and IPv6 bound port sets
Allow to bind IPv4 and IPv6 ports to tap, namespace or init separately.

Port numbers of TCP ports that are bound in a namespace are also bound
for UDP for convenience (e.g. iperf3), and IPv4 ports are always bound
if the corresponding IPv6 port is bound (socket might not have the
IPV6_V6ONLY option set). This will also be configurable later.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-21 17:44:39 +02:00
Stefano Brivio
b508079c4c tcp: Replace source address also if it's the same as the guest address
...not just for loopback addresses, with the address of the default
gateway. Otherwise, the guest might receive packets with source and
destination set to the same address.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-21 12:05:58 +02:00
Stefano Brivio
1642a04f48 tcp: Increase maximum window scaling factor from 8 to 9
This is actually reasonable in terms of memory consumption and
allows for better performance with local services.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-21 12:05:03 +02:00
Stefano Brivio
64a0ba3b27 udp: Introduce recvmmsg()/sendmmsg(), zero-copy path from socket
Packets are received directly onto pre-cooked, static buffers
for IPv4 (with partial checksum pre-calculation) and IPv6 frames,
with pre-filled Ethernet addresses and, partially, IP headers,
and sent out from the same buffers with sendmmsg(), for both
passt and pasta (non-local traffic only) modes.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-21 12:01:04 +02:00
Stefano Brivio
7fa3e90290 ndp: Store link-local or global address on any NDP message received
The guest might not send other types of traffic before we try to
communicate to it, so take also this chance to store its configured
addresses.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-21 10:04:17 +02:00
Stefano Brivio
5ec9ad7e9d doc/demo.sh: Set MTU to 65535 for both veth interfaces
There's no reason to limit the MTU here to any lower value.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-21 10:03:29 +02:00
Stefano Brivio
38a4fae186 dhcp: Set MTU option (26) to 65520 bytes
This value should work for all tap-like interfaces and is rather
convenient for performance testing. It will be configurable later
on.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-21 09:59:26 +02:00
Stefano Brivio
a9c8d4d924 ndp: Fix calculation of length for DNS Search List option (31)
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-17 17:58:03 +02:00
Stefano Brivio
4667226bb0 tcp: Fix partial (ACK) message coalescing, ACK timeout, MSG_MORE flag setting
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-17 17:57:11 +02:00
Stefano Brivio
33482d5bf2 passt: Add PASTA mode, major rework
PASTA (Pack A Subtle Tap Abstraction) provides quasi-native host
connectivity to an otherwise disconnected, unprivileged network
and user namespace, similarly to slirp4netns. Given that the
implementation is largely overlapping with PASST, no separate binary
is built: 'pasta' (and 'passt4netns' for clarity) both link to
'passt', and the mode of operation is selected depending on how the
binary is invoked. Usage example:

	$ unshare -rUn
	# echo $$
	1871759

	$ ./pasta 1871759	# From another terminal

	# udhcpc -i pasta0 2>/dev/null
	# ping -c1 pasta.pizza
	PING pasta.pizza (64.190.62.111) 56(84) bytes of data.
	64 bytes from 64.190.62.111 (64.190.62.111): icmp_seq=1 ttl=255 time=34.6 ms

	--- pasta.pizza ping statistics ---
	1 packets transmitted, 1 received, 0% packet loss, time 0ms
	rtt min/avg/max/mdev = 34.575/34.575/34.575/0.000 ms
	# ping -c1 spaghetti.pizza
	PING spaghetti.pizza(2606:4700:3034::6815:147a (2606:4700:3034::6815:147a)) 56 data bytes
	64 bytes from 2606:4700:3034::6815:147a (2606:4700:3034::6815:147a): icmp_seq=1 ttl=255 time=29.0 ms

	--- spaghetti.pizza ping statistics ---
	1 packets transmitted, 1 received, 0% packet loss, time 0ms
	rtt min/avg/max/mdev = 28.967/28.967/28.967/0.000 ms

This entails a major rework, especially with regard to the storage of
tracked connections and to the semantics of epoll(7) references.

Indexing TCP and UDP bindings merely by socket proved to be
inflexible and unsuitable to handle different connection flows: pasta
also provides Layer-2 to Layer-2 socket mapping between init and a
separate namespace for local connections, using a pair of splice()
system calls for TCP, and a recvmmsg()/sendmmsg() pair for UDP local
bindings. For instance, building on the previous example:

	# ip link set dev lo up
	# iperf3 -s

	$ iperf3 -c ::1 -Z -w 32M -l 1024k -P2 | tail -n4
	[SUM]   0.00-10.00  sec  52.3 GBytes  44.9 Gbits/sec  283             sender
	[SUM]   0.00-10.43  sec  52.3 GBytes  43.1 Gbits/sec                  receiver

	iperf Done.

epoll(7) references now include a generic part in order to
demultiplex data to the relevant protocol handler, using 24
bits for the socket number, and an opaque portion reserved for
usage by the single protocol handlers, in order to track sockets
back to corresponding connections and bindings.

A number of fixes pertaining to TCP state machine and congestion
window handling are also included here.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-17 11:04:22 +02:00
Stefano Brivio
28fca04eb9 qrap: Skip pci.2 bus for pc-q35, add proper error reporting for probing
On pc-q35, pci.2 is usually configured by libvirt as a hotplug bus,
so we can't use address 0x0 there. Look for free busses starting from
pci.3 instead.

While at it, add proper error reporting for passt probing, and add
some comments to structs that were previously missing.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-17 08:28:38 +02:00
Stefano Brivio
69c8e5b598 doc/demo.sh: Support IPv4-only environments too
If no IPv6 global addresses are available, proceed with just IPv4
addresses and routes.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-07-17 08:27:32 +02:00
Stefano Brivio
90078ebc59 tcp: Add support for kernels not exporting tcpi_snd_wnd via TCP_INFO
Before commit 8f7baad7f035 ("tcp: Add snd_wnd to TCP_INFO"), the
kernel didn't export tcpi_snd_wnd via TCP_INFO, which means we don't
know what's the window size of the receiver, socket-side.

To get TCP connections working in that case, ignore this value if
it's zero during handshake, and use the initial window value as
suggested by RFC 6928 (14 600 bytes, instead of 4 380 bytes), to
keep network performance usable.

To make the TCP dynamic responsive enough in this case, also check
the socket for available data whenever we get an ACK segment from
tap, instead of waiting until all the data from the tap is dequeued.

While at it, fix the window scaling value sent for SYN and SYN, ACK
segments: we want to increase the data pointer after writing the
option, not the value itself.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-06-08 02:20:28 +02:00
Stefano Brivio
8b39b0b47f tcp: Fix window size in initial SYN, ACK segment to guest
During handshake, the initial SYN, ACK segment to the guest, send as
a response to the SYN segment, needs to report the unscaled value for
the window, given that the handshake hasn't completed yet.

While at it, fix the endianness for the window value in case TCP
parameters can't be queried via TCP_INFO and we need to use the
default value.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-06-05 14:54:12 +02:00
Stefano Brivio
46b799c077 passt: When probing for an existing instance, also accept ENOENT on connect()
The most common case is actually that no other instance created a
socket with that name -- and that also means there is no other
instance.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-23 15:43:31 +02:00
Stefano Brivio
7ab1b2a97a util: On -DDEBUG, log to stderr with timestamps
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:22:09 +02:00
Stefano Brivio
84a62b79a2 passt: Also log to stderr, don't fork to background if not interactive
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:22:04 +02:00
Stefano Brivio
9311ceb8b6 icmp: Implement lazy bind for ping sockets
It turns out that binding ICMP/ICMPv6 echo sockets takes a long
time. Instead of binding all of them (one for each possible echo
identification number, that is, 2^17) at start-up, bind them as
ICMP/ICMPv6 packets are sent by the guest.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:53 +02:00
Stefano Brivio
5fd6db7751 ndp: Always answer neighbour solicitations with the requested target address
The guest might try to resolve hosts other than the main host
namespace (i.e. the gateway) -- just recycle the target address from
the request and resolve it to the MAC address of the gateway.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:52 +02:00
Stefano Brivio
ad4a85c860 qrap: Connect to the first available instance of passt, probe via ARP request
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:52 +02:00
Stefano Brivio
19d254bbbb passt: Add support for multiple instances in different network namespaces
...sharing the same filesystem. Instead of a fixed path for the UNIX
domain socket, passt now uses a path with a counter, probing for
existing instances, and picking the first free one.

The demo script is updated accordingly -- it can now be started several
times to create multiple namespaces with an instance of passt each,
with addressing reflecting separate subnets, and NDP proxying between
them.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:51 +02:00
Stefano Brivio
8ce188ecb0 tcp: Properly initialise parameters for SO_ACCEPTCONN getsockopt()
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:51 +02:00
Stefano Brivio
bd5aaaac7f tcp: Actually enforce MAX_CONNS limit
and, given that the connection table is indexed by socket number,
we also need to increase MAX_CONNS now as the ICMP implementation
needs 2^17 sockets, that will be opened before TCP connections are
accepted.

This needs to be changed later: the connection table should be
indexed by a translated number -- we're wasting 2^17 table entries
otherwise. Move initialisation of TCP listening sockets as last
per-protocol initialisation, this will make it easier.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:50 +02:00
Stefano Brivio
d303cfdd55 icmp: Implement ping tracking based on echo identifiers
Open and bind a socket for each possible ICMP/ICMPv6 echo identifier,
and add a tracking mechanism. Otherwise, multiple pings in parallel
won't work, and a single ping to a different destination would make
an existing ping sequence stop working.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:50 +02:00
Stefano Brivio
af243857fa qrap: Silence gcc -O3 warning about strncpy() buffer length
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:49 +02:00
Stefano Brivio
4adae47c40 passt: Close UNIX domain socket on failure before accepting new connections
The socket isn't necessarily closed, make sure we close it before
getting a new one from accept(), so that we don't mix it up with
protocol sockets numbering.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:49 +02:00
Stefano Brivio
17337a736f passt: Introduce packet capture implementation
With -DDEBUG, passt now saves guest-side traffic captures in
pcap format at /tmp/passt_<ISO8601 timestamp>.pcap. The timestamp
refers to time and date of start-up.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:48 +02:00
Stefano Brivio
59182924f0 qrap: Also drop -device e1000e,... from qemu command line
As libvirt can pass e1000e (not just e1000) devices as well,
make sure we also drop those network devices from the command
line before adding the parameters we need.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:48 +02:00
Stefano Brivio
9010054ea4 dhcp, ndp, dhcpv6: Support for multiple DNS servers, search list
Add support for a variable amount of DNS servers, including zero,
from /etc/resolv.conf, in DHCP, NDP and DHCPv6 implementations.

Introduce support for domain search list for DHCP (RFC 3397),
NDP (RFC 8106), and DHCPv6 (RFC 3646), also sourced from
/etc/resolv.conf.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:47 +02:00
Stefano Brivio
0231ac1c86 dhcp: Increase lease time to maximum allowed value
...to make things simpler at least for the moment being.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:46 +02:00
Stefano Brivio
8754878fbf passt: With -DDEBUG, also print protocol number for unsupported protocols
...otherwise, we have no idea what's going on if we receive something
unexpected.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:46 +02:00
Stefano Brivio
7e3e36b2c2 tcp: Close socket on EPOLLHUP or EPOLLRDHUP in non-data state
If the peer doesn't shut down orderly, this might happen: just close
the socket then.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-21 11:14:42 +02:00
Stefano Brivio
8358400b3f qrap: Lowest usable PCI bus number for pc-q35 is actually 1
...3 was a left-over from a test.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-12 16:20:56 +02:00
Stefano Brivio
5307faa059 qrap: Strip network devices from command line, set them up according to machine
The previous approach wasn't really robust: adding a -netdev option
without libvirt knowing could result in clashes with other devices.

Drop network devices from command line, check the available busses
and addresses from all -device options according to the -machine
parameter, and add a virtio-net device using an available address
or bus. Then, add a corresponding -netdev socket option.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-12 08:35:36 +02:00
Stefano Brivio
7503332a1e qrap: Adapt -net/-netdev command-line mangling to existing arguments
If a socket netdev parameter is already passed, don't touch the command
line. If it's not, add it, taking the id= reference from a netdev=
parameter, if any.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-11 17:08:15 +02:00
Stefano Brivio
f98e3589b4 qrap: Fix qemu name-guessing loop, add /usr/libexec/qemu-kvm as full path too
The name-guessing loop should iterate over names, not single
characters. Also add /usr/libexec/qemu-kvm as full path for
execvp(): execvp() won't find it if it's not in $PATH, which
is the reason why it shouldn't be under /usr/libexec/, but this
seems to be the case for some current version of Fedora.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-10 14:00:30 +02:00
Stefano Brivio
0328e2a1f7 passt: Don't fork into background until the UNIX domain socket isn't listening
Once passt forks to background, it should be guaranteed that the UNIX
domain socket is available, otherwise, if qemu is started right after
it, it might fail to connect.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-05-10 13:15:52 +02:00