Provide an AVX2-based function using compiler intrinsics for
TCP/IP-style checksums. The load/unpack/add idea and implementation
is largely based on code from BESS (the Berkeley Extensible Software
Switch) licensed as 3-Clause BSD, with a number of modifications to
further decrease pipeline stalls and to minimise cache pollution.
This speeds up considerably data paths from sockets to tap
interfaces, decreasing overhead for checksum computation, with
16-64KiB packet buffers, from approximately 11% to 7%. The rest is
just syscalls at this point.
While at it, provide convenience targets in the Makefile for avx2,
avx2_debug, and debug targets -- these simply add target-specific
CFLAGS to the build.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
dhcpv6_opt() already reflects consumed bytes on the remaining length,
and that we're not exceeding the message length. At this point, the
remaining length is usually zero.
While at it, drop a useless __packed__ attribute that triggers a gcc
warning.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Allow to bind IPv4 and IPv6 ports to tap, namespace or init separately.
Port numbers of TCP ports that are bound in a namespace are also bound
for UDP for convenience (e.g. iperf3), and IPv4 ports are always bound
if the corresponding IPv6 port is bound (socket might not have the
IPV6_V6ONLY option set). This will also be configurable later.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...not just for loopback addresses, with the address of the default
gateway. Otherwise, the guest might receive packets with source and
destination set to the same address.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This is actually reasonable in terms of memory consumption and
allows for better performance with local services.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Packets are received directly onto pre-cooked, static buffers
for IPv4 (with partial checksum pre-calculation) and IPv6 frames,
with pre-filled Ethernet addresses and, partially, IP headers,
and sent out from the same buffers with sendmmsg(), for both
passt and pasta (non-local traffic only) modes.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The guest might not send other types of traffic before we try to
communicate to it, so take also this chance to store its configured
addresses.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This value should work for all tap-like interfaces and is rather
convenient for performance testing. It will be configurable later
on.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
PASTA (Pack A Subtle Tap Abstraction) provides quasi-native host
connectivity to an otherwise disconnected, unprivileged network
and user namespace, similarly to slirp4netns. Given that the
implementation is largely overlapping with PASST, no separate binary
is built: 'pasta' (and 'passt4netns' for clarity) both link to
'passt', and the mode of operation is selected depending on how the
binary is invoked. Usage example:
$ unshare -rUn
# echo $$
1871759
$ ./pasta 1871759 # From another terminal
# udhcpc -i pasta0 2>/dev/null
# ping -c1 pasta.pizza
PING pasta.pizza (64.190.62.111) 56(84) bytes of data.
64 bytes from 64.190.62.111 (64.190.62.111): icmp_seq=1 ttl=255 time=34.6 ms
--- pasta.pizza ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 34.575/34.575/34.575/0.000 ms
# ping -c1 spaghetti.pizza
PING spaghetti.pizza(2606:4700:3034::6815:147a (2606:4700:3034::6815:147a)) 56 data bytes
64 bytes from 2606:4700:3034::6815:147a (2606:4700:3034::6815:147a): icmp_seq=1 ttl=255 time=29.0 ms
--- spaghetti.pizza ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 28.967/28.967/28.967/0.000 ms
This entails a major rework, especially with regard to the storage of
tracked connections and to the semantics of epoll(7) references.
Indexing TCP and UDP bindings merely by socket proved to be
inflexible and unsuitable to handle different connection flows: pasta
also provides Layer-2 to Layer-2 socket mapping between init and a
separate namespace for local connections, using a pair of splice()
system calls for TCP, and a recvmmsg()/sendmmsg() pair for UDP local
bindings. For instance, building on the previous example:
# ip link set dev lo up
# iperf3 -s
$ iperf3 -c ::1 -Z -w 32M -l 1024k -P2 | tail -n4
[SUM] 0.00-10.00 sec 52.3 GBytes 44.9 Gbits/sec 283 sender
[SUM] 0.00-10.43 sec 52.3 GBytes 43.1 Gbits/sec receiver
iperf Done.
epoll(7) references now include a generic part in order to
demultiplex data to the relevant protocol handler, using 24
bits for the socket number, and an opaque portion reserved for
usage by the single protocol handlers, in order to track sockets
back to corresponding connections and bindings.
A number of fixes pertaining to TCP state machine and congestion
window handling are also included here.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
On pc-q35, pci.2 is usually configured by libvirt as a hotplug bus,
so we can't use address 0x0 there. Look for free busses starting from
pci.3 instead.
While at it, add proper error reporting for passt probing, and add
some comments to structs that were previously missing.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Before commit 8f7baad7f035 ("tcp: Add snd_wnd to TCP_INFO"), the
kernel didn't export tcpi_snd_wnd via TCP_INFO, which means we don't
know what's the window size of the receiver, socket-side.
To get TCP connections working in that case, ignore this value if
it's zero during handshake, and use the initial window value as
suggested by RFC 6928 (14 600 bytes, instead of 4 380 bytes), to
keep network performance usable.
To make the TCP dynamic responsive enough in this case, also check
the socket for available data whenever we get an ACK segment from
tap, instead of waiting until all the data from the tap is dequeued.
While at it, fix the window scaling value sent for SYN and SYN, ACK
segments: we want to increase the data pointer after writing the
option, not the value itself.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
During handshake, the initial SYN, ACK segment to the guest, send as
a response to the SYN segment, needs to report the unscaled value for
the window, given that the handshake hasn't completed yet.
While at it, fix the endianness for the window value in case TCP
parameters can't be queried via TCP_INFO and we need to use the
default value.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The most common case is actually that no other instance created a
socket with that name -- and that also means there is no other
instance.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
It turns out that binding ICMP/ICMPv6 echo sockets takes a long
time. Instead of binding all of them (one for each possible echo
identification number, that is, 2^17) at start-up, bind them as
ICMP/ICMPv6 packets are sent by the guest.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The guest might try to resolve hosts other than the main host
namespace (i.e. the gateway) -- just recycle the target address from
the request and resolve it to the MAC address of the gateway.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...sharing the same filesystem. Instead of a fixed path for the UNIX
domain socket, passt now uses a path with a counter, probing for
existing instances, and picking the first free one.
The demo script is updated accordingly -- it can now be started several
times to create multiple namespaces with an instance of passt each,
with addressing reflecting separate subnets, and NDP proxying between
them.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
and, given that the connection table is indexed by socket number,
we also need to increase MAX_CONNS now as the ICMP implementation
needs 2^17 sockets, that will be opened before TCP connections are
accepted.
This needs to be changed later: the connection table should be
indexed by a translated number -- we're wasting 2^17 table entries
otherwise. Move initialisation of TCP listening sockets as last
per-protocol initialisation, this will make it easier.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Open and bind a socket for each possible ICMP/ICMPv6 echo identifier,
and add a tracking mechanism. Otherwise, multiple pings in parallel
won't work, and a single ping to a different destination would make
an existing ping sequence stop working.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The socket isn't necessarily closed, make sure we close it before
getting a new one from accept(), so that we don't mix it up with
protocol sockets numbering.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
With -DDEBUG, passt now saves guest-side traffic captures in
pcap format at /tmp/passt_<ISO8601 timestamp>.pcap. The timestamp
refers to time and date of start-up.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
As libvirt can pass e1000e (not just e1000) devices as well,
make sure we also drop those network devices from the command
line before adding the parameters we need.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Add support for a variable amount of DNS servers, including zero,
from /etc/resolv.conf, in DHCP, NDP and DHCPv6 implementations.
Introduce support for domain search list for DHCP (RFC 3397),
NDP (RFC 8106), and DHCPv6 (RFC 3646), also sourced from
/etc/resolv.conf.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The previous approach wasn't really robust: adding a -netdev option
without libvirt knowing could result in clashes with other devices.
Drop network devices from command line, check the available busses
and addresses from all -device options according to the -machine
parameter, and add a virtio-net device using an available address
or bus. Then, add a corresponding -netdev socket option.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
If a socket netdev parameter is already passed, don't touch the command
line. If it's not, add it, taking the id= reference from a netdev=
parameter, if any.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The name-guessing loop should iterate over names, not single
characters. Also add /usr/libexec/qemu-kvm as full path for
execvp(): execvp() won't find it if it's not in $PATH, which
is the reason why it shouldn't be under /usr/libexec/, but this
seems to be the case for some current version of Fedora.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Once passt forks to background, it should be guaranteed that the UNIX
domain socket is available, otherwise, if qemu is started right after
it, it might fail to connect.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
It might be impractical to pass options to qrap when using libvirt,
because the <emulator/> tag expects a path to an executable, without
further arguments.
If the first argument is not a plausible socket number, and the
second argument is not a valid executable, look up a qemu command
from a list of possible names, then start it patching the command line
to include the -netdev fd= parameter corresponding to the AF_UNIX
domain socket we just opened.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
If net.ipv4.ping_group_range doesn't include our PID, we'll fail
to open sockets for ICMP and ICMPv6 echo. Warn instead of
exiting, this is not fatal.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Multiple arrays, one for each address, were needed with a single
fprintf(). Now that it's replaced by info(), we can have just one
for each protocol version.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
With glibc, we can't reliably build a static binary with
getprotobynumber(), which is currently used with -DDEBUG.
Replace that with a small array of protocol strings.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This is in preparation for scatter-gather IO on the UDP receive path:
save a getsockname() syscall by setting a flag if we get the numbering
of all bound sockets in a strict sequence (expected, in practice) and
repurpose the tap buffer to be also a socket receive buffer, passing
it down to protocol handlers.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
As we support UDP forwarding for packets that are sent to local
ports, we actually need some kind of connection tracking for UDP.
While at it, this commit introduces a number of vaguely related fixes
for issues observed while trying this out. In detail:
- implement an explicit, albeit minimalistic, connection tracking
for UDP, to allow usage of ephemeral ports by the guest and by
the host at the same time, by binding them dynamically as needed,
and to allow mapping address changes for packets with a loopback
address as destination
- set the guest MAC address whenever we receive a packet from tap
instead of waiting for an ARP request, and set it to broadcast on
start, otherwise DHCPv6 might not work if all DHCPv6 requests time
out before the guest starts talking IPv4
- split context IPv6 address into address we assign, global or site
address seen on tap, and link-local address seen on tap, and make
sure we use the addresses we've seen as destination (link-local
choice depends on source address). Similarly, for IPv4, split into
address we assign and address we observe, and use the address we
observe as destination
- introduce a clock_gettime() syscall right after epoll_wait() wakes
up, so that we can remove all the other ones and pass the current
timestamp to tap and socket handlers -- this is additionally needed
by UDP to time out bindings to ephemeral ports and mappings between
loopback address and a local address
- rename sock_l4_add() to sock_l4(), no semantic changes intended
- include <arpa/inet.h> in passt.c before kernel headers so that we
can use <netinet/in.h> macros to check IPv6 address types, and
remove a duplicate <linux/ip.h> inclusion
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
