Now that the back end allows passt/pasta to use different external
interfaces for IPv4 and IPv6, use that to do the right thing in the case
that the host has IPv4 and IPv6 connectivity via different interfaces.
If the user hasn't explicitly chosen an interface, separately search for
a suitable external interface for each protocol.
As a bonus, this substantially simplifies the external interface probe. It
also eliminates a subtle confusing case where in some circumstances we
would pick the first interface in interface index order, and sometimes in
order of routes returned from netlink. On some network configurations that
could cause tests to fail, because the logic in the tests was subtly
different (it always used route order).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
By default, passt itself attaches to the first host interface with a
default route. However, when determining the host interface name the tests
implicitly select the *last* host interface: they use a jq expression which
will list all interfaces with default routes, but the way output detection
works in the scripts, it will only pick up the last line.
If there are multiple interfaces with default routes on the host, and they
each have a different address, this can cause spurious test failures.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
It's quite plausible for a host to have both IPv4 and IPv6 connectivity,
but only via different interfaces. For example, this will happen in the
case that IPv6 connectivity is via a tunnel (e.g. 6in4 or 6rd). It would
also happen in the case that IPv4 access is via a tunnel on an otherwise
IPv6 only local network, which is a setup that might become more common in
the post IPv4 address exhaustion world.
In turns out there's no real need for passt/pasta to get its IPv4 and IPv6
connectivity via the same interface, so we can handle this situation fairly
easily. Change the core to allow eparate external interfaces for IPv4 and
IPv6. We don't actually set these separately for now.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
A couple of days ago, we started running out of space there as we're
about to install gcc -- about 50 MiB are missing.
Given that virt-resize (which could be conveniently invoked by the
Makefile for tests) reorders partitions if we expand the first one,
resize the image using qemu-img from the test script itself, and then
take care of expanding root partition and filesystem online later.
This is probably a temporary hack, so I'm not looking for a more
generic or elegant solution at the moment.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Otherwise, if the current PID has fewer digits than a previously
written one, the content will be wrong.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
For some reason using https to clone from the passt git repo is very slow,
at least from network-distant places. Use git protocol in the demo instead
to avoid a tedious wait to get the source.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
With pasta, the namespace interface name is generally the same as the host
interface name. We already rely on this in the dhcp/pasta tests, but for
no clear reason ndp/pasta separately determines the host interface name.
Remove this unnecessary step.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The iperf based test commands create a bunch of .bw and .pid files for
each iperf client and server. The server side .bw files are cleaned
up afterwards, but the pid files are not, and none of the client side
files are cleaned up. The latter doesn't really matter when the
client is run on ephemeral guests, but sometimes we run it in a
namespace that shares the filesystem with the host.
Clean up all of these files after the tests.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Before starting the guests, these tests configure addresses in a pasta
namespace using dhclient. However, because it's a user namespace, it's
not running as "real" root and can't write to the dhclient pid file.
This doesn't stop it working, but causes an ugly error message which we
can avoid by using the --no-pid option.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
All the UDP tests use :> to truncate some temporary data files. This
appears to be so that they're empty before writing data to them with tee.
However tee, by default, truncates its output file anyway (you need tee -a
to append). So drop the unnecessary truncations.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
teardown_passt_in_ns() sends a ^D to the NS pane, which appears to be
intended to terminate the nsenter running there, leaving the namespace.
However, we've also sent a ^D to the PASST pane which will exit the pasta
instance which created the namespace. With the namespace destroyed the
nsenter in the NS pane will be killed, so it does not need to be exited
explicitly.
In fact sending the extra ^D can be harmful, since it will exit the shell
in which the nsenter was run, causing the whole pane to be closed. That
can then mean that the "pane_wait NS" hangs indefinitely. I believe this
will sometimes work, because there's a race between the various options
here, but it should be more reliable without the extra ^D.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Commit 41c02e10 ("tests: Use nmap-ncat instead of openbsd netcat for pasta
tests") updated the pasta tests to use the nmap version of ncat instead of
the openbsd version, for greater portability.
For some upcoming changes, however, we'll be wanting to use socat.
"socat" can do everything "ncat" can and more, so let's move all the
tests using host tools (either directly on the host or via mbuto
generated images) to using socat instead.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio: Fix a typo in port specification, 31337 instead of x31337]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Some versions of valgrind (such as the version on my Fedora laptop -
valgrind-3.19.0-3.fc36.x86_64) use futexes. But futex is currently not
allowed in the seccomp filter, even with the extra calls added for
valgrind builds. Add it, to avoid spurious valgrind failures.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
In what looks like a copy-and-paste error from the TCP script, the
udp/passt test script creates a test file called '__TEMP_BIG__', while
the commands it use the variable __TEMP__. Correct this so that a) we
actually transfer the data we created for the purpose and b) we don't
leave a stale __TEMP_BIG__ file in the current directory.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The dhcp/passt and dhcp/passt_in_ns tests at least, and maybe others
use 'hout' commands that need to be able to detect empty output.
However, we don't set PS1, which means the screen-scraping logic which
detects this may not be reliable. In addition, if the host is using a
recent bash, it will have bracketed paste mode enabled which will also
add escape codes which will mess up the empty output detection.
Set the prompt and disable bracketed paste mode from the passt and
passt_in_ns setups to avoid these problems.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently our small custom dhclient-script only handles the 'domain-name'
option, which can just list a single domain, not the 'domain-search'
option, which can handle several. Correct it to handle both.
We also weren't emptying the resolv.conf file before we began, which
could lead to surprising contents after multiple DHCP transactions.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We previously introduced a change to passt to handle the case where the
host machine is its own nameserver - so resolv.conf points to 127.0.0.1.
In this case we advertize the gateway as the DNS server for the guest,
which in turn will be redirected back to the host by existing passt logic.
The dhcp/passt doesn't handle this case correctly, so add some logic to
account for it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
To check publishing of DNS information via DHCP, we need to extract a list
of nameservers and/or search domains from resolv.conf in the test script.
The current version (usually) leaves the result with a trailing ','.
That's usually ok because it happens on both guest and host sides. However
it's kind of confusing, and might stop working if the host had a
resolv.conf without a trailing \n on the last line. It also makes some
later changes we'll need more difficult.
So, normalize the output from resolv.conf a bit further, removing any
trailing ','. It turns out we can do this with a slightly less complex
sed expression than the one we already have.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Although it can operate without them, dhclient can issue errors if it
doesn't have /var/run to write a pid file and /var/lib to write a leases
file. Create those in mbuto.img to stop it complaining.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We now supply a minimal dhclient-script of our own in the mbuto boot image.
There are some problems with it, so add some basic logging to help debug
it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Modern Fedora (and RHEL) systems have /sbin as a symlink to /usr/sbin
(along with a number of similar links). Along with that it expects to
find dhclient-script in /usr/sbin/dhclient-script rather than
/sbin/dhclient-script.
Link them together in our mbuto image so that the Fedora build of dhclient
can find it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
AFAICT the symlink we created in mbuto from /usr/bin/bash to /bin/sh was
for the benefit of a dhclient-script which used /usr/bin/bash as its
interpreter (e.g. in Fedora). That was a bit risky if the script really
did require bash and we linked it to dash or another shell.
We now supply our own custom dhclient-script, so we don't need the
link any more.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Similar case as the one fixed by David's patch "tests: Remove
unnecessary ^D in passt_in_ns teardown": we happen to pseudo-randomly
close panes by unnecessarily exiting the parent shells there, and
subsequent pane_wait directives hang.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
For some reason, I now have to update some "vendored" dependencies
on a fresh git clone, at least in my environment, before building.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This was dependent on my own environment where I usually have /sbin
in $PATH. If that's missing, given that we're running dhclient as
user, we won't find it.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Parsing pstree's output is somewhat unreliable: there might be
multiple pasta instances running on the same host, and depending on
the overall output width pstree might truncate some branches.
Ask pasta to save its PID to file, and use that as parameter for
pgrep to find the PID of the interactive shell whose user and network
namespaces we want to join.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
For a while now, passt disables ARP functionality completely if IPv4
is disabled. If qrap sends an ARP request as a probe in that case, it
will receive no answer and move on, trying to find another instance.
Add a second probe frame, a hardcoded neighbour solicitation, so that
we get a neighbour advertisement if IPv6 is enabled.
Without this change, IPv6-only operation is completely broken.
Reported-by: Wenli Quan <wquan@redhat.com>
Reported-by: Alona Paz <alkaplan@redhat.com>
Link: https://bugzilla.redhat.com/show_bug.cgi?id=2106257
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
I forgot to reset the range endpoints after parsing an item of the
comma-separated list in commit 220759efb8 ("conf: Allow to specify
ranges and ports excluded from given ranges") -- fix that.
Fixes: 220759efb8 ("conf: Allow to specify ranges and ports excluded from given ranges")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This is useful in environments where we want to forward a large
number of ports, or all non-ephemeral ones, and some other service
running on the host needs a few selected ports.
I'm using ~ as prefix for the specification of excluded ranges and
ports to avoid the need for explicit command line quoting.
Ranges and ports can be excluded from given ranges by adding them
in the comma-separated list, prefixed by ~. Some quick examples:
-t 5000-6000,~5555: forward ports 5000 to 6000, but not 5555
-t ~20000-20010: forward all non-ephemeral, allowed ports, except
for ports 20000 to 20010
...more details in usage message and man page.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
In commit 675174d4ba ("conf, tap: Split netlink and pasta
functions, allow interface configuration"), I broke the initial
setting of the observed IPv6 addresses in two ways:
- the size copied from the configured addresses corresponds to an
IPv4 address, not to an IPv6 address
- the observed link-local address is initialised to the configured
unicast address, not the link-local one
If we haven't seen the guest using some type of addresses yet, we
should default to the configured values, hence these initial
settings: fix both.
This resulted in UDP flows to the guest from a unique local address
on the network not working before the guest shows passt a valid
address itself, as reported by Alona.
Reported-by: Alona Paz <alkaplan@redhat.com>
Link: https://bugs.passt.top/show_bug.cgi?id=16
Fixes: 675174d4ba ("conf, tap: Split netlink and pasta functions, allow interface configuration")
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
We handle SIGQUIT and SIGTERM calling exit(), which is usually
implemented with the exit_group() system call.
If we don't allow exit_group(), we'll get a SIGSYS while handling
SIGQUIT and SIGTERM, which means a misleading non-zero exit code.
Reported-by: Wenli Quan <wquan@redhat.com>
Link: https://bugzilla.redhat.com/show_bug.cgi?id=2101990
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
...instead of argv[0], which might or might not contain a valid path
to the executable itself. Instead of mangling argv[0], use the same
link to find out if we're already running the AVX2 build where
supported.
Alternatively, we could use execvpe(), but that might result in
running a different installed version, in case e.g. the set of
binaries is present in both /usr/bin and /usr/local/bin, with both
being in $PATH.
Reported-by: Wenli Quan <wquan@redhat.com>
Link: https://bugzilla.redhat.com/show_bug.cgi?id=2101310
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The Fedora test file extracts some information from the host resolv.conf
into a DNS6 variable which is then never used. Remove this unnecessary
step, which is presumably a leftover from an earlier iteration.
This was the only user of 'head' and 'sed' in the test file, so those can
also be removed from the required tools. The debian and ubuntu test files
also listed 'head' and 'sed' as tools, although they don't use them,
I'm guessing because of an earlier version which had the same DNS6 code.
Remove those as well.
The opensuse test file still actually uses DNS6, so leave it there for now.
The DNS handling and network config handling for SuSE looks to be kind of
broken, but fixing that is a job for another day.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Before booting the guest images, the distro test cases need to modify the
guest images, using virt-edit and guestfish, to boot in the way we need.
At present this gets repeated on every test run, even though it's not
really doing anything we want to test for.
In addition many of the images have the same preparation steps leading to
a lot of duplicated stages in the tests. A number of additional images can
be prepared using common steps, even if the ones used now have small
differences.
Therefore move the preparation of most of the guest images to the asset
build phase, where they can be done a single time for multiple test runs,
using a common preparation script. We can even avoid making a copy of the
disk image for booting, by using qemu's -snapshot option.
A few of the distros (openSUSE and older Ubuntu) do need different steps.
For now we don't chage how they are run, they could possibly be handled
more like this in future.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Rather than directly download distro images from the test scripts, handle
all the downloads during the test asset build, then just clone them for
the tests themselves. This avoids repeated downloads which can be very
slow when debugging failing tests.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio: Add OPENSUSE_IMGS to DOWNLOAD_ASSETS in Makefile, and note
that xzcat doesn't take a -O option in test/distro/opensuse]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Currently test/run uses wildcards to run all of the tests in a directory.
However, that wildcard list is filtered down by the "onlyfor" directives
in the test files... usually to a single file.
Therefore, just explicitly list the files we *really* want to run for this
test mode. This makes it easier to see at the top level what tests will
be executed, and to change that list temporarily while debugging specific
failures.
This means the "onlyfor" directive no longer has any purpose, and we can
remove it. "onlyfor" was also the only used of the $MODE variable, so we
can remove that too.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The top level listing control of which tests to run is in test/run, however
it uses the test() function which runs an entire directory of test files,
filtered by some criteria. This makes it awkward to narrow down to a
subset of tests when debugging a specific failure.
To make this easier, have test() take an explicit list of test files to
run, and have the caller in test/run handle the directory traversal. The
construct we use for this is pretty awkward to handle the fact that we're
in the source tree root directory rather than test/ at this point in
test/run. Later cleanups will improve that.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The test scripts support a "req" directive which requires one test script
to be run before another. It's implemented by doing a topological sort
based on these directives in the runner scripts, which is about as awkward
as you'd expect in Bourne shell.
It turns out we only use this functionality in one place - to make the
"make install" test run after the plain "make" test. We also already have
a simpler way of making sure tests run in a specific order: just put them
into the same test script file.
So, remove support for the "req" directive and just fold the build/all and
build/install test scripts together.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
If the tests are interrupted at the right point a passt.pid file can be
left over. Clean it up with "make clean" and add it to .gitignore so it
doesn't get accidentally committed.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Apparently qemu's ARM virt machine needs to be explicitly given a firmware
image, rather than just supplying a sane default. Unfortunately the EDK2
firmware image we need isn't in the same place on all host distros.
Currently the test scripts hardcode the Debian location, meaning it will
break on hosts that have it somewhere else. This patch searches multiple
locations for the firmware, and creates a local link during the asset build
phase, which the tests can then use.
For now it only searches the locations used by Debian and Fedora, but
that's a small improvement in robustness already, and can be later improved
further if we need to.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Move the download of mbuto and using it to create a sample initramfs to
the asset build makefile, rather than embedding it in the test scripts
themselves.
The two_guests tests used to use two separate copies of the mbuto
image. As an initramfs the mbuto image is strictly readonly though,
so that's not necessary. So, also use the same image for both guests.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
A number of passt/pasta testcases have initial steps which are just about
building images or other assets we need for the test proper. Repeating
these for each test run can be quite costly.
This patch makes a start on moving this sort of test asset building to
a separate phase before running the tests proper. For now just add a
Makefile to handle the asset building (although it doesn't build
anything yet), and make the path where we'll be building the assets
available to the tests.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
A lot of tests and examples invoke qemu with the command "kvm". However,
as far as I can tell, "kvm" being aliased to the appropriate qemu system
binary is Debian specific. The binary names from qemu upstream -
qemu-system-$ARCH - also aren't universal, but they are more common (they
should be good for both Debian and Fedora at least).
In order to still get KVM acceleration when available, we use the option
"-M accel=kvm:tcg" to tell qemu to try using either KVM or TCG in that
order
A number of the places we invoked "kvm" are expecting specifically an x86
guest, and so it's also safer to explicitly invoke qemu-system-x86_64.
Some others appear to be independent of the target arch (just wanting the
same arch as the host to allow KVM acceleration). Although I suspect there
may be more subtle x86 specific options in the qemu command lines, attempt
to preserve arch independence by using $(uname -m).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Several tests run pp64le guests using "qemu-system-ppc64le". But, at the
system level there's no difference between ppc64 and ppc64le - it's the
same hardware, just placed into different endian modes by OS early boot
code. Reflecting that, qemu only supplies a single "qemu-system-ppc64".
Some distros alias qemu-system-ppc64le to qemu-system-ppc64 (Debian does),
but it's best not to count on this (Fedora doesn't, for example).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
