Commit graph

196 commits

Author SHA1 Message Date
Stefano Brivio
dfddfdd8f8 test: Fix Podman build in Podman demo
For some reason, I now have to update some "vendored" dependencies
on a fresh git clone, at least in my environment, before building.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-07-22 19:41:42 +02:00
Stefano Brivio
506b4ede55 test: In pasta demo, issue /sbin/dhclient instead of dhclient
This was dependent on my own environment where I usually have /sbin
in $PATH. If that's missing, given that we're running dhclient as
user, we won't find it.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-07-22 19:41:42 +02:00
Stefano Brivio
a4778b3b81 test: In demos, use pgrep instead of pstree to find namespace PID
Parsing pstree's output is somewhat unreliable: there might be
multiple pasta instances running on the same host, and depending on
the overall output width pstree might truncate some branches.

Ask pasta to save its PID to file, and use that as parameter for
pgrep to find the PID of the interactive shell whose user and network
namespaces we want to join.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-07-22 19:41:18 +02:00
Stefano Brivio
95f7b4ec3f test: In passt demo, bring up eth0 in guest, not in namespace pane
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-07-21 13:50:50 +02:00
Stefano Brivio
8fc8fb7afe demo/passt: Bring interface up before starting dhclient in guest
...I forgot about one occurrence of this.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-07-14 12:38:42 +02:00
David Gibson
30ac86823b tests: Remove unused DNS6 calculation from fedora tests
The Fedora test file extracts some information from the host resolv.conf
into a DNS6 variable which is then never used.  Remove this unnecessary
step, which is presumably a leftover from an earlier iteration.

This was the only user of 'head' and 'sed' in the test file, so those can
also be removed from the required tools.  The debian and ubuntu test files
also listed 'head' and 'sed' as tools, although they don't use them,
I'm guessing because of an earlier version which had the same DNS6 code.
Remove those as well.

The opensuse test file still actually uses DNS6, so leave it there for now.
The DNS handling and network config handling for SuSE looks to be kind of
broken, but fixing that is a job for another day.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:36:05 +02:00
David Gibson
d2802ec874 tests: Prepare distro images during asset build phase
Before booting the guest images, the distro test cases need to modify the
guest images, using virt-edit and guestfish, to boot in the way we need.
At present this gets repeated on every test run, even though it's not
really doing anything we want to test for.

In addition many of the images have the same preparation steps leading to
a lot of duplicated stages in the tests.  A number of additional images can
be prepared using common steps, even if the ones used now have small
differences.

Therefore move the preparation of most of the guest images to the asset
build phase, where they can be done a single time for multiple test runs,
using a common preparation script.  We can even avoid making a copy of the
disk image for booting, by using qemu's -snapshot option.

A few of the distros (openSUSE and older Ubuntu) do need different steps.
For now we don't chage how they are run, they could possibly be handled
more like this in future.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:36:02 +02:00
David Gibson
32c5e05479 tests: Move distro image download to asset build makefile
Rather than directly download distro images from the test scripts, handle
all the downloads during the test asset build, then just clone them for
the tests themselves.  This avoids repeated downloads which can be very
slow when debugging failing tests.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio: Add OPENSUSE_IMGS to DOWNLOAD_ASSETS in Makefile, and note
 that xzcat doesn't take a -O option in test/distro/opensuse]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-07-14 01:34:37 +02:00
David Gibson
a832a44e67 tests: Explicitly list test files in test/run, remove "onlyfor" support
Currently test/run uses wildcards to run all of the tests in a directory.
However, that wildcard list is filtered down by the "onlyfor" directives
in the test files... usually to a single file.

Therefore, just explicitly list the files we *really* want to run for this
test mode.  This makes it easier to see at the top level what tests will
be executed, and to change that list temporarily while debugging specific
failures.

This means the "onlyfor" directive no longer has any purpose, and we can
remove it.  "onlyfor" was also the only used of the $MODE variable, so we
can remove that too.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:32:42 +02:00
David Gibson
544f790bf8 tests: Don't automatically traverse directories of test files
The top level listing control of which tests to run is in test/run, however
it uses the test() function which runs an entire directory of test files,
filtered by some criteria.  This makes it awkward to narrow down to a
subset of tests when debugging a specific failure.

To make this easier, have test() take an explicit list of test files to
run, and have the caller in test/run handle the directory traversal.  The
construct we use for this is pretty awkward to handle the fact that we're
in the source tree root directory rather than test/ at this point in
test/run.  Later cleanups will improve that.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:32:42 +02:00
David Gibson
5d7688d26f tests: Remove not-very-useful "req" directive
The test scripts support a "req" directive which requires one test script
to be run before another.  It's implemented by doing a topological sort
based on these directives in the runner scripts, which is about as awkward
as you'd expect in Bourne shell.

It turns out we only use this functionality in one place - to make the
"make install" test run after the plain "make" test.  We also already have
a simpler way of making sure tests run in a specific order: just put them
into the same test script file.

So, remove support for the "req" directive and just fold the build/all and
build/install test scripts together.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:32:42 +02:00
David Gibson
eb2e86dec0 tests: Remove unused set_mode() function
This utility function is never called.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:32:42 +02:00
David Gibson
b44e16ed6c tests: Search multiple places for aarch64 EDK2 bios image
Apparently qemu's ARM virt machine needs to be explicitly given a firmware
image, rather than just supplying a sane default.  Unfortunately the EDK2
firmware image we need isn't in the same place on all host distros.

Currently the test scripts hardcode the Debian location, meaning it will
break on hosts that have it somewhere else.  This patch searches multiple
locations for the firmware, and creates a local link during the asset build
phase, which the tests can then use.

For now it only searches the locations used by Debian and Fedora, but
that's a small improvement in robustness already, and can be later improved
further if we need to.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:32:42 +02:00
David Gibson
2297637251 tests: Move mbuto download and execution to asset build
Move the download of mbuto and using it to create a sample initramfs to
the asset build makefile, rather than embedding it in the test scripts
themselves.

The two_guests tests used to use two separate copies of the mbuto
image.  As an initramfs the mbuto image is strictly readonly though,
so that's not necessary.  So, also use the same image for both guests.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:32:42 +02:00
David Gibson
db551e5de0 tests: Introduce makefile for building test assets
A number of passt/pasta testcases have initial steps which are just about
building images or other assets we need for the test proper.  Repeating
these for each test run can be quite costly.

This patch makes a start on moving this sort of test asset building to
a separate phase before running the tests proper.  For now just add a
Makefile to handle the asset building (although it doesn't build
anything yet), and make the path where we'll be building the assets
available to the tests.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:32:42 +02:00
David Gibson
7bcc5930a6 Invoke specific qemu-system-* binaries
A lot of tests and examples invoke qemu with the command "kvm".  However,
as far as I can tell, "kvm" being aliased to the appropriate qemu system
binary is Debian specific.  The binary names from qemu upstream -
qemu-system-$ARCH - also aren't universal, but they are more common (they
should be good for both Debian and Fedora at least).

In order to still get KVM acceleration when available, we use the option
"-M accel=kvm:tcg" to tell qemu to try using either KVM or TCG in that
order

A number of the places we invoked "kvm" are expecting specifically an x86
guest, and so it's also safer to explicitly invoke qemu-system-x86_64.

Some others appear to be independent of the target arch (just wanting the
same arch as the host to allow KVM acceleration).  Although I suspect there
may be more subtle x86 specific options in the qemu command lines, attempt
to preserve arch independence by using $(uname -m).

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:32:42 +02:00
David Gibson
c4d8a77512 tests: qemu-system-ppc64le isn't a thing
Several tests run pp64le guests using "qemu-system-ppc64le".  But, at the
system level there's no difference between ppc64 and ppc64le - it's the
same hardware, just placed into different endian modes by OS early boot
code.  Reflecting that, qemu only supplies a single "qemu-system-ppc64".

Some distros alias qemu-system-ppc64le to qemu-system-ppc64 (Debian does),
but it's best not to count on this (Fedora doesn't, for example).

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-07-14 01:32:42 +02:00
Stefano Brivio
4d777144fd test: Embed script for dhclient(8) in mbuto(1) profile
David reports that dhclient-script(8) on Fedora needs a number of
binaries that are not included in PROGS of the current mbuto profile,
and we would also need to include hostnamectl(1) there, which will
fail without a systemd init.

Embed a minimal script for dhclient(8) in the profile itself, written
to /sbin/dhclient-script at boot, to just check what we need to check
out of DHCP and DHCPv6 functionality.

While at it, drop busybox and logger from PROGS, as we don't need them,
and add hostname(1). While DHCP option 12 isn't supported yet by the
DHCP implementation in passt, we should probably add it soon.

Note: owing to the simplicity of this script, we now need to bring up
the interface before starting dhclient: add this in test scripts where
it's not the case yet.

Reported-by: David Gibson <david@gibson.dropbear.id.au>
Suggested-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>

note that we need to bring up the interface before starting dhclient
2022-07-14 01:31:28 +02:00
Stefano Brivio
20c418f1f9 test: Add external mbuto profile, drop udhcpc, and switch to it
This depends on a future change in mbuto to accept external profile
files. Add a file defining what we need for tests and demos, dropping
udhcpc and script as they're not needed anymore, and switch to it.

Suggested-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-07-06 08:09:26 +02:00
David Gibson
25f515831c Makefile: Use $(BIN) and $(MANPAGES) variable to simplify several targets
There are several places which explicitly list the various generated
binaries, even though a $(BIN) variable already lists them.  There are
several more places that list all the manpage files, introduce a
$(MANPAGES) variable to remove that repetition as well.

Tweak the generation of pasta.1 as a link to passt.1 so it's not just made
as a side effect of the pasta target.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio: add passt.1 and qrap.1 to guest files for distro tests]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-06-18 09:06:00 +02:00
David Gibson
41c02e10db tests: Use nmap-ncat instead of openbsd netcat for pasta tests
A number of the testcases use options specific the OpenBSD version of
netcat.  That's available in Debian, but not easily available in Fedora.
Switch the pasta tests to using the nmap version of netcat (a.k.a. ncat).
This is easily available in both Debian and Fedora, and appears to be a
bit more modern and maintained as well.

ncat generally requires explicit listen addresses (which is good for
clarity anywhere).  Its default options appear to remove the need for the
-N and -q options.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio: changed one ncat listening address to IPv6 loopback]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-06-18 09:05:06 +02:00
David Gibson
e48373382f Use dhclient instead of udhcpc
For some reason, the passt/pasta tests and examples use dhclient for
DHCPv6, but in most cases use udhcpc for DHCPv4.  Change it to use dhclient
for both DHCPv4 and DHCPv6.  This means one less tool we need for testing,
plus dhclient is easily available on Fedora whereas udhcpc is not.

Note that the passt tests still rely on udhcpc indirectly because mbuto
wants to put it into the guest images it generates.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-06-15 09:38:10 +02:00
David Gibson
ca0c33ae5b Tweak dhclient arguments for readability
A number of tests and examples use dhclient in both IPv4 and IPv6 modes.
We use "dhclient -6" for IPv6, but usually just "dhclient" for IPv4.  Add
an explicit "-4" argument to make it more clear and explicit.

In addition, when dhclient is run from within pasta it usually won't be
"real" root, and so will not have access to write the default global pid
file.  This results in a mostly harmless but irritating error:
    Can't create /var/run/dhclient.pid: Permission denied
We can avoid that by using the --no-pid flag to dhclient.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-06-15 09:38:10 +02:00
David Gibson
2320ac3349 Don't abbreviate ip(8) arguments in examples and tests
ip(8)'s ability to take abbreviated arguments (e.g. "li sh" instead of
"link show") is very handy when using it interactively, but it doesn't make
for very readable scripts and examples when shown that way.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-06-15 09:38:10 +02:00
David Gibson
6703da44c1 tests: Use more explicit netcat options for distro/fedora tests
distro/fedora contains two versions of the basic tests, used for different
Fedora versions.  One uses explicit listening address for netcat in some
extra places, the other does not.  Apparently the older netcat versions
didn't require the explicit addresses.  Not supplying addresses doesn't
test anything useful though, just a detail in netcat's behaviour.  So,
it's cleaner to just always supply explicit addresses.

In addition, we're explicitly expecting the nmap version of ncat, also
known as "ncat".  So, it's more explicit what we're after if we invoke it
via that name rather than "nc", which will go via an /etc/alternatives
link.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio: Fix port argument in distro_quick_pasta_test{,_fedora34} too]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-06-15 09:37:03 +02:00
Stefano Brivio
3d4c2a44a6 tests: Don't check exit code for every command in demo mode
Having all those 'echo $?' is rather distracting in demos.

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Sefano Brivio <sbrivio@redhat.com>
2022-05-19 15:24:22 +02:00
Stefano Brivio
fe536d78ba tests: Don't count number of test units for demos
...there are no 'test' directives in demo, and this causes a
script failure.

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-05-19 15:24:19 +02:00
Stefano Brivio
c447344158 demo/pasta: Fix bad sleep directive
'sleep' always needs an argument, this was meant to introduce
a 2 seconds delay.

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-05-19 15:24:15 +02:00
Stefano Brivio
b4d41ac4b6 test/run: Return 0 from run(), exit value already reflects failures
There's no need to return non-zero if there have been failures in
run(), because the exit value is already determined from the number
of failures reported in the log file.

Return zero, so that this doesn't cause the script to fail, given we
now run it with -e.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-05-19 15:24:15 +02:00
Stefano Brivio
cfa8a78a39 test/perf/pasta_udp: Drop redundant assignment of ::1 to loopback interface
There are a few occurrences of this assignment, which are needed to
re-add ::1 as loopback address after the MTU has been increased
back from a value below 1280 bytes.

This one, however, is redundant, and causes an error in the
execution.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-05-19 15:24:15 +02:00
David Gibson
21e9cf7b95 tests: Simplify explicit checks for command success
A number of individual test cases use '*out' commands to check for success
of specific commands they've issued.  Now that the test harness is testing
for success of all issued commands as a matter of course, we no longer need
to do this.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-19 15:24:15 +02:00
David Gibson
4643b9b74c tests: Simplify *tools commands using pane_status
Now that we have pane_status to check the success of commands issued to
panes, we can more easily check for the success of the 'which' commands
used to check tool availability, rather than constructing, then parsing
special "skip" output.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-19 15:24:15 +02:00
David Gibson
13ad716f30 tests: Add pane_status command to check for success of issued commands
When we use pane_wait to wait for a command issued to a tmux pane to finish
we have no idea whether the command succeeded or not.  This means that the
test scripts can keep running long after the point something vital has
failed, making it difficult to work out what went wrong.

Add a new pane_status command that checks for success of the issued command
and use it in most places instead of pane_wait.  We still need explicit
pane_wait where we're gathering explicit output with pane_parse, because
the way we check the status with 'echo $?' means we lose track of that
output.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio:
 - instead of quitting the script, make a test fail if a command
    issued in a pane fails during a test, and loop until the status code is
    numeric in pane_status() as a hack to make it a bit more robust
 - retain usage of pane_wait() in iperf3 and teardown functions as we
   interrupt iperf3, passt, and pasta, so a non-zero exit code is expected
 - drop bogus ns_{1,2}_wait() calls in teardown_two_guests(), those
   functions were never implemented
 - use pane_status() for "guest" test directives too
]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-05-19 15:24:15 +02:00
David Gibson
3e0641f91f tests: Don't ignore errors during script
Most commands issued during the testing scripts aren't explicitly checked
for errors.  Therefore, if they fail, the shell will just keep on
executing.  This makes it difficult to figure out where things started
going wrong if things fall over.

Run the whole script with the set -e mode so that it will exit in the case
of any (unchecked) failing command.  To make this work we do need to add
explicit checks / fallbacks for some commands which we expect to fail.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio: use sh -e instead of setting -e later, so that we don't miss
 anything before set -e is issued]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-05-19 15:24:15 +02:00
David Gibson
c638129a9e tests: Improve control character filtering in pane_parse
pane_parse() attempts to grab the output from the last command issued
into a tmux pane.  It strips out control characters using tr, which in
particular includes the final \r\n.  However, this won't fully strip
out terminal escape sequences.  In particular this breaks if the shell
in the pane is bash, with enable-bracketed-paste enabled in readline.
That issues terminal sequences to enable and disable bracketed paste
mode around every shell prompt.

We can work around this because these escapes are followed by a \r
(CR).  More generally, it seems reasonable to assume that any terminal
shenanigans followed by a CR, but not an LF is supposed to be hidden.
So, use sed to strip everything before the second last CR.  We still
need the tr to remove the final \r\n from the string (sed processes a
line at a time, and doesn't consider the CRLF part of the buffer it's
processing).

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio: modify regexp to keep foo\r\r\n unchanged, by matching on at
 least one CR and a non-CR afterwards: that's the usual output pattern
 for bash on Debian 8 and Debian 9]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-05-19 15:24:15 +02:00
David Gibson
ae83999b75 tests: Don't globally set tmux default-shell
run_term() uses tmux set-option -g to globally set the default shell.
Unfortunately this hits a chicken-and-egg problem that's common with many
of tmux's session options.  If there isn't already a tmux server running,
we can't connect to set the option.  If we attempt this after starting our
session (and therefore the server), then the session will already be
started with the previous default shell.

In any case it's not a good idea to set tmux global options, since that
might interfere with whatever else the user is doing in tmux.  So, instead
set the default-shell option locally to the session after starting it.  To
make sure we get the right shell for our initial script, explicitly invoke
/bin/sh to interpret it.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-19 15:24:15 +02:00
David Gibson
14cbbd602c tests: Don't use tmux update-environment
The semantics of tmux's update-environment option are a bit confusing.
It says it means the given variables are copied into the session
environment from the source environment, but it's not entirely clear
what the "source" environment means.

From my experimentation it appeast to be the environment from which
the tmux *server* is launched, not the one issuing the 'new-session'
command.  That makes it pretty much useles, certainly in our case where
we have no way of knowing if the user has pre-existing tmux sessions.

Instead use the new-session -e option to explicitly pass in the variables
we want to propagate.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-19 15:24:15 +02:00
David Gibson
5f637a2060 tests: Add some debugging output for the test scripts themselves
The DEBUG option for tests/run enables debugging options to passt/pasta,
however that doesn't help with debugging the test scripts themselves, which
are fairly fragile.

Extend the DEBUG option so it also prints information on each command in
the test scripts to make it easier to work out where things are falling
over.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-19 15:24:15 +02:00
David Gibson
81c50acb21 tests: Remove unused XVFB variable
The XVFB variable is initialized at the beginning of test/run then never
used again.  I'm assuming it's a leftover from some ealier iteration.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-19 15:24:15 +02:00
Stefano Brivio
8e898ea71e tests: Update mbuto git URLs
The project is now at mbuto.sh, and git transport is enabled.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-05-19 15:24:15 +02:00
David Gibson
d3e842afc0 Add basic .gitignore files
Ignore various files generated during build or test.

Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2022-05-19 15:24:09 +02:00
Stefano Brivio
32210fb64f demo: podman: New port forwarding behaviour for pasta, minor fixes
Reflect the recent changes in the Podman adaptation (no port
forwarding by default).

It turns out that by running two iperf3 processes, sometimes
slirp4netns blocks the second connection until the first test is
done, thus doubling the throughput. Use a single process for
slirp4netns with slirp4netns port handling.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-05-02 23:14:37 +02:00
Stefano Brivio
6e9464ff0a test/lib: Add small delay before trying to parse output
Don't fetch the log too early, we might get output from previous
commands.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-04-07 11:44:35 +02:00
Stefano Brivio
a20d269630 test/distro: Set unprivileged_userns_clone on Debian Buster and earlier
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-04-07 11:44:35 +02:00
Stefano Brivio
081f7c8f4c test/lib: Consistent cols, rows, poster attributes for asciinema player
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-04-07 11:44:35 +02:00
Stefano Brivio
0ebaac9747 test/perf: Work-around for virtio_net hang before long streams from guest
I didn't have time to investigate the root cause for the virtio_net
TX hang yet. Add a quick work-around for the moment being.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-03-29 15:35:38 +02:00
Stefano Brivio
66a95e331e test, seccomp, Makefile: Switch to valgrind runs for passt functional tests
Pass to seccomp.sh a list of additional syscalls valgrind needs as
EXTRA_SYSCALLS in a new 'valgrind' make target, and add corresponding
support in seccomp.sh itself.

In test setup functions, start passt with valgrind, but not for
performance tests.

Add tests checking that valgrind exits without errors after all the
other tests in the group are done.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-03-29 15:35:38 +02:00
Stefano Brivio
a3bcca864e test: Add asciinema(1) as requirement for CI in README
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-03-28 17:11:40 +02:00
Stefano Brivio
700ce1f875 test/lib/video: Fill in href attributes of video shortcuts
...so that they can be indexed.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-03-28 17:11:40 +02:00
Stefano Brivio
d2e40bb8d9 conf, util, tap: Implement --trace option for extra verbose logging
--debug can be a bit too noisy, especially as single packets or
socket messages are logged: implement a new option, --trace,
implying --debug, that enables all debug messages.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-03-25 13:21:13 +01:00
Stefano Brivio
bec6d3e084 test/lib/setup: Unshare PID namespace in pasta_setup()
...otherwise, we'll leave processes (dhclient) around.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-03-02 05:00:21 +01:00
Stefano Brivio
763f281155 demo/pasta: Clean up before rebuilding with -g
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-03-01 21:32:08 +01:00
Stefano Brivio
213c397492 passt, pasta: Run-time selection of AVX2 build
Build-time selection of AVX2 flags and routines is not practical for
distributions, but limiting AVX2 usage to checksum routines with
specific run-time detection doesn't allow for easy performance gains
from auto-vectorisation of batched packet handling routines.

For x86_64, build non-AVX2 and AVX2 binaries, and implement a simple
wrapper replacing the current executable with the AVX2 build if it's
available, and if AVX2 is supported by the current CPU.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-28 16:46:28 +01:00
Stefano Brivio
deca1ebe50 test/distro/opensuse: Add Tumbleweed armv7l test
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-28 04:25:53 +01:00
Stefano Brivio
7992995d35 test/lib/term: Don't run demo when started as ./run
I changed this in a previous commit by mistake, restore the original
command.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-28 04:24:54 +01:00
Stefano Brivio
28fb960451 demo/pasta: Exit namespace in 'ns' pane before restarting pasta
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-28 02:50:25 +01:00
Stefano Brivio
08b7a2ec38 test: Fix name of CI asciinema player in perf links handler
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-23 13:21:52 +01:00
Stefano Brivio
337f55166f demo, ci: Switch to asciinema(1) for terminal recordings
For demos, cool-retro-term(1) looked fancier, but several threads of
that and ffmpeg(1) are just messing up with performance testing.

The CI videos started getting really big as well, and they were
difficult to read.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-22 18:36:24 +01:00
Stefano Brivio
be2a7898e9 test: Add demo for Podman with pasta
...showing setup steps, some peculiarities as --net option, and a
general side-to-side comparison with slirp4netns(1), including
"quick" TCP and UDP throughput and latency benchmarks.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-22 18:34:44 +01:00
Stefano Brivio
edb64ecbad test/perf/passt_udp: Drop threshold for 256B test
That test fails sometimes, it looks like iperf3 is still sending
initial messages that are too big. I'll need to figure out why,
but given that 256 bytes is not really an expected MTU, drop the
thresholds to zero for the moment being.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-21 13:41:13 +01:00
Stefano Brivio
550e2744ab test/distro/ubuntu: Use DEBIAN_FRONTEND=noninteractive for apt on 22.04
Removing the needrestart package doesn't seem to work anymore, and
I'm getting again prompts to restart services after installing gcc
and make: export DEBIAN_FRONTEND=noninteractive before installing
packages to avoid that.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-21 13:41:13 +01:00
Stefano Brivio
0515adceaa passt, pasta: Namespace-based sandboxing, defer seccomp policy application
To reach (at least) a conceptually equivalent security level as
implemented by --enable-sandbox in slirp4netns, we need to create a
new mount namespace and pivot_root() into a new (empty) mountpoint, so
that passt and pasta can't access any filesystem resource after
initialisation.

While at it, also detach IPC, PID (only for passt, to prevent
vulnerabilities based on the knowledge of a target PID), and UTS
namespaces.

With this approach, if we apply the seccomp filters right after the
configuration step, the number of allowed syscalls grows further. To
prevent this, defer the application of seccomp policies after the
initialisation phase, before the main loop, that's where we expect bad
things to happen, potentially. This way, we get back to 22 allowed
syscalls for passt and 34 for pasta, on x86_64.

While at it, move #syscalls notes to specific code paths wherever it
conceptually makes sense.

We have to open all the file handles we'll ever need before
sandboxing:

- the packet capture file can only be opened once, drop instance
  numbers from the default path and use the (pre-sandbox) PID instead

- /proc/net/tcp{,v6} and /proc/net/udp{,v6}, for automatic detection
  of bound ports in pasta mode, are now opened only once, before
  sandboxing, and their handles are stored in the execution context

- the UNIX domain socket for passt is also bound only once, before
  sandboxing: to reject clients after the first one, instead of
  closing the listening socket, keep it open, accept and immediately
  discard new connection if we already have a valid one

Clarify the (unchanged) behaviour for --netns-only in the man page.

To actually make passt and pasta processes run in a separate PID
namespace, we need to unshare(CLONE_NEWPID) before forking to
background (if configured to do so). Introduce a small daemon()
implementation, __daemon(), that additionally saves the PID file
before forking. While running in foreground, the process itself can't
move to a new PID namespace (a process can't change the notion of its
own PID): mention that in the man page.

For some reason, fork() in a detached PID namespace causes SIGTERM
and SIGQUIT to be ignored, even if the handler is still reported as
SIG_DFL: add a signal handler that just exits.

We can now drop most of the pasta_child_handler() implementation,
that took care of terminating all processes running in the same
namespace, if pasta started a shell: the shell itself is now the
init process in that namespace, and all children will terminate
once the init process exits.

Issuing 'echo $$' in a detached PID namespace won't return the
actual namespace PID as seen from the init namespace: adapt
demo and test setup scripts to reflect that.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-21 13:41:13 +01:00
Stefano Brivio
70c05e678e test/perf/passt_tcp: Drop iperf3 window size for host-to-guest tests
With a recent 5.15 kernel, passing a huge window size to iperf3 with
lower MTUs makes iperf3 stop sending packets after a few seconds --
I haven't investigated this in detail, but the window size will be
adjusted dynamically anyway and not passing it doesn't actually
affect throughput, so simply drop the option.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-02-01 17:12:24 +01:00
Stefano Brivio
154216d483 test/distro/ubuntu: Skip apt-get update for 16.04 on powerpc
Some recent change to xenial-updates broke dependencies for gcc,
it can't be installed anymore. Skipping apt-get update leaves gcc
dependencies in a consistent state, though.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-30 04:06:43 +01:00
Stefano Brivio
ec7b65fd79 test/distro: Avoid race between display of ns_msg and netcat exiting
The shell might report 'nc -6 -l -p 9999 > /tmp/ns_msg' as done
even after the subsequent 'echo' is done: wait one second before
reading out /tmp/ns_msg, to ensure we read that instead of the
"Done" message.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-30 03:09:46 +01:00
Stefano Brivio
b1f5688495 test: Add basic documentation about test suite, and cool-retro-term profile
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-28 18:51:50 +01:00
Stefano Brivio
6e61b4040a test: Add distribution tests for several architectures and kernel versions
The new tests check build and a simple case with pasta sending a
short message in both directions (namespace to init, init to
namespace).

Tests cover a mix of Debian, Fedora, OpenSUSE and Ubuntu combinations
on aarch64, i386, ppc64, ppc64le, s390x, x86_64.

Builds tested starting from approximately glibc 2.19, gcc 4.7, and
actual functionality approximately from 4.4 kernels, glibc 2.25,
gcc 4.8, all the way up to current glibc/gcc/kernel versions.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-28 18:51:50 +01:00
Stefano Brivio
808ab390a0 test/lib/test: Introduce 'def' directive for frequently used patterns
For distribution tests, we'll repeat some tests frequently. Add a
'def' directive that starts a block, ended by 'endef', whose
execution can then be triggered by simply giving its name as a
directive itself.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-28 02:05:19 +01:00
Stefano Brivio
8ca491e7c0 test/lib/term: Allow for a wider variety of prompt characters in pane_wait()
We might have highlighting and slightly different prompts across
different distributions, allow a more reasonable set of prompt
strings to be accepted as prompts.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-28 02:05:19 +01:00
Stefano Brivio
fb6fa99ca3 test/lib/video: Drop -preset ultrafast from ffmpeg arguments
It's not really needed on a reasonably powered CPU, and makes the
video contents way less readable.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-26 16:30:59 +01:00
Stefano Brivio
4ac48b6eff test/lib/setup: Don't rely on IFS to properly separate qemu arguments
...this gets needlessly annoying while playing with test cases.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-26 16:30:59 +01:00
Stefano Brivio
b8307a0a88 perf/passt_udp: Lower failure throughput thresholds with big MTUs
The throughput results in this test look quite variable, slightly
lower figures look reasonable anyway.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-26 16:30:59 +01:00
Stefano Brivio
c48758bec6 test/two_guests: Drop stray spaces after sleep directives
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-26 16:30:59 +01:00
Stefano Brivio
6910c1ef28 demo/pasta: Don't wait for pasta to return to a prompt
Debug information might be printed after a prompt is seen,
just wait those 3 seconds and be done with it.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2022-01-26 16:30:59 +01:00
Stefano Brivio
a77c5ef93a README, perf_report: Markdown and CSS fixes
Updating md2html on the server needs a few adjustments.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-22 14:52:47 +02:00
Stefano Brivio
2d75a3d71c test/two_guests: Fix sleep command for DAD
An inline comment prefixed by a space doesn't mean the space
is dropped, and sleep(1) will get a blank in its argument.

Move the comment on its own line.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-21 20:10:23 +02:00
Stefano Brivio
e934da3c81 test/two_guests: Let the guests end DAD before starting the DHCPv6 client
They'll start DAD as we bring up the interface, and the DHCPv6
client might be unreasonably delayed if we start it too early.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-21 17:54:03 +02:00
Stefano Brivio
627e18fa8a passt: Add cppcheck target, test, and address resulting warnings
...mostly false positives, but a number of very relevant ones too,
in tcp_get_sndbuf(), tcp_conn_from_tap(), and siphash PREAMBLE().

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-21 09:41:13 +02:00
Stefano Brivio
c3f8e4d2cd test/perf: Actually load passt enough to test UDP performance
With recent improvements, we're not CPU-bound at all while testing
UDP performance. Give the VM more memory and CPUs, forward two
additional ports, start up to four threads in parallel, and give
single iperf3 threads higher bandwidth targets.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-21 04:54:05 +02:00
Stefano Brivio
1f3d6f96b5 test/lib/test: Wait a bit longer before terminating iperf3 processes
Sometimes tests run a few seconds longer than expected, wait a few
more seconds.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-21 04:51:39 +02:00
Stefano Brivio
6257a2752e test/perf: Try sourcing maximum scaling frequency from cpufreq
On most recent CPUs, that's a better indication of all-core turbo
frequency, or non-turbo frequency, than /proc/cpuinfo.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-21 01:24:22 +02:00
Stefano Brivio
12cfa6444c passt: Add clang-tidy Makefile target and test, take care of warnings
Most are just about style and form, but a few were actually
serious mistakes (NDP-related).

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-20 08:34:22 +02:00
Stefano Brivio
7f1e7019cb test/demo: Don't wait for # after pasta is started by perf report
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-20 08:34:22 +02:00
Stefano Brivio
087b5f4dbb LICENSES: Add license text files, add missing notices, fix SPDX tags
SPDX tags don't replace license files. Some notices were missing and
some tags were not according to the SPDX specification, too.

Now reuse --lint from the REUSE tool (https://reuse.software/) passes.

Reported-by: Martin Hauke <mardnh@gmx.de>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-20 08:29:30 +02:00
Stefano Brivio
f154a0489a Makefile: Install man pages to /usr/share/man instead of /usr/man
Reported-by: Martin Hauke <mardnh@gmx.de>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-20 08:29:30 +02:00
Stefano Brivio
9df5027129 perf/passt_udp: Don't overshoot UDP bandwidth excessively on larger MTUs
...performance with 64KiB MTUs might look worse than with 9000bytes
on some configurations.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-19 09:30:42 +02:00
Stefano Brivio
7aaff3387a perf/passt_tcp: Don't exceed typical L3 cache sizes with buffers
...we might see misleading rate drops with larger MTUs otherwise.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-19 09:28:44 +02:00
Stefano Brivio
e8ac8a3b7c test/perf: Use CPU frequency from /proc/cpuinfo instead of cpupower(1)
Get it to work also in nested virtualisation environments.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-19 09:25:29 +02:00
Stefano Brivio
6231422dfb demo/pasta: Swap init>ns and ns>init flows
...make those short performance tests actually match table headers.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-15 20:46:17 +02:00
Stefano Brivio
f318174a93 test: Drop debugging left-overs in lib/util
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-14 13:15:12 +02:00
Giuseppe Scrivano
9a175cc2ce pasta: Allow specifying paths and names of namespaces
Based on a patch from Giuseppe Scrivano, this adds the ability to:

- specify paths and names of target namespaces to join, instead of
  a PID, also for user namespaces, with --userns

- request to join or create a network namespace only, without
  entering or creating a user namespace, with --netns-only

- specify the base directory for netns mountpoints, with --nsrun-dir

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
[sbrivio: reworked logic to actually join the given namespaces when
 they're not created, implemented --netns-only and --nsrun-dir,
 updated pasta demo script and man page]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-07 04:05:15 +02:00
Stefano Brivio
a26722b875 test/lib/term: Export PCAP and DEBUG variables for tmux sessions globally
Otherwise, this would depend on the local tmux configuration.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-05 20:02:03 +02:00
Stefano Brivio
8ec5adc989 test/lib/setup: Increase --max-stackframe in commented-out valgrind command
...so that I don't keep fighting with this for pasta clone() calls.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-05 20:02:03 +02:00
Stefano Brivio
529b245d2b demo/pasta: Enter the right directory before issuing perf report -g
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-10-04 22:21:21 +02:00
Stefano Brivio
4e5129719d test: Record CI and demo videos in Xvfb by default, fix demo setup sequence
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-09-29 16:45:26 +02:00
Stefano Brivio
061519b562 test: Add CI/demo scripts
Not really quick, definitely dirty.

Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
2021-09-27 15:10:35 +02:00