Having the "subcommand" first is more conventional and will make it more
natural for future extensions I have planned.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
In preparation for extending what it does.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
context_run() has a race condition if two commands are run in close
proximity (generally involving at least one in the background). Because we
always use the same name for the temporary fifo files, if another command
is issued while the fifos for the first still exist, mkfifo will fail,
typically causing the entire test script to jam.
Create unique names for the temporary fifos to avoid this problem.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
In practical terms, passt doesn't benefit from the additional
protection offered by the AGPL over the GPL, because it's not
suitable to be executed over a computer network.
Further, restricting the distribution under the version 3 of the GPL
wouldn't provide any practical advantage either, as long as the passt
codebase is concerned, and might cause unnecessary compatibility
dilemmas.
Change licensing terms to the GNU General Public License Version 2,
or any later version, with written permission from all current and
past contributors, namely: myself, David Gibson, Laine Stump, Andrea
Bolognani, Paul Holzinger, Richard W.M. Jones, Chris Kuhn, Florian
Weimer, Giuseppe Scrivano, Stefan Hajnoczi, and Vasiliy Ulyanov.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Fedora 32-35 are now old enough that they're not on all mirrors. Fetch
them from the archive server instead.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The current debian cloud images no longer include ppc64. Change to using
the latest snapshot which does include ppc64.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
On shell 'exit' commands, running shells from pasta, we might get:
Cannot set tty process group (No such process)
as some TTY devices might be unaccessible. This is harmless, but
after commit "pasta: propagate exit code from child command", we'll
get test failures there, at least with dash.
Ignore those explicitly with a ugly workaround: we can't simply do
something like:
exit || :
because the failure is reported by the shell itself once it exits,
regardless of the command evaluation.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Similarly to UDP cases, these were missing as it wasn't clear, when
the other tests were introduced, if using the global address of a
namespace, from the host, should have resulted in connections being
routed via the tap interface.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
These were missing as it wasn't clear, when the other tests were
introduced, if using the global address of a namespace, from the
host, should have resulted in traffic being routed via the tap
interface (as opposed to the loopback interface). We now clarified
that's actually the case.
Use same values and thresholds as the tests for loopback traffic, as
throughput figures currently indicate there isn't much difference.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
...instead of doing it after the test. Now that we have pre-built
guest images, we might also have old JSON files from previous,
interrupted test runs.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Now that we install the binary in /bin, and we have a link from
/usr/bin, change the path in the test itself as well. Otherwise
it works with bash but not with dash for some reason.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Now that we require 13c6be96618c ("net: stream: add unix socket")
in qemu to run the tests, we can also assume that commit df8d07081718
("virtio-net: fix bottom-half packet TX on asynchronous completion")
is present, as it was merged before that one.
This fixes the issue we attempted to work around in passt TCP and
UDP performance tests: finally drop that stuff.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
qemu commit 13c6be96618c ("net: stream: add unix socket") introduces
support for native AF_UNIX support, finally making qrap useless.
We can't quite drop that yet until a qemu release includes it, and
then we'll need to wait a while for users to switch anyway, but at
least for tests, we can use that support.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
As pasta now configures that target network namespace with
--config-net, we need to wait for addresses and routes to be actually
present. Just sending netlink messages doesn't mean this is done
synchronously.
A more elegant alternative, which probably makes sense regardless of
this test setup, would be to query, from pasta, addresses and routes
we added, and wait until they're there, before proceeding.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
These show a summary of memory usage in kernel and userspace with
different port forwarding configurations, details of userspace usage
using 'nm' (passt only uses statically allocated memory), and details
of kernel memory from slab reporting facilities.
This adds a new test image, mbuto.mem.img, with harcoded IPv4 and
IPv6 addresses and routes, and just the tools we need to start and
stop passt, to report from /proc/slabinfo, /proc/meminfo, and to
print and parse symbol sizes using nm(1).
passt can't pivot_root() for sandboxing purposes on ramfs, so we need
to create another filesystem and chroot into it, first.
We don't want to use pane context functions, as we're checking memory
usage for sockets: resort to screen-scraping.
Configure a dummy interface to provide passt with an appearance of
working IPv4 and IPv6 connectivity, contributed by David.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
This can be used for generic cell values with an arbitrary scale.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Instead of just disabling performance reports if running in demo
mode. This allows us to use table functions outside of performance
reports.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
I'm going to add yet another one of those, for which I have no quick
solution. It's a regression in some sense, but at least if we make
this regression more observable and defined, it should be easier to
find a comprehensive solution later, within this or another testing
framework.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
They're too slow to cope with current release cycles, and they
haven't found bugs in months, also because clang-tidy and cppcheck
would find most of them earlier.
Disable them for the moment. We should pre-install gcc and make in
non-x86 images, as those run on my test machine with qemu TCG, and
that's the real slow-down here. Then we can re-enable them.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
To test log files on a tmpfs mount, we need to unshare the mount
namespace, which means using a context for the passt pane is not
really practical at the moment, as we can't open a shell there, so
we would have to encapsulate all the commands under 'unshare -rUm',
plus the "inner" pasta command, running in turn a tcp_rr server.
It might be worth fixing this by e.g. detecting we are trying to
spawn an interactive shell and adding a special path in the context
setup with some form of stdin redirection -- I'm not sure it's doable
though.
For this reason, add a new layout, using a context only for the host
pane, while keeping the old command dispatch mechanism for the passt
pane.
We also need a new setup function that doesn't start pasta: we want
to start and restart it with different options.
Further, we need a 'pint' directive, to send an interrupt to the
passt pane: add that in lib/test.
All the tests before the one involving tmpfs and a detached mount
namespace were also tested with the context mechanism. To make an
eventual conversion easier, pass tcp_crr directly as a command on
pasta's command line where feasible.
While at it, fix the comment to the teardown_pasta() function.
The new test set can be semi-conveniently run as:
./run pasta_options/log_to_file
and it checks basic log creation, size of the log file after flooding
it with debug entries, rotations, and basic consistency after
rotations, on both an existing filesystem and a tmpfs, chosen as
it doesn't support collapsing data ranges via fallocate(), hence
triggering the fall-back mechanism for logging rotation.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The distro and performance tests are by far the slowest part of the passt
testsuite. Move them to the end of the testsuite run, so that it's easier
to do a quick test during development by letting the other tests run then
interrupting the test runner.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
To keep this simple, only support tests that have corresponding setup
and teardown functions implied by their path. For example:
./run passt/ndp
will trigger the 'passt' setup and teardown functions.
This is not really elegant, but it looks robust, and while David is
considering proper alternatives, it should be quite useful.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Add the -Wextra -pedantic and -std=c99 flags when compiling the nsholder
test helper to get extra compiler checks, like we already use for the
main source code.
While we're there, fix some %d (signed) printf descriptors being used
for unsigned values (uid_t and gid_t). Pointed out by cppcheck.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
This loop goes through and gives a numeric label to each pane, even though
we name the panes properly shortly thereafter. Looks like a leftover from
some earlier version. Remove it.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Many of our tests are based around performing transfers of sample data
across passt/pasta created links. The data flow here can be a bit
hard to follow since, e.g. we create a file transfer it to the guest,
then transfer it back to the host across several different tests.
This also means that the test cases aren't independent of each other.
Because we don't have the original file available at both ends in some
cases, we compare them by generating md5sums at each end and comparing
them, which is a bit complicated.
Make a number of changes to simplify this:
1. Pre-generate the sample data files as a test asset, rather than
building them on the fly during the tests proper
2. Include the sample data files in the mbuto guest image
3. Because we have good copies of the original data available in all
contexts, we can now simply use 'cmp' to check if the transfer
has worked, avoiding md5sum complications.
4. Similarly we can always use the original copy of the sample data
on the send side of each transfer, meaning that the tests become
more independent of each other.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
The setup functions for passt_in_ns and two_guests perform some fairly slow
dhclient calls to configure the network in the namespace before starting
the guest. This isn't really part of the tests, just necessary for the
operations later.
We can simplify and speed this up a bit by using pasta's '--config-net'
option to configure the networking for us. As a bonus this means we have
at least a minimal test of the --config-net option itself.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
When we start passt or pasta, it may take a short time to be ready to
handle packets, especially if running under valgrind. We have a
number of semi-arbitrary fixed sleeps to account for this.
We can do this more robustly by exploiting the fact that pasta/passt
doesn't write its pidfile until it's ready to go, so if we wait for
the pidfile to be created, we can proceed with confidence.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
These are hangovers from older ways of shutting down the pasta/passt
processes and no longer serve any purpose.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Add a shell helper function to wait for some command to succeed - typically
a test for something to be done by a background process. Use it in the
context code which waits for the guest to respond to ssh-over-vsock
connections.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
It's not used anymore. While at it, fix the function name in the
comment to perf_report_append_js().
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
That's the default port for iperf3, which also means that it's quite
likely in use on my test machine. Use different port numbers: recycle
the scheme we use in tests for passt and pasta's demo, use 5221-5224
(a bit shorter) for the slirp4netns container in Podman's demo.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
A cd to __STATEDIR__ results in a rather long command, that's not
very readable. Jump between directories using .. and relative paths,
once we're there.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
We won't necessarily run mbuto as part of regular tests: it can also
be used for demos or out-of-tree tests.
To keep the profile simple, leave the whole sshd setup there, which
is otherwise harmless, but don't fail if guest-key.pub is missing in
the current directory.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
If we use dhclient without creating a complete network configuration,
systemd-resolved will stop working after a while, and this sometimes
happens while we're still installing packages.
Disable it, together with systemd-networkd, while taking care of
removing the dhclient hook that prevents overriding /etc/resolv.conf.
While at it, it looks like removing snapd and needrestart actually
takes more time than keeping them: drop that line.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
I'm not sure why, but dhclient hangs otherwise. This reflects what we
do in the passt_in_ns setup steps.
Eventually, this whole block could go away if we let pasta configure
this network namespace with --config-net.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Starting tcp_rr, tcp_crr, udp_rr servers in the guest takes a bit
longer than starting the corresponding clients on the host, and we
end up starting clients before servers unless we add a delay there.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
If we start another server on the same port right away, we might fail
to bind the port. A small delay appears to be needed -- I'm not
entirely sure why at this point.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Unfortunately, this partially counters recent efforts by David to
speed up these tests, but it looks like iperf3 clients don't reliably
terminate, in some rare cases I couldn't isolate yet.
For the moment being, reintroduce the time-based wait approach, now
using the configurable test duration, and terminate the servers at
the end of it, in case they're stuck. There's no point in keeping
the 'sleep 2' later, so drop that, and while at it, make sure that
the stuck servers have time to flush the JSON output before we use
it.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
It appears that if we run throughput tests with one-second periodic
reports, the sending side of the vhost channel used for SSH-based
command dispatch occasionally stops working altogether. I haven't
investigated this further, all I see is that output is truncated
at some point, and doesn't resume.
If we use gzip compression (ssh -C) this happens less frequently,
but it still happens, seemingly indicating the issue is probably
related to vhost itself.
Disable periodic reports in iperf3 clients. The -i options were
actually redundant, so remove them from both test files as well as
from test_iperf3().
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
An iperf3 client might fail to send the control message indicating
the end of the test, if the kernel buffer doesn't accept it, and exit
without having sent it, as the control socket is non-blocking. Should
this happen, the server will just wait forever for this message,
instead of terminating.
Restore some of the behaviour that went away with the
"test: Rewrite test_iperf3" patch: instead of waiting on servers to
terminate, wait on the clients. When they are done, wait 2 seconds,
and then send SIGINT to the servers, which make them still write
out the JSON report before terminating.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
If we don't, guest command dispatch will fail altogether, given that
we use cat(1) on the enter file, which contains spaces.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
It looks like the workaround for the virtio_net TX hang issue is
working less reliably with the new command dispatch mechanism, I'm
not sure why. Switch to 10 seconds, at least for the moment.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
There are some 'sleep 1' commands between starting the socat server
and its corresponding client to avoid races due to the server not
being ready as we start sending data.
However, those don't cover all the cases where we might need them,
and in some cases the sleep command actually ended up being before
the server even starts.
This fixes occasional failures in TCP and UDP simple transfer tests,
that became apparent with the new command dispatch mechanism.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Otherwise, we're depending on having /sbin in $PATH. For some reason
I didn't completely grasp, with the new command dispatch mechanism
that's not the case anymore, even if I have /sbin in $PATH in the
parent shell.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
We use the [ "$x" -eq "$x" ] syntax to check if $x is a number. The
behaviour is clearly implied by POSIX, but some shells might actually
report the (intended) error, and dash floods script.log with
"Illegal number" error messages. Hide them.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Because UDP is connectionless we don't have an in-built end-of-stream
signal for our connectivity tests. We work around this by explicitly
adding an end marker to our sample data and killing the listening end once
it is seen.
However, socat has some built-in options - null-eof and shut-null - which
can be used to signal the end of stream with a zero-length UDP packet.
Use these to simplify how the UDP tests are implemented.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The tests generate a performance report in $BASEPATH/perf.js and
hooks/pre-push copies it to the website. To avoid cluttering the working
directory, instead put perf.js in $LOGDIR/web, since it's a test output
artefact. Update hooks/pre-push to copy from its new location.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The asciinema video handling creates a number of temporary files (.uncat,
.start, .stop) which currently go into the source tree. Put them in the
temporary state directory to avoid clutter.
The final processed output is now placed into test_logs/web/ along with the
corresponding .js file with links, since they're essentially test
artefacts. hooks/pre-push is updated to look for those files in the new
location when updating the web site.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Avoiding putting them in bare /tmp means they will be automatically
cleaned up with everything else.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently they go in the passt source tree with a fixed names, which means
their presence can mess with subsequent test runs.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The capture files are more or less a different form of log output from the
tests, so place them in $LOGDIR.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Instead of using the 'temp' and 'tempdir' DSL directives to create
temporary files, use fixed paths relative to __STATEDIR__. This has two
advantages:
1) The files are automatically cleaned up if the tests fail (and even if
that doesn't work they're easier to clean up manuall)
2) When debugging tests it's easier to figure out which of the temporary
files are relevant to whatever's going wrong
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently the context command dispatch subsystem creates a bunch of
temporary files in $LOGDIR, which is messy. Store them in $STATEDIR which
is for precisely this purpose. The logs from each context still go into
$LOGDIR.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We use this fifo to send messages to the information pane. Put it in the
state directory so it doesn't need its own cleanup.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The test scripts create a bunch of temporary files to keep track of
internal state. Some are made in /tmp with individual mktemp calls, some
go in the passt source directory, and some go in $LOGDIR. This can
sometimes make it messy to clean up after failed test runs.
Start cleaning this up by creating a single "state" directory ($STATEBASE)
in /tmp for all the state or temporary files used by a single test run.
Clean it up automatically in cleanup() - except when DEBUG==1, because
those files can be useful for debugging test script failures.
We create subdirectories under $STATEBASE for each setup function, exposed
as $STATESETUP. We also create subdirectories for each test script and
expose those to the scripts as __STATEDIR__.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We install a cleanup() function with 'trap' in order to clean up temporary
files we generate during the tests. However, we deinstall it after
run_term, which means it won't run in most of the cases where it would be
useful. Even if "run from_term" exits with an error, that error will be
hidden from the run_term wrapper because it's within a tmux session, so we
will return from run_term normally, uninstall the trap and never clean up.
In fact there's no reason to uninstall the trap at all, it works just as
well on the success exit path as an error exit path.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For example, passt/dhcp rather than dhcp/passt. This is more
consistent with the two_guests and other test groups, and makes some
other cleanups simpler.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Put the pieces together to use the new style context based dispatch for
the passt_in_pasta tests.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Now that we have all the pieces we need for issuing commands both into
namespaces and into guests, we can use those to convert the two_guests to
using only the new style context command issue.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Extends the context system in the test scripts to allow executing commands
within a guest. Do this without requiring an existing network in the guest
by using socat to run ssh via a vsock connection.
We do need some additional "sleep"s in the tests, because the new
faster dispatch means that sometimes we attempt to connect before
socat has managed to listen.
For now, only use this for the plain "passt" tests. The "passt_in_ns" and
other tests have additional complications we still need to deal with.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Extend the context system to allow commands to be run in a namespace
created with unshare, and use it for the namespace used in the pasta tests.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
In our test scripts we need to do some ugly parsing of /proc and/or pstree
output in order to get the PIDs of processes running in namespaces so that
we can connect to those namespaces with nsenter or pasta.
This is actually a pretty tricky problem with standard tools. To determine
the PID from the outside of the namespace we need to know how the process
of interest is related to the unshare or pasta process (child? one of
several children? grandchild?) as well as then parsing /proc or ps output.
This is slightly awkward now, and will get worse with future changes I'd
like to make to have processes are dispatched.
The obvious solution would be to have the process of interest (which we
control) report its own PID, but that doesn't work easily, because it is in
a PID namepace and sees only its local PID not the global PID we need to
address it from outside.
To handle this, add a small custom tool, "nsholder". This takes a path
and a mode parameter. In "hold" mode it will create a unix domain socket
bound to the path and listening. In "pid" mode it will get the "hold"ing
process's pid via the unix socket using SO_PEERCRED, which translates
between PID namespaces. In "stop" mode it will send a message to the
socket causing the "hold"ing process to clean up and exit.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Convert the pasta and passt tests to use new-style context execution
for the things that run in the "passt" frame. Don't touch the
passt_in_ns or two_guests tests yet, because they run passt inside a
namespace which introduces some additional complications we have yet
to handle.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Convert most of the tests to use the new-style system for issuing commands
for all host commands. We leave the distro tests for now: they use
the same pane for both host and guest commands which we'll need some more
things to deal with.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We're creating a system for tests to more reliably execute commands in
various contexts (e.g. host, guest, namespace). That transition is going
to happen over a number of steps though, so in the meantime we need to deal
with both the old-style issuing of commands via typing into and screen
scraping tmux panels, and the new-style system for executing commands in
context.
Introduce some transitional helpers which will issue a command via context
if the requested context is initialized, but will otherwise fall back to
the old style tmux panel based method. Re-implement the various test DSL
commands in terms of these new helpers.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We're moving to a new way of the tests dispatching commands to running in
contexts (host, guest, namespace, etc.). As we make this transition,
though, we still want the user to be able to watch the commands running
in a context, as they previously could from the commands issued in the
pane.
Add a helper to set up a pane to watch a context's log to allow this. In
some cases we currently issue commands from several different logical
contexts in the same pane, so allow a pane to watch several contexts at
once. Also use tail's --retry option to allow starting the watch before
we've initialized the context which will be useful in some cases.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For the tests, we need to run commands in various contexts: in the host,
in a guest or in a namespace. Currently we do this by running each context
in a tmux pane, and using tmux commands to type the commands into the
relevant pane, then screen-scrape the output for the results if we need
them.
This is very fragile, because we have to make various assumptions to parse
the output. Those can break if a shell doesn't have the prompt we expect,
if the tmux pane is too small or in various other conditions.
This starts some library functions for a new "context" system, that
provides a common way to invoke commands in a given context, in a way that
properly preserves stdout, stderr and the process return code.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Our test DSL has a number of paired commands to run something in the
background in a pane, then later to wait for it to complete. However, in
some of the tests we have these mismatched - starting a command in one
pane, then waiting for it in another.
We appear to get away with this for some reason, but it's not correct and
future changes make it cause more problems.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
test_iperf3() is a pretty inscrutable mess of nested background processes.
It has a number of ugly sleeps needed to wait for things to complete.
Rewrite it to be cleaner:
* Use the construct (a & b & wait) to run 'a' and 'b' in parallel, but
then wait for them both to complete before continuing
* This allows us to wait for both the server and client to finish, rather
than sleeping
* Use jq to do all the math we need to get the final result, rather than
jq followed by some complicated 'bc' mangling
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently all the throughput tests are run for 30s. This is reflected in
both the actual parameters given to the iperf commands, but also in the
matching sleeps in test_iperf3.
Allow this to be adjusted more easily with a new parameter to test_iperf3.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio: Reflect new parameter in comment to test_iperf3()]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
These two commands in the DSL to run an iperf client and server are always
used together, and some of the parameters must match between them. The
iperf3s must also be run more or less immediately after iperf3c, since
iperf3c will run a client in the background after a sleep and requires a
server to be running before it will work.
A bunch of things can be made cleaner if we make a single DSL command that
runs both sides of the test. For now make the combined command work
exactly like the two commands together did, warts and all.
This does lose the ability for the DSL scripts to give additional options
to the iperf3 server, but we weren't using that anyway.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
On new Ubuntu 22.04 images, stopping systemd-resolved to get the
dhclient script override resolv.conf doesn't work anymore. I
originally used that hack to avoid introducing a delay which is
needed when running it on TCG.
Keep systemd-resolved running instead, and wait for it to be ready
by retrying to resolve a domain a few times before installing
packages, so that we don't add another ugly delay that might
unnecessarily slow down things even further.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Performance tests use iperf3(1) with large windows, and these sysctl
entries are needed to run them unmodified.
The passt demo uses perf(1) to report syscall overhead, and that
needs access to hardware performance counters for unprivileged
users.
Reported-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
We start getting prompts about restarting outdated services: we're
using daily images but they might have been cached for a while now.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Currently in at least some of the testcases we kill qemu processes we're
done with by issuing a Control-C to the tmux panel it's running in. That
makes things harder as we try to move towards allowing "headless" testing
without tmux.
So, instead always use an explicit kill on a pid derived from a pidfile
for killing qemu. Note that we don't need to remove the pidfiles
afterwards, because qemu does that itself when terminated.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The test scripts run with sh -e, which means they will stop if any commands
return an error. That's generally desirable, because we won't continue
after things are hopeless due to an earlier step failing.
Unfortunately, the tmux setup we run the script in means it's not obvious
where any error messages related to such a failure will go. Depending on
exactly where the error occurs they might go to the original terminal
hidden behind tmux, or they might go to a tmux panel that's not visible in
the normal layouts.
To make it easier to find such error message, redirect direct output and
errors from the test script itself to a 'script.log' file in the logs
directory. When in DEBUG=1 mode, additionaly 'set -x' so we log all the
commands we execute to that file.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
For the passt and passt_in_ns tests we have a "shutdown" testcase that
checks for any errors from the passt process we were using (including
valgrind warnings). Do the same for pasta tests, so that we catch any
error codes from the pasta process.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The "valgrind" test cases are designed to pick up errors reported when
passt is running under valgrind. But what it actually does is just kill
the passt process, then see if it had a non-zero exit code. That means it
will equally well pick up any other problems which caused passt to exit
with an error status: either something detected within passt or as a result
of passt being killed by an unexpected signal.
The fact that the "valgrind" test is actually responsible for shutting down
the passt process is non-obvious and can lead to problems when selectively
running tests during debugging.
Rename the "valgrind" tests to "shutdown" tests and run it regardless of
whether we're using valgrind or not. This allows us to remove an ugly
speacial case in the passt_in_ns teardown code.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The queries we use in the test scripts to locate the external interface
or gateway can return multiple results. We get away with this because the
way we parse command output only looks at the last line. It's not really
correct, though, and improvements to our handling of command output will
mean it breaks.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Currently the build tests and distro tests share a common setup function.
That works for now, but changes we want to make will mean they need
slightly different setup, so split the setup functions in preparation.
Currently, neither build nor distro tests have any teardown function.
Again, future changes are going to mean we need to do some teardown, so
create some empty for now teardown functions in preparation.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
When recording tests or demos with asciinema we generate several temporary
files during post-processing. Add these to the .gitignore file so they're
not accidentally comitted.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The DEMO_XTERM and CI_XTERM variables defined in test/lib/term aren't used
anywhere. Remove them.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Both clang-tidy and cppcheck linting are handled by the same test file,
test/build/static_checkers. The two linters are independent of each other
though, and each one takes quite a long time. Split them into separate
files to make it easier to control which are executed from the top level
test script.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
We've recently converted most of our tests to use socat instead of
nc/netcat/ncat, because socat is more powerful and we don't need to deal
with the several possible variants of netcat.
We still use nc or ncat for the distro tests. Because there we control
the guest environment and can pick our tools, there isn't the same reason
to switch to socat. However, using socat here as well makes the tests
a bit easier to read, and doesn't require people reading or modifying them
to become familiar with an additional tool.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
[sbrivio: keep using netcat-openbsd in Ubuntu 16.04 ppc64 test, as socat
is unavailable there]
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Distribution packages reasonably expect to have a human-readable
Markdown version of the README under /usr/share/doc/, but all we have
right now is a heavily web-oriented version.
Introduce a ugly hack to strip web-oriented parts from the current
README and install it.
It should probably work the other way around: a human-readable README
could be used as a source for the web page. But cgit needs a file
that's in the tree, not something that can be built, and
https://passt.top/ is based on cgit. It should eventually be doable
to work around this in cgit, instead.
Reported-by: Benson Muite <benson_muite@emailplus.org>
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Given that a three-way git merge was enough to cope with context
changes in man pages, it's probably a good idea to enable that for
'git am' in the demo too.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
Now that the back end allows passt/pasta to use different external
interfaces for IPv4 and IPv6, use that to do the right thing in the case
that the host has IPv4 and IPv6 connectivity via different interfaces.
If the user hasn't explicitly chosen an interface, separately search for
a suitable external interface for each protocol.
As a bonus, this substantially simplifies the external interface probe. It
also eliminates a subtle confusing case where in some circumstances we
would pick the first interface in interface index order, and sometimes in
order of routes returned from netlink. On some network configurations that
could cause tests to fail, because the logic in the tests was subtly
different (it always used route order).
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
By default, passt itself attaches to the first host interface with a
default route. However, when determining the host interface name the tests
implicitly select the *last* host interface: they use a jq expression which
will list all interfaces with default routes, but the way output detection
works in the scripts, it will only pick up the last line.
If there are multiple interfaces with default routes on the host, and they
each have a different address, this can cause spurious test failures.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
A couple of days ago, we started running out of space there as we're
about to install gcc -- about 50 MiB are missing.
Given that virt-resize (which could be conveniently invoked by the
Makefile for tests) reorders partitions if we expand the first one,
resize the image using qemu-img from the test script itself, and then
take care of expanding root partition and filesystem online later.
This is probably a temporary hack, so I'm not looking for a more
generic or elegant solution at the moment.
Signed-off-by: Stefano Brivio <sbrivio@redhat.com>
For some reason using https to clone from the passt git repo is very slow,
at least from network-distant places. Use git protocol in the demo instead
to avoid a tedious wait to get the source.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
With pasta, the namespace interface name is generally the same as the host
interface name. We already rely on this in the dhcp/pasta tests, but for
no clear reason ndp/pasta separately determines the host interface name.
Remove this unnecessary step.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
The iperf based test commands create a bunch of .bw and .pid files for
each iperf client and server. The server side .bw files are cleaned
up afterwards, but the pid files are not, and none of the client side
files are cleaned up. The latter doesn't really matter when the
client is run on ephemeral guests, but sometimes we run it in a
namespace that shares the filesystem with the host.
Clean up all of these files after the tests.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Before starting the guests, these tests configure addresses in a pasta
namespace using dhclient. However, because it's a user namespace, it's
not running as "real" root and can't write to the dhclient pid file.
This doesn't stop it working, but causes an ugly error message which we
can avoid by using the --no-pid option.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
All the UDP tests use :> to truncate some temporary data files. This
appears to be so that they're empty before writing data to them with tee.
However tee, by default, truncates its output file anyway (you need tee -a
to append). So drop the unnecessary truncations.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
teardown_passt_in_ns() sends a ^D to the NS pane, which appears to be
intended to terminate the nsenter running there, leaving the namespace.
However, we've also sent a ^D to the PASST pane which will exit the pasta
instance which created the namespace. With the namespace destroyed the
nsenter in the NS pane will be killed, so it does not need to be exited
explicitly.
In fact sending the extra ^D can be harmful, since it will exit the shell
in which the nsenter was run, causing the whole pane to be closed. That
can then mean that the "pane_wait NS" hangs indefinitely. I believe this
will sometimes work, because there's a race between the various options
here, but it should be more reliable without the extra ^D.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
