Merge #243: Modularize Tests

1cc432a136 examples/deploy-container: use new extra-container features (Erik Arvstedt)
16b2783ae7 examples: enable running outside of working dir (Erik Arvstedt)
04075b108c test: use QEMU from stable nixpkgs (Erik Arvstedt)
03f8dbba47 test: add non-secure-node eval test (Erik Arvstedt)
bb763d6a26 run-tests: add 'eval' command (Erik Arvstedt)
36358066e4 spark-wallet: don't disable tor when onion-service is disabled (Erik Arvstedt)
c9251e72a1 README: add run-tests.sh to examples (Erik Arvstedt)
5a565dff66 netns test: use netns ips from config (Erik Arvstedt)
ac95fe7c82 netns test: don't test recurring-donations (Erik Arvstedt)
84744f38d7 netns test: disable backup test (Erik Arvstedt)
fcc67da9f4 test: add container support (Erik Arvstedt)
e99b7edb8e run-tests: pass script args verbatim to command without word splitting (Erik Arvstedt)
e7c397a485 run-tests: rename scriptDir -> testDir (Erik Arvstedt)
b552d17d55 run-tests: fix arg error messages (Erik Arvstedt)
572967d3ad extra-container: pre-release -> 0.5-pre (Erik Arvstedt)
ac6cee5c12 pkgs: add extra-container (Erik Arvstedt)
be2127ae5b test: fix noConnections configs (Erik Arvstedt)
1e18d3ea3b test: improve modularization (Erik Arvstedt)
14d2d97ba6 test: rename scenario withnetns -> netns (Erik Arvstedt)
9bf77ee3e8 backups test: simplify and speed up (Erik Arvstedt)
fcda69e8b6 netns test: connect from main netns (Erik Arvstedt)
24069aa2c6 electrs: add option 'monitoringPort' (Erik Arvstedt)
611cfe5a28 electrs: remove redundant daemonrpc option (Erik Arvstedt)
a19d3b07c2 electrs: add variable 'bitcoind' (Erik Arvstedt)
a6dde36b87 electrs: use consistent args formatting (Erik Arvstedt)
45bcbf683d test: rename test.nix -> tests.nix (Erik Arvstedt)
c92e85f707 test: rename base.py -> tests.py (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  nixbitcoin:
    ACK 1cc432a136
  jonasnick:
    ACK 1cc432a136

Tree-SHA512: 388f195c85e740937f9e2cecbd672a1f1e64e6ce06a75ba8167aba4fc77b70e07d14282e9ca117b31e9085e37ed4cf286f90f1204cdaa4c0360141039b1cae95
This commit is contained in:
Jonas Nick 2020-10-16 14:32:51 +00:00
commit 79067156da
No known key found for this signature in database
GPG Key ID: 4861DBF262123605
22 changed files with 835 additions and 568 deletions

View File

@ -15,7 +15,8 @@ env:
- secure: "xXCFZ7g+k5YmCGm8R8l3bZElVmt+RD1KscG3kGr5w4HyyDPTzFetPo+sT8bUpysDU0u3HWhfVhHtpog2mhNhwVl3tQwKXea3dHKC1i6ypBg3gjDngmJRR5wo++ocYDpK8qPaU7m/jHQTNFnTA4CbmMcc05GcYx/1Ai/ZGkNwWFjdIcVeOUoiol33gykMOXIGDg2qlXudt33wP53FHbX8L4fxzodWfAuxKK4AoGprxy5eSnU7LCaXxxJmu4HwuV+Ux2U1NfE/E33cvhlUvTQCswVSZFG06mg8rwhMG1ozsDvlL2itZlu/BeUQH5y3XMMlnJIUXUazkRBibf1w/ebVjpOF+anqkqmq8tcbFEa7T+RJeVTIsvP+L8rE8fcmuZtdg9hNmgRnLmaeT0vVwD1L2UqW9HdRyujdoS0jPYuoc1W7f1JQWfAPhBPQ1SrtKyNNqcbVJ34aN7b+4vCzRpQL1JTbmjzQIWhkiKN1qMo1v/wbIydW8yka4hc4JOfdQLaAJEPI1eAC1MLotSAegMnwKWE1dzm66MuPSipksYjZrvsB28cV4aCVUffIuRhrSr1i2afRHwTpNbK9U4/576hah15ftUdR79Sfkcoi1ekSQTFGRvkRIPYtkKLYwFa3jVA41qz7+IIZCf4TsApy3XDdFx91cRub7yPq9BeZ83A+qYQ="
jobs:
- TestModules=1 STABLE=1 SCENARIO=default
- TestModules=1 STABLE=1 SCENARIO=withnetns
- TestModules=1 STABLE=1 SCENARIO=netns
- EvalModules=1 STABLE=1
- PKG=hwi STABLE=1
- PKG=hwi STABLE=0
- PKG=lightning-charge STABLE=1
@ -34,6 +35,12 @@ env:
- PKG=joinmarket STABLE=0
script:
- printf '%s (%s)\n' "$NIX_PATH" "$VER"
#
- |
if [[ $EvalModules ]]; then
test/run-tests.sh --scenario full eval
travis_terminate 0
fi
- |
getBuildExpr() {
if [[ $TestModules ]]; then

View File

@ -24,11 +24,11 @@ The goal is to make it easy to deploy a reasonably secure Bitcoin node with a us
It should allow managing bitcoin (the currency) effectively and providing public infrastructure.
It should be a reproducible and extensible platform for applications building on Bitcoin.
Example
Examples
---
The easiest way to try out nix-bitcoin is to use one of the provided examples.
```
```bash
git clone https://github.com/fort-nix/nix-bitcoin
cd nix-bitcoin/examples/
nix-shell
@ -39,7 +39,7 @@ shut down immediately. They leave no traces (outside of `/nix/store`) on the hos
- [`./deploy-container.sh`](examples/deploy-container.sh) creates a [NixOS container](https://github.com/erikarvstedt/extra-container).\
This is the fastest way to set up a node.\
Requires: [NixOS](https://nixos.org/)
Requires: [Nix](https://nixos.org/), a systemd-based Linux distro and root privileges
- [`./deploy-qemu-vm.sh`](examples/deploy-qemu-vm.sh) creates a QEMU VM.\
Requires: [Nix](https://nixos.org/nix/)
@ -48,6 +48,28 @@ shut down immediately. They leave no traces (outside of `/nix/store`) on the hos
NixOps can be used to deploy to various other backends like cloud providers.\
Requires: [Nix](https://nixos.org/nix/), [VirtualBox](https://www.virtualbox.org)
#### Tests
The internal test suite is also useful for exploring features.
The following `run-tests.sh` commands leave no traces (outside of `/nix/store`) on
the host system.
```bash
git clone https://github.com/fort-nix/nix-bitcoin
cd nix-bitcoin/test
# Run a Python test shell inside a VM node
./run-tests.sh debug
print(succeed("systemctl status bitcoind"))
# Run a node in a container. Requires systemd and root privileges.
./run-tests.sh container
c systemctl status bitcoind
# Explore a single feature
./run-tests.sh --scenario electrs container
```
See [`run-tests.sh`](test/run-tests.sh) for a complete documentation.
Available modules
---
By default the `configuration.nix` provides:

View File

@ -9,67 +9,22 @@ set -euo pipefail
# script in the interactive shell.
if [[ $(sysctl -n net.ipv4.ip_forward) != 1 ]]; then
echo "Error: IP forwarding (net.ipv4.ip_forward) is not enabled"
exit 1
fi
if [[ ! -e /run/current-system/nixos-version ]]; then
echo "Error: This script needs NixOS to run"
echo "Error: IP forwarding (net.ipv4.ip_forward) is not enabled."
echo "Needed for container WAN access."
exit 1
fi
if [[ ! -v IN_NIX_SHELL ]]; then
echo "Running script in nix shell env..."
cd "${BASH_SOURCE[0]%/*}"
exec nix-shell --run "${BASH_SOURCE[0]}"
fi
# Cleanup on exit
cleanup() {
echo
echo "Deleting container..."
sudo extra-container destroy demo-node
}
trap "cleanup" EXIT
# Build container.
# You can re-run this command with a changed container config.
# The running container is then switched to the new config.
# Learn more: https://github.com/erikarvstedt/extra-container
#
sudo extra-container create --start <<'EOF'
{ pkgs, lib, ... }: let
containerName = "demo-node"; # container name length is limited to 11 chars
localAddress = "10.250.0.2"; # container address
hostAddress = "10.250.0.1";
in {
containers.${containerName} = {
privateNetwork = true;
inherit localAddress hostAddress;
config = { pkgs, config, lib, ... }: {
imports = [
<nix-bitcoin/examples/configuration.nix>
<nix-bitcoin/modules/secrets/generate-secrets.nix>
];
# Speed up evaluation
documentation.nixos.enable = false;
};
};
# Allow WAN access
systemd.services."container@${containerName}" = {
preStart = "${pkgs.iptables}/bin/iptables -w -t nat -A POSTROUTING -s ${localAddress} -j MASQUERADE";
# Delete rule
postStop = "${pkgs.iptables}/bin/iptables -w -t nat -D POSTROUTING -s ${localAddress} -j MASQUERADE || true";
};
}
EOF
# Run command in container
c() {
if [[ $# > 0 ]]; then
sudo extra-container run demo-node -- "$@" | cat;
else
sudo nixos-container root-login demo-node
fi
}
# Uncomment to start a container shell session
# interactive=1
# These commands can also be executed interactively in a shell session
demoCmds='
echo
echo "Bitcoind service:"
c systemctl status bitcoind
@ -85,8 +40,32 @@ c nodeinfo
echo
echo "Bitcoind data dir:"
sudo ls -al /var/lib/containers/demo-node/var/lib/bitcoind
'
# Uncomment to start a shell session here
# . start-bash-session.sh
if [[ ${interactive:-} ]]; then
runCmd=
else
runCmd=(--run bash -c "$demoCmds")
fi
# Cleanup happens at exit (see above)
# Build container.
# Learn more: https://github.com/erikarvstedt/extra-container
#
read -d '' src <<'EOF' || true
{ pkgs, lib, ... }: {
containers.demo-node = {
extra.addressPrefix = "10.250.0";
extra.enableWAN = true;
config = { pkgs, config, lib, ... }: {
imports = [
<nix-bitcoin/examples/configuration.nix>
<nix-bitcoin/modules/secrets/generate-secrets.nix>
];
};
};
}
EOF
$([[ $EUID = 0 ]] || echo sudo "PATH=$PATH" "NIX_PATH=$NIX_PATH") \
$(type -P extra-container) shell -E "$src" "${runCmd[@]}"
# The container is automatically deleted at exit

View File

@ -10,6 +10,7 @@ set -euo pipefail
if [[ ! -v IN_NIX_SHELL ]]; then
echo "Running script in nix shell env..."
cd "${BASH_SOURCE[0]%/*}"
exec nix-shell --run "${BASH_SOURCE[0]}"
fi

View File

@ -13,6 +13,7 @@ set -euo pipefail
if [[ ! -v IN_NIX_SHELL ]]; then
echo "Running script in nix shell env..."
cd "${BASH_SOURCE[0]%/*}"
exec nix-shell --run "${BASH_SOURCE[0]}"
fi

View File

@ -11,11 +11,6 @@ let
nixpkgs = import nixpkgs-path {};
nix-bitcoin = nixpkgs.callPackage nix-bitcoin-path {};
extraContainer = nixpkgs.callPackage (builtins.fetchTarball {
url = "https://github.com/erikarvstedt/extra-container/archive/6cced2c26212cc1c8cc7cac3547660642eb87e71.tar.gz";
sha256 = "0qr41mma2iwxckdhqfabw3vjcbp2ffvshnc3k11kwriwj14b766v";
}) {};
nix-bitcoin-unpacked = (import <nixpkgs> {}).runCommand "nix-bitcoin-src" {} ''
mkdir $out; tar xf ${builtins.fetchurl nix-bitcoin-release} -C $out
'';
@ -25,7 +20,7 @@ with nixpkgs;
stdenv.mkDerivation rec {
name = "nix-bitcoin-environment";
buildInputs = [ nix-bitcoin.nixops19_09 figlet extraContainer ];
buildInputs = [ nix-bitcoin.nixops19_09 nix-bitcoin.extra-container figlet ];
shellHook = ''
export NIX_PATH="nixpkgs=${nixpkgs-path}:nix-bitcoin=${toString nix-bitcoin-path}:."

View File

@ -5,6 +5,7 @@ let
cfg = config.services.electrs;
inherit (config) nix-bitcoin-services;
secretsDir = config.nix-bitcoin.secretsDir;
bitcoind = config.services.bitcoind;
in {
options.services.electrs = {
enable = mkEnableOption "electrs";
@ -33,19 +34,17 @@ in {
address = mkOption {
type = types.str;
default = "127.0.0.1";
description = "RPC listening address.";
description = "RPC and monitoring listening address.";
};
port = mkOption {
type = types.port;
default = 50001;
description = "RPC port.";
};
daemonrpc = mkOption {
type = types.str;
default = "127.0.0.1:8332";
description = ''
Bitcoin daemon JSONRPC 'addr:port' to connect
'';
monitoringPort = mkOption {
type = types.port;
default = 4224;
description = "Prometheus monitoring port.";
};
extraArgs = mkOption {
type = types.separatedString " ";
@ -57,7 +56,7 @@ in {
config = mkIf cfg.enable {
assertions = [
{ assertion = config.services.bitcoind.prune == 0;
{ assertion = bitcoind.prune == 0;
message = "electrs does not support bitcoind pruning.";
}
];
@ -74,7 +73,7 @@ in {
requires = [ "bitcoind.service" ];
after = [ "bitcoind.service" ];
preStart = ''
echo "cookie = \"${config.services.bitcoind.rpc.users.public.name}:$(cat ${secretsDir}/bitcoin-rpcpassword-public)\"" \
echo "cookie = \"${bitcoind.rpc.users.public.name}:$(cat ${secretsDir}/bitcoin-rpcpassword-public)\"" \
> electrs.toml
'';
serviceConfig = nix-bitcoin-services.defaultHardening // {
@ -84,22 +83,25 @@ in {
ExecStart = ''
${pkgs.nix-bitcoin.electrs}/bin/electrs -vvv \
${if cfg.high-memory then
traceIf (!config.services.bitcoind.dataDirReadableByGroup) ''
traceIf (!bitcoind.dataDirReadableByGroup) ''
Warning: For optimal electrs syncing performance, enable services.bitcoind.dataDirReadableByGroup.
Note that this disables wallet support in bitcoind.
'' ""
else
"--jsonrpc-import --index-batch-size=10"
} \
--db-dir '${cfg.dataDir}' --daemon-dir '${config.services.bitcoind.dataDir}' \
--electrum-rpc-addr=${toString cfg.address}:${toString cfg.port} \
--daemon-rpc-addr=${toString cfg.daemonrpc} ${cfg.extraArgs}
--db-dir='${cfg.dataDir}' \
--daemon-dir='${bitcoind.dataDir}' \
--electrum-rpc-addr=${cfg.address}:${toString cfg.port} \
--monitoring-addr=${cfg.address}:${toString cfg.monitoringPort} \
--daemon-rpc-addr=${builtins.elemAt bitcoind.rpcbind 0}:${toString bitcoind.rpc.port} \
${cfg.extraArgs}
'';
User = cfg.user;
Group = cfg.group;
Restart = "on-failure";
RestartSec = "10s";
ReadWritePaths = "${cfg.dataDir} ${if cfg.high-memory then "${config.services.bitcoind.dataDir}" else ""}";
ReadWritePaths = "${cfg.dataDir} ${if cfg.high-memory then "${bitcoind.dataDir}" else ""}";
} // (if cfg.enforceTor
then nix-bitcoin-services.allowTor
else nix-bitcoin-services.allowAnyIP

View File

@ -298,10 +298,7 @@ in {
cliExec = mkCliExec "liquidd";
};
services.electrs = {
address = netns.electrs.address;
daemonrpc = "${netns.bitcoind.address}:${toString config.services.bitcoind.rpc.port}";
};
services.electrs.address = netns.electrs.address;
services.spark-wallet = {
host = netns.spark-wallet.address;

View File

@ -71,9 +71,6 @@ in {
};
users.groups.spark-wallet = {};
services.tor.enable = cfg.onion-service;
# requires client functionality for Bitcoin rate lookup
services.tor.client.enable = true;
services.tor.hiddenServices.spark-wallet = mkIf cfg.onion-service {
map = [{
port = 80; toPort = 9737; toHost = cfg.host;

View File

@ -13,6 +13,7 @@
nixops19_09 = pkgs.callPackage ./nixops { };
netns-exec = pkgs.callPackage ./netns-exec { };
lightning-loop = pkgs.callPackage ./lightning-loop { };
extra-container = pkgs.callPackage ./extra-container { };
pinned = import ./pinned.nix;

View File

@ -0,0 +1,36 @@
{ stdenv, lib, nixos-container, openssh
, glibcLocales
}:
stdenv.mkDerivation rec {
name = "extra-container-${version}";
version = "0.5-pre";
src = builtins.fetchTarball {
url = "https://github.com/erikarvstedt/extra-container/archive/${version}.tar.gz";
sha256 = "0gdy2dpqrdv7f4kyqz88j34x1p2fpav04kznv41hwqq88hmzap90";
};
buildCommand = ''
install -D $src/extra-container $out/bin/extra-container
patchShebangs $out/bin
share=$out/share/extra-container
install $src/eval-config.nix -Dt $share
# Use existing PATH for systemctl and machinectl (for nixos-container)
scriptPath="export PATH=${lib.makeBinPath [ nixos-container openssh ]}:\$PATH"
sed -i \
-e "s|evalConfig=.*|evalConfig=$share/eval-config.nix|" \
-e "s|LOCALE_ARCHIVE=.*|LOCALE_ARCHIVE=${glibcLocales}/lib/locale/locale-archive|" \
-e "2i$scriptPath" \
$out/bin/extra-container
'';
meta = with lib; {
description = "Run declarative containers without full system rebuilds";
homepage = https://github.com/erikarvstedt/extra-container;
license = licenses.mit;
maintainers = [ maintainers.earvstedt ];
};
}

View File

@ -1,193 +0,0 @@
is_interactive = "is_interactive" in vars()
def succeed(*cmds):
"""Returns the concatenated output of all cmds"""
return machine.succeed(*cmds)
def assert_matches(cmd, regexp):
out = succeed(cmd)
if not re.search(regexp, out):
raise Exception(f"Pattern '{regexp}' not found in '{out}'")
def assert_full_match(cmd, regexp):
out = succeed(cmd)
if not re.fullmatch(regexp, out):
raise Exception(f"Pattern '{regexp}' doesn't match '{out}'")
def log_has_string(unit, str):
return f"journalctl -b --output=cat -u {unit} --grep='{str}'"
def assert_no_failure(unit):
"""Unit should not have failed since the system is running"""
machine.fail(log_has_string(unit, "Failed with result"))
def assert_running(unit):
machine.wait_for_unit(unit)
assert_no_failure(unit)
def run_tests(extra_tests):
"""
:param extra_tests: Test functions that hook into the testing code below
:type extra_tests: Dict[str, Callable[]]
"""
# Don't execute the following test suite when this script is running in interactive mode
if is_interactive:
raise Exception()
test_security()
assert_running("bitcoind")
machine.wait_until_succeeds("bitcoin-cli getnetworkinfo")
assert_matches("su operator -c 'bitcoin-cli getnetworkinfo' | jq", '"version"')
# RPC access for user 'public' should be restricted
machine.fail(
"bitcoin-cli -rpcuser=public -rpcpassword=$(cat /secrets/bitcoin-rpcpassword-public) stop"
)
machine.wait_until_succeeds(
log_has_string("bitcoind", "RPC User public not allowed to call method stop")
)
assert_running("electrs")
extra_tests.pop("electrs")()
# Check RPC connection to bitcoind
machine.wait_until_succeeds(log_has_string("electrs", "NetworkInfo"))
# Stop electrs from spamming the test log with 'wait for bitcoind sync' messages
succeed("systemctl stop electrs")
assert_running("liquidd")
machine.wait_until_succeeds("elements-cli getnetworkinfo")
assert_matches("su operator -c 'elements-cli getnetworkinfo' | jq", '"version"')
succeed("su operator -c 'liquidswap-cli --help'")
assert_running("clightning")
assert_matches("su operator -c 'lightning-cli getinfo' | jq", '"id"')
assert_running("lnd")
assert_matches("su operator -c 'lncli getinfo' | jq", '"version"')
assert_no_failure("lnd")
assert_running("lightning-loop")
assert_matches("su operator -c 'loop --version'", "version")
# Check that lightning-loop fails with the right error, making sure
# lightning-loop can connect to lnd
machine.wait_until_succeeds(
log_has_string(
"lightning-loop",
"Waiting for lnd to be fully synced to its chain backend, this might take a while",
)
)
assert_running("nbxplorer")
machine.wait_until_succeeds(log_has_string("nbxplorer", "BTC: RPC connection successful"))
extra_tests.pop("nbxplorer")()
assert_running("btcpayserver")
machine.wait_until_succeeds(log_has_string("btcpayserver", "Listening on"))
extra_tests.pop("btcpayserver")()
assert_running("spark-wallet")
extra_tests.pop("spark-wallet")()
assert_running("lightning-charge")
extra_tests.pop("lightning-charge")()
assert_running("nanopos")
extra_tests.pop("nanopos")()
assert_running("onion-chef")
assert_running("joinmarket")
machine.wait_until_succeeds(
log_has_string("joinmarket", "P2EPDaemonServerProtocolFactory starting on 27184")
)
machine.wait_until_succeeds(
log_has_string("joinmarket-yieldgenerator", "Failure to get blockheight",)
)
# FIXME: use 'wait_for_unit' because 'create-web-index' always fails during startup due
# to incomplete unit dependencies.
# 'create-web-index' implicitly tests 'nodeinfo'.
machine.wait_for_unit("create-web-index")
assert_running("nginx")
extra_tests.pop("web-index")()
machine.wait_until_succeeds(log_has_string("bitcoind-import-banlist", "Importing node banlist"))
assert_no_failure("bitcoind-import-banlist")
### Additional tests
# Current time in µs
pre_restart = succeed("date +%s.%6N").rstrip()
# Sanity-check system by restarting all services
succeed(
"systemctl restart bitcoind clightning lnd lightning-loop spark-wallet lightning-charge nanopos liquidd"
)
# Now that the bitcoind restart triggered a banlist import restart, check that
# re-importing already banned addresses works
machine.wait_until_succeeds(
log_has_string(f"bitcoind-import-banlist --since=@{pre_restart}", "Importing node banlist")
)
assert_no_failure("bitcoind-import-banlist")
extra_tests.pop("prestop")()
### Test duplicity
succeed("systemctl stop bitcoind")
succeed("systemctl start duplicity")
machine.wait_until_succeeds(log_has_string("duplicity", "duplicity.service: Succeeded."))
# Make sure files in duplicity backup and /var/lib are identical
assert_matches(
"export $(cat /secrets/backup-encryption-env); duplicity verify '--archive-dir' '/var/lib/duplicity' 'file:///var/lib/localBackups' '/var/lib'",
"0 differences found",
)
# Make sure duplicity backup includes important files
assert_matches(
"export $(cat /secrets/backup-encryption-env); duplicity list-current-files 'file:///var/lib/localBackups'",
"var/lib/clightning/bitcoin/hsm_secret",
)
assert_matches(
"export $(cat /secrets/backup-encryption-env); duplicity list-current-files 'file:///var/lib/localBackups'",
"secrets/lnd-seed-mnemonic",
)
assert_matches(
"export $(cat /secrets/backup-encryption-env); duplicity list-current-files 'file:///var/lib/localBackups'",
"secrets/jm-wallet-seed",
)
assert_matches(
"export $(cat /secrets/backup-encryption-env); duplicity list-current-files 'file:///var/lib/localBackups'",
"var/lib/bitcoind/wallet.dat",
)
assert_matches(
"export $(cat /secrets/backup-encryption-env); duplicity list-current-files 'file:///var/lib/localBackups'",
"var/backup/postgresql/btcpaydb.sql.gz",
)
### Check that all extra_tests have been run
assert len(extra_tests) == 0
def test_security():
assert_running("setup-secrets")
# Unused secrets should be inaccessible
succeed('[[ $(stat -c "%U:%G %a" /secrets/dummy) = "root:root 440" ]]')
# Access to '/proc' should be restricted
machine.succeed("grep -Fq hidepid=2 /proc/mounts")
machine.wait_for_unit("bitcoind")
# `systemctl status` run by unprivileged users shouldn't leak cgroup info
assert_matches(
"sudo -u electrs systemctl status bitcoind 2>&1 >/dev/null",
"Failed to dump process list for 'bitcoind.service', ignoring: Access denied",
)
# The 'operator' with group 'proc' has full access
assert_full_match("sudo -u operator systemctl status bitcoind 2>&1 >/dev/null", "")

89
test/lib/make-container.sh Executable file
View File

@ -0,0 +1,89 @@
#!/usr/bin/env bash
# Usage:
#
# run-tests.sh [--scenario|-s <scenario>] container
#
# Start container and start a shell session with helper commands
# for accessing the container.
# A short command documentation is printed at the start of the session.
# The container is destroyed after exiting the shell.
# An existing container is destroyed before starting.
#
# Supported arguments:
#
# --destroy|-d to destroy
#
# When `run-tests.sh container` from inside an existing shell session,
# the current container is updated without restarting by switching
# its NixOS configuration.
# Use this arg to destroy and restart the container instead.
#
# --no-destroy|-n
#
# By default, all commands destroy an existing container before starting and,
# when appropriate, before exiting.
# This ensures that containers start with no leftover filesystem state from
# previous runs and that containers don't consume system resources after use.
# This args disables auto-destructing containers.
#
#
# run-tests.sh container --run|-r c systemctl status bitcoind
#
# Run a command in the shell session environmentand exit.
# Destroy the container afterwards.
# All arguments following `--run` are used as a command.
# Supports argument '--no-destroy|-n' (see above for an explanation).
#
# Example: Start shell inside container
# run-tests.sh container --run c
#
#
# run-tests.sh [--scenario|-s <scenario>] container --command|--c
#
# Provide a custom extra-container command.
#
# Example:
# run-tests.sh container --command create -s
# Create and start a container without a shell.
#
#
# All extra args are passed to extra-container (unless --command is used):
# run-tests.sh container --build-args --builders 'ssh://worker - - 8'
set -euo pipefail
if [[ $EUID != 0 ]]; then
# NixOS containers require root permissions.
# By using sudo here and not at the user's call-site extra-container can detect if it is running
# inside an existing shell session (by checking an internal environment variable).
exec sudo scenario="$scenario" testDir="$testDir" NIX_PATH="$NIX_PATH" PATH="$PATH" \
scenarioOverridesFile="${scenarioOverridesFile:-}" "$testDir/lib/make-container.sh" "$@"
fi
export containerName=nb-test
containerCommand=shell
while [[ $# > 0 ]]; do
case $1 in
--command|-c)
shift
containerCommand=$1
shift
;;
*)
break
esac
done
containerBin=$(type -P extra-container) || true
if [[ ! ($containerBin && $(realpath $containerBin) == *extra-container-0.5*) ]]; then
echo "Building extra-container. Skip this step by adding extra-container 0.5 to PATH."
nix-build --out-link /tmp/extra-container "$testDir"/../pkgs -A extra-container >/dev/null
export PATH="/tmp/extra-container/bin${PATH:+:}$PATH"
fi
read -d '' src <<EOF || true
(import "$testDir/tests.nix" { scenario = "$scenario"; }).container
EOF
exec extra-container $containerCommand -E "$src" "$@"

View File

@ -3,11 +3,6 @@ testArgs:
let
pkgs = import <nixpkgs> { config = {}; overlays = []; };
pkgs19_09 = import (pkgs.fetchzip {
url = "https://github.com/NixOS/nixpkgs-channels/archive/a7ceb2536ab11973c59750c4c48994e3064a75fa.tar.gz";
sha256 = "0hka65f31njqpq7i07l22z5rs7lkdfcl4pbqlmlsvnysb74ynyg1";
}) { config = {}; overlays = []; };
test = (import "${pkgs.path}/nixos/tests/make-test-python.nix") testArgs;
fixedTest = { system ? builtins.currentSystem, ... }@args:
@ -23,11 +18,6 @@ let
exec ${pkgs.python3Packages.black}/bin/black $extraArgs "$@"
'';
};
# QEMU 4.20 from unstable fails on Travis build nodes with message
# "error: failed to set MSR 0x48b to 0x159ff00000000"
# Use version 4.0.1 instead.
inherit (pkgs19_09) qemu_test;
};
in
test (args // { pkgs = pkgsFixed; });

51
test/lib/make-test.nix Normal file
View File

@ -0,0 +1,51 @@
scenario: testConfig:
{
vm = import ./make-test-vm.nix {
name = "nix-bitcoin-${scenario}";
machine = {
imports = [ testConfig ];
# Needed because duplicity requires 270 MB of free temp space, regardless of backup size
virtualisation.diskSize = 1024;
};
testScript = nodes: let
cfg = nodes.nodes.machine.config;
data = {
data = cfg.test.data;
tests = cfg.tests;
};
dataFile = builtins.toFile "test-data" (builtins.toJSON data);
initData = ''
import json
with open("${dataFile}") as f:
data = json.load(f)
enabled_tests = set(test for (test, enabled) in data["tests"].items() if enabled)
test_data = data["data"]
'';
in
builtins.concatStringsSep "\n\n" [
initData
(builtins.readFile ./../tests.py)
# Don't run tests in interactive mode.
# is_interactive is set in ../run-tests.sh
''
if not "is_interactive" in vars():
run_tests()
''
];
};
container = {
# The container name has a 11 char length limit
containers.nb-test = { config, ...}: {
config = {
extra = config.config.test.container;
config = testConfig;
};
};
};
}

39
test/lib/test-lib.nix Normal file
View File

@ -0,0 +1,39 @@
{ config, lib, ... }:
with lib;
{
options = {
test = {
noConnections = mkOption {
type = types.bool;
default = !config.test.container.enableWAN;
description = ''
Whether services should be configured to not connect to external hosts.
This can silence some warnings while running the test in an offline environment.
'';
};
data = mkOption {
type = types.attrs;
default = {};
description = ''
Attrs that are available in the Python test script under the global
dictionary variable 'test_data'. The data is exported via JSON.
'';
};
container = {
# Forwarded to extra-container. For descriptions, see
# https://github.com/erikarvstedt/extra-container/blob/master/eval-config.nix
addressPrefix = mkOption { default = "10.225.255"; };
enableWAN = mkOption { default = false; };
firewallAllowHost = mkOption { default = true; };
exposeLocalhost = mkOption { default = false; };
};
};
tests = mkOption {
type = with types; attrsOf bool;
default = {};
description = "Python tests that should be run.";
};
};
}

View File

@ -1,14 +1,20 @@
#!/usr/bin/env bash
# Modules integration test runner.
# The test (./test.nix) uses the NixOS testing framework and is executed in a VM.
# The tests (./tests.nix) use the NixOS testing framework and are executed in a VM.
#
# Usage:
# Run all tests
# ./run-tests.sh
#
# Test specific scenario
# ./run-tests.sh --scenario <scenario>
# ./run-tests.sh --scenario|-s <scenario>
#
# When <scenario> is undefined, the test is run with an adhoc scenario
# where services.<scenario> is enabled.
#
# Example:
# ./run-tests.sh -s electrs
#
# Run test and link results to avoid garbage collection
# ./run-tests.sh [--scenario <scenario>] --out-link-prefix /tmp/nix-bitcoin-test build
@ -20,7 +26,17 @@
# ./run-tests.sh [--scenario <scenario>] debug
#
# This starts the testing VM and drops you into a Python REPL where you can
# manually execute the tests from ./test-script.py
# manually execute the tests from ./tests.py
#
# Run a test scenario in a container
# sudo ./run-tests.sh [--scenario <scenario>] container
#
# This is useful for quick experiments; containers start much faster than VMs.
# Running the Python test suite in containers is not yet supported.
# For now, creating NixOS containers requires root permissions.
# See ./lib/make-container.sh for a complete documentation.
#
# To add custom scenarios, set the environment variable `scenarioOverridesFile`.
set -eo pipefail
@ -34,7 +50,7 @@ while :; do
shift
shift
else
>&2 echo 'Error: "$1" requires an argument.'
>&2 echo "Error: $1 requires an argument."
exit 1
fi
;;
@ -44,7 +60,7 @@ while :; do
shift
shift
else
>&2 echo 'Error: "$1" requires an argument.'
>&2 echo "Error: $1 requires an argument."
exit 1
fi
;;
@ -57,9 +73,9 @@ numCPUs=${numCPUs:-$(nproc)}
# Min. 800 MiB needed to avoid 'out of memory' errors
memoryMiB=${memoryMiB:-2048}
scriptDir=$(cd "${BASH_SOURCE[0]%/*}" && pwd)
testDir=$(cd "${BASH_SOURCE[0]%/*}" && pwd)
export NIX_PATH=nixpkgs=$(nix eval --raw -f "$scriptDir/../pkgs/nixpkgs-pinned.nix" nixpkgs)
export NIX_PATH=nixpkgs=$(nix eval --raw -f "$testDir/../pkgs/nixpkgs-pinned.nix" nixpkgs)
# Run the test. No temporary files are left on the host system.
run() {
@ -67,20 +83,14 @@ run() {
export TMPDIR=$(mktemp -d /tmp/nix-bitcoin-test.XXX)
trap "rm -rf $TMPDIR" EXIT
nix-build --out-link $TMPDIR/driver -E "import \"$scriptDir/test.nix\" { scenario = \"$scenario\"; }" -A driver
nix-build --out-link $TMPDIR/driver -E "(import \"$testDir/tests.nix\" { scenario = \"$scenario\"; }).vm" -A driver
# Variable 'tests' contains the Python code that is executed by the driver on startup
if [[ $1 == --interactive ]]; then
echo "Running interactive testing environment"
tests=$(
echo 'is_interactive = True'
# The test script raises an error when 'is_interactive' is defined so
# that it just loads the initial helper functions and stops before
# executing the actual tests
echo 'try:'
echo 'exec(os.environ["testScript"])'
echo 'except:'
echo ' pass'
# Start VM
echo 'start_all()'
# Start REPL
@ -109,6 +119,15 @@ debug() {
run --interactive
}
evalTest() {
nix eval --raw "($(vmTestNixExpr)).outPath"
echo # nix eval doesn't print a newline
}
container() {
. "$testDir/lib/make-container.sh" "$@"
}
# Run the test by building the test derivation
buildTest() {
if [[ $outLinkPrefix ]]; then
@ -130,14 +149,19 @@ exprForCI() {
((memAvailableMiB < memoryMiB)) && memoryMiB=$memAvailableMiB
>&2 echo "VM stats: CPUs: $numCPUs, memory: $memoryMiB MiB"
>&2 echo "Host memory total: $((memTotalKiB / 1024)) MiB, available: $memAvailableMiB MiB"
vmTestNixExpr
# VMX is usually not available on CI nodes due to recursive virtualisation.
# Explicitly disable VMX, otherwise QEMU 4.20 fails with message
# "error: failed to set MSR 0x48b to 0x159ff00000000"
vmTestNixExpr "-cpu host,-vmx"
}
vmTestNixExpr() {
extraQEMUOpts="$1"
cat <<EOF
(import "$scriptDir/test.nix" { scenario = "$scenario"; } {}).overrideAttrs (old: rec {
((import "$testDir/tests.nix" { scenario = "$scenario"; }).vm {}).overrideAttrs (old: rec {
buildCommand = ''
export QEMU_OPTS="-smp $numCPUs -m $memoryMiB"
export QEMU_OPTS="-smp $numCPUs -m $memoryMiB $extraQEMUOpts"
echo "VM stats: CPUs: $numCPUs, memory: $memoryMiB MiB"
'' + old.buildCommand;
})
@ -149,7 +173,8 @@ build() {
buildTest "$@"
else
scenario=default buildTest "$@"
scenario=withnetns buildTest "$@"
scenario=netns buildTest "$@"
scenario=full evalTest "$@"
fi
}
@ -158,4 +183,9 @@ if [[ $1 && $1 != build ]]; then
: ${scenario:=default}
fi
eval "${@:-build}"
command="${1:-build}"
shift || true
if [[ $command == eval ]]; then
command=evalTest
fi
$command "$@"

View File

@ -1,56 +0,0 @@
def electrs():
machine.wait_for_open_port(4224) # prometeus metrics provider
def nbxplorer():
machine.wait_for_open_port(24444)
def btcpayserver():
machine.wait_for_open_port(23000)
# test lnd custom macaroon
assert_matches(
'sudo -u btcpayserver curl -s --cacert /secrets/lnd-cert --header "Grpc-Metadata-macaroon: $(xxd -ps -u -c 1000 /run/lnd/btcpayserver.macaroon)" -X GET https://127.0.0.1:8080/v1/getinfo | jq',
'"version"',
)
def spark_wallet():
machine.wait_for_open_port(9737)
spark_auth = re.search("login=(.*)", succeed("cat /secrets/spark-wallet-login"))[1]
assert_matches(f"curl -s {spark_auth}@localhost:9737", "Spark")
def lightning_charge():
machine.wait_for_open_port(9112)
charge_auth = re.search("API_TOKEN=(.*)", succeed("cat /secrets/lightning-charge-env"))[1]
assert_matches(f"curl -s api-token:{charge_auth}@localhost:9112/info | jq", '"id"')
def nanopos():
machine.wait_for_open_port(9116)
assert_matches("curl localhost:9116", "tshirt")
def web_index():
machine.wait_for_open_port(80)
assert_matches("curl localhost", "nix-bitcoin")
assert_matches("curl -L localhost/store", "tshirt")
def prestop():
pass
extra_tests = {
"electrs": electrs,
"nbxplorer": nbxplorer,
"btcpayserver": btcpayserver,
"spark-wallet": spark_wallet,
"lightning-charge": lightning_charge,
"nanopos": nanopos,
"web-index": web_index,
"prestop": prestop,
}
run_tests(extra_tests)

View File

@ -1,126 +0,0 @@
# netns IP addresses
bitcoind_ip = "169.254.1.12"
clightning_ip = "169.254.1.13"
lnd_ip = "169.254.1.14"
liquidd_ip = "169.254.1.15"
electrs_ip = "169.254.1.16"
sparkwallet_ip = "169.254.1.17"
lightningcharge_ip = "169.254.1.18"
nanopos_ip = "169.254.1.19"
recurringdonations_ip = "169.254.1.20"
nginx_ip = "169.254.1.21"
lightningloop_ip = "169.254.1.22"
nbxplorer_ip = "169.254.1.23"
btcpayserver_ip = "169.254.1.24"
def electrs():
machine.wait_until_succeeds(
"ip netns exec nb-electrs nc -z localhost 4224"
) # prometeus metrics provider
def nbxplorer():
machine.wait_until_succeeds("ip netns exec nb-nbxplorer nc -z %s 24444" % nbxplorer_ip)
def btcpayserver():
machine.wait_until_succeeds("ip netns exec nb-btcpayserver nc -z %s 23000" % btcpayserver_ip)
# test lnd custom macaroon
assert_matches(
'ip netns exec nb-btcpayserver sudo -u btcpayserver curl -s --cacert /secrets/lnd-cert --header "Grpc-Metadata-macaroon: $(xxd -ps -u -c 1000 /run/lnd/btcpayserver.macaroon)" -X GET https://%s:8080/v1/getinfo | jq'
% lnd_ip,
'"version"',
)
def spark_wallet():
machine.wait_until_succeeds("ip netns exec nb-spark-wallet nc -z %s 9737" % sparkwallet_ip)
spark_auth = re.search("login=(.*)", succeed("cat /secrets/spark-wallet-login"))[1]
assert_matches(
f"ip netns exec nb-spark-wallet curl -s {spark_auth}@%s:9737" % sparkwallet_ip, "Spark"
)
def lightning_charge():
machine.wait_until_succeeds("ip netns exec nb-nanopos nc -z %s 9112" % lightningcharge_ip)
charge_auth = re.search("API_TOKEN=(.*)", succeed("cat /secrets/lightning-charge-env"))[1]
assert_matches(
f"ip netns exec nb-nanopos curl -s api-token:{charge_auth}@%s:9112/info | jq"
% lightningcharge_ip,
'"id"',
)
def nanopos():
machine.wait_until_succeeds("ip netns exec nb-lightning-charge nc -z %s 9116" % nanopos_ip)
assert_matches("ip netns exec nb-lightning-charge curl %s:9116" % nanopos_ip, "tshirt")
def web_index():
machine.wait_until_succeeds("ip netns exec nb-nginx nc -z localhost 80")
assert_matches("ip netns exec nb-nginx curl localhost", "nix-bitcoin")
assert_matches("ip netns exec nb-nginx curl -L localhost/store", "tshirt")
def prestop():
ping_bitcoind = "ip netns exec nb-bitcoind ping -c 1 -w 1"
ping_nanopos = "ip netns exec nb-nanopos ping -c 1 -w 1"
ping_nbxplorer = "ip netns exec nb-nbxplorer ping -c 1 -w 1"
# Positive ping tests (non-exhaustive)
machine.succeed(
"%s %s &&" % (ping_bitcoind, bitcoind_ip)
+ "%s %s &&" % (ping_bitcoind, clightning_ip)
+ "%s %s &&" % (ping_bitcoind, lnd_ip)
+ "%s %s &&" % (ping_bitcoind, liquidd_ip)
+ "%s %s &&" % (ping_bitcoind, nbxplorer_ip)
+ "%s %s &&" % (ping_nbxplorer, btcpayserver_ip)
+ "%s %s &&" % (ping_nanopos, lightningcharge_ip)
+ "%s %s &&" % (ping_nanopos, nanopos_ip)
+ "%s %s" % (ping_nanopos, nginx_ip)
)
# Negative ping tests (non-exhaustive)
machine.fail(
"%s %s ||" % (ping_bitcoind, sparkwallet_ip)
+ "%s %s ||" % (ping_bitcoind, lightningloop_ip)
+ "%s %s ||" % (ping_bitcoind, lightningcharge_ip)
+ "%s %s ||" % (ping_bitcoind, nanopos_ip)
+ "%s %s ||" % (ping_bitcoind, recurringdonations_ip)
+ "%s %s ||" % (ping_bitcoind, nginx_ip)
+ "%s %s ||" % (ping_nanopos, bitcoind_ip)
+ "%s %s ||" % (ping_nanopos, clightning_ip)
+ "%s %s ||" % (ping_nanopos, lnd_ip)
+ "%s %s ||" % (ping_nanopos, lightningloop_ip)
+ "%s %s ||" % (ping_nanopos, liquidd_ip)
+ "%s %s ||" % (ping_nanopos, electrs_ip)
+ "%s %s ||" % (ping_nanopos, sparkwallet_ip)
+ "%s %s ||" % (ping_nanopos, recurringdonations_ip)
+ "%s %s" % (ping_nanopos, btcpayserver_ip)
)
# test that netns-exec can't be run for unauthorized namespace
machine.fail("netns-exec nb-electrs ip a")
# test that netns-exec drops capabilities
assert_full_match(
"su operator -c 'netns-exec nb-bitcoind capsh --print | grep Current '", "Current: =\n"
)
# test that netns-exec can not be executed by users that are not operator
machine.fail("sudo -u clightning netns-exec nb-bitcoind ip a")
extra_tests = {
"electrs": electrs,
"nbxplorer": nbxplorer,
"btcpayserver": btcpayserver,
"spark-wallet": spark_wallet,
"lightning-charge": lightning_charge,
"nanopos": nanopos,
"web-index": web_index,
"prestop": prestop,
}
run_tests(extra_tests)

View File

@ -1,76 +0,0 @@
# Integration test, can be run without internet access.
# Make sure to update build() in ./run-tests.sh when adding new scenarios
{ scenario ? "default" }:
import ./make-test.nix rec {
name = "nix-bitcoin-${scenario}";
hardened = {
imports = [ <nixpkgs/nixos/modules/profiles/hardened.nix> ];
security.allowUserNamespaces = true; # re-enable disabled option
};
machine = { pkgs, lib, ... }: with lib; {
imports = [
../modules/presets/secure-node.nix
../modules/secrets/generate-secrets.nix
# using the hardened profile increases total test duration by ~50%, so disable it for now
# hardened
];
# needed because duplicity requires 270 MB of free temp space, regardless of backup size.
virtualisation.diskSize = 1024;
nix-bitcoin.netns-isolation.enable = (scenario == "withnetns");
services.bitcoind.extraConfig = mkForce "connect=0";
services.clightning.enable = true;
services.spark-wallet.enable = true;
services.lightning-charge.enable = true;
services.nanopos.enable = true;
services.lnd.enable = true;
services.lnd.listenPort = 9736;
services.lightning-loop.enable = true;
services.electrs.enable = true;
services.liquidd = {
enable = true;
listen = mkForce false;
extraConfig = "noconnect=1";
};
services.nix-bitcoin-webindex.enable = true;
services.hardware-wallets = {
trezor = true;
ledger = true;
};
services.backups.enable = true;
services.btcpayserver.enable = true;
services.btcpayserver.lightningBackend = "lnd";
# needed to test macaroon creation
environment.systemPackages = with pkgs; [ openssl xxd ];
services.joinmarket.enable = true;
services.joinmarket.yieldgenerator = {
enable = true;
customParameters = ''
txfee = 200
cjfee_a = 300
'';
};
# to test that unused secrets are made inaccessible by 'setup-secrets'
systemd.services.generate-secrets.postStart = ''
install -o nobody -g nogroup -m777 <(:) /secrets/dummy
'';
};
testScript =
builtins.readFile ./base.py + "\n\n" + builtins.readFile "${./.}/scenarios/${scenario}.py";
}

151
test/tests.nix Normal file
View File

@ -0,0 +1,151 @@
# Integration tests, can be run without internet access.
{ scenario ? "default" }:
import ./lib/make-test.nix scenario (
{ config, pkgs, lib, ... }: with lib;
let testEnv = rec {
cfg = config.services;
mkIfTest = test: mkIf (config.tests.${test} or false);
baseConfig = {
imports = [
./lib/test-lib.nix
../modules/modules.nix
../modules/secrets/generate-secrets.nix
];
config = {
tests.bitcoind = cfg.bitcoind.enable;
services.bitcoind = {
enable = true;
extraConfig = mkIf config.test.noConnections "connect=0";
};
tests.clightning = cfg.clightning.enable;
tests.spark-wallet = cfg.spark-wallet.enable;
tests.nanopos = cfg.nanopos.enable;
tests.lnd = cfg.lnd.enable;
services.lnd.listenPort = 9736;
tests.lightning-loop = cfg.lightning-loop.enable;
tests.electrs = cfg.electrs.enable;
tests.liquidd = cfg.liquidd.enable;
services.liquidd.extraConfig = mkIf config.test.noConnections "connect=0";
tests.btcpayserver = cfg.btcpayserver.enable;
services.btcpayserver.lightningBackend = "lnd";
# Needed to test macaroon creation
environment.systemPackages = mkIfTest "btcpayserver" (with pkgs; [ openssl xxd ]);
tests.joinmarket = cfg.joinmarket.enable;
services.joinmarket.yieldgenerator = {
enable = config.services.joinmarket.enable;
customParameters = ''
txfee = 200
cjfee_a = 300
'';
};
tests.backups = cfg.backups.enable;
# To test that unused secrets are made inaccessible by 'setup-secrets'
systemd.services.generate-secrets.postStart = mkIfTest "security" ''
install -o nobody -g nogroup -m777 <(:) /secrets/dummy
'';
};
};
scenarios = {
base = baseConfig; # Included in all scenarios
default = scenarios.secureNode;
# All available basic services and tests
full = {
tests.security = true;
services.clightning.enable = true;
services.spark-wallet.enable = true;
services.lightning-charge.enable = true;
services.nanopos.enable = true;
services.lnd.enable = true;
services.lightning-loop.enable = true;
services.electrs.enable = true;
services.liquidd.enable = true;
services.btcpayserver.enable = true;
services.joinmarket.enable = true;
services.backups.enable = true;
services.hardware-wallets = {
trezor = true;
ledger = true;
};
};
secureNode = {
imports = [
scenarios.full
../modules/presets/secure-node.nix
];
services.nix-bitcoin-webindex.enable = true;
tests.secure-node = true;
tests.banlist-and-restart = true;
};
netns = {
imports = [ scenarios.secureNode ];
nix-bitcoin.netns-isolation.enable = true;
test.data.netns = config.nix-bitcoin.netns-isolation.netns;
tests.netns-isolation = true;
# This test is rather slow and unaffected by netns settings
tests.backups = mkForce false;
};
## Examples / debug helper
# Run a selection of tests in scenario 'netns'
selectedTests = {
imports = [ scenarios.netns ];
tests = mkForce {
btcpayserver = true;
netns-isolation = true;
};
};
# Container-specific features
containerFeatures = {
# Container has WAN access and bitcoind connects to external nodes
test.container.enableWAN = true;
# See ./lib/test-lib.nix for a description
test.container.exposeLocalhost = true;
};
adhoc = {
# <Add your config here>
# You can also set the env var `scenarioOverridesFile` (used below) to define custom scenarios.
};
};
};
in
let
overrides = builtins.getEnv "scenarioOverridesFile";
scenarios = testEnv.scenarios // (optionalAttrs (overrides != "") (import overrides {
inherit testEnv config pkgs lib;
}));
autoScenario = {
services.${scenario}.enable = true;
};
in {
imports = [
scenarios.base
(scenarios.${scenario} or autoScenario)
];
}
)

330
test/tests.py Normal file
View File

@ -0,0 +1,330 @@
from collections import OrderedDict
def succeed(*cmds):
"""Returns the concatenated output of all cmds"""
return machine.succeed(*cmds)
def assert_matches(cmd, regexp):
out = succeed(cmd)
if not re.search(regexp, out):
raise Exception(f"Pattern '{regexp}' not found in '{out}'")
def assert_full_match(cmd, regexp):
out = succeed(cmd)
if not re.fullmatch(regexp, out):
raise Exception(f"Pattern '{regexp}' doesn't match '{out}'")
def log_has_string(unit, str):
return f"journalctl -b --output=cat -u {unit} --grep='{str}'"
def assert_no_failure(unit):
"""Unit should not have failed since the system is running"""
machine.fail(log_has_string(unit, "Failed with result"))
def assert_running(unit):
with machine.nested(f"waiting for unit: {unit}"):
machine.wait_for_unit(unit)
assert_no_failure(unit)
def wait_for_open_port(address, port):
def is_port_open(_):
status, _ = machine.execute(f"nc -z {address} {port}")
return status == 0
with log.nested(f"Waiting for TCP port {address}:{port}"):
retry(is_port_open)
### Test runner
tests = OrderedDict()
def test(name):
def x(fn):
tests[name] = fn
return x
def run_tests():
enabled = enabled_tests.copy()
to_run = []
for test in tests:
if test in enabled:
enabled.remove(test)
to_run.append(test)
if enabled:
raise RuntimeError(f"The following tests are enabled but not defined: {enabled}")
machine.connect() # Visually separate boot output from the test output
for test in to_run:
with log.nested(f"test: {test}"):
tests[test]()
def run_test(test):
tests[test]()
### Tests
# All tests are executed in the order they are defined here
@test("security")
def _():
assert_running("setup-secrets")
# Unused secrets should be inaccessible
succeed('[[ $(stat -c "%U:%G %a" /secrets/dummy) = "root:root 440" ]]')
if "secure-node" in enabled_tests:
# Access to '/proc' should be restricted
machine.succeed("grep -Fq hidepid=2 /proc/mounts")
machine.wait_for_unit("bitcoind")
# `systemctl status` run by unprivileged users shouldn't leak cgroup info
assert_matches(
"sudo -u electrs systemctl status bitcoind 2>&1 >/dev/null",
"Failed to dump process list for 'bitcoind.service', ignoring: Access denied",
)
# The 'operator' with group 'proc' has full access
assert_full_match("sudo -u operator systemctl status bitcoind 2>&1 >/dev/null", "")
@test("bitcoind")
def _():
assert_running("bitcoind")
machine.wait_until_succeeds("bitcoin-cli getnetworkinfo")
assert_matches("su operator -c 'bitcoin-cli getnetworkinfo' | jq", '"version"')
# RPC access for user 'public' should be restricted
machine.fail(
"bitcoin-cli -rpcuser=public -rpcpassword=$(cat /secrets/bitcoin-rpcpassword-public) stop"
)
machine.wait_until_succeeds(
log_has_string("bitcoind", "RPC User public not allowed to call method stop")
)
# Impure: Stops electrs
@test("electrs")
def _():
assert_running("electrs")
wait_for_open_port(ip("electrs"), 4224) # prometeus metrics provider
# Check RPC connection to bitcoind
machine.wait_until_succeeds(log_has_string("electrs", "NetworkInfo"))
# Stop electrs from spamming the test log with 'wait for bitcoind sync' messages
succeed("systemctl stop electrs")
@test("liquidd")
def _():
assert_running("liquidd")
machine.wait_until_succeeds("elements-cli getnetworkinfo")
assert_matches("su operator -c 'elements-cli getnetworkinfo' | jq", '"version"')
succeed("su operator -c 'liquidswap-cli --help'")
@test("clightning")
def _():
assert_running("clightning")
assert_matches("su operator -c 'lightning-cli getinfo' | jq", '"id"')
@test("lnd")
def _():
assert_running("lnd")
assert_matches("su operator -c 'lncli getinfo' | jq", '"version"')
assert_no_failure("lnd")
@test("lightning-loop")
def _():
assert_running("lightning-loop")
assert_matches("su operator -c 'loop --version'", "version")
# Check that lightning-loop fails with the right error, making sure
# lightning-loop can connect to lnd
machine.wait_until_succeeds(
log_has_string(
"lightning-loop",
"Waiting for lnd to be fully synced to its chain backend, this might take a while",
)
)
@test("btcpayserver")
def _():
assert_running("nbxplorer")
machine.wait_until_succeeds(log_has_string("nbxplorer", "BTC: RPC connection successful"))
wait_for_open_port(ip("nbxplorer"), 24444)
assert_running("btcpayserver")
machine.wait_until_succeeds(log_has_string("btcpayserver", "Listening on"))
wait_for_open_port(ip("btcpayserver"), 23000)
# test lnd custom macaroon
assert_matches(
"sudo -u btcpayserver curl -s --cacert /secrets/lnd-cert "
'--header "Grpc-Metadata-macaroon: $(xxd -ps -u -c 1000 /run/lnd/btcpayserver.macaroon)" '
f"-X GET https://{ip('lnd')}:8080/v1/getinfo | jq",
'"version"',
)
@test("spark-wallet")
def _():
assert_running("spark-wallet")
wait_for_open_port(ip("spark-wallet"), 9737)
spark_auth = re.search("login=(.*)", succeed("cat /secrets/spark-wallet-login"))[1]
assert_matches(f"curl -s {spark_auth}@{ip('spark-wallet')}:9737", "Spark")
@test("lightning-charge")
def _():
assert_running("lightning-charge")
wait_for_open_port(ip("lightning-charge"), 9112)
machine.wait_until_succeeds(f"nc -z {ip('lightning-charge')} 9112")
charge_auth = re.search("API_TOKEN=(.*)", succeed("cat /secrets/lightning-charge-env"))[1]
assert_matches(
f"curl -s api-token:{charge_auth}@{ip('lightning-charge')}:9112/info | jq", '"id"'
)
@test("nanopos")
def _():
assert_running("nanopos")
wait_for_open_port(ip("nanopos"), 9116)
assert_matches(f"curl {ip('nanopos')}:9116", "tshirt")
@test("joinmarket")
def _():
assert_running("joinmarket")
machine.wait_until_succeeds(
log_has_string("joinmarket", "P2EPDaemonServerProtocolFactory starting on 27184")
)
machine.wait_until_succeeds(
log_has_string("joinmarket-yieldgenerator", "Failure to get blockheight",)
)
@test("secure-node")
def _():
assert_running("onion-chef")
# FIXME: use 'wait_for_unit' because 'create-web-index' always fails during startup due
# to incomplete unit dependencies.
# 'create-web-index' implicitly tests 'nodeinfo'.
machine.wait_for_unit("create-web-index")
assert_running("nginx")
wait_for_open_port(ip("nginx"), 80)
assert_matches(f"curl {ip('nginx')}", "nix-bitcoin")
assert_matches(f"curl -L {ip('nginx')}/store", "tshirt")
# Run this test before the following tests that shut down services
# (and their corresponding network namespaces).
@test("netns-isolation")
def _():
ping_bitcoind = "ip netns exec nb-bitcoind ping -c 1 -w 1"
ping_nanopos = "ip netns exec nb-nanopos ping -c 1 -w 1"
ping_nbxplorer = "ip netns exec nb-nbxplorer ping -c 1 -w 1"
# Positive ping tests (non-exhaustive)
machine.succeed(
"%s %s &&" % (ping_bitcoind, ip("bitcoind"))
+ "%s %s &&" % (ping_bitcoind, ip("clightning"))
+ "%s %s &&" % (ping_bitcoind, ip("lnd"))
+ "%s %s &&" % (ping_bitcoind, ip("liquidd"))
+ "%s %s &&" % (ping_bitcoind, ip("nbxplorer"))
+ "%s %s &&" % (ping_nbxplorer, ip("btcpayserver"))
+ "%s %s &&" % (ping_nanopos, ip("lightning-charge"))
+ "%s %s &&" % (ping_nanopos, ip("nanopos"))
+ "%s %s" % (ping_nanopos, ip("nginx"))
)
# Negative ping tests (non-exhaustive)
machine.fail(
"%s %s ||" % (ping_bitcoind, ip("spark-wallet"))
+ "%s %s ||" % (ping_bitcoind, ip("lightning-loop"))
+ "%s %s ||" % (ping_bitcoind, ip("lightning-charge"))
+ "%s %s ||" % (ping_bitcoind, ip("nanopos"))
+ "%s %s ||" % (ping_bitcoind, ip("nginx"))
+ "%s %s ||" % (ping_nanopos, ip("bitcoind"))
+ "%s %s ||" % (ping_nanopos, ip("clightning"))
+ "%s %s ||" % (ping_nanopos, ip("lnd"))
+ "%s %s ||" % (ping_nanopos, ip("lightning-loop"))
+ "%s %s ||" % (ping_nanopos, ip("liquidd"))
+ "%s %s ||" % (ping_nanopos, ip("electrs"))
+ "%s %s ||" % (ping_nanopos, ip("spark-wallet"))
+ "%s %s" % (ping_nanopos, ip("btcpayserver"))
)
# test that netns-exec can't be run for unauthorized namespace
machine.fail("netns-exec nb-electrs ip a")
# test that netns-exec drops capabilities
assert_full_match(
"su operator -c 'netns-exec nb-bitcoind capsh --print | grep Current '", "Current: =\n"
)
# test that netns-exec can not be executed by users that are not operator
machine.fail("sudo -u clightning netns-exec nb-bitcoind ip a")
# Impure: stops bitcoind (and dependent services)
@test("backups")
def _():
succeed("systemctl stop bitcoind")
succeed("systemctl start duplicity")
machine.wait_until_succeeds(log_has_string("duplicity", "duplicity.service: Succeeded."))
run_duplicity = "export $(cat /secrets/backup-encryption-env); duplicity"
# Files in backup and /var/lib should be identical
assert_matches(
f"{run_duplicity} verify --archive-dir /var/lib/duplicity file:///var/lib/localBackups /var/lib",
"0 differences found",
)
# Backup should include important files
files = succeed(f"{run_duplicity} list-current-files file:///var/lib/localBackups")
assert "var/lib/clightning/bitcoin/hsm_secret" in files
assert "secrets/lnd-seed-mnemonic" in files
assert "secrets/jm-wallet-seed" in files
assert "var/lib/bitcoind/wallet.dat" in files
assert "var/backup/postgresql/btcpaydb.sql.gz" in files
# Impure: restarts services
@test("banlist-and-restart")
def _():
machine.wait_until_succeeds(log_has_string("bitcoind-import-banlist", "Importing node banlist"))
assert_no_failure("bitcoind-import-banlist")
# Current time in µs
pre_restart = succeed("date +%s.%6N").rstrip()
# Sanity-check system by restarting all services
succeed(
"systemctl restart bitcoind clightning lnd lightning-loop spark-wallet lightning-charge nanopos liquidd"
)
# Now that the bitcoind restart triggered a banlist import restart, check that
# re-importing already banned addresses works
machine.wait_until_succeeds(
log_has_string(f"bitcoind-import-banlist --since=@{pre_restart}", "Importing node banlist")
)
assert_no_failure("bitcoind-import-banlist")
if "netns-isolation" in enabled_tests:
def ip(name):
return test_data["netns"][name]["address"]
else:
def ip(_):
return "127.0.0.1"