3781a85c9b joinmarket: enable Agora as a third IRC server (nixbitcoin)
ced1637d07 joinmarket: share IRC server definitions between jm and ob-watcher (Erik Arvstedt)
59fc003ebd joinmarket: 0.9.1 -> 0.9.2 (nixbitcoin)
Pull request description:
ACKs for top commit:
erikarvstedt:
ACK 3781a85c9b
Tree-SHA512: 5ec919d2291ecf96fb4ca880f3dbeabff13f2bab71822db893ebbaba1b95463666b098ccc1412a1b56f327a231e10c1f2d47feb0f520fce349ab243d398bf7b4
- `waitfornewblock` was previously not included in the public RPC
whitelist because it's reserved for testing and marked as hidden
in bitcoind.
- electrs changed its verbosity settings. `-vv` is now the best choice
for normal usage.
- bitcoind option `dataDirReadableByGroup` is now unused.
Because it can be valuable for other use cases and implementing
it is intricate, we're keeping it for now.
- test: keep `nc` connection open because otherwise the electrs
RPC server would now close the connection before sending a response.
Previously, Tor was always enabled because `cfg` was always nonempty
(via definitions at `Set sensible defaults for some services`).
Now only enable Tor if there are active onion services.
Also rename var `services` -> `onionServices` to improve readability in
section `Set getPublicAddressCmd ...` where the same name is also used for
option `config.services`.
Previously, the glob (*) returned '*' when no files existed in the
secrets dir, leading to error `chown: cannot access '*'`.
Now `unprocessedFiles` is empty when there are no secrets.
Also remove the unneeded sorting of `unprocessedFiles` and
remove redundant leading zero in the default mode.
- `discover` is automatically disabled by bitcoind because we're
setting `externalip` via the `nix-bitcoin.onionServices` mechanism
- `bech32` is bitcoind's default addresstype
- Add nbxplorer to whitelists.
This is recommended by the nbxplorer docs and guarantees that nbxplorer
can always p2p-connect to bitcoind/liquidd.
- Enable bitcoind/liquidd p2p servers via `listen`.
- bitcoind: Remove obsolete defaultText
- clightning: Fix description
Option `address` can't be used to specify a socket path because it's
used explicitly as an IP address in many places.
- lnd: Break up overlong line
This is required by commit `services: support 0.0.0.0/:: in `address` options`
- nix-bitcoin.nix: Formatting
- secrets: Improve descriptions
Benefits of adding top-level variables for used services:
- Makes it obvious which other services are referenced by a service
- Less code
We already do this in many other places.
These are insignificant, generic options; place them above readonly options.
We already do this in other services.
Also move user/group config to bottom in spark-wallet.
This greatly improves readability and makes it easier to discover options.
This commit was genereated by running the following script inside the
repo root dir:
#!/usr/bin/env ruby
def transform(src)
return false if src.include?('inherit options;')
success = false
options = nil
src.sub!(/^ options.*?^ }.*?;/m) do |match|
options = match
" inherit options;"
end
return false if !options
src.sub!(/^with lib;\s*let\n+/m) do |match|
success = true
<<~EOF
with lib;
let
#{options}
EOF
end
success
end
Dir['modules/**/*.nix'].each do |f|
src = File.read(f)
if transform(src)
puts "Changed file #{f}"
File.write(f, src)
end
end
`generate-secrets` is no longer a monolithic script. Instead, it's
composed of the values of option `nix-bitcoin.generateSecretsCmds`.
This has the following advantages:
- generate-secrets is now extensible by users
- Only secrets of enabled services are generated
- RPC IPs in the `lnd` and `loop` certs are no longer hardcoded.
Secrets are no longer automatically generated when entering nix-shell.
Instead, they are generated before deployment (via `krops-deploy`)
because secrets generation is now dependant on the node configuration.
- btcpayserver: remove unneeded trailing semicolons
- krops/get-sha256:
`tail` is unneeded because `nix-prefetch-url` just outputs a single
line containing the hash.
Now that service uid, gid mappings are included in the backups, along
with the service data dirs, we can remove 'chown -R' for
clightning and liquidd data dirs.
Note that we used 'chown -R' only for these two services, while this
approach would have been relevant for all services with data dirs.
nix-bitcoin.nix is now no longer dependent on clightning.nix and lnd.nix.
Due to condition '!(config.services ? clightning)' lnd.nix still
doesn't depend on clightning.nix.
Also fix the assertion message by renaming clightning.bindPort to clightning.port.
NixOS option `security.hideProcessInformation` for globally restricting
access to /proc has been removed.
Use per-service restrictions via 'ProtectProc' instead.
Rename
`nix-bitcoin.security.hideProcessInformation` to
`nix-bitcoin.security.dbusHideProcessInformation`
because this option now only implements the dbus restriction.
Starting with 0.21.0, bitcoin no longer automatically creates and loads
a default wallet.
This was being ignored because of a JoinMarket issue [1] in CI builds prior
to this version. Now a watch-only Bitcoin Core wallet is created in ExecStartPost.
[1] https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/812
Don't copy bitcoin-rpcpassword-privileged as root, instead run service
with group "bitcoin".
Same effect, less complexity. Note, PoLP still obeyed for joinmarket-ob-watcher.
Due to a possible NixOS bug, this commit has no effect on NixOS 20.09
where `RestrictAddressFamilies` is a no-op.
It's only relevant for NixOS unstable with cgroups v2.
bitcoind+zmq: instead of allowing all address families, only add the required
AF_NETLINK family.
lnd: lnd only runs a zmq client, not a server, therefore it requires
no additional address families.
lightning-pool, clightning-plugin-zmq: add AF_NETLINK.
- Improve readability by using minutes
- set `TimeoutStopSec` like in bitcoin/contrib/init/bitcoind.service.
Stopping bitcoind can exceed the default timeout during IBD.
Keeping the secrets dir read-only is more simple and robust.
- lnd seed mnemonic creation and joinmarket wallet creation can be
run as the regular service user instead of root.
- It is easier to switch to a third-party secrets deployment
method in the future.
Don't create a seed mnemonic for lnd when a wallet exists.
This avoids creating unused mnemonics and helps simplifying
the migration command in `versioning.nix`.
- Fail at evaluation when secrets setup is not configured.
Previously, bitcoind failed at runtime due to the missing secrets target.
- Fail at evaluation when conflicting secrets setup methods are used.
This happens when `secretsSetupMethod` has more than one definition.
With krops or nixops the secrets target is always restarted during
the deployment process.
This previously caused unnecessary restarts of all nix-bitcoin services.
Move this feature from a module preset to a regular option, so that it's
easily discoverable and accessible.
Simplify the implementation of `generateSecrets` by adding it to the
existing `setup-secrets` service script.
Also rename option setup-secrets -> setupSecrets.
This commit fixes an issue with LND, in which if both
nix-bitcoin.onionServices.lnd.public &
services.lnd.restOnionService.enable were enabled, one would try to
create a file named `lnd` and the other would try to create a directory
named `lnd` with a file named `lnd-rest` inside it. This would obiously
cause an error and fail the LND service.
When running as root, use runuser instead of sudo.
As opposed to sudo or doas, runuser is a standalone
binary that needs no external configuration.
Also, it's a bit faster.
Use the following order of definitions for all services:
- assertions
- configuration of other services
- environment.systemPackages
- tmpfiles
- own service
- users
- secrets
Journal entries now look like
`joinmarket-yieldgenerator[9795]: User data location: /var/lib/joinmarket`
instead of
`bash[9795]: User data location: /var/lib/joinmarket`
DynamicUser simplifies services that don't need a persistent uid/gid,
like joinmarket-ob-watcher.
For existing installations the data dir migration to dynamic users
is automatically handled by systemd.
- Fix jm-wallet-seed being globally readable.
- Handle seed extraction failures.
If seed extraction fails, remove the newly created wallet.
This guarantees that wallets always have an accompanying seed.
Systemd's `Description` option is a misnomer (as confessed by `man systemd.unit`):
Its value is used by user-facing tools in place of the unit file name, so this option
could have been more aptly named `label` or `name`.
`Description` should only be set if the unit file name is not sufficient for naming a unit.
This is not the case for our services, except for `systemd.services.nb-netns-bridge`
whose description has been kept.
As an example how this affects users, weird journal lines like
```
nb-test systemd[1]: Starting Run clightningd...
```
are now replaced by
```
nb-test systemd[1]: Starting clightning.service...
```
Previously, onionAddresses definitions in onionServices were of the form
onionAddresses.access.<service> = [<service>];
This caused failures for configurations where a service user name was
overridden or for bitcoind whose default user is 'bitcoin' instead of 'bitcoind'.
Now set the equivalent of:
onionAddresses.access.<actualServiceUser> = [<service>];
Implement this via a new option `onionAddresses.services` to make things more
readable and to work around an infinite recursion error in onionServices.
- enable usage outside of secure-node.nix
- use json as the output format
- show ports
- also show local addresses, which is particularly useful when
netns-isolation is enabled
- only show enabled services
Move 'enforceTor' and onion-service definitions from secure-node.nix.
Use the onionServices module to define onion services.
Onion services now automatically work for services that bind to an INADDR_ANY (`0.0.0.0`) address.
- use -e to check for existence of /var/lib/tor/state, use shorter
polling interval
- clear existing dataDir contents to avoid accumulating obsolete data
- use concatMapStrings instead of foldl'
The 'scudo' memory allocator set by the 'hardened' profile breaks some
services on 20.09.
The fix for NixOS unstable (https://github.com/NixOS/nixpkgs/pull/104052)
is ineffective on 20.09.
As a workaround, add a custom 'hardened' preset that uses the default allocator.
Under normal circumstances, service-specific netns should never exist
before the netns setup service starts.
An existing netns is a genuine error that should not be silently ignored.
A short time after `netns delete` finishes, the peer link in the main
netns is automatically removed.
When `link del` is run before that, it fails with
`Cannot find device "nb-veth-br-*"` and the netns service enters a failed state.
This also enables the test scenario 'netnsRegtest' introduced in a
later commit by fixing the following bug:
For unknown reasons, when tor.active=true and tor is not running, lnd
fails with a tor connection error on netns-isolation, but runs fine
without netns-isolation.
