ZNC

I've been an off-and-on IRC user for years, and running Wallops on System 6 reminded me how helpful these text-based communities can be. On my MacBook I've used IRSSI, but I'm not an avid enough IRC-er to remember the commands. I've also used the Matrix bridge, which has its own challenges and has recently been disabled. At least one large IRC network, Mozilla, has migrated from IRC to Matrix as of March 2020.

Lately I've used Textual which is paid but also open source. There are some issues with IRC that Matrix solved though, namely losing the history every time your client sleeps or restarts. The age-old solution to this is ZNC.

I set up ZNC on the Fedora VM I use for miscelaneous services, misc.home.arpa. First, we can install znc from Fedora's package repos:

sudo dnf install znc

Then, we need to run the init command:

sudo -u znc znc --makeconf

This will ask you a series of questions such as the admin username and password, nick, etc. I set up a dedicated admin user with a random password (thanks 1Password) and left most things blank. It's best practice to have a dedicated admin account, and then create a standard user account for yourself to use with your IRC client. When choosing a port, I choose 6697 as proposed by RFC 7194 and enabled SSL.

As the wizard warns, some browsers will not open :6697. I run Nginx on this server, so I added /etc/nginx/conf.d/znc.conf to reverse proxy to it from :443 when the host is znc.home.arpa:

server {
    listen       80;
    listen       [::]:80;
    server_name  znc.home.arpa;
    root         /usr/share/nginx/html;

    return 301 https://$host$request_uri;
}

# Settings for a TLS enabled server.
server {
    listen       443 ssl http2;
    listen       [::]:443 ssl http2;
    server_name  znc.home.arpa;
    root         /usr/share/nginx/html;

    ssl_certificate "/etc/pki/nginx/server.crt";
    ssl_certificate_key "/etc/pki/nginx/private/server.key";
    ssl_session_cache shared:SSL:1m;
    ssl_session_timeout  10m;
    ssl_ciphers PROFILE=SYSTEM;
    ssl_prefer_server_ciphers on;

    location / {
        proxy_pass https://127.0.0.1:6697/;
        proxy_set_header Host "127.0.0.1";
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forward-For $proxy_add_x_forwarded_for;
    }

    error_page 404 /404.html;
    location = /404.html {
    }

    error_page 500 502 503 504 /50x.html;
    location = /50x.html {
    }
}

You need a couple of things for this to work:

• Some unique DNS (e.g. znc.home.arpa) should point to your server's IP, I set this up via pfSense under Services, DNS Resolver, then at the bottom under Host Overrides I added a new record which points to the static IP I have DHCP configured to assign my VM:

  

• A cert that your computer trusts, or you can skip TLS altogether. I use cfssl to manage my TLS certificates. You'll need to create a CA¹ and Intermediate CA, which I placed in a certs folder, and then a CSR file like this for your server:

  {
      "CN": "misc.home.arpa",
      "key": {
          "algo": "rsa",
          "size": 2048
      },
      "names": [
          {
              "C": "US",
              "ST": "Arkansas",
              "L": "Little Rock",
              "O": "Heavy Computer",
              "OU": "Heavy Computer Registry"
          }
      ],
      "hosts": [
          "misc.home.arpa",
          "cups.home.arpa",
          "znc.home.arpa",
          "localhost",
          "10.0.3.3"
      ]
  }
  

  I use the same certificate for all services on the VM, but you can be more granular if you'd like. To generate the cert from the CSR, run:

  ; cfssl gencert -ca ../../intermediate-ca.pem -ca-key ../../intermediate-ca-key.pem -config ../../cfssl.json -profile=server misc.home.arpa.json | cfssljson -bare misc.home.arpa-server
  

  Then copy the public key along with its intermediate public key to form a chain:

  ; cat misc.home.arpa-server.pem ../../intermediate-ca.pem | pbcopy
  

  And place it at /etc/pki/nginx/server.crt on your server, then copy the key file:

  ; cat misc.home.arpa-server-key.pem | pbcopy
  

  And place it at /etc/pki/nginx/private/server.key. Then adjust the permissions appropriately, so that Nginx can use it:

  ; sudo ls -l /etc/pki/nginx/server.crt /etc/pki/nginx/private/server.key
  -r--------. 1 nginx nginx 1676 Sep  5 16:49 /etc/pki/nginx/private/server.key
  -r--------. 1 nginx nginx 3165 Sep  5 16:48 /etc/pki/nginx/server.crt
  

You'll also need the firwall to allow HTTP/HTTPS traffic to Nginx:

; sudo firewall-cmd --permanent --add-service=http
; sudo firewall-cmd --permanent --add-service=https
; sudo firewall-cmd --reload

And we should enable the ZNC service:

; sudo systemctl enable --now znc.service

A quirk of ZNC on my system is that systemctl stop znc.service doesn't work, you need to pkill znc and then systemctl start znc.service to restart it.

Now, we should have a UI available at https://znc.home.arpa! From there, I logged in as admin and did the following:

• Under Global Settings, we can add a new Listen Port that only the UI uses (only HTTP and IPv4 are checked), say 6668, which won't be exposed to the network -- set Bind Host to 127.0.0.1 to signify this. Then we can edit our Nginx config to use http://127.0.0.1:6668/ as our upstream, so that we can only access it via Nginx.
• We can update our 6697 port so that it doesn't have the HTTP checkbox.
• We can add a 6667 port for unencrypted connections from clients like Wallops.

We also want our clients to be able to talk to ZNC over TLS without self-signed cert errors, so we can use our same cfssl certs from Nginx for ZNC by copying them to /etc/pki/znc/server.crt and /etc/pki/znc/private/server.key and then chown -R znc:znc /etc/pki/znc so that ZNC can read them. Then, we can update the config at /var/lib/znc/.znc/configs/znc.conf with:

SSLCertFile = /etc/pki/znc/server.crt
// SSLDHParamFile = /etc/pki/znc/server.crt
SSLKeyFile = /etc/pki/znc/private/server.key

At this point you may need to restart ZNC (in my case with pkill znc followed by systemctl start znc.service).

Now, we should expose these ports to the network via firewall:

; sudo firewall-cmd --permanent --add-service=ircs
; sudo firewall-cmd --permanent --add-service=irc
; sudo firewall-cmd --reload

Next, you should add a new user for yourself in the UI. Then, under that user add a Network. I use Liberachat, with these options:

I use CertFP to authenticate, and for that we'll need to check the cert module's box on both the User and Network page. Then, go to the User Modules > Certificate page and paste your certificate.

We'll also need to enable the sasl module on the Network page, which is required for CertFP to work for some networks, and from our client run /msg *sasl Mechanism EXTERNAL from each network.

Once connected, you can check that the certificate is being used by running /whois <your username> (after you've authenticated via another method), where you should see

cptaffe has client certificate fingerprint ...

If so, you can add that fingerprint to your account with /msg NickServer CERT ADD. Once your certificate is added to your account, and your SASL is set to EXTERNAL, you can reconnect with /msg *status jump to ensure you are correctly authenticated.

To connect to ZNC, configure your client's username and password fields:

You can configure multiple connections, one for each network.

Time Zone

There is a bug in ZNC where server time is not round-tripped appropriately if the server isn't running in UTC. It's good practice to run servers in UTC anyway, we can swap our system time to UTC:

; sudo timedatectl set-timezone UTC

and then reboot. As described in this comment, the playback module uses timestamps to determine what messages to deliver, so if the server time isn't synced properly then you may get duplicate messages or gaps.

I ran into this issue initially since my server was in my local time zone, which resulted in duplicate messages from the buffer on reconnect, and missing self-messages when reconnecting. This was most apparent in Palaver since it must reconnect so often.

Keeping a Nick

In most cases the initial connection will authenticate you and your nick will be assigned to you. In rare cases, for instance when your connection is interrupted, you may not be able to claim your nick because the server still has it assigned to your old connection. Enabling the keepnick module will configure ZNC to keep trying to get your original nick. In the case of a disconnection, your nick will be available soon when the old connection times out on the server end. I've only experienced this when connecting over Tor, but it's likely good practice generally.

Cloaks

To prevent users seeing your IP address, you can use a cloak.

On Libera, just /join #libera-cloak and send !cloakme, now your /whois {user} response should look like:

[14:22:25] cptaffe has userhost ~ZNC@user/cptaffe and real name "Connor Taffe"

On OFTC, send /msg NickServ SET CLOAK ON.

On Ergo, users are automatically cloaked:

By default, all hostnames on ergo.chat are cryptographically “cloaked” so that your IP address information is not visible to other users (although it is visible to server administrators).

They note, you can connect even more anonymously:

If you would like to anonymize your connection against the administrators as well, we are accessible via the Tor network, although you may be banned from some channels until you register a nickname:

Field	Value
Host	vrw7zcuarwx4oeju3iikiz3jffrvuijsysyznqf53mxizxrebomfnrid.onion
Port	6667
SSL/TLS	false

On System 6

To get Wallops to connect, I needed to create a new account with only one channel configured so that the joins wouldn't overwhelm the Macintosh SE. Wallops also doesn't have a username field where we could pass the network, so we can pass it in the password field as:

{username}/{network}:{password}

With that, Wallops can connect to ZNC and by proxy use CertFP for authentication!

We can also use the chanfilter module to hide channels so that Wallops only sees one channel, enabling us to use a single account. See the section below on multiple clients.

First, add a new client id to chanfilter, I called it wallops:

/msg *chanfilter AddClient wallops

Then, join from an IRC client which can handle all your channels using the client identifier. You can place the identifier in your username: for Textual I use shared@wallops/libera, but for Palaver (which has a dedicated network field for ZNC) I use shared@wallops. Once connected, leave all channels you wish to hide from wallops (all but one) -- the /part will be intercepted by chanfilter (on a second /part, ZNC will leave the channel).

To check that the channels you want to hide are hidden:

/msg *chanfilter ListChans wallops

Now in Wallops, our password field will look like:

{username}@{client identifier}/{network}:{password}

for example, user@wallops/libera:hunter2.

Over the Internet

To access our ZNC bouncer outside of the network, we need to creat a NAT rule. I followed these instructions to port-forward 6697 from my DNS address to my VM via pfSense, then followed these instructions to enable NAT Reflection so I could reach it from inside my network as well as outside.

I quickly realized that my internal certs won't work with my external DNS name, so I decided to scratch port-forward in favor of using the HAProxy plug-in. I've already configured it to work with the ACME plug-in to automatically issue and renew certificates for my domain names with Let's Encrypt. To do this, we add a new backend in HAProxy, znc, with:

We also need to tune the timeout settings because IRC connectios are long-lived and often quiet, unlike HTTP connections. The timeout must be longer than the interval between PINGs. I set mine to one day:

From the HAProxy docs:

The timeout connect setting configures the time that HAProxy will wait for a TCP connection to a backend server to be established. The timeout client setting measures inactivity during periods that we would expect the client to be speaking, or in other words sending TCP segments. The timeout server setting measures inactivity when we’d expect the backend server to be speaking. When a timeout expires, the connection is closed.

We forward to the unencrypted port to avoid the extra SSL overhead on our local network, but SSL can be used as well. Then create a new front-end irc-6697 with:

We also need to tune the timeout settings again:

Then under Actions, choose Use Backend and the znc backend.

Next, under Firewall > Rules, create a new rule:

Now we can easily configure our clients to use our external DNS address.

With our public ZNC service, we can use IRC on the move. I installed Palaver on my iPhone and configured it against my IRC bouncer to do just that.

Multiple Clients

Using multiple clients on a single ZNC account can introduce issues with the playback buffer not forwarding to all clients, so that the history is choppy in any one clinet. I asked on #palaver on irc.ergo.chat (which can be set up just like Libera, and supports CertFP), and kylef gave me this advice:

I would recommend the znc-playback module to solve the history sync per device, depending on which other clients you use and if they support it though. Make sure that the "auto clear" buffer features are not enabled in ZNC otherwise it will clear buffers on each connection. Having separate users would work but its complicated and this would solve it. As for push notifications, I would recommend installing the clientaway module, then configure all your clients to auto away you when you are not there. That provides the best experience as then when you are using one client actively, your other devices are not receiving messages you've read.

Under "Your Settings" for the user your clints login as, uncheck "Auto Clear Chan Buffer" and "Auto Clear Query Buffer." You may need to do this for each channel under each network as well, if there are any already configured.

You'll also want to enable route_replies for all networks, so that client request responses (such as /who, etc.) are routed to the client which sent the request. See the Multiple Clients wiki page.

Playback

To install the playback module, we follow the directions in compiling modules:

; git clone https://github.com/jpnurmi/znc-playback.git
; cd znc-playback/

We need the znc-buildmod command, available in the developement package:

; sudo dnf install znc-devel

Now we can build the module:

; znc-buildmod playback.cpp

Now we have playback.so, we can place it in a .znc/modules directory:

; sudo mkdir /var/lib/znc/.znc/modules
; sudo mv playback.so /var/lib/znc/.znc/modules
; sudo chown -R znc:znc /var/lib/znc/.znc/modules
; sudo chmod 700 /var/lib/znc/.znc/modules
; sudo chmod 700 /var/lib/znc/.znc/modules/playback.so

Now to load the module you can either message *status if you are an admin:

/msg *status LoadMod --type=global playback

or edit /var/lib/znc/.znc/configs/znc.conf directly, and add:

LoadModule = playback

After restarting ZNC, the module should appear.

Palaver

To install the znc-palaver module which will enable push notifications to Palaver, we can follow similar directions:

; git clone https://github.com/cocodelabs/znc-palaver
; cd znc-palaver/
; znc-buildmod palaver.cpp
; sudo mv palaver.so /var/lib/znc/.znc/modules
; sudo chown znc:znc /var/lib/znc/.znc/modules/palaver.so
; sudo chmod 700 /var/lib/znc/.znc/modules/palaver.so

Now either message *status if you are an admin:

/msg *status LoadMod --type=global palaver

or edit /var/lib/znc/.znc/configs/znc.conf, and add:

LoadModule = palaver

Upon restarting znc, you should see a "Connected!" push notification come through Palaver, you can also run /msg *palaver info for connected device info.

Client Away

For the clientaway module, instructions are very similar:

; git clone https://github.com/kylef-archive/znc-contrib.git
; cd znc-contrib/
; znc-buildmod clientaway.cpp
; sudo mv clientaway.so /var/lib/znc/.znc/modules
; sudo chown znc:znc /var/lib/znc/.znc/modules/clientaway.so
; sudo chmod 700 /var/lib/znc/.znc/modules/clientaway.so

This module is configure per-user instead of globally, so when messaging *status (admin not required):

/msg *status LoadMod --type=user clientaway

Or via the config file, the LoadModule clientaway statement is added under a user in the config (or toggled on in the Web UI after a restart):

<User your-user>
    ...
    LoadModule clientaway

You may also need to enable this per-network, via the UI or *status on each network:

/msg *status LoadMod --type=network clientaway

Chan Filter

This module is helpful if you don't want all channels to be visible on all clients, see the section on System 6 for usage.

For the chanfilter module, instructions are very similar:

; git clone https://github.com/jpnurmi/znc-chanfilter.git
; cd znc-chanfilter/
; znc-buildmod chanfilter.cpp
; sudo mv chanfilter.so /var/lib/znc/.znc/modules
; sudo chown znc:znc /var/lib/znc/.znc/modules/chanfilter.so
; sudo chmod 700 /var/lib/znc/.znc/modules/chanfilter.so

XMPP

The znc-xmpp module adds an XMPP (Jabber) interface to ZNC, for XMPP clients like iChat on mid-2000s versions of OS X. The module requires the libxml2 library and headers, install it via:

; sudo dnf install libxml2-devel

then proceed as usual:

; git clone https://github.com/kylef-archive/znc-xmpp.git
; cd znc-xmpp

The C++ compiler (my version of g++ and clang++) complains about the use of vector<...> without a using namespace std; statement. I ran grep vector -r src/ to find all usages of vector and changed them to std::vector.

; make
; sudo mv xmpp.so /var/lib/znc/.znc/modules
; sudo chown znc:znc /var/lib/znc/.znc/modules/xmpp.so
; sudo chmod 700 /var/lib/znc/.znc/modules/xmpp.so

When connected as an admin user (no network required), message *status:

/msg *status LoadMod --type=global xmpp znc.home.arpa

where znc.home.arpa is the host I want it to listen for XMPP connections (default is localhost).

Now we'll need to add a firewall rule:

; sudo firewall-cmd --permanent --new-service xmpp
; sudo firewall-cmd --permanent --service xmpp --add-port 5222/tcp
; sudo firewall-cmd --permanent --add-service xmpp
; sudo firewall-cmd --reload

Tor

You can access networks more anonymously via Tor, several networks have onion services:

Add /etc/yum.repo.d/tor.repo as:

[tor]
name=Tor for Fedora $releasever - $basearch
baseurl=https://rpm.torproject.org/fedora/$releasever/$basearch
enabled=1
gpgcheck=1
gpgkey=https://rpm.torproject.org/fedora/public_gpg.key
cost=100

then run

; sudo dnf install tor

Adding mappings allows us to connect using TLS without certificate pinning, since the host name will match the certificate. TLS is required for CertFP authentication, which is required over Tor by Libera. Edit /etc/tor/torrc to include:

MapAddress palladium.libera.chat libera75jm6of4wxpxt4aynol3xjmbtxgfyjpu34ss4d7r7q2v5zrpyd.onion
MapAddress irc.oftc.net oftcnet6xg6roj6d7id4y4cu6dchysacqj2ldgea73qzdagufflqxrid.onion
MapAddress irc.ergo.chat vrw7zcuarwx4oeju3iikiz3jffrvuijsysyznqf53mxizxrebomfnrid.onion

then run

; sudo systemctl enable --now tor.service

Since ZNC doesn't support SOCKS proxies natively, you'll need proxychains:

; sudo dnf install proxychains-ng

which by default is configured to proxy to Tor over localhost:9050 using SOCKS4.

Then change the ExecStart line in /usr/lib/systemd/system/znc.service to:

ExecStart=/usr/bin/proxychains /usr/bin/znc -f

and add

Requires=tor.service

Then, you should update your networks in ZNC to point to the MapAddress'd addresses above:

At first I had OFTC mapped to graviton.oftc.net, but they load balance between several servers. When it moved to dacia.oftc.net, TLS stopped working and my connection dropped. Since irc.oftc.net is an alternative name in all server certificates, it's valid without certificate pinning on all servers.

This means that all connections will happen over Tor, so networks that don't have an onion service will use Tor exit nodes, which are blocked on many networks. The suggestion from #znc is to use two ZNC servers, one specifically for Tor connections. You may also be able to connect one to the other so that only one ZNC service need be exposed. I've been told implementing proxy support requires a big refactor of the ZNC networking code.

Fixing restarts

If not using proxychains, you can add a PidFile to /var/lib/znc/.znc/configs/znc.conf,

PidFile /var/lib/znc/.znc/znc.pid

Then remove the -f flag so that znc forks, and add Type=forking to the service file at /usr/lib/systemd/system/znc.service:

[Unit]
Description=ZNC, an advanced IRC bouncer
After=network.target

[Service]
Type=forking
ExecStart=/usr/bin/znc
User=znc
PIDFile=/var/lib/znc/.znc/znc.pid

[Install]
WantedBy=multi-user.target

which should enable systemctl restart znc.service.

Other Networks

You can add networks via the web UI, or via *status with:

/msg *status AddNetwork undernet

then connect to that network, and you can add servers:

/msg *status AddServer irc.libera.chat +6697

you can also load modules at the network level:

/msg *status LoadMod --type=network sasl

or

/msg *status LoadMod --type=network nickserv

and configure the module via its user:

/msg *sasl Mechanism PLAIN

or

/msg *nickserv SetCommand IDENTIFY PRIVMSG NickServ :IDENTIFY {password}

Undernet

Undernet doesn't support TLS, CertFP, or even SASL. It doesn't allow registering nicks. Once you've signed up for an account, you need to provide

+x! <username> <password>

in the networks's server password field, this is called Login on Connect. The +x! will cloak your IP upon connection:

+x!: Only connect me when X is online and hide my IP address

X is the Undernet Channel Services bot, which should always be online.

Undernet also doesn't support registering nicks, only usernames registered through the Undernet Channel Service, known as CService. The CService website supports 2FA via OTP codes, but it is required for IRC login as well if enabled which is not supported in ZNC, so I don't recommend enabling it.

Undernet blocks Tor exit node IPs, so it won't connect if using proxychains.

IRCnet

IRCnet supports TLS but also doesn't support registering nicks, and doesn't support SASL or CertFP. If you sign up for a cloak, you can then provide a server password to ssl.cloak.ircnet.io, but you must have a static IP address or CIDR to sign up for one.

IRCnet does provide an onion service at IRCnet3mh2zfmpn3zcgwtrjnh37zcnyvjmsvoig577isjmy6m24auqqd.onion on port 6667.