Enhance FreeBSD Sylve to run backups

In my previous post  Migrate TrueNAS Core to Sylve I ended with an exercise for the reader to "Configure Samba and ZFS send-receive".

Well, I succeeded in adding this and even added some more backup options.

All my backup types

  1. ZFS send/receive over SSH
  2. Time Machine of Apple MacOS on a Samba share
  3. Rsync of UFS filesystem

ZFS send/receive over SSH

I have a couple of FreeBSD machines, ranging from laptops to desktops to Raspberry Pi's. Most of these run ZFS and I just want to send/receive the complete ZFS pool to my backup server.

TrueNAS had this feature integrated using zrepl. In Sylve I only found a feature to backup the Sylve server itself, not to backup other hosts to the Sylve server. 

 I didn't want to write a custom script, so I tried some existing ones. I ended up with sysutils/sanoid (pkg install sanoid). This package contains an application syncoid which is very flexible with only command line switches. I now have this in cron on the Sylve server:

SYNCOID_ARGS="--no-privilege-elevation --compress none --no-sync-snap --sendoptions=Lce -r"
1 0 * * * syncoid $SYNCOID_ARGS backup@rpi4.thuis.klop.ws:zrpi4 freenas-data/backup-rpi4 2>&1 | ts | tee /var/tmp/backup-rpi4.log
2 0 * * * syncoid $SYNCOID_ARGS backup@rpi4.thuis.klop.ws:zdata4 freenas-data/backup-rpi4-zdata4 2>&1 | ts | tee /var/tmp/backup-rpi4-zdata4.log
3 0 * * * syncoid $SYNCOID_ARGS root@rpi5.thuis.klop.ws:zroot freenas-data/backup-rpi5 2>&1 | ts | tee /var/tmp/backup-rpi5.log

This pulls the ZFS snapshots from my rpi4 and rpi5 and stores them on the local disk. Access is managed via ssh.
On rpi4 I set "zfs allow backup send zrpi4" so I can use an unprivileged user "backup" to send the backups. Unfortunately this didn't work on rpi5 because I have some jailed zfs filesystems which need the "root" user for the sending. Something to investigate for the future.

Snapshots are taken locally on the machines. I don't want the tools that sends/receives mixed with my snapshot strategy.

Time Machine of Apple MacOS on a Samba share

Sylve has support for Samba and MacOS can use that for Time Machine. But Samba needs some extra settings for Time Machine to work.
Fortunately the Sylve developer is pretty active and open for suggestions. This future was added quickly: [Feature] Let Samba share support Apple Time Machine #113.

Rsync of UFS filesystem

I have one rpi3 which uses UFS as filesystem. It can create snapshots, but does not have a send/receive feature like ZFS does. Luckily net/rsync (pkg install rsync) exists.

In TrueNAS Core it was possible to start an rsync daemon with some basic configuration. The GUI was pretty much a free form text field so you could make up your own configuration. Sylve doesn't have this. But it is very easy to setup rsync for this purpose.

An overview of my setup.

Enable rsyncd_enable="YES" in /etc/rc.conf. 
In Sylve configure a filesystem /freenas-data/Rsync.
Add these files to your system:
# cat /usr/local/etc/rsync/rsyncd.conf
use chroot = yes
max connections = 4
pid file = /var/run/rsyncd.pid
[RsyncBackup]
        path = /freenas-data/Rsync
        max connections = 0
        uid = root
        gid = nobody
        write only = true
        read only = false
        auth users = ronald
        secrets file = /usr/local/etc/rsync/secrets.RsyncBackup

# cat /usr/local/etc/rsync/secrets.RsyncBackup 
ronald:<mypassword>

The client calls "rsync <options> / ronald@sylve.host::RsyncBackup/rpi3-root/".

This starts a daemon that chroots to a filesystem and the client can rsync anything into it. The connection is authenticated by user + password, but it is not encrypted. So use only in your trusted network or enhance this with ssh or some tunnelling. For my use case this is fine.

Conclusion

With these three backup options I implemented all backup options I needed. It uses Sylve to manage the filesystems and Samba.

I didn't boot back into TrueNAS Core for a couple of weeks now.

Comments

Post a Comment

Popular posts from this blog

Migrate TrueNAS Core to Sylve

For years I ran TrueNAS Core on my home fileserver. This was an awesome, easy way of running FreeBSD + ZFS + tools without a lot of manual maintenance. Unfortunately TrueNAS decided to start using Linux as the basis of TrueNAS and I love using FreeBSD. Fortunately a new project is on the horizon which looks promising:   Sylve  I'm eager to try out Sylve and will write down my plan and the results in this blog. Some useful facts TrueNAS Core installs full images containing the OS, middleware and UI. The last version was based on FreeBSD 13.3 and uses Boot Environments to manage versions. Sylve wants FreeBSD 15.0 or later pre-installed and only installs middleware and UI on top of it. I use my NAS mainly for Samba (Time Machine) and ZRepl (zfs+ssh send-receive backups). The machine:  dmesg . root@freenas[~]# gpart show =>       40  234441568  ada0  GPT  (112G)          40       1024...
Read more

Waiting for clang. Forever and ever.

FreeBSD compiles with LLVM /clang by default. Part of this is compiling the compiler to accommodate upgrades of the compiler. On my laptop FreeBSD compiles in about 2 hours. But clang takes 2 hours on its own. The culprit is that now and than a small fix is made to the compiler which triggers 2 hours of extra building. It is also possible to install llvm as a pkg. This saves you hours of compiling. I tried to built base with the port, but it did not succeed yet. I now have setup FreeBSD in a container (FreeBSD jail) which helps greatly in fixing mistakes instead of a non bootable system without a working compiler I can now just reinstall the container from a working host. My changes to make this working: This is on: FreeBSD sjakie 13.0-CURRENT FreeBSD 13.0-CURRENT #11 r364932M: Sat Aug 29 03:21:18 CEST 2020 Add to /etc/src.conf : WITHOUT_CLANG=true WITHOUT_CLANG_IS_CC=true WITHOUT_CLANG_BOOTSTRAP=true WITHOUT_LLD=true WITHOUT_LLD_IS_LD=true WITHOUT_LLD_BOOTSTRAP=true WITHOUT_LLDB=true...
Read more

FreeBSD running vlan and bridge on one shared interface

 I have this setup on FreeBSD 13.2 on my Raspberry PI: vlans_ue0="vlan3" create_args_vlan3="vlan 3" cloned_interfaces="bridge0 bridge3" ifconfig_bridge0="addm ue0 SYNCDHCP" ifconfig_bridge0_ipv6="inet6 auto_linklocal accept_rtadv" ifconfig_ue0="up" ifconfig_bridge3="addm vlan3" ifconfig_vlan3="up" So: +--vlan3--bridge3--jailProxy | ue0-+ | +--bridge0--jailWifi Networking from the host works fine. Networking from jailWifi works fine. But... jailProxy can send DHCP to the network in vlan 3. But the reply goes to jailWifi. Does bridge0 have precedence over vlan3? Could I let vlan3 handle packages with the proper vlan tag only and let the rest of the traffic go to bridge0? Ok. While typing this I came up with a solution which I tested and works. ue0--bridge0-+--jailWifi | +--epair0a--epair0b--vlan3--bridge3--jailProxy So now I have this DMZ bridge3 on its own VLAN. NB: ...
Read more