ROSE DS Drive Hotspot?
Hi All, Does anyone know or have experience with ROSE DS and if the drives can be hot swappable? Regards -- Steve Sugden *Director* *Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: 447 Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, QLD, 4226 Phone: 1300 458 136 | Mobile 0413 440 479 Email: steve@valvenetworks.com.au | Web: www.valvenetworks.com.au
Hi Steve, My testing has found I could hotplug in new drives, but removing a drive and re-inserting required a reboot to recognise the disk again.. However I have used the Eject button for a disk first and then successfully removed and inserted the disk again. So I recomened using the Eject button first. I have to fail a raid disk yet and rebuild it. Interesting though, there is no log of me ejecting the disk and removing it, only adding it back in. Darren. On 23/10/2025 2:55 pm, Steve Sugden via Public wrote:
Hi All,
Does anyone know or have experience with ROSE DS and if the drives can be hot swappable?
Regards
--
Steve Sugden *Director*
*Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: 447 Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, QLD, 4226 Phone: 1300 458 136 | Mobile 0413 440 479 Email: steve@valvenetworks.com.au | Web: www.valvenetworks.com.au _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Hi All, We also provided some feedback to Mikrotik and were informed that XFS support may soon be added for rose-storage. Regards, Darren On 23/10/2025 2:55 pm, Steve Sugden via Public wrote:
Hi All,
Does anyone know or have experience with ROSE DS and if the drives can be hot swappable?
Regards
--
Steve Sugden *Director*
*Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: 447 Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, QLD, 4226 Phone: 1300 458 136 | Mobile 0413 440 479 Email: steve@valvenetworks.com.au | Web: www.valvenetworks.com.au _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Is ZFS supported?
On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public <public@talk.mikrotik.com.au> wrote:
Hi All,
We also provided some feedback to Mikrotik and were informed that XFS support may soon be added for rose-storage.
Regards, Darren
On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: Hi All,
Does anyone know or have experience with ROSE DS and if the drives can be hot swappable?
Regards
--
Steve Sugden *Director*
*Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: 447 Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, QLD, 4226 Phone: 1300 458 136 | Mobile 0413 440 479 Email: steve@valvenetworks.com.au | Web: www.valvenetworks.com.au _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
I second that. ZFS is essential for anything I look at for storage or where data integrity is essential. Cheers, Andrew
On 23 Oct 2025, at 5:00 pm, Craig Askings via Public <public@talk.mikrotik.com.au> wrote:
Is ZFS supported?
On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public <public@talk.mikrotik.com.au> wrote:
Hi All,
We also provided some feedback to Mikrotik and were informed that XFS support may soon be added for rose-storage.
Regards, Darren
On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: Hi All,
Does anyone know or have experience with ROSE DS and if the drives can be hot swappable?
Regards
--
Steve Sugden *Director*
*Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: 447 Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, QLD, 4226 Phone: 1300 458 136 | Mobile 0413 440 479 Email: steve@valvenetworks.com.au | Web: www.valvenetworks.com.au _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
G'day! Unfortunately not :-l And unlikely to ever be, apparently, due to "license incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL. What it means is that when ZFS is compiled into any license kernel project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow? Cheers! ------------------------------------------------------------------------------------ Why Choose DuxTel for all your MikroTik needs? 10 good reasons: http://duxtel.com/why_duxtel ------------------------------------------------------------------------------------ Follow our tweets for news and updates: http://twitter.com/duxtel
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au> Sent: Thursday, 23 October 2025 5:52 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Craig Askings <craig@askings.com.au> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Is ZFS supported?
On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public <public@talk.mikrotik.com.au> wrote:
Hi All,
We also provided some feedback to Mikrotik and were informed that XFS support may soon be added for rose-storage.
Regards, Darren
On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: Hi All,
Does anyone know or have experience with ROSE DS and if the drives can be hot swappable?
Regards
--
Steve Sugden *Director*
*Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: 447 Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, QLD, 4226 Phone: 1300 458 136 | Mobile 0413 440 479 Email: steve@valvenetworks.com.au | Web: www.valvenetworks.com.au _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Thanks everyone! Also realised my typo in subject as Hotspot instead of Hotswap! Regards, Steve On Fri, 24 Oct 2025 at 09:07, Mike Everest via Public < public@talk.mikrotik.com.au> wrote:
G'day!
Unfortunately not :-l
And unlikely to ever be, apparently, due to "license incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL.
What it means is that when ZFS is compiled into any license kernel project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow?
Cheers! ------------------------------------------------------------------------------------
Why Choose DuxTel for all your MikroTik needs? 10 good reasons: http://duxtel.com/why_duxtel ------------------------------------------------------------------------------------
Follow our tweets for news and updates: http://twitter.com/duxtel
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au> Sent: Thursday, 23 October 2025 5:52 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Craig Askings <craig@askings.com.au> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Is ZFS supported?
On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public <public@talk.mikrotik.com.au> wrote:
Hi All,
We also provided some feedback to Mikrotik and were informed that XFS support may soon be added for rose-storage.
Regards, Darren
On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: Hi All,
Does anyone know or have experience with ROSE DS and if the drives can be hot swappable?
Regards
--
Steve Sugden *Director*
*Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: 447 Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, QLD, 4226 Phone: 1300 458 136 | Mobile 0413 440 479 Email: steve@valvenetworks.com.au | Web: www.valvenetworks.com.au _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
:-D Would you believe I never even noticed that – I read it from the beginning as ‘hot swap’… weird how that works…! By the way, what is the big deal with ZFS anyway, and why is it actually so important/necessary? I’m not being captious, but genuinely don't understand the ins and outs in this context and would like to continue applying some 'encouraging pressure' on MT to add it in future :-} Cheers! ------------------------------------------------------------------------------------ Why Choose DuxTel for all your MikroTik needs? 10 good reasons: <http://duxtel.com/why_duxtel> http://duxtel.com/why_duxtel ------------------------------------------------------------------------------------ Follow our tweets for news and updates: <http://twitter.com/duxtel> http://twitter.com/duxtel From: Steve Sugden <steve@valvenetworks.com.au> Sent: Friday, 24 October 2025 10:30 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Mike Everest <mike@duxtel.com> Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot? Thanks everyone! Also realised my typo in subject as Hotspot instead of Hotswap! Regards, Steve On Fri, 24 Oct 2025 at 09:07, Mike Everest via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote: G'day! Unfortunately not :-l And unlikely to ever be, apparently, due to "license incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL. What it means is that when ZFS is compiled into any license kernel project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow? Cheers! ------------------------------------------------------------------------------------ Why Choose DuxTel for all your MikroTik needs? 10 good reasons: http://duxtel.com/why_duxtel ------------------------------------------------------------------------------------ Follow our tweets for news and updates: http://twitter.com/duxtel
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Sent: Thursday, 23 October 2025 5:52 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Cc: Craig Askings <craig@askings.com.au <mailto:craig@askings.com.au> > Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Is ZFS supported?
On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote:
Hi All,
We also provided some feedback to Mikrotik and were informed that XFS support may soon be added for rose-storage.
Regards, Darren
On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: Hi All,
Does anyone know or have experience with ROSE DS and if the drives can be hot swappable?
Regards
--
Steve Sugden *Director*
*Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: 447 Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, QLD, 4226 Phone: 1300 458 136 | Mobile 0413 440 479 Email: steve@valvenetworks.com.au <mailto:steve@valvenetworks.com.au> | Web: www.valvenetworks.com.au <http://www.valvenetworks.com.au> _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
Hi Mike, Craig and others can probably answer this more eloquently than me but ZFS has a number of major advantages: Absolute data integrity. Data corruption is almost non-existent. For it to occur all mirrors or slices need to fail. And it knows whether any piece of data is good or not so you aren’t left guessing. People are often more worried about cosmic rays and using ECC RAM than about ZFS or the disks. Utter reliability. BTRFS is still a long way off comparatively. Yes BTRFS is good from everything I’ve seen but it’s just not at the same level. 99.99999% is not good enough for critical storage. Snapshots. BTRFS has them too (and other filesystems) but ZFS’s snapshot speed, size, functionality, etc and the ability to send and receive them all over the place (across different OSs even) is game changing. I even use them as backups sent to removable disks. Standardised and stable. We use Proxmox for our clusters. I can send ZFS snapshots and backups to anything else with ZFS. I can send them across Proxmox versions and clusters and to FreeBSD systems. Tools. They are a little esoteric to learn but consistent and very functional. Speed and performance is also amazing. Basically you just don’t think about it anymore. Adding a ZIL and L2ARC on fast flash storage in front of spinning rust is really quite incredible with what it achieves. And the memory based ARC is a piece of black magic. Instantly resizable zvols acting as block devices (or quotas on datasets) makes VM maintenance a modern miracle. Did I mention data integrity and reliability? I’ve lost data once in 20 years of ZFS use and that was because it was on a system with a single NVMe and I screwed up. Of course ZFS told me it had a problem and what was lost. It meant I had to re-provision a few VMs but the system never held “data” in any case as they were DNS secondaries and the like. So having said all of this, I can also say I don’t know BTRFS. I’ve looked at it multiple times over the years but it has never been close to ZFS on the data integrity. And at the end of the day that’s what storage is for. I’ve also looked at Ceph many times, and it looks to be quite similar to ZFS in it’s data integrity concepts but from what I can see it’s performance is awful in comparison no matter how much network you throw at it. Of course if that doesn’t matter too much it looks absolutely incredible. Lastly, with regards to the licensing, Proxmox has worked out how to make it work and they are no longer some small little company that is ignored by the various communities and legal people. I’d hope Mikrotik could do the same. They probably have a lot less incentive, since it would mean working it out for what is at the moment just one product. But if they could, I’d buy a RDS in a heartbeat. Until then, I’ll keep stuffing storage into my Proxmox servers. Cheers, Andrew
On 24 Oct 2025, at 9:51 am, Mike Everest via Public <public@talk.mikrotik.com.au> wrote:
:-D
Would you believe I never even noticed that – I read it from the beginning as ‘hot swap’… weird how that works…!
By the way, what is the big deal with ZFS anyway, and why is it actually so important/necessary? I’m not being captious, but genuinely don't understand the ins and outs in this context and would like to continue applying some 'encouraging pressure' on MT to add it in future :-}
Cheers!
------------------------------------------------------------------------------------
Why Choose DuxTel for all your MikroTik needs?
10 good reasons: <http://duxtel.com/why_duxtel> http://duxtel.com/why_duxtel
------------------------------------------------------------------------------------
Follow our tweets for news and updates: <http://twitter.com/duxtel> http://twitter.com/duxtel
From: Steve Sugden <steve@valvenetworks.com.au> Sent: Friday, 24 October 2025 10:30 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Mike Everest <mike@duxtel.com> Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Thanks everyone!
Also realised my typo in subject as Hotspot instead of Hotswap!
Regards,
Steve
On Fri, 24 Oct 2025 at 09:07, Mike Everest via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote:
G'day!
Unfortunately not :-l
And unlikely to ever be, apparently, due to "license incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL.
What it means is that when ZFS is compiled into any license kernel project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow?
Cheers! ------------------------------------------------------------------------------------ Why Choose DuxTel for all your MikroTik needs? 10 good reasons: http://duxtel.com/why_duxtel ------------------------------------------------------------------------------------ Follow our tweets for news and updates: http://twitter.com/duxtel
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Sent: Thursday, 23 October 2025 5:52 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Cc: Craig Askings <craig@askings.com.au <mailto:craig@askings.com.au> > Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Is ZFS supported?
On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote:
Hi All,
We also provided some feedback to Mikrotik and were informed that XFS support may soon be added for rose-storage.
Regards, Darren
On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: Hi All,
Does anyone know or have experience with ROSE DS and if the drives can be hot swappable?
Regards
--
Steve Sugden *Director*
*Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: 447 Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, QLD, 4226 Phone: 1300 458 136 | Mobile 0413 440 479 Email: steve@valvenetworks.com.au <mailto:steve@valvenetworks.com.au> | Web: www.valvenetworks.com.au <http://www.valvenetworks.com.au> _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Andrew, That was a better answer than I would have given. The killer feature for me is how lightweight snapshotting is and the ability to ship snapshots between servers. Makes offsite point in time replication a breeze. You get into the habit of making snapshots all the time. Updating your DB schema? Snapshot! Distribution upgrade? Snapshot! YOLOing that Netbox upgrade? Snapshot! Bulk deleting files by hand? Snapshot! Etc.
On 24 Oct 2025, at 10:54 am, Andrew Radke via Public <public@talk.mikrotik.com.au> wrote:
Hi Mike,
Craig and others can probably answer this more eloquently than me but ZFS has a number of major advantages:
Absolute data integrity. Data corruption is almost non-existent. For it to occur all mirrors or slices need to fail. And it knows whether any piece of data is good or not so you aren’t left guessing. People are often more worried about cosmic rays and using ECC RAM than about ZFS or the disks. Utter reliability. BTRFS is still a long way off comparatively. Yes BTRFS is good from everything I’ve seen but it’s just not at the same level. 99.99999% is not good enough for critical storage. Snapshots. BTRFS has them too (and other filesystems) but ZFS’s snapshot speed, size, functionality, etc and the ability to send and receive them all over the place (across different OSs even) is game changing. I even use them as backups sent to removable disks. Standardised and stable. We use Proxmox for our clusters. I can send ZFS snapshots and backups to anything else with ZFS. I can send them across Proxmox versions and clusters and to FreeBSD systems. Tools. They are a little esoteric to learn but consistent and very functional. Speed and performance is also amazing. Basically you just don’t think about it anymore. Adding a ZIL and L2ARC on fast flash storage in front of spinning rust is really quite incredible with what it achieves. And the memory based ARC is a piece of black magic. Instantly resizable zvols acting as block devices (or quotas on datasets) makes VM maintenance a modern miracle.
Did I mention data integrity and reliability? I’ve lost data once in 20 years of ZFS use and that was because it was on a system with a single NVMe and I screwed up. Of course ZFS told me it had a problem and what was lost. It meant I had to re-provision a few VMs but the system never held “data” in any case as they were DNS secondaries and the like.
So having said all of this, I can also say I don’t know BTRFS. I’ve looked at it multiple times over the years but it has never been close to ZFS on the data integrity. And at the end of the day that’s what storage is for.
I’ve also looked at Ceph many times, and it looks to be quite similar to ZFS in it’s data integrity concepts but from what I can see it’s performance is awful in comparison no matter how much network you throw at it. Of course if that doesn’t matter too much it looks absolutely incredible.
Lastly, with regards to the licensing, Proxmox has worked out how to make it work and they are no longer some small little company that is ignored by the various communities and legal people. I’d hope Mikrotik could do the same. They probably have a lot less incentive, since it would mean working it out for what is at the moment just one product. But if they could, I’d buy a RDS in a heartbeat. Until then, I’ll keep stuffing storage into my Proxmox servers.
Cheers, Andrew
On 24 Oct 2025, at 9:51 am, Mike Everest via Public <public@talk.mikrotik.com.au> wrote:
:-D
Would you believe I never even noticed that – I read it from the beginning as ‘hot swap’… weird how that works…!
By the way, what is the big deal with ZFS anyway, and why is it actually so important/necessary? I’m not being captious, but genuinely don't understand the ins and outs in this context and would like to continue applying some 'encouraging pressure' on MT to add it in future :-}
Cheers!
------------------------------------------------------------------------------------
Why Choose DuxTel for all your MikroTik needs?
10 good reasons: <http://duxtel.com/why_duxtel> http://duxtel.com/why_duxtel
------------------------------------------------------------------------------------
Follow our tweets for news and updates: <http://twitter.com/duxtel> http://twitter.com/duxtel
From: Steve Sugden <steve@valvenetworks.com.au> Sent: Friday, 24 October 2025 10:30 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Mike Everest <mike@duxtel.com> Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Thanks everyone!
Also realised my typo in subject as Hotspot instead of Hotswap!
Regards,
Steve
On Fri, 24 Oct 2025 at 09:07, Mike Everest via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote:
G'day!
Unfortunately not :-l
And unlikely to ever be, apparently, due to "license incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL.
What it means is that when ZFS is compiled into any license kernel project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow?
Cheers! ------------------------------------------------------------------------------------ Why Choose DuxTel for all your MikroTik needs? 10 good reasons: http://duxtel.com/why_duxtel ------------------------------------------------------------------------------------ Follow our tweets for news and updates: http://twitter.com/duxtel
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Sent: Thursday, 23 October 2025 5:52 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Cc: Craig Askings <craig@askings.com.au <mailto:craig@askings.com.au> > Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Is ZFS supported?
On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote:
Hi All,
We also provided some feedback to Mikrotik and were informed that XFS support may soon be added for rose-storage.
Regards, Darren
On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: Hi All,
Does anyone know or have experience with ROSE DS and if the drives can be hot swappable?
Regards
--
Steve Sugden *Director*
*Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: 447 Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, QLD, 4226 Phone: 1300 458 136 | Mobile 0413 440 479 Email: steve@valvenetworks.com.au <mailto:steve@valvenetworks.com.au> | Web: www.valvenetworks.com.au <http://www.valvenetworks.com.au> _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
OK! Thanks Crag, Thanks Andrew - good to know. I still don't like the chances that ZFS will ever come - the licensing issue is a significant deal-breaker for RouterOS. Maybe proxmox solution is one of those that are open to legal challenge, which is one way to approach it - holding out hope that 'our lawyers are better than yours' (and 'we can afford to pay them for longer' ; ) - or maybe they have negotiated some kind of license exception. Either way, it is not likely that we'll see it (though not impossible! : ) What is far more likely, is that there will be additional FS support that address the deficiencies (real and perceived) of currently supported filesystems - such as those mentioned (thank you!) Which is why I asked (so that they know : ) Cheers!
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au> Sent: Friday, 24 October 2025 1:16 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Craig Askings <craig@askings.com.au> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Andrew,
That was a better answer than I would have given.
The killer feature for me is how lightweight snapshotting is and the ability to ship snapshots between servers.
Makes offsite point in time replication a breeze.
You get into the habit of making snapshots all the time.
Updating your DB schema? Snapshot! Distribution upgrade? Snapshot! YOLOing that Netbox upgrade? Snapshot! Bulk deleting files by hand? Snapshot! Etc.
On 24 Oct 2025, at 10:54 am, Andrew Radke via Public <public@talk.mikrotik.com.au> wrote:
Hi Mike,
Craig and others can probably answer this more eloquently than me but ZFS has a number of major advantages:
Absolute data integrity. Data corruption is almost non-existent. For it to occur all mirrors or slices need to fail. And it knows whether any piece of data is good or not so you aren’t left guessing. People are often more worried about cosmic rays and using ECC RAM than about ZFS or the disks. Utter reliability. BTRFS is still a long way off comparatively. Yes BTRFS is good from everything I’ve seen but it’s just not at the same level. 99.99999% is not good enough for critical storage. Snapshots. BTRFS has them too (and other filesystems) but ZFS’s snapshot speed, size, functionality, etc and the ability to send and receive them all over the place (across different OSs even) is game changing. I even use them as backups sent to removable disks. Standardised and stable. We use Proxmox for our clusters. I can send ZFS snapshots and backups to anything else with ZFS. I can send them across Proxmox versions and clusters and to FreeBSD systems. Tools. They are a little esoteric to learn but consistent and very functional. Speed and performance is also amazing. Basically you just don’t think about it anymore. Adding a ZIL and L2ARC on fast flash storage in front of spinning rust is really quite incredible with what it achieves. And the memory based ARC is a piece of black magic. Instantly resizable zvols acting as block devices (or quotas on datasets) makes VM maintenance a modern miracle.
Did I mention data integrity and reliability? I’ve lost data once in 20 years of ZFS use and that was because it was on a system with a single NVMe and I screwed up. Of course ZFS told me it had a problem and what was lost. It meant I had to re-provision a few VMs but the system never held “data” in any case as they were DNS secondaries and the like.
So having said all of this, I can also say I don’t know BTRFS. I’ve looked at it multiple times over the years but it has never been close to ZFS on the data integrity. And at the end of the day that’s what storage is for.
I’ve also looked at Ceph many times, and it looks to be quite similar to ZFS in it’s data integrity concepts but from what I can see it’s performance is awful in comparison no matter how much network you throw at it. Of course if that doesn’t matter too much it looks absolutely incredible.
Lastly, with regards to the licensing, Proxmox has worked out how to make it work and they are no longer some small little company that is ignored by the various communities and legal people. I’d hope Mikrotik could do the same. They probably have a lot less incentive, since it would mean working it out for what is at the moment just one product. But if they could, I’d buy a RDS in a heartbeat. Until then, I’ll keep stuffing storage into my Proxmox servers.
Cheers, Andrew
On 24 Oct 2025, at 9:51 am, Mike Everest via Public <public@talk.mikrotik.com.au> wrote:
:-D
Would you believe I never even noticed that – I read it from the beginning as ‘hot swap’… weird how that works…!
By the way, what is the big deal with ZFS anyway, and why is it actually so important/necessary? I’m not being captious, but genuinely don't understand the ins and outs in this context and would like to continue applying some 'encouraging pressure' on MT to add it in future :-}
Cheers!
--------------------------------------------------------------------- ---------------
Why Choose DuxTel for all your MikroTik needs?
10 good reasons: <http://duxtel.com/why_duxtel> http://duxtel.com/why_duxtel
--------------------------------------------------------------------- ---------------
Follow our tweets for news and updates: <http://twitter.com/duxtel> http://twitter.com/duxtel
From: Steve Sugden <steve@valvenetworks.com.au> Sent: Friday, 24 October 2025 10:30 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Mike Everest <mike@duxtel.com> Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Thanks everyone!
Also realised my typo in subject as Hotspot instead of Hotswap!
Regards,
Steve
On Fri, 24 Oct 2025 at 09:07, Mike Everest via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote:
G'day!
Unfortunately not :-l
And unlikely to ever be, apparently, due to "license incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL.
What it means is that when ZFS is compiled into any license kernel project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow?
Cheers! --------------------------------------------------------------------- --------------- Why Choose DuxTel for all your MikroTik needs? 10 good reasons: http://duxtel.com/why_duxtel --------------------------------------------------------------------- --------------- Follow our tweets for news and updates: http://twitter.com/duxtel
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Sent: Thursday, 23 October 2025 5:52 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Cc: Craig Askings <craig@askings.com.au <mailto:craig@askings.com.au> > Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Is ZFS supported?
On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote:
Hi All,
We also provided some feedback to Mikrotik and were informed that XFS support may soon be added for rose-storage.
Regards, Darren
> On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: > Hi All, > > Does anyone know or have experience with ROSE DS and if the > drives can be hot swappable? > > > Regards > > > -- > > Steve Sugden > *Director* > > > > *Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: > 447 > Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, > QLD, > 4226 > Phone: 1300 458 136 | Mobile 0413 440 479 > Email: steve@valvenetworks.com.au > <mailto:steve@valvenetworks.com.au> | Web: > www.valvenetworks.com.au <http://www.valvenetworks.com.au> > _______________________________________________ > Public mailing list -- public@talk.mikrotik.com.au > <mailto:public@talk.mikrotik.com.au> To unsubscribe send an > email to public-leave@talk.mikrotik.com.au > <mailto:public-leave@talk.mikrotik.com.au> _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
It's a sad one, since Ubuntu / Debian / TrueNAS and others are distributing it as a kernel module and have for years...
In 2016 Ubuntu announced that a legal review resulted in the conclusion that it is legally acceptable to use ZFS as binary kernel module in Linux. (As opposed to building it into the kernel image itself.)
On 2025-10-24 12:59 Mike Everest via Public wrote:
OK!
Thanks Crag, Thanks Andrew - good to know.
I still don't like the chances that ZFS will ever come - the licensing issue is a significant deal-breaker for RouterOS.
Maybe proxmox solution is one of those that are open to legal challenge, which is one way to approach it - holding out hope that 'our lawyers are better than yours' (and 'we can afford to pay them for longer' ; ) - or maybe they have negotiated some kind of license exception. Either way, it is not likely that we'll see it (though not impossible! : )
What is far more likely, is that there will be additional FS support that address the deficiencies (real and perceived) of currently supported filesystems - such as those mentioned (thank you!) Which is why I asked (so that they know : )
Cheers!
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au> Sent: Friday, 24 October 2025 1:16 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Craig Askings <craig@askings.com.au> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Andrew,
That was a better answer than I would have given.
The killer feature for me is how lightweight snapshotting is and the ability to ship snapshots between servers.
Makes offsite point in time replication a breeze.
You get into the habit of making snapshots all the time.
Updating your DB schema? Snapshot! Distribution upgrade? Snapshot! YOLOing that Netbox upgrade? Snapshot! Bulk deleting files by hand? Snapshot! Etc.
On 24 Oct 2025, at 10:54 am, Andrew Radke via Public <public@talk.mikrotik.com.au> wrote:
Hi Mike,
Craig and others can probably answer this more eloquently than me but ZFS has a number of major advantages:
Absolute data integrity. Data corruption is almost non-existent. For it to occur all mirrors or slices need to fail. And it knows whether any piece of data is good or not so you aren’t left guessing. People are often more worried about cosmic rays and using ECC RAM than about ZFS or the disks. Utter reliability. BTRFS is still a long way off comparatively. Yes BTRFS is good from everything I’ve seen but it’s just not at the same level. 99.99999% is not good enough for critical storage. Snapshots. BTRFS has them too (and other filesystems) but ZFS’s snapshot speed, size, functionality, etc and the ability to send and receive them all over the place (across different OSs even) is game changing. I even use them as backups sent to removable disks. Standardised and stable. We use Proxmox for our clusters. I can send ZFS snapshots and backups to anything else with ZFS. I can send them across Proxmox versions and clusters and to FreeBSD systems. Tools. They are a little esoteric to learn but consistent and very functional. Speed and performance is also amazing. Basically you just don’t think about it anymore. Adding a ZIL and L2ARC on fast flash storage in front of spinning rust is really quite incredible with what it achieves. And the memory based ARC is a piece of black magic. Instantly resizable zvols acting as block devices (or quotas on datasets) makes VM maintenance a modern miracle.
Did I mention data integrity and reliability? I’ve lost data once in 20 years of ZFS use and that was because it was on a system with a single NVMe and I screwed up. Of course ZFS told me it had a problem and what was lost. It meant I had to re-provision a few VMs but the system never held “data” in any case as they were DNS secondaries and the like.
So having said all of this, I can also say I don’t know BTRFS. I’ve looked at it multiple times over the years but it has never been close to ZFS on the data integrity. And at the end of the day that’s what storage is for.
I’ve also looked at Ceph many times, and it looks to be quite similar to ZFS in it’s data integrity concepts but from what I can see it’s performance is awful in comparison no matter how much network you throw at it. Of course if that doesn’t matter too much it looks absolutely incredible.
Lastly, with regards to the licensing, Proxmox has worked out how to make it work and they are no longer some small little company that is ignored by the various communities and legal people. I’d hope Mikrotik could do the same. They probably have a lot less incentive, since it would mean working it out for what is at the moment just one product. But if they could, I’d buy a RDS in a heartbeat. Until then, I’ll keep stuffing storage into my Proxmox servers.
Cheers, Andrew
On 24 Oct 2025, at 9:51 am, Mike Everest via Public <public@talk.mikrotik.com.au> wrote:
:-D
Would you believe I never even noticed that – I read it from the beginning as ‘hot swap’… weird how that works…!
By the way, what is the big deal with ZFS anyway, and why is it actually so important/necessary? I’m not being captious, but genuinely don't understand the ins and outs in this context and would like to continue applying some 'encouraging pressure' on MT to add it in future :-}
Cheers!
--------------------------------------------------------------------- ---------------
Why Choose DuxTel for all your MikroTik needs?
10 good reasons: <http://duxtel.com/why_duxtel> http://duxtel.com/why_duxtel
--------------------------------------------------------------------- ---------------
Follow our tweets for news and updates: <http://twitter.com/duxtel> http://twitter.com/duxtel
From: Steve Sugden <steve@valvenetworks.com.au> Sent: Friday, 24 October 2025 10:30 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Mike Everest <mike@duxtel.com> Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Thanks everyone!
Also realised my typo in subject as Hotspot instead of Hotswap!
Regards,
Steve
On Fri, 24 Oct 2025 at 09:07, Mike Everest via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote:
G'day!
Unfortunately not :-l
And unlikely to ever be, apparently, due to "license incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL.
What it means is that when ZFS is compiled into any license kernel project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow?
Cheers! --------------------------------------------------------------------- --------------- Why Choose DuxTel for all your MikroTik needs? 10 good reasons: http://duxtel.com/why_duxtel --------------------------------------------------------------------- --------------- Follow our tweets for news and updates: http://twitter.com/duxtel
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Sent: Thursday, 23 October 2025 5:52 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Cc: Craig Askings <craig@askings.com.au <mailto:craig@askings.com.au> > Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Is ZFS supported?
On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote:
Hi All,
We also provided some feedback to Mikrotik and were informed that XFS support may soon be added for rose-storage.
Regards, Darren
>> On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: >> Hi All, >> >> Does anyone know or have experience with ROSE DS and if the >> drives can be hot swappable? >> >> >> Regards >> >> >> -- >> >> Steve Sugden >> *Director* >> >> >> >> *Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: >> 447 >> Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, >> QLD, >> 4226 >> Phone: 1300 458 136 | Mobile 0413 440 479 >> Email: steve@valvenetworks.com.au >> <mailto:steve@valvenetworks.com.au> | Web: >> www.valvenetworks.com.au <http://www.valvenetworks.com.au> >> _______________________________________________ >> Public mailing list -- public@talk.mikrotik.com.au >> <mailto:public@talk.mikrotik.com.au> To unsubscribe send an >> email to public-leave@talk.mikrotik.com.au >> <mailto:public-leave@talk.mikrotik.com.au> > _______________________________________________ > Public mailing list -- public@talk.mikrotik.com.au > <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email > to public-leave@talk.mikrotik.com.au > <mailto:public-leave@talk.mikrotik.com.au>
Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Yeah, I think that the difference is that RouterOS is proprietary closed source and not like any of those other platforms. If they were to release a loadable module, they would need to release the source of the module which will probably disclose some of their proprietary work - which I guess is what they want to avoid. Proxmox is Debian, I think, so all open source anyway, and TrueNAS is FreeBDS(?) so it's essentially the same as CDDL. So RDS is stuck with alternatives until something else changes, or something new comes along ; ) Cheers!
-----Original Message----- From: James Hodgkinson <yaleman@ricetek.net> Sent: Friday, 24 October 2025 4:30 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Mike Everest <mike@duxtel.com> Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot?
It's a sad one, since Ubuntu / Debian / TrueNAS and others are distributing it as a kernel module and have for years...
In 2016 Ubuntu announced that a legal review resulted in the conclusion that it is legally acceptable to use ZFS as binary kernel module in Linux. (As opposed to building it into the kernel image itself.)
On 2025-10-24 12:59 Mike Everest via Public wrote:
OK!
Thanks Crag, Thanks Andrew - good to know.
I still don't like the chances that ZFS will ever come - the licensing issue is a significant deal-breaker for RouterOS.
Maybe proxmox solution is one of those that are open to legal challenge, which is one way to approach it - holding out hope that 'our lawyers are better than yours' (and 'we can afford to pay them for longer' ; ) - or maybe they have negotiated some kind of license exception. Either way, it is not likely that we'll see it (though not impossible! : )
What is far more likely, is that there will be additional FS support that address the deficiencies (real and perceived) of currently supported filesystems - such as those mentioned (thank you!) Which is why I asked (so that they know : )
Cheers!
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au> Sent: Friday, 24 October 2025 1:16 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Craig Askings <craig@askings.com.au> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Andrew,
That was a better answer than I would have given.
The killer feature for me is how lightweight snapshotting is and the ability to ship snapshots between servers.
Makes offsite point in time replication a breeze.
You get into the habit of making snapshots all the time.
Updating your DB schema? Snapshot! Distribution upgrade? Snapshot! YOLOing that Netbox upgrade? Snapshot! Bulk deleting files by hand? Snapshot! Etc.
On 24 Oct 2025, at 10:54 am, Andrew Radke via Public <public@talk.mikrotik.com.au> wrote:
Hi Mike,
Craig and others can probably answer this more eloquently than me but ZFS has a number of major advantages:
Absolute data integrity. Data corruption is almost non-existent. For it to occur all mirrors or slices need to fail. And it knows whether any piece of data is good or not so you aren’t left guessing. People are often more worried about cosmic rays and using ECC RAM than about ZFS or the disks. Utter reliability. BTRFS is still a long way off comparatively. Yes BTRFS is good from everything I’ve seen but it’s just not at the same level. 99.99999% is not good enough for critical storage. Snapshots. BTRFS has them too (and other filesystems) but ZFS’s snapshot speed, size, functionality, etc and the ability to send and receive them all over the place (across different OSs even) is game changing. I even use them as backups sent to removable disks. Standardised and stable. We use Proxmox for our clusters. I can send ZFS snapshots and backups to anything else with ZFS. I can send them across Proxmox versions and clusters and to FreeBSD systems. Tools. They are a little esoteric to learn but consistent and very functional. Speed and performance is also amazing. Basically you just don’t think about it anymore. Adding a ZIL and L2ARC on fast flash storage in front of spinning rust is really quite incredible with what it achieves. And the memory based ARC is a piece of black magic. Instantly resizable zvols acting as block devices (or quotas on datasets) makes VM maintenance a modern miracle.
Did I mention data integrity and reliability? I’ve lost data once in 20 years of ZFS use and that was because it was on a system with a single NVMe and I screwed up. Of course ZFS told me it had a problem and what was lost. It meant I had to re-provision a few VMs but the system never held “data” in any case as they were DNS secondaries and the like.
So having said all of this, I can also say I don’t know BTRFS. I’ve looked at it multiple times over the years but it has never been close to ZFS on the data integrity. And at the end of the day that’s what storage is for.
I’ve also looked at Ceph many times, and it looks to be quite similar to ZFS in it’s data integrity concepts but from what I can see it’s performance is awful in comparison no matter how much network you throw at it. Of course if that doesn’t matter too much it looks absolutely incredible.
Lastly, with regards to the licensing, Proxmox has worked out how to make it work and they are no longer some small little company that is ignored by the various communities and legal people. I’d hope Mikrotik could do the same. They probably have a lot less incentive, since it would mean working it out for what is at the moment just one product. But if they could, I’d buy a RDS in a heartbeat. Until then, I’ll keep stuffing storage into my Proxmox servers.
Cheers, Andrew
On 24 Oct 2025, at 9:51 am, Mike Everest via Public <public@talk.mikrotik.com.au> wrote:
:-D
Would you believe I never even noticed that – I read it from the beginning as ‘hot swap’… weird how that works…!
By the way, what is the big deal with ZFS anyway, and why is it actually so important/necessary? I’m not being captious, but genuinely don't understand the ins and outs in this context and would like to continue applying some 'encouraging pressure' on MT to add it in future :-}
Cheers!
------------------------------------------------------------------ --- ---------------
Why Choose DuxTel for all your MikroTik needs?
10 good reasons: <http://duxtel.com/why_duxtel> http://duxtel.com/why_duxtel
------------------------------------------------------------------ --- ---------------
Follow our tweets for news and updates: <http://twitter.com/duxtel> http://twitter.com/duxtel
From: Steve Sugden <steve@valvenetworks.com.au> Sent: Friday, 24 October 2025 10:30 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Mike Everest <mike@duxtel.com> Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Thanks everyone!
Also realised my typo in subject as Hotspot instead of Hotswap!
Regards,
Steve
On Fri, 24 Oct 2025 at 09:07, Mike Everest via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote:
G'day!
Unfortunately not :-l
And unlikely to ever be, apparently, due to "license incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL.
What it means is that when ZFS is compiled into any license kernel project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow?
Cheers! ------------------------------------------------------------------ --- --------------- Why Choose DuxTel for all your MikroTik needs? 10 good reasons: http://duxtel.com/why_duxtel ------------------------------------------------------------------ --- --------------- Follow our tweets for news and updates: http://twitter.com/duxtel
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Sent: Thursday, 23 October 2025 5:52 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > Cc: Craig Askings <craig@askings.com.au <mailto:craig@askings.com.au> > Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Is ZFS supported?
> On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote: > > Hi All, > > We also provided some feedback to Mikrotik and were informed > that XFS support may soon be added for rose-storage. > > Regards, > Darren > >>> On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: >>> Hi All, >>> >>> Does anyone know or have experience with ROSE DS and if the >>> drives can be hot swappable? >>> >>> >>> Regards >>> >>> >>> -- >>> >>> Steve Sugden >>> *Director* >>> >>> >>> >>> *Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: >>> 447 >>> Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, >>> QLD, >>> 4226 >>> Phone: 1300 458 136 | Mobile 0413 440 479 >>> Email: steve@valvenetworks.com.au >>> <mailto:steve@valvenetworks.com.au> | Web: >>> www.valvenetworks.com.au <http://www.valvenetworks.com.au> >>> _______________________________________________ >>> Public mailing list -- public@talk.mikrotik.com.au >>> <mailto:public@talk.mikrotik.com.au> To unsubscribe send an >>> email to public-leave@talk.mikrotik.com.au >>> <mailto:public-leave@talk.mikrotik.com.au> >> _______________________________________________ >> Public mailing list -- public@talk.mikrotik.com.au >> <mailto:public@talk.mikrotik.com.au> To unsubscribe send an >> email to public-leave@talk.mikrotik.com.au >> <mailto:public-leave@talk.mikrotik.com.au> _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
TrueNAS Scale is debian based and it has zfs also. Which on another point TrueNAS dropped glusterfs support the day after Redhat dropped support, Redhat dropped btrfs years ago also, hence Enterprise users today consider it no longer supported. Using that logic, btrfs probably should never had gone into the Rose package in the first place. ZFS also has problems of it's own, zfs requires quite a bit of memory? They say 1GB for every 1TB, on top of the minimum, since the RDS has 32GB, you'll need every bit to run a large array, potentially placing a limit of around 20TB or so. I also wouldn't give up completely on zfs being introduced in the future, but the resource drain alone will be discouraging, especially with limited resources on other devices, you'd likely to be limited to the RDS or CHR/x86 platforms only which further highlights the issue as why would Mikrotik spend time on adding a niche option evnet though there would be a advantage. Has anyone considered the use of the containers to run a distribtued filesystem over the top? Cephfs and glusterfs for example can run in a ARM container, and being cross platform also. You'd run the filesystem for files with live replication, the VM hosts are snapshoted on their hosts system with the files copied off to the RDS for replication indeplendantly. On 24/10/2025 5:33 pm, Mike Everest via Public wrote:
Yeah, I think that the difference is that RouterOS is proprietary closed source and not like any of those other platforms. If they were to release a loadable module, they would need to release the source of the module which will probably disclose some of their proprietary work - which I guess is what they want to avoid.
Proxmox is Debian, I think, so all open source anyway, and TrueNAS is FreeBDS(?) so it's essentially the same as CDDL.
So RDS is stuck with alternatives until something else changes, or something new comes along ; )
Cheers!
-----Original Message----- From: James Hodgkinson <yaleman@ricetek.net> Sent: Friday, 24 October 2025 4:30 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Mike Everest <mike@duxtel.com> Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot?
It's a sad one, since Ubuntu / Debian / TrueNAS and others are distributing it as a kernel module and have for years...
In 2016 Ubuntu announced that a legal review resulted in the conclusion that it is legally acceptable to use ZFS as binary kernel module in Linux. (As opposed to building it into the kernel image itself.) On 2025-10-24 12:59 Mike Everest via Public wrote: OK!
Thanks Crag, Thanks Andrew - good to know.
I still don't like the chances that ZFS will ever come - the licensing issue is a significant deal-breaker for RouterOS.
Maybe proxmox solution is one of those that are open to legal challenge, which is one way to approach it - holding out hope that 'our lawyers are better than yours' (and 'we can afford to pay them for longer' ; ) - or maybe they have negotiated some kind of license exception. Either way, it is not likely that we'll see it (though not impossible! : )
What is far more likely, is that there will be additional FS support that address the deficiencies (real and perceived) of currently supported filesystems - such as those mentioned (thank you!) Which is why I asked (so that they know : )
Cheers!
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au> Sent: Friday, 24 October 2025 1:16 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Craig Askings <craig@askings.com.au> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Andrew,
That was a better answer than I would have given.
The killer feature for me is how lightweight snapshotting is and the ability to ship snapshots between servers.
Makes offsite point in time replication a breeze.
You get into the habit of making snapshots all the time.
Updating your DB schema? Snapshot! Distribution upgrade? Snapshot! YOLOing that Netbox upgrade? Snapshot! Bulk deleting files by hand? Snapshot! Etc.
On 24 Oct 2025, at 10:54 am, Andrew Radke via Public <public@talk.mikrotik.com.au> wrote: Hi Mike,
Craig and others can probably answer this more eloquently than me but ZFS has a number of major advantages: Absolute data integrity. Data corruption is almost non-existent. For it to occur all mirrors or slices need to fail. And it knows whether any piece of data is good or not so you aren’t left guessing. People are often more worried about cosmic rays and using ECC RAM than about ZFS or the disks. Utter reliability. BTRFS is still a long way off comparatively. Yes BTRFS is good from everything I’ve seen but it’s just not at the same level. 99.99999% is not good enough for critical storage. Snapshots. BTRFS has them too (and other filesystems) but ZFS’s snapshot speed, size, functionality, etc and the ability to send and receive them all over the place (across different OSs even) is game changing. I even use them as backups sent to removable disks. Standardised and stable. We use Proxmox for our clusters. I can send ZFS snapshots and backups to anything else with ZFS. I can send them across Proxmox versions and clusters and to FreeBSD systems. Tools. They are a little esoteric to learn but consistent and very functional. Speed and performance is also amazing. Basically you just don’t think about it anymore. Adding a ZIL and L2ARC on fast flash storage in front of spinning rust is really quite incredible with what it achieves. And the memory based ARC is a piece of black magic. Instantly resizable zvols acting as block devices (or quotas on datasets) makes VM maintenance a modern miracle. Did I mention data integrity and reliability? I’ve lost data once in 20 years of ZFS use and that was because it was on a system with a single NVMe and I screwed up. Of course ZFS told me it had a problem and what was lost. It meant I had to re-provision a few VMs but the system never held “data” in any case as they were DNS secondaries and the like. So having said all of this, I can also say I don’t know BTRFS. I’ve looked at it multiple times over the years but it has never been close to ZFS on the data integrity. And at the end of the day that’s what storage is for. I’ve also looked at Ceph many times, and it looks to be quite similar to ZFS in it’s data integrity concepts but from what I can see it’s performance is awful in comparison no matter how much network you throw at it. Of course if that doesn’t matter too much it looks absolutely incredible. Lastly, with regards to the licensing, Proxmox has worked out how to make it work and they are no longer some small little company that is ignored by the various communities and legal people. I’d hope Mikrotik could do the same. They probably have a lot less incentive, since it would mean working it out for what is at the moment just one product. But if they could, I’d buy a RDS in a heartbeat. Until then, I’ll keep stuffing storage into my Proxmox servers. Cheers, Andrew
On 24 Oct 2025, at 9:51 am, Mike Everest via Public <public@talk.mikrotik.com.au> wrote: :-D
Would you believe I never even noticed that – I read it from the beginning as ‘hot swap’… weird how that works…! By the way, what is the big deal with ZFS anyway, and why is it actually so important/necessary? I’m not being captious, but genuinely don't understand the ins and outs in this context and would like to continue applying some 'encouraging pressure' on MT to add it in future :-}
Cheers!
------------------------------------------------------------------ --- ---------------
Why Choose DuxTel for all your MikroTik needs?
10 good reasons: <http://duxtel.com/why_duxtel> http://duxtel.com/why_duxtel
------------------------------------------------------------------ --- ---------------
Follow our tweets for news and updates: <http://twitter.com/duxtel> http://twitter.com/duxtel
From: Steve Sugden <steve@valvenetworks.com.au> Sent: Friday, 24 October 2025 10:30 AM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Mike Everest <mike@duxtel.com> Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Thanks everyone!
Also realised my typo in subject as Hotspot instead of Hotswap!
Regards,
Steve
On Fri, 24 Oct 2025 at 09:07, Mike Everest via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote: G'day!
Unfortunately not :-l
And unlikely to ever be, apparently, due to "license incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL. What it means is that when ZFS is compiled into any license kernel project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow? Cheers! ------------------------------------------------------------------ --- --------------- Why Choose DuxTel for all your MikroTik needs? 10 good reasons: http://duxtel.com/why_duxtel ------------------------------------------------------------------ --- --------------- Follow our tweets for news and updates: http://twitter.com/duxtel
> -----Original Message----- > From: Craig Askings via Public <public@talk.mikrotik.com.au > <mailto:public@talk.mikrotik.com.au> > > Sent: Thursday, 23 October 2025 5:52 PM > To: MikroTik Australia Public List <public@talk.mikrotik.com.au > <mailto:public@talk.mikrotik.com.au> > > Cc: Craig Askings <craig@askings.com.au > <mailto:craig@askings.com.au> > > Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot? > > Is ZFS supported? > >> On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public > <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> wrote: >> Hi All, >> >> We also provided some feedback to Mikrotik and were informed >> that XFS > support may soon be added for rose-storage. >> Regards, >> Darren >> >>>> On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: >>>> Hi All, >>>> >>>> Does anyone know or have experience with ROSE DS and if the >>>> drives can be hot swappable? >>>> >>>> >>>> Regards >>>> >>>> >>>> -- >>>> >>>> Steve Sugden >>>> *Director* >>>> >>>> >>>> >>>> *Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: >>>> 447 >>>> Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, >>>> QLD, >>>> 4226 >>>> Phone: 1300 458 136 | Mobile 0413 440 479 >>>> Email: steve@valvenetworks.com.au >>>> <mailto:steve@valvenetworks.com.au> | Web: >>>> www.valvenetworks.com.au <http://www.valvenetworks.com.au> >>>> _______________________________________________ >>>> Public mailing list -- public@talk.mikrotik.com.au >>>> <mailto:public@talk.mikrotik.com.au> To unsubscribe send an >>>> email to public-leave@talk.mikrotik.com.au >>>> <mailto:public-leave@talk.mikrotik.com.au> >>> _______________________________________________ >>> Public mailing list -- public@talk.mikrotik.com.au >>> <mailto:public@talk.mikrotik.com.au> To unsubscribe send an >>> email to public-leave@talk.mikrotik.com.au >>> <mailto:public-leave@talk.mikrotik.com.au> > _______________________________________________ > Public mailing list -- public@talk.mikrotik.com.au > <mailto:public@talk.mikrotik.com.au> To unsubscribe send an > email to public-leave@talk.mikrotik.com.au > <mailto:public-leave@talk.mikrotik.com.au> _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> To unsubscribe send an email to public-leave@talk.mikrotik.com.au <mailto:public-leave@talk.mikrotik.com.au>
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Yeah, I think the whole argument against ZFS and Linux has pretty much been settled. Proxmox uses a standard kernel module and I’m guessing does TrueNAS. I haven’t checked but I’d be pretty surprised if RouterOS doesn’t use kernel modules at which point it’s pretty much sorted. Licensing may require that the source code is provided but that’s easy to deal with. But, I’d guess there would be a fair bit of pushback from Mikrotik at this point in time. We can only hope. As to the RAM requirements, the old rule of thumb of 1GB pre 1TB of storage is probably a bit long in the tooth now. Really you are only going to be worrying about the ARC allocation and 32GB holds a *lot* of ARC. Considering everything in the RDS is flash I wouldn’t be too stressed about ARC either. It should be able to take care of almost almost all small reads with 16GB allocated to it and it would ignore large reads anyway. This is a huge generalisation, but once you are past the ARC RAM availability of the RDS you are almost certainly *way* past the speed it can deal with getting things off of disk with it’s capabilities, and you are going to be looking at a much bigger system. But if they can get ZFS into the RDS, there would be very little that could compete with it at it’s price point.
On 27 Oct 2025, at 10:06 am, Darren Clissold via Public <public@talk.mikrotik.com.au> wrote:
TrueNAS Scale is debian based and it has zfs also. Which on another point TrueNAS dropped glusterfs support the day after Redhat dropped support, Redhat dropped btrfs years ago also, hence Enterprise users today consider it no longer supported. Using that logic, btrfs probably should never had gone into the Rose package in the first place.
ZFS also has problems of it's own, zfs requires quite a bit of memory? They say 1GB for every 1TB, on top of the minimum, since the RDS has 32GB, you'll need every bit to run a large array, potentially placing a limit of around 20TB or so.
I also wouldn't give up completely on zfs being introduced in the future, but the resource drain alone will be discouraging, especially with limited resources on other devices, you'd likely to be limited to the RDS or CHR/x86 platforms only which further highlights the issue as why would Mikrotik spend time on adding a niche option evnet though there would be a advantage.
Has anyone considered the use of the containers to run a distribtued filesystem over the top? Cephfs and glusterfs for example can run in a ARM container, and being cross platform also. You'd run the filesystem for files with live replication, the VM hosts are snapshoted on their hosts system with the files copied off to the RDS for replication indeplendantly.
On 24/10/2025 5:33 pm, Mike Everest via Public wrote:
Yeah, I think that the difference is that RouterOS is proprietary closed source and not like any of those other platforms. If they were to release a loadable module, they would need to release the source of the module which will probably disclose some of their proprietary work - which I guess is what they want to avoid.
Proxmox is Debian, I think, so all open source anyway, and TrueNAS is FreeBDS(?) so it's essentially the same as CDDL.
So RDS is stuck with alternatives until something else changes, or something new comes along ; )
Cheers!
-----Original Message----- From: James Hodgkinson <yaleman@ricetek.net> Sent: Friday, 24 October 2025 4:30 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Mike Everest <mike@duxtel.com> Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot?
It's a sad one, since Ubuntu / Debian / TrueNAS and others are distributing it as a kernel module and have for years...
In 2016 Ubuntu announced that a legal review resulted in the conclusion that it is legally acceptable to use ZFS as binary kernel module in Linux. (As opposed to building it into the kernel image itself.) On 2025-10-24 12:59 Mike Everest via Public wrote: OK!
Thanks Crag, Thanks Andrew - good to know.
I still don't like the chances that ZFS will ever come - the licensing issue is a significant deal-breaker for RouterOS.
Maybe proxmox solution is one of those that are open to legal challenge, which is one way to approach it - holding out hope that 'our lawyers are better than yours' (and 'we can afford to pay them for longer' ; ) - or maybe they have negotiated some kind of license exception. Either way, it is not likely that we'll see it (though not impossible! : )
What is far more likely, is that there will be additional FS support that address the deficiencies (real and perceived) of currently supported filesystems - such as those mentioned (thank you!) Which is why I asked (so that they know : )
Cheers!
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au> Sent: Friday, 24 October 2025 1:16 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Craig Askings <craig@askings.com.au> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Andrew,
That was a better answer than I would have given.
The killer feature for me is how lightweight snapshotting is and the ability to ship snapshots between servers.
Makes offsite point in time replication a breeze.
You get into the habit of making snapshots all the time.
Updating your DB schema? Snapshot! Distribution upgrade? Snapshot! YOLOing that Netbox upgrade? Snapshot! Bulk deleting files by hand? Snapshot! Etc.
On 24 Oct 2025, at 10:54 am, Andrew Radke via Public <public@talk.mikrotik.com.au> wrote: Hi Mike,
Craig and others can probably answer this more eloquently than me but ZFS has a number of major advantages: Absolute data integrity. Data corruption is almost non-existent. For it to occur all mirrors or slices need to fail. And it knows whether any piece of data is good or not so you aren’t left guessing. People are often more worried about cosmic rays and using ECC RAM than about ZFS or the disks. Utter reliability. BTRFS is still a long way off comparatively. Yes BTRFS is good from everything I’ve seen but it’s just not at the same level. 99.99999% is not good enough for critical storage. Snapshots. BTRFS has them too (and other filesystems) but ZFS’s snapshot speed, size, functionality, etc and the ability to send and receive them all over the place (across different OSs even) is game changing. I even use them as backups sent to removable disks. Standardised and stable. We use Proxmox for our clusters. I can send ZFS snapshots and backups to anything else with ZFS. I can send them across Proxmox versions and clusters and to FreeBSD systems. Tools. They are a little esoteric to learn but consistent and very functional. Speed and performance is also amazing. Basically you just don’t think about it anymore. Adding a ZIL and L2ARC on fast flash storage in front of spinning rust is really quite incredible with what it achieves. And the memory based ARC is a piece of black magic. Instantly resizable zvols acting as block devices (or quotas on datasets) makes VM maintenance a modern miracle. Did I mention data integrity and reliability? I’ve lost data once in 20 years of ZFS use and that was because it was on a system with a single NVMe and I screwed up. Of course ZFS told me it had a problem and what was lost. It meant I had to re-provision a few VMs but the system never held “data” in any case as they were DNS secondaries and the like. So having said all of this, I can also say I don’t know BTRFS. I’ve looked at it multiple times over the years but it has never been close to ZFS on the data integrity. And at the end of the day that’s what storage is for. I’ve also looked at Ceph many times, and it looks to be quite similar to ZFS in it’s data integrity concepts but from what I can see it’s performance is awful in comparison no matter how much network you throw at it. Of course if that doesn’t matter too much it looks absolutely incredible. Lastly, with regards to the licensing, Proxmox has worked out how to make it work and they are no longer some small little company that is ignored by the various communities and legal people. I’d hope Mikrotik could do the same. They probably have a lot less incentive, since it would mean working it out for what is at the moment just one product. But if they could, I’d buy a RDS in a heartbeat. Until then, I’ll keep stuffing storage into my Proxmox servers. Cheers, Andrew
> On 24 Oct 2025, at 9:51 am, Mike Everest via Public <public@talk.mikrotik.com.au> wrote: > :-D > > > > Would you believe I never even noticed that – I read it from the > beginning as ‘hot swap’… weird how that works…! > By the way, what is the big deal with ZFS anyway, and why is it > actually so important/necessary? I’m not being captious, but > genuinely don't understand the ins and outs in this context and > would like to continue applying some 'encouraging pressure' on MT > to add it in future :-} > > Cheers! > > ------------------------------------------------------------------ > --- > --------------- > > Why Choose DuxTel for all your MikroTik needs? > > 10 good reasons: <http://duxtel.com/why_duxtel> > http://duxtel.com/why_duxtel > > ------------------------------------------------------------------ > --- > --------------- > > Follow our tweets for news and updates: > <http://twitter.com/duxtel> http://twitter.com/duxtel > > > > From: Steve Sugden <steve@valvenetworks.com.au> > Sent: Friday, 24 October 2025 10:30 AM > To: MikroTik Australia Public List <public@talk.mikrotik.com.au> > Cc: Mike Everest <mike@duxtel.com> > Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot? > > > > Thanks everyone! > > > > Also realised my typo in subject as Hotspot instead of Hotswap! > > > > Regards, > > Steve > > > > > > On Fri, 24 Oct 2025 at 09:07, Mike Everest via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote: > G'day! > > Unfortunately not :-l > > And unlikely to ever be, apparently, due to "license > incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL. > What it means is that when ZFS is compiled into any license kernel > project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow? > Cheers! > ------------------------------------------------------------------ > --- > --------------- Why Choose DuxTel for all your MikroTik needs? > 10 good reasons: http://duxtel.com/why_duxtel > ------------------------------------------------------------------ > --- > --------------- Follow our tweets for news and updates: > http://twitter.com/duxtel > >> -----Original Message----- >> From: Craig Askings via Public <public@talk.mikrotik.com.au >> <mailto:public@talk.mikrotik.com.au> > >> Sent: Thursday, 23 October 2025 5:52 PM >> To: MikroTik Australia Public List <public@talk.mikrotik.com.au >> <mailto:public@talk.mikrotik.com.au> > >> Cc: Craig Askings <craig@askings.com.au >> <mailto:craig@askings.com.au> > >> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot? >> >> Is ZFS supported? >> >>> On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public >> <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> wrote: >>> Hi All, >>> >>> We also provided some feedback to Mikrotik and were informed >>> that XFS >> support may soon be added for rose-storage. >>> Regards, >>> Darren >>> >>>>> On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: >>>>> Hi All, >>>>> >>>>> Does anyone know or have experience with ROSE DS and if the >>>>> drives can be hot swappable? >>>>> >>>>> >>>>> Regards >>>>> >>>>> >>>>> -- >>>>> >>>>> Steve Sugden >>>>> *Director* >>>>> >>>>> >>>>> >>>>> *Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: >>>>> 447 >>>>> Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, >>>>> QLD, >>>>> 4226 >>>>> Phone: 1300 458 136 | Mobile 0413 440 479 >>>>> Email: steve@valvenetworks.com.au >>>>> <mailto:steve@valvenetworks.com.au> | Web: >>>>> www.valvenetworks.com.au <http://www.valvenetworks.com.au> >>>>> _______________________________________________ >>>>> Public mailing list -- public@talk.mikrotik.com.au >>>>> <mailto:public@talk.mikrotik.com.au> To unsubscribe send an >>>>> email to public-leave@talk.mikrotik.com.au >>>>> <mailto:public-leave@talk.mikrotik.com.au> >>>> _______________________________________________ >>>> Public mailing list -- public@talk.mikrotik.com.au >>>> <mailto:public@talk.mikrotik.com.au> To unsubscribe send an >>>> email to public-leave@talk.mikrotik.com.au >>>> <mailto:public-leave@talk.mikrotik.com.au> >> _______________________________________________ >> Public mailing list -- public@talk.mikrotik.com.au >> <mailto:public@talk.mikrotik.com.au> To unsubscribe send an >> email to public-leave@talk.mikrotik.com.au >> <mailto:public-leave@talk.mikrotik.com.au> > _______________________________________________ > Public mailing list -- public@talk.mikrotik.com.au > <mailto:public@talk.mikrotik.com.au> > To unsubscribe send an email to public-leave@talk.mikrotik.com.au > <mailto:public-leave@talk.mikrotik.com.au> > > _______________________________________________ > Public mailing list -- public@talk.mikrotik.com.au To unsubscribe > send an email to public-leave@talk.mikrotik.com.au _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Am I remembering correctly that the 1GB per TB requirement only applies if you're using de-duplication? ________________________________ From: Andrew Radke via Public <public@talk.mikrotik.com.au> Sent: Monday, 27 October 2025 16:34 To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Andrew Radke <andrew@deepport.net> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot? Yeah, I think the whole argument against ZFS and Linux has pretty much been settled. Proxmox uses a standard kernel module and I’m guessing does TrueNAS. I haven’t checked but I’d be pretty surprised if RouterOS doesn’t use kernel modules at which point it’s pretty much sorted. Licensing may require that the source code is provided but that’s easy to deal with. But, I’d guess there would be a fair bit of pushback from Mikrotik at this point in time. We can only hope. As to the RAM requirements, the old rule of thumb of 1GB pre 1TB of storage is probably a bit long in the tooth now. Really you are only going to be worrying about the ARC allocation and 32GB holds a *lot* of ARC. Considering everything in the RDS is flash I wouldn’t be too stressed about ARC either. It should be able to take care of almost almost all small reads with 16GB allocated to it and it would ignore large reads anyway. This is a huge generalisation, but once you are past the ARC RAM availability of the RDS you are almost certainly *way* past the speed it can deal with getting things off of disk with it’s capabilities, and you are going to be looking at a much bigger system. But if they can get ZFS into the RDS, there would be very little that could compete with it at it’s price point.
On 27 Oct 2025, at 10:06 am, Darren Clissold via Public <public@talk.mikrotik.com.au> wrote:
TrueNAS Scale is debian based and it has zfs also. Which on another point TrueNAS dropped glusterfs support the day after Redhat dropped support, Redhat dropped btrfs years ago also, hence Enterprise users today consider it no longer supported. Using that logic, btrfs probably should never had gone into the Rose package in the first place.
ZFS also has problems of it's own, zfs requires quite a bit of memory? They say 1GB for every 1TB, on top of the minimum, since the RDS has 32GB, you'll need every bit to run a large array, potentially placing a limit of around 20TB or so.
I also wouldn't give up completely on zfs being introduced in the future, but the resource drain alone will be discouraging, especially with limited resources on other devices, you'd likely to be limited to the RDS or CHR/x86 platforms only which further highlights the issue as why would Mikrotik spend time on adding a niche option evnet though there would be a advantage.
Has anyone considered the use of the containers to run a distribtued filesystem over the top? Cephfs and glusterfs for example can run in a ARM container, and being cross platform also. You'd run the filesystem for files with live replication, the VM hosts are snapshoted on their hosts system with the files copied off to the RDS for replication indeplendantly.
On 24/10/2025 5:33 pm, Mike Everest via Public wrote:
Yeah, I think that the difference is that RouterOS is proprietary closed source and not like any of those other platforms. If they were to release a loadable module, they would need to release the source of the module which will probably disclose some of their proprietary work - which I guess is what they want to avoid.
Proxmox is Debian, I think, so all open source anyway, and TrueNAS is FreeBDS(?) so it's essentially the same as CDDL.
So RDS is stuck with alternatives until something else changes, or something new comes along ; )
Cheers!
-----Original Message----- From: James Hodgkinson <yaleman@ricetek.net> Sent: Friday, 24 October 2025 4:30 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Mike Everest <mike@duxtel.com> Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot?
It's a sad one, since Ubuntu / Debian / TrueNAS and others are distributing it as a kernel module and have for years...
In 2016 Ubuntu announced that a legal review resulted in the conclusion that it is legally acceptable to use ZFS as binary kernel module in Linux. (As opposed to building it into the kernel image itself.) On 2025-10-24 12:59 Mike Everest via Public wrote: OK!
Thanks Crag, Thanks Andrew - good to know.
I still don't like the chances that ZFS will ever come - the licensing issue is a significant deal-breaker for RouterOS.
Maybe proxmox solution is one of those that are open to legal challenge, which is one way to approach it - holding out hope that 'our lawyers are better than yours' (and 'we can afford to pay them for longer' ; ) - or maybe they have negotiated some kind of license exception. Either way, it is not likely that we'll see it (though not impossible! : )
What is far more likely, is that there will be additional FS support that address the deficiencies (real and perceived) of currently supported filesystems - such as those mentioned (thank you!) Which is why I asked (so that they know : )
Cheers!
-----Original Message----- From: Craig Askings via Public <public@talk.mikrotik.com.au> Sent: Friday, 24 October 2025 1:16 PM To: MikroTik Australia Public List <public@talk.mikrotik.com.au> Cc: Craig Askings <craig@askings.com.au> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Andrew,
That was a better answer than I would have given.
The killer feature for me is how lightweight snapshotting is and the ability to ship snapshots between servers.
Makes offsite point in time replication a breeze.
You get into the habit of making snapshots all the time.
Updating your DB schema? Snapshot! Distribution upgrade? Snapshot! YOLOing that Netbox upgrade? Snapshot! Bulk deleting files by hand? Snapshot! Etc.
On 24 Oct 2025, at 10:54 am, Andrew Radke via Public <public@talk.mikrotik.com.au> wrote: Hi Mike,
Craig and others can probably answer this more eloquently than me but ZFS has a number of major advantages: Absolute data integrity. Data corruption is almost non-existent. For it to occur all mirrors or slices need to fail. And it knows whether any piece of data is good or not so you aren’t left guessing. People are often more worried about cosmic rays and using ECC RAM than about ZFS or the disks. Utter reliability. BTRFS is still a long way off comparatively. Yes BTRFS is good from everything I’ve seen but it’s just not at the same level. 99.99999% is not good enough for critical storage. Snapshots. BTRFS has them too (and other filesystems) but ZFS’s snapshot speed, size, functionality, etc and the ability to send and receive them all over the place (across different OSs even) is game changing. I even use them as backups sent to removable disks. Standardised and stable. We use Proxmox for our clusters. I can send ZFS snapshots and backups to anything else with ZFS. I can send them across Proxmox versions and clusters and to FreeBSD systems. Tools. They are a little esoteric to learn but consistent and very functional. Speed and performance is also amazing. Basically you just don’t think about it anymore. Adding a ZIL and L2ARC on fast flash storage in front of spinning rust is really quite incredible with what it achieves. And the memory based ARC is a piece of black magic. Instantly resizable zvols acting as block devices (or quotas on datasets) makes VM maintenance a modern miracle. Did I mention data integrity and reliability? I’ve lost data once in 20 years of ZFS use and that was because it was on a system with a single NVMe and I screwed up. Of course ZFS told me it had a problem and what was lost. It meant I had to re-provision a few VMs but the system never held “data” in any case as they were DNS secondaries and the like. So having said all of this, I can also say I don’t know BTRFS. I’ve looked at it multiple times over the years but it has never been close to ZFS on the data integrity. And at the end of the day that’s what storage is for. I’ve also looked at Ceph many times, and it looks to be quite similar to ZFS in it’s data integrity concepts but from what I can see it’s performance is awful in comparison no matter how much network you throw at it. Of course if that doesn’t matter too much it looks absolutely incredible. Lastly, with regards to the licensing, Proxmox has worked out how to make it work and they are no longer some small little company that is ignored by the various communities and legal people. I’d hope Mikrotik could do the same. They probably have a lot less incentive, since it would mean working it out for what is at the moment just one product. But if they could, I’d buy a RDS in a heartbeat. Until then, I’ll keep stuffing storage into my Proxmox servers. Cheers, Andrew
> On 24 Oct 2025, at 9:51 am, Mike Everest via Public <public@talk.mikrotik.com.au> wrote: > :-D > > > > Would you believe I never even noticed that – I read it from the > beginning as ‘hot swap’… weird how that works…! > By the way, what is the big deal with ZFS anyway, and why is it > actually so important/necessary? I’m not being captious, but > genuinely don't understand the ins and outs in this context and > would like to continue applying some 'encouraging pressure' on MT > to add it in future :-} > > Cheers! > > ------------------------------------------------------------------ > --- > --------------- > > Why Choose DuxTel for all your MikroTik needs? > > 10 good reasons: <https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fduxtel.com%2Fwhy_duxtel&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729357886%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=MUH80RgGPkx9iIpR5PhAN4cGcTgj0%2F9on%2Bqu6RQt1qA%3D&reserved=0<http://duxtel.com/why_duxtel>> > https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fduxtel.com%2Fwhy_duxtel&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729373437%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=qblVazPXKKzcXQaN6ji%2FvjVg2wi%2FEdp8vOLrmDY6KO0%3D&reserved=0<http://duxtel.com/why_duxtel> > > ------------------------------------------------------------------ > --- > --------------- > > Follow our tweets for news and updates: > <https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftwitter.com%2Fduxtel&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729383201%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=AzgMjoItF%2FxGO5Yvuvy6%2FT97GDJaTz%2BZe6nMgpKnciw%3D&reserved=0<http://twitter.com/duxtel>> https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftwitter.com%2Fduxtel&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729392491%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=pffL%2F77Jqm28EApqEvB9nO5Hb1WHMcsU7cJ4DadFBaU%3D&reserved=0<http://twitter.com/duxtel> > > > > From: Steve Sugden <steve@valvenetworks.com.au> > Sent: Friday, 24 October 2025 10:30 AM > To: MikroTik Australia Public List <public@talk.mikrotik.com.au> > Cc: Mike Everest <mike@duxtel.com> > Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot? > > > > Thanks everyone! > > > > Also realised my typo in subject as Hotspot instead of Hotswap! > > > > Regards, > > Steve > > > > > > On Fri, 24 Oct 2025 at 09:07, Mike Everest via Public <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> > wrote: > G'day! > > Unfortunately not :-l > > And unlikely to ever be, apparently, due to "license > incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL. > What it means is that when ZFS is compiled into any license kernel > project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow? > Cheers! > ------------------------------------------------------------------ > --- > --------------- Why Choose DuxTel for all your MikroTik needs? > 10 good reasons: https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fduxtel.com%2Fwhy_duxtel&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729401883%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=jczWJ8XfEVVhKAgN1kMAIKiwF70Q5WetsjU9i%2F61ZNo%3D&reserved=0<http://duxtel.com/why_duxtel> > ------------------------------------------------------------------ > --- > --------------- Follow our tweets for news and updates: > https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftwitter.com%2Fduxtel&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729411063%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=jtEokgc7tRer1beh34SF7HmVqMu%2FeeRA%2FPEocDd1xxU%3D&reserved=0<http://twitter.com/duxtel> > >> -----Original Message----- >> From: Craig Askings via Public <public@talk.mikrotik.com.au >> <mailto:public@talk.mikrotik.com.au> > >> Sent: Thursday, 23 October 2025 5:52 PM >> To: MikroTik Australia Public List <public@talk.mikrotik.com.au >> <mailto:public@talk.mikrotik.com.au> > >> Cc: Craig Askings <craig@askings.com.au >> <mailto:craig@askings.com.au> > >> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot? >> >> Is ZFS supported? >> >>> On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public >> <public@talk.mikrotik.com.au <mailto:public@talk.mikrotik.com.au> wrote: >>> Hi All, >>> >>> We also provided some feedback to Mikrotik and were informed >>> that XFS >> support may soon be added for rose-storage. >>> Regards, >>> Darren >>> >>>>> On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: >>>>> Hi All, >>>>> >>>>> Does anyone know or have experience with ROSE DS and if the >>>>> drives can be hot swappable? >>>>> >>>>> >>>>> Regards >>>>> >>>>> >>>>> -- >>>>> >>>>> Steve Sugden >>>>> *Director* >>>>> >>>>> >>>>> >>>>> *Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: >>>>> 447 >>>>> Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, >>>>> QLD, >>>>> 4226 >>>>> Phone: 1300 458 136 | Mobile 0413 440 479 >>>>> Email: steve@valvenetworks.com.au >>>>> <mailto:steve@valvenetworks.com.au> | Web: >>>>> https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.valvenetworks.com.au%2F&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729420067%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=IGzDSqxWzg79YMXEWG2MDjqosI5sIbO%2F%2BmUn%2B6YrNVc%3D&reserved=0<http://www.valvenetworks.com.au/> <https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.valvenetworks.com.au%2F&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729429706%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=kji530MyRcEePz6zUHvYub7aVECxXujQ0joIulFIgMw%3D&reserved=0<http://www.valvenetworks.com.au/>> >>>>> _______________________________________________ >>>>> Public mailing list -- public@talk.mikrotik.com.au >>>>> <mailto:public@talk.mikrotik.com.au> To unsubscribe send an >>>>> email to public-leave@talk.mikrotik.com.au >>>>> <mailto:public-leave@talk.mikrotik.com.au> >>>> _______________________________________________ >>>> Public mailing list -- public@talk.mikrotik.com.au >>>> <mailto:public@talk.mikrotik.com.au> To unsubscribe send an >>>> email to public-leave@talk.mikrotik.com.au >>>> <mailto:public-leave@talk.mikrotik.com.au> >> _______________________________________________ >> Public mailing list -- public@talk.mikrotik.com.au >> <mailto:public@talk.mikrotik.com.au> To unsubscribe send an >> email to public-leave@talk.mikrotik.com.au >> <mailto:public-leave@talk.mikrotik.com.au> > _______________________________________________ > Public mailing list -- public@talk.mikrotik.com.au > <mailto:public@talk.mikrotik.com.au> > To unsubscribe send an email to public-leave@talk.mikrotik.com.au > <mailto:public-leave@talk.mikrotik.com.au> > > _______________________________________________ > Public mailing list -- public@talk.mikrotik.com.au To unsubscribe > send an email to public-leave@talk.mikrotik.com.au _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
On 27 Oct 2025, at 10:06 am, Darren Clissold via Public wrote:
TrueNAS Scale is debian based and it has zfs also. Which on another point TrueNAS dropped glusterfs support the day after Redhat dropped support, Redhat dropped btrfs years ago also, hence Enterprise users today consider it no longer supported. Using
De-duplication requires a *huge* amount of resources. I'd be surprised if 1GB per TB would be enough. And it is harsh on CPU. It used to be a "rule" that you would need 1GB of RAM per TB of storage. But it really doesn't matter much anymore. The work done over the last 10-15 years on OppenZFS is truly amazing. Performance improvements are fantastic. And a massive one is that if you loose your ZIL you no longer loose you pool, only the data that was in the ZIL. I'd still mirror it no matter what. I also have thoroughly settled into using RAID1 or RAID10 as when a disk fails or starts to fail the load on the other disks during rebuild is very light. RAIDZ and RAIDZ2 require everything to be rebalanced and that means you are putting strain on your other disks right as you have a degraded pool. That's probably been my only heart stopper with ZFS ever. Watching a disk failing that had previously been "good" as a RAIDZ was resyncing. It made it but it took days and everything was slow as hell during that time. ZFS saw lots of bad reads and retried them over and over until it got a good CRC and I didn't loose a single byte but it was unpleasant. The resync to replace the second disk only took a couple of hours from memory. The reason for mentioning this though, is that some of the work on RAIDZ, and now dRAID, has been equally impressive. I haven't played with them to be able to comment properly. dRAID really looks like the way to go if you have lots of disks. From memory it isn't efficient until about 8-10 disks though. And one of the recent updates allows for complete rebalancing of RAID1, RAID0 and RAID10 using cow for each item. This is fantastic news for those of us that have very old pools that have seen multiple disk size upgrades over the years. And through all of this ZFS has kept its utter reliability for data integrity. Ultimately that's the key, and I think why so many companies have been moving from BTRFS to ZFS. It also feels like BTRFS is consistently years behind. It's still cool that it's there but it feels a little like another ReiserFS or XFS. Regards Andrew Radke On Tuesday, 28-10-2025 at 8:43 Russell Hurren via Public wrote: Am I remembering correctly that the 1GB per TB requirement only applies if you're using de-duplication? ________________________________ From: Andrew Radke via Public Sent: Monday, 27 October 2025 16:34 To: MikroTik Australia Public List Cc: Andrew Radke Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot? Yeah, I think the whole argument against ZFS and Linux has pretty much been settled. Proxmox uses a standard kernel module and I’m guessing does TrueNAS. I haven’t checked but I’d be pretty surprised if RouterOS doesn’t use kernel modules at which point it’s pretty much sorted. Licensing may require that the source code is provided but that’s easy to deal with. But, I’d guess there would be a fair bit of pushback from Mikrotik at this point in time. We can only hope. As to the RAM requirements, the old rule of thumb of 1GB pre 1TB of storage is probably a bit long in the tooth now. Really you are only going to be worrying about the ARC allocation and 32GB holds a *lot* of ARC. Considering everything in the RDS is flash I wouldn’t be too stressed about ARC either. It should be able to take care of almost almost all small reads with 16GB allocated to it and it would ignore large reads anyway. This is a huge generalisation, but once you are past the ARC RAM availability of the RDS you are almost certainly *way* past the speed it can deal with getting things off of disk with it’s capabilities, and you are going to be looking at a much bigger system. But if they can get ZFS into the RDS, there would be very little that could compete with it at it’s price point. that logic, btrfs probably should never had gone into the Rose package in the first place.
ZFS also has problems of it's own, zfs requires quite a bit of
They say 1GB for every 1TB, on top of the minimum, since the RDS has 32GB, you'll need every bit to run a large array, potentially placing a limit of around 20TB or so.
I also wouldn't give up completely on zfs being introduced in the future, but the resource drain alone will be discouraging, especially with limited resources on other devices, you'd likely to be limited to
memory? the RDS or CHR/x86 platforms only which further highlights the issue as why would Mikrotik spend time on adding a niche option evnet though there would be a advantage.
Has anyone considered the use of the containers to run a distribtued
Cephfs and glusterfs for example can run in a ARM container, and being cross platform also. You'd run the filesystem for files with live replication, the VM hosts are snapshoted on their hosts system with the files copied off to the RDS for replication indeplendantly.
On 24/10/2025 5:33 pm, Mike Everest via Public wrote:
Yeah, I think that the difference is that RouterOS is proprietary closed source and not like any of those other platforms. If they were to release a loadable module, they would need to release the
Proxmox is Debian, I think, so all open source anyway, and TrueNAS
is FreeBDS(?) so it's essentially the same as CDDL.
So RDS is stuck with alternatives until something else changes, or
something new comes along ; )
Cheers!
-----Original Message----- From: James Hodgkinson Sent: Friday, 24 October 2025 4:30 PM To: MikroTik Australia Public List Cc: Mike Everest Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot?
It's a sad one, since Ubuntu / Debian / TrueNAS and others are
distributing it
as a kernel module and have for years...
In 2016 Ubuntu announced that a legal review resulted in the conclusion that it is legally acceptable to use ZFS as binary kernel module in Linux. (As opposed to building it into the kernel image itself.) On 2025-10-24 12:59 Mike Everest via Public wrote: OK!
Thanks Crag, Thanks Andrew - good to know.
I still don't like the chances that ZFS will ever come - the
issue is a significant deal-breaker for RouterOS.
Maybe proxmox solution is one of those that are open to legal challenge, which is one way to approach it - holding out hope
'our lawyers are better than yours' (and 'we can afford to pay
for longer' ; ) - or maybe they have negotiated some kind of
exception. Either way, it is not likely that we'll see it (though not impossible! : )
What is far more likely, is that there will be additional FS support that address the deficiencies (real and perceived) of currently supported filesystems - such as those mentioned (thank you!) Which is why I asked (so that they know : )
Cheers!
-----Original Message----- From: Craig Askings via Public Sent: Friday, 24 October 2025 1:16 PM To: MikroTik Australia Public List Cc: Craig Askings Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot?
Andrew,
That was a better answer than I would have given.
The killer feature for me is how lightweight snapshotting is and
ability to ship snapshots between servers.
Makes offsite point in time replication a breeze.
You get into the habit of making snapshots all the time.
Updating your DB schema? Snapshot! Distribution upgrade? Snapshot! YOLOing that Netbox upgrade? Snapshot! Bulk deleting files by hand? Snapshot! Etc.
On 24 Oct 2025, at 10:54 am, Andrew Radke via Public wrote: Hi Mike,
Craig and others can probably answer this more eloquently than me but ZFS has a number of major advantages: Absolute data integrity. Data corruption is almost non-existent. For it to occur all mirrors or slices need to fail. And it knows whether any piece of data is good or not so you aren’t left guessing. People are often more worried about cosmic rays and using ECC RAM than about ZFS or the disks. Utter reliability. BTRFS is still a long way off comparatively. Yes BTRFS is good from everything I’ve seen but it’s just not at the same level. 99.99999% is not good enough for critical storage. Snapshots. BTRFS has them too (and other filesystems) but ZFS’s snapshot speed, size, functionality, etc and the ability to send and receive them all over the place (across different OSs even) is game changing. I even use them as backups sent to removable disks. Standardised and stable. We use Proxmox for our clusters. I can send ZFS snapshots and backups to anything else with ZFS. I can send them across Proxmox versions and clusters and to FreeBSD systems. Tools. They are a little esoteric to learn but consistent and very functional. Speed and performance is also amazing. Basically you just don’t think about it anymore. Adding a ZIL and L2ARC on fast flash storage in front of spinning rust is really quite incredible with what it achieves. And the memory based ARC is a piece of black magic. Instantly resizable zvols acting as block devices (or quotas on datasets) makes VM maintenance a modern miracle. Did I mention data integrity and reliability? I’ve lost data once in 20 years of ZFS use and that was because it was on a system with a single NVMe and I screwed up. Of course ZFS told me it had a problem and what was lost. It meant I had to re-provision a few VMs but the system never held “data” in any case as they were DNS secondaries and the
So having said all of this, I can also say I don’t know BTRFS. I’ve looked at it multiple times over the years but it has never been close to ZFS on the data integrity. And at the end of the day that’s what storage is for. I’ve also looked at Ceph many times, and it looks to be quite similar to ZFS in it’s data integrity concepts but from what I can see it’s
Lastly, with regards to the licensing, Proxmox has worked out how to make it work and they are no longer some small little company that is ignored by the various communities and legal people. I’d hope Mikrotik could do the same. They probably have a lot less incentive, since it would mean working it out for what is at the moment just one
is awful in comparison no matter how much network you throw at it. Of course if that doesn’t matter too much it looks absolutely incredible. product. But if they could, I’d buy a RDS in a heartbeat. Until then, I’ll keep stuffing storage into my Proxmox servers.
Cheers, Andrew
> On 24 Oct 2025, at 9:51 am, Mike Everest via Public wrote: > :-D > > > > Would you believe I never even noticed that – I read it from
filesystem over the top? source of the module which will probably disclose some of their proprietary work - which I guess is what they want to avoid. licensing that them license the like. performance the
> beginning as ‘hot swap’… weird how that works…! > By the way, what is the big deal with ZFS anyway, and why is it > actually so important/necessary? I’m not being captious, but > genuinely don't understand the ins and outs in this context and > would like to continue applying some 'encouraging pressure' on MT > to add it in future :-} > > Cheers! > >
> --- > --------------- > > Why Choose DuxTel for all your MikroTik needs? > > 10 good reasons: > https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fduxtel.com%2Fwhy_duxtel&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729373437%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=qblVazPXKKzcXQaN6ji%2FvjVg2wi%2FEdp8vOLrmDY6KO0%3D&reserved=0 > >
> --- > --------------- > > Follow our tweets for news and updates: > https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftwitter.com%2Fduxtel&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729392491%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=pffL%2F77Jqm28EApqEvB9nO5Hb1WHMcsU7cJ4DadFBaU%3D&reserved=0 > > > > From: Steve Sugden > Sent: Friday, 24 October 2025 10:30 AM > To: MikroTik Australia Public List > Cc: Mike Everest > Subject: Re: [MT-AU Public] Re: ROSE DS Drive Hotspot? > > > > Thanks everyone! > > > > Also realised my typo in subject as Hotspot instead of Hotswap! > > > > Regards, > > Steve > > > > > > On Fri, 24 Oct 2025 at 09:07, Mike Everest via Public
wrote:
> G'day! > > Unfortunately not :-l > > And unlikely to ever be, apparently, due to "license > incompatibility" of ZFS license terms: ZFS is CDDL, Linux kernel (hence RouterOS) is GPL. > What it means is that when ZFS is compiled into any license kernel > project, then the resulting distro becomes also CDDL. Arguably, ZFS could be released as an independent routerOS module, but the jury is out as to whether that is a technically valid work-around, and even so, it would no longer be part of the main kernel code so potentially some loss of performance(?) which kind of defeats the purpose of using that filesystem anyhow? > Cheers! >
> --- > --------------- Why Choose DuxTel for all your MikroTik needs? > 10 good reasons: https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fduxtel.com%2Fwhy_duxtel&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729401883%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=jczWJ8XfEVVhKAgN1kMAIKiwF70Q5WetsjU9i%2F61ZNo%3D&reserved=0 >
> --- > --------------- Follow our tweets for news and updates: > https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftwitter.com%2Fduxtel&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729411063%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=jtEokgc7tRer1beh34SF7HmVqMu%2FeeRA%2FPEocDd1xxU%3D&reserved=0 > >> -----Original Message----- >> From: Craig Askings via Public >>>>>>> > >> Sent: Thursday, 23 October 2025 5:52 PM >> To: MikroTik Australia Public List >>>>>>> > >> Cc: Craig Askings >>>>>>> > >> Subject: [MT-AU Public] Re: ROSE DS Drive Hotspot? >> >> Is ZFS supported? >> >>> On 23 Oct 2025, at 3:35 pm, Darren Clissold via Public >> >>>> wrote: >>> Hi All, >>> >>> We also provided some feedback to Mikrotik and were informed >>> that XFS >> support may soon be added for rose-storage. >>> Regards, >>> Darren >>> >>>>> On 23/10/2025 2:55 pm, Steve Sugden via Public wrote: >>>>> Hi All, >>>>> >>>>> Does anyone know or have experience with ROSE DS and if the >>>>> drives can be hot swappable? >>>>> >>>>> >>>>> Regards >>>>> >>>>> >>>>> -- >>>>> >>>>> Steve Sugden >>>>> *Director* >>>>> >>>>> >>>>> >>>>> *Valve Networks Pty Ltd*ABN: 31 168 442 288 ACMA Carrier Licence: >>>>> 447 >>>>> Office: Level 3, Space Building, 328 Scottsdale Drive, Robina, >>>>> QLD, >>>>> 4226 >>>>> Phone: 1300 458 136 | Mobile 0413 440 479 >>>>> Email: steve@valvenetworks.com.au >>>>> | Web: >>>>> https://aus01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.valvenetworks.com.au%2F&data=05%7C02%7Crussell%40zeropointnetworks.com%7C81c62089461840e099d708de15374b18%7C11ff14afea684cb1af94b4c5be201f2d%7C0%7C0%7C638971524729420067%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C40000%7C%7C%7C&sdata=IGzDSqxWzg79YMXEWG2MDjqosI5sIbO%2F%2BmUn%2B6YrNVc%3D&reserved=0
>>>>> _______________________________________________ >>>>> Public mailing list -- public@talk.mikrotik.com.au >>>>> To unsubscribe send an >>>>> email to public-leave@talk.mikrotik.com.au >>>>> >>>> _______________________________________________ >>>> Public mailing list -- public@talk.mikrotik.com.au >>>> To unsubscribe send an >>>> email to public-leave@talk.mikrotik.com.au >>>> >> _______________________________________________ >> Public mailing list -- public@talk.mikrotik.com.au >> To unsubscribe send an >> email to public-leave@talk.mikrotik.com.au >> > _______________________________________________ > Public mailing list -- public@talk.mikrotik.com.au > > To unsubscribe send an email to public-leave@talk.mikrotik.com.au > > > _______________________________________________ > Public mailing list -- public@talk.mikrotik.com.au To unsubscribe > send an email to public-leave@talk.mikrotik.com.au _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
_______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au _______________________________________________ Public mailing list -- public@talk.mikrotik.com.au To unsubscribe send an email to public-leave@talk.mikrotik.com.au
participants (7)
-
Andrew Radke -
Craig Askings -
Darren Clissold -
James Hodgkinson -
Mike Everest -
Russell Hurren -
Steve Sugden