Conversation

Notices

1. Embed this notice
   þernia (pernia@cum.salon)'s status on Saturday, 19-Apr-2025 08:13:04 JST þernia
   rclone vfs is the greatest invention since sliced bread
   • Embed this notice
     þernia (pernia@cum.salon)'s status on Saturday, 19-Apr-2025 08:12:59 JST þernia

     in reply to

     • meeper

     @meeper

     without caching the sequential write is pretty much link speed:

     ~ $ dd if=/dev/zero of=/mnt/nas/media/test1gb bs=1M count=1024 oflag=direct 1024+0 records in 1024+0 records out 1073741824 bytes (1.0GB) copied, 109.953206 seconds, 9.3MB/s

     the sequential read is also link speed:

     ~ $ dd if=/mnt/nas/media/test1gb of=test1gb bs=1M oflag=direct 1024+0 records in 1024+0 records out 1073741824 bytes (1.0GB) copied, 103.523645 seconds, 9.9MB/s

     i'm also using zerotier to give a static ip address to my nas and other devices, but the overhead from that should be negligible (its p2p and i'm at home, next to my nas).

     pistolero likes this.
   • Embed this notice
     þernia (pernia@cum.salon)'s status on Saturday, 19-Apr-2025 08:13:00 JST þernia

     in reply to

     • meeper

     @meeper yea with cache its asyncronous, so it'll download the part of the file that you need for reading first (you can configure the chunksize so that you download less if you want), and continue with the rest of the download. for writing it will first write the file to cache, and upload it a certain time (5 seconds default, also configurable) after you've last opened the file for writing, and it will use all bandwidth (picrel)

     command: fio --filename=/mnt/nas/media/testrand10gb --size=10GB --direct=1 --rw=randrw --bs=64k --ioengine=libaio --iodepth= 64 --runtime=120 --numjobs=4 --time_based --group_reporting --name=throughput-test-job --eta-newline=1

     output:

     throughput-test-job: (groupid=0, jobs=4): err= 0: pid=31412: Fri Apr 18 16:51:46 2025 read: IOPS=3770, BW=236MiB/s (247MB/s)(27.6GiB/120004msec) slat (nsec): min=1523, max=435830k, avg=347584.43, stdev=2238101.58 clat (msec): min=2, max=2218, avg=33.07, stdev=50.25 lat (msec): min=9, max=2218, avg=33.41, stdev=50.66 clat percentiles (msec): | 1.00th=[ 19], 5.00th=[ 21], 10.00th=[ 22], 20.00th=[ 23], | 30.00th=[ 24], 40.00th=[ 24], 50.00th=[ 25], 60.00th=[ 26], | 70.00th=[ 27], 80.00th=[ 28], 90.00th=[ 32], 95.00th=[ 68], | 99.00th=[ 264], 99.50th=[ 313], 99.90th=[ 409], 99.95th=[ 776], | 99.99th=[ 2089] bw ( KiB/s): min= 640, max=347520, per=100.00%, avg=241365.90, stdev=19273.68, samples=959 iops : min= 10, max= 5430, avg=3771.32, stdev=301.15, samples=959 write: IOPS=3773, BW=236MiB/s (247MB/s)(27.6GiB/120004msec); 0 zone resets slat (usec): min=2, max=342667, avg=706.11, stdev=2809.83 clat (usec): min=741, max=2219.7k, avg=33742.59, stdev=54258.44 lat (msec): min=6, max=2227, avg=34.45, stdev=54.89 clat percentiles (msec): | 1.00th=[ 19], 5.00th=[ 21], 10.00th=[ 22], 20.00th=[ 23], | 30.00th=[ 24], 40.00th=[ 25], 50.00th=[ 26], 60.00th=[ 26], | 70.00th=[ 27], 80.00th=[ 29], 90.00th=[ 33], 95.00th=[ 70], | 99.00th=[ 268], 99.50th=[ 321], 99.90th=[ 409], 99.95th=[ 1062], | 99.99th=[ 2089] bw ( KiB/s): min= 640, max=344064, per=100.00%, avg=241565.77, stdev=19171.92, samples=959 iops : min= 10, max= 5376, avg=3774.45, stdev=299.56, samples=959 lat (usec) : 750=0.01%, 1000=0.01% lat (msec) : 4=0.01%, 10=0.01%, 20=4.32%, 50=88.68%, 100=4.27% lat (msec) : 250=1.49%, 500=1.18%, 750=0.02%, 1000=0.01%, 2000=0.03% lat (msec) : >=2000=0.02% cpu : usr=1.16%, sys=2.06%, ctx=1195357, majf=27, minf=53 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0% issued rwts: total=452421,452803,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=64 Run status group 0 (all jobs): READ: bw=236MiB/s (247MB/s), 236MiB/s-236MiB/s (247MB/s-247MB/s), io=27.6GiB (29.6GB), run=120004-120004msec WRITE: bw=236MiB/s (247MB/s), 236MiB/s-236MiB/s (247MB/s-247MB/s), io=27.6GiB (29.7GB), run=120004-120004msec
   • Embed this notice
     þernia (pernia@cum.salon)'s status on Saturday, 19-Apr-2025 08:13:01 JST þernia

     in reply to

     • meeper
     @meeper sftp has very little overhead, atleast compared to samba and nfs. last i tested it it was about as fast as my network i think.
     
     and the rclone vfs layer makes it so that it caches reads and writes, so its even nicer. i like that feature because if i disconnect for whatever reason i know i can just reconnect and be fine
   • Embed this notice
     meeper@mai.waifuism.life's status on Saturday, 19-Apr-2025 08:13:02 JST meeper

     in reply to

     @pernia@cum.salon sshfs sucks lol, also encryption and decryption slows things to a snails pace, it's on a local network and I am not syncing sensitive info lol

   • Embed this notice
     meeper@mai.waifuism.life's status on Saturday, 19-Apr-2025 08:13:03 JST meeper

     in reply to

     @pernia@cum.salon Is it better than synced thing

   • Embed this notice
     þernia (pernia@cum.salon)'s status on Saturday, 19-Apr-2025 08:13:03 JST þernia

     in reply to

     • meeper
     @meeper haven't tried synced thing.
     
     i'm using rclone instead of sshfs. and it really beats its ass
   • Embed this notice
     pwm (pwm@darkdork.dev)'s status on Saturday, 19-Apr-2025 13:09:01 JST pwm

     in reply to
     @pernia pedophile
   • Embed this notice
     Lina Inver?e (lina@eientei.org)'s status on Saturday, 19-Apr-2025 14:46:56 JST Lina Inver?e

     in reply to

     • Churkia
     @pernia @churkia goku
   • Embed this notice
     þernia (pernia@cum.salon)'s status on Saturday, 19-Apr-2025 14:46:58 JST þernia

     in reply to

     • Churkia
     @churkia its freagging goated :goku:
   • Embed this notice
     Churkia (churkia@cum.salon)'s status on Saturday, 19-Apr-2025 14:46:59 JST Churkia

     in reply to
     @boyperpia actually going to use rclone bcuz of u