原文说:https://www.liujason.com/article/800.html
新建loop虚拟盘 本地磁盘就是NVME的,如果可以重新分区那是最好,不过我这里嫌麻烦就直接用loop虚拟盘了,性能会差一点点但是不会太多(毕竟本地已经格式化为Ext4了)。
创建镜像文件 大家都喜欢用dd,不过fallocate我觉得更舒服。这里新建两个镜像文件,分别是读缓存和写缓存的说:
要注意的是,读缓存损坏后数据不会丢失,写缓存损坏的话会导致数据丢失,慎用!
挂载为loop 这个没啥好说的,用losetup即可说:
检查挂载情况
添加到zpool 查看zpool 首先检查一下zpool的情况说:
添加缓存 分别添加读缓存和写缓存说:
检查缓存情况
缓存效果 用zpool iostat -v可以明显看到SSD的读写缓存已经生效了,后续热文件将直接从cache中读取,降低机械盘的负载。
- root@HPC-HEL1-S101 ~ # zpool iostat -v
- capacity operations bandwidth
- pool alloc free read write read write
- ———- —– —– —– —– —– —–
- Hel1-S101 132G 90.8T 0 89 3.30K 1.91M
- raidz1 132G 90.8T 0 81 3.30K 991K
- sda – – 0 8 333 99.5K
- sdb – – 0 7 315 98.6K
- sdc – – 0 8 349 99.6K
- sdd – – 0 7 334 98.7K
- sde – – 0 8 362 99.7K
- sdf – – 0 8 326 98.7K
- sdg – – 0 8 354 99.6K
- sdh – – 0 8 328 98.6K
- sdi – – 0 8 349 99.6K
- sdj – – 0 8 322 98.5K
- logs – – – – – –
- loop1 11.8M 298G 0 10 8 1.21M
- cache – – – – – –
- loop2 224M 300G 0 0 49 81.3K
- ———- —– —– —– —– —– —–
复制代码
fio测试-16k小文件随机 测试代码说:fio -filename=/Hel1-S101/fiotest.bin -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=16k -size=10G -numjobs=30 -runtime=10 -group_reporting -name=zfs-cache-test
- Jobs: 30 (f=30): [w(30)][100.0%][w=153MiB/s][w=9782 IOPS][eta 00m:00s]
- zfs-cache-test: (groupid=0, jobs=30): err= 0: pid=21034: Thu Jun 4 18:49:56 2020
- write: IOPS=11.1k, BW=173MiB/s (182MB/s)(1734MiB/10002msec); 0 zone resets
- clat (usec): min=5, max=88218, avg=2700.66, stdev=2176.33
- lat (usec): min=6, max=88218, avg=2700.98, stdev=2176.36
- clat percentiles (usec):
- | 1.00th=[ 10], 5.00th=[ 45], 10.00th=[ 52], 20.00th=[ 816],
- | 30.00th=[ 1696], 40.00th=[ 2311], 50.00th=[ 2737], 60.00th=[ 3130],
- | 70.00th=[ 3556], 80.00th=[ 3982], 90.00th=[ 4883], 95.00th=[ 6063],
- | 99.00th=[ 7373], 99.50th=[ 8291], 99.90th=[18220], 99.95th=[26346],
- | 99.99th=[53740]
- bw ( KiB/s): min= 3552, max=40271, per=3.33%, avg=5913.96, stdev=4550.90, samples=592
- iops : min= 222, max= 2516, avg=369.60, stdev=284.39, samples=592
- lat (usec) : 10=1.10%, 20=0.79%, 50=7.09%, 100=5.98%, 250=2.23%
- lat (usec) : 500=0.92%, 750=1.26%, 1000=3.18%
- lat (msec) : 2=11.54%, 4=46.17%, 10=19.38%, 20=0.29%, 50=0.07%
- lat (msec) : 100=0.01%
- cpu : usr=0.16%, sys=2.21%, ctx=205649, majf=0, minf=0
- IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=0,110988,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=1
-
- Run status group 0 (all jobs):
- WRITE: bw=173MiB/s (182MB/s), 173MiB/s-173MiB/s (182MB/s-182MB/s), io=1734MiB (1818MB), run=10002-10002msec
复制代码
fio测试-大文件1G连续 测试代码说:fio -filename=/Hel1-S101/fiotest.bin -direct=1 -iodepth 1 -thread -rw=randwrite -ioengine=psync -bs=1G -size=20G -numjobs=30 -runtime=10 -group_reporting -name=zfs-cache-test
- zfs-cache-test: (groupid=0, jobs=30): err= 0: pid=6117: Thu Jun 4 18:52:20 2020
- write: IOPS=0, BW=785MiB/s (823MB/s)(30.0GiB/39131msec); 0 zone resets
- clat (msec): min=14642, max=38803, avg=24851.81, stdev=8958.50
- lat (msec): min=14770, max=39124, avg=25138.77, stdev=8987.50
- clat percentiles (msec):
- | 1.00th=[14697], 5.00th=[14697], 10.00th=[14832], 20.00th=[15771],
- | 30.00th=[16174], 40.00th=[16845], 50.00th=[17113], 60.00th=[17113],
- | 70.00th=[17113], 80.00th=[17113], 90.00th=[17113], 95.00th=[17113],
- | 99.00th=[17113], 99.50th=[17113], 99.90th=[17113], 99.95th=[17113],
- | 99.99th=[17113]
- cpu : usr=0.54%, sys=1.83%, ctx=264186, majf=0, minf=0
- IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
- submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
- issued rwts: total=0,30,0,0 short=0,0,0,0 dropped=0,0,0,0
- latency : target=0, window=0, percentile=100.00%, depth=1
-
- Run status group 0 (all jobs):
- WRITE: bw=785MiB/s (823MB/s), 785MiB/s-785MiB/s (823MB/s-823MB/s), io=30.0GiB (32.2GB), run=39131-39131msec
复制代码
|
评论前必须登录!
注册