proxmox更新到7版後

在管理介面 mount nfs 可以選 v3 或 v4

用fio測一下 v3 和 v4 的效能差別

config 如下

# This job file tries to mimic the Intel IOMeter File Server Access Pattern

[global]

description=Emulation of Intel IOmeter File Server Access Pattern

[iometer]

bssplit=512/10:1k/5:2k/5:4k/60:8k/2:16k/4:32k/4:64k/10

filename=PhysicalDrive1:PhysicalDrive2:PhysicalDrive3

size=10G

rw=randrw

#set read 50% write 50%

rwmixread=50

direct=1

runtime=60

# IOMeter defines the server loads as the following:

# iodepth=1                    Linear

# iodepth=4                    Very Light

# iodepth=8                    Light

# iodepth=64                    Moderate

# iodepth=256                    Heavy

iodepth=64

測三次 取最後一次的值如下

nfs v4

iometer: (g=0): rw=randrw, bs=(R) 512B-64.0KiB, (W) 512B-64.0KiB, (T) 512B-64.0KiB, ioengine=psync, iodepth=64

fio-3.25

Starting 1 process

Jobs: 1 (f=3): [m(1)][100.0%][r=168KiB/s,w=107KiB/s][r=11,w=12 IOPS][eta 00m:00s]

iometer: (groupid=0, jobs=1): err= 0: pid=2342174: Sun Oct 31 07:28:44 2021

  Description  : [Emulation of Intel IOmeter File Server Access Pattern]

  read: IOPS=55, BW=610KiB/s (625kB/s)(35.7MiB/60016msec)

    clat (usec): min=430, max=339828, avg=10019.94, stdev=13081.64

     lat (usec): min=431, max=339828, avg=10020.43, stdev=13081.64

    clat percentiles (usec):

     |  1.00th=[   775],  5.00th=[  3228], 10.00th=[  4015], 20.00th=[  5473],

     | 30.00th=[  6587], 40.00th=[  7832], 50.00th=[  9110], 60.00th=[ 10159],

     | 70.00th=[ 11338], 80.00th=[ 12387], 90.00th=[ 13304], 95.00th=[ 14222],

     | 99.00th=[ 43254], 99.50th=[ 99091], 99.90th=[208667], 99.95th=[299893],

     | 99.99th=[341836]

   bw (  KiB/s): min=   72, max= 1180, per=100.00%, avg=613.90, stdev=270.71, samples=119

   iops        : min=    8, max=   90, avg=56.03, stdev=18.14, samples=119

  write: IOPS=57, BW=592KiB/s (606kB/s)(34.7MiB/60016msec); 0 zone resets

    clat (usec): min=316, max=218978, avg=7624.96, stdev=11376.82

     lat (usec): min=316, max=218979, avg=7626.12, stdev=11376.98

    clat percentiles (usec):

     |  1.00th=[   400],  5.00th=[   457], 10.00th=[   490], 20.00th=[   586],

     | 30.00th=[  1172], 40.00th=[  4113], 50.00th=[  6128], 60.00th=[  8094],

     | 70.00th=[ 10028], 80.00th=[ 11994], 90.00th=[ 13960], 95.00th=[ 18482],

     | 99.00th=[ 37487], 99.50th=[ 85459], 99.90th=[152044], 99.95th=[212861],

     | 99.99th=[219153]

   bw (  KiB/s): min=   18, max= 1278, per=100.00%, avg=595.21, stdev=271.81, samples=119

   iops        : min=    6, max=  106, avg=58.24, stdev=19.98, samples=119

  lat (usec)   : 500=6.12%, 750=8.19%, 1000=1.17%

  lat (msec)   : 2=1.67%, 4=7.70%, 10=39.02%, 20=33.10%, 50=2.17%

  lat (msec)   : 100=0.41%, 250=0.41%, 500=0.03%

  cpu          : usr=0.16%, sys=0.57%, ctx=6838, majf=0, minf=14

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued rwts: total=3339,3476,0,0 short=0,0,0,0 dropped=0,0,0,0

     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):

   READ: bw=610KiB/s (625kB/s), 610KiB/s-610KiB/s (625kB/s-625kB/s), io=35.7MiB (37.5MB), run=60016-60016msec

  WRITE: bw=592KiB/s (606kB/s), 592KiB/s-592KiB/s (606kB/s-606kB/s), io=34.7MiB (36.4MB), run=60016-60016msec

nfs v3

iometer: (g=0): rw=randrw, bs=(R) 512B-64.0KiB, (W) 512B-64.0KiB, (T) 512B-64.0KiB, ioengine=psync, iodepth=64

fio-3.25

Starting 1 process

Jobs: 1 (f=3): [m(1)][100.0%][r=581KiB/s,w=609KiB/s][r=51,w=42 IOPS][eta 00m:00s]

iometer: (groupid=0, jobs=1): err= 0: pid=2347167: Sun Oct 31 07:35:09 2021

  Description  : [Emulation of Intel IOmeter File Server Access Pattern]

  read: IOPS=49, BW=548KiB/s (561kB/s)(32.1MiB/60012msec)

    clat (usec): min=394, max=285769, avg=9756.36, stdev=10269.42

     lat (usec): min=394, max=285770, avg=9756.86, stdev=10269.42

    clat percentiles (usec):

     |  1.00th=[   889],  5.00th=[  3294], 10.00th=[  4113], 20.00th=[  5407],

     | 30.00th=[  6718], 40.00th=[  7898], 50.00th=[  8979], 60.00th=[ 10290],

     | 70.00th=[ 11338], 80.00th=[ 12387], 90.00th=[ 13435], 95.00th=[ 14484],

     | 99.00th=[ 30278], 99.50th=[ 56361], 99.90th=[206570], 99.95th=[233833],

     | 99.99th=[287310]

   bw (  KiB/s): min=    8, max= 1118, per=100.00%, avg=548.19, stdev=256.60, samples=119

   iops        : min=    2, max=   86, avg=49.50, stdev=15.73, samples=119

  write: IOPS=51, BW=522KiB/s (534kB/s)(30.6MiB/60012msec); 0 zone resets

    clat (usec): min=357, max=221586, avg=10035.07, stdev=12250.59

     lat (usec): min=358, max=221587, avg=10036.14, stdev=12250.75

    clat percentiles (usec):

     |  1.00th=[   482],  5.00th=[   627], 10.00th=[  1172], 20.00th=[  4146],

     | 30.00th=[  5735], 40.00th=[  7111], 50.00th=[  8586], 60.00th=[ 10159],

     | 70.00th=[ 11600], 80.00th=[ 13042], 90.00th=[ 16188], 95.00th=[ 21103],

     | 99.00th=[ 47449], 99.50th=[ 93848], 99.90th=[168821], 99.95th=[196084],

     | 99.99th=[221250]

   bw (  KiB/s): min=   34, max= 1039, per=100.00%, avg=523.79, stdev=223.74, samples=119

   iops        : min=    6, max=   82, avg=51.63, stdev=15.50, samples=119

  lat (usec)   : 500=0.73%, 750=4.03%, 1000=0.87%

  lat (msec)   : 2=0.71%, 4=7.69%, 10=44.35%, 20=37.52%, 50=3.37%

  lat (msec)   : 100=0.38%, 250=0.33%, 500=0.02%

  cpu          : usr=0.25%, sys=0.59%, ctx=6076, majf=0, minf=15

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued rwts: total=2968,3090,0,0 short=0,0,0,0 dropped=0,0,0,0

     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):

   READ: bw=548KiB/s (561kB/s), 548KiB/s-548KiB/s (561kB/s-561kB/s), io=32.1MiB (33.7MB), run=60012-60012msec

  WRITE: bw=522KiB/s (534kB/s), 522KiB/s-522KiB/s (534kB/s-534kB/s), io=30.6MiB (32.1MB), run=60012-60012msec

初步看起來 v4 稍微好一些

在all ssd 的 nas 上用 fio 測了一次io

config 如下

# This job file tries to mimic the Intel IOMeter File Server Access Pattern
[global]
description=Emulation of Intel IOmeter File Server Access Pattern

[iometer]
bssplit=512/10:1k/5:2k/5:4k/60:8k/2:16k/4:32k/4:64k/10
filename=\\.\PhysicalDrive1:\\.\PhysicalDrive2:\\.\PhysicalDrive3
size=1G
rw=randrw
#set read 50% write 50%
rwmixread=50
direct=1
runtime=60                  
# IOMeter defines the server loads as the following:
# iodepth=1                    Linear
# iodepth=4                    Very Light
# iodepth=8                    Light
# iodepth=64                    Moderate
# iodepth=256                    Heavy
iodepth=64

4個模式結果如下

default no cache

iometer: (g=0): rw=randrw, bs=(R) 512B-64.0KiB, (W) 512B-64.0KiB, (T) 512B-64.0KiB, ioengine=psync, iodepth=64

fio-3.7

Starting 1 process

iometer: Laying out IO files (3 files / total 1023MiB)

Jobs: 1 (f=3): [m(1)][100.0%][r=12.1MiB/s,w=12.3MiB/s][r=2912,w=2900 IOPS][eta 00m:00s]

iometer: (groupid=0, jobs=1): err= 0: pid=1059: Sun Oct 24 21:59:29 2021

  Description  : [Emulation of Intel IOmeter File Server Access Pattern]

   read: IOPS=2643, BW=15.0MiB/s (16.7MB/s)(518MiB/32459msec)

    clat (usec): min=98, max=3023, avg=151.62, stdev=52.08

     lat (usec): min=100, max=3025, avg=153.64, stdev=52.78

    clat percentiles (usec):

     |  1.00th=[  108],  5.00th=[  113], 10.00th=[  116], 20.00th=[  120],

     | 30.00th=[  124], 40.00th=[  129], 50.00th=[  137], 60.00th=[  145],

     | 70.00th=[  153], 80.00th=[  167], 90.00th=[  217], 95.00th=[  262],

     | 99.00th=[  326], 99.50th=[  367], 99.90th=[  498], 99.95th=[  644],

     | 99.99th=[ 1074]

   bw (  KiB/s): min=   17, max=29187, per=100.00%, avg=16368.95, stdev=5319.54, samples=64

   iops        : min=    6, max= 3237, avg=2636.11, stdev=553.54, samples=64

  write: IOPS=2632, BW=15.6MiB/s (16.4MB/s)(506MiB/32459msec)

    clat (usec): min=109, max=269781, avg=208.42, stdev=1809.64

     lat (usec): min=111, max=269787, avg=210.60, stdev=1809.68

    clat percentiles (usec):

     |  1.00th=[  117],  5.00th=[  122], 10.00th=[  126], 20.00th=[  131],

     | 30.00th=[  137], 40.00th=[  141], 50.00th=[  151], 60.00th=[  159],

     | 70.00th=[  167], 80.00th=[  182], 90.00th=[  233], 95.00th=[  277],

     | 99.00th=[  343], 99.50th=[  388], 99.90th=[13435], 99.95th=[32637],

     | 99.99th=[77071]

   bw (  KiB/s): min=   25, max=27107, per=100.00%, avg=16004.09, stdev=5117.56, samples=64

   iops        : min=    8, max= 3423, avg=2627.02, stdev=552.46, samples=64

  lat (usec)   : 100=0.01%, 250=93.02%, 500=6.83%, 750=0.06%, 1000=0.02%

  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.03%

  lat (msec)   : 100=0.01%, 250=0.01%, 500=0.01%

  cpu          : usr=4.79%, sys=12.52%, ctx=171263, majf=0, minf=36

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued rwts: total=85804,85461,0,0 short=0,0,0,0 dropped=0,0,0,0

     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):

   READ: bw=15.0MiB/s (16.7MB/s), 15.0MiB/s-15.0MiB/s (16.7MB/s-16.7MB/s), io=518MiB (543MB), run=32459-32459msec

  WRITE: bw=15.6MiB/s (16.4MB/s), 15.6MiB/s-15.6MiB/s (16.4MB/s-16.4MB/s), io=506MiB (531MB), run=32459-32459msec

Disk stats (read/write):

  vda: ios=85701/85498, merge=0/34, ticks=11506/17166, in_queue=28672, util=99.78%

===================================================================

direct sync

iometer: (g=0): rw=randrw, bs=(R) 512B-64.0KiB, (W) 512B-64.0KiB, (T) 512B-64.0KiB, ioengine=psync, iodepth=64

fio-3.7

Starting 1 process

Jobs: 1 (f=3): [m(1)][100.0%][r=9873KiB/s,w=9.85MiB/s][r=2326,w=2311 IOPS][eta 00m:00s]

iometer: (groupid=0, jobs=1): err= 0: pid=1024: Sun Oct 24 22:01:57 2021

  Description  : [Emulation of Intel IOmeter File Server Access Pattern]

   read: IOPS=2588, BW=15.6MiB/s (16.4MB/s)(518MiB/33143msec)

    clat (usec): min=101, max=2854, avg=153.73, stdev=51.58

     lat (usec): min=103, max=2855, avg=155.67, stdev=52.05

    clat percentiles (usec):

     |  1.00th=[  109],  5.00th=[  115], 10.00th=[  118], 20.00th=[  123],

     | 30.00th=[  126], 40.00th=[  133], 50.00th=[  141], 60.00th=[  147],

     | 70.00th=[  155], 80.00th=[  169], 90.00th=[  215], 95.00th=[  260],

     | 99.00th=[  334], 99.50th=[  375], 99.90th=[  519], 99.95th=[  660],

     | 99.99th=[ 1045]

   bw (  KiB/s): min= 8901, max=29142, per=100.00%, avg=16021.95, stdev=4651.25, samples=66

   iops        : min= 1870, max= 3118, avg=2589.53, stdev=300.89, samples=66

  write: IOPS=2578, BW=15.3MiB/s (16.0MB/s)(506MiB/33143msec)

    clat (usec): min=119, max=123840, avg=214.82, stdev=1482.62

     lat (usec): min=121, max=123842, avg=216.99, stdev=1482.65

    clat percentiles (usec):

     |  1.00th=[  127],  5.00th=[  133], 10.00th=[  137], 20.00th=[  143],

     | 30.00th=[  147], 40.00th=[  153], 50.00th=[  163], 60.00th=[  174],

     | 70.00th=[  182], 80.00th=[  194], 90.00th=[  241], 95.00th=[  289],

     | 99.00th=[  367], 99.50th=[  424], 99.90th=[ 5866], 99.95th=[29492],

     | 99.99th=[80217]

   bw (  KiB/s): min= 9162, max=26677, per=100.00%, avg=15655.61, stdev=4532.61, samples=66

   iops        : min= 1860, max= 3100, avg=2577.86, stdev=315.86, samples=66

  lat (usec)   : 250=92.52%, 500=7.29%, 750=0.10%, 1000=0.02%

  lat (msec)   : 2=0.01%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.02%

  lat (msec)   : 100=0.01%, 250=0.01%

  cpu          : usr=4.74%, sys=11.81%, ctx=171269, majf=0, minf=36

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued rwts: total=85804,85461,0,0 short=0,0,0,0 dropped=0,0,0,0

     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):

   READ: bw=15.6MiB/s (16.4MB/s), 15.6MiB/s-15.6MiB/s (16.4MB/s-16.4MB/s), io=518MiB (543MB), run=33143-33143msec

  WRITE: bw=15.3MiB/s (16.0MB/s), 15.3MiB/s-15.3MiB/s (16.0MB/s-16.0MB/s), io=506MiB (531MB), run=33143-33143msec

Disk stats (read/write):

  vda: ios=85795/85513, merge=0/20, ticks=11798/16745, in_queue=28543, util=99.74%

===============================================================

write through

iometer: (g=0): rw=randrw, bs=(R) 512B-64.0KiB, (W) 512B-64.0KiB, (T) 512B-64.0KiB, ioengine=psync, iodepth=64

fio-3.7

Starting 1 process

Jobs: 1 (f=3): [m(1)][100.0%][r=13.8MiB/s,w=14.2MiB/s][r=3275,w=3321 IOPS][eta 00m:00s]

iometer: (groupid=0, jobs=1): err= 0: pid=1025: Sun Oct 24 22:03:47 2021

  Description  : [Emulation of Intel IOmeter File Server Access Pattern]

   read: IOPS=2467, BW=14.9MiB/s (15.6MB/s)(518MiB/34776msec)

    clat (usec): min=28, max=2270, avg=105.88, stdev=76.86

     lat (usec): min=30, max=2272, avg=107.86, stdev=77.07

    clat percentiles (usec):

     |  1.00th=[   35],  5.00th=[   36], 10.00th=[   37], 20.00th=[   38],

     | 30.00th=[   40], 40.00th=[   43], 50.00th=[   81], 60.00th=[  135],

     | 70.00th=[  151], 80.00th=[  165], 90.00th=[  194], 95.00th=[  241],

     | 99.00th=[  330], 99.50th=[  371], 99.90th=[  482], 99.95th=[  570],

     | 99.99th=[  971]

   bw (  KiB/s): min= 9946, max=20245, per=99.96%, avg=15240.57, stdev=2279.39, samples=69

   iops        : min= 1582, max= 3690, avg=2457.10, stdev=587.94, samples=69

  write: IOPS=2457, BW=14.6MiB/s (15.3MB/s)(506MiB/34776msec)

    clat (usec): min=135, max=78912, avg=281.99, stdev=1239.11

     lat (usec): min=136, max=78913, avg=284.13, stdev=1239.15

    clat percentiles (usec):

     |  1.00th=[  147],  5.00th=[  151], 10.00th=[  157], 20.00th=[  165],

     | 30.00th=[  178], 40.00th=[  192], 50.00th=[  210], 60.00th=[  245],

     | 70.00th=[  285], 80.00th=[  330], 90.00th=[  375], 95.00th=[  424],

     | 99.00th=[  570], 99.50th=[  644], 99.90th=[ 9110], 99.95th=[28443],

     | 99.99th=[63177]

   bw (  KiB/s): min= 9128, max=20043, per=99.95%, avg=14897.84, stdev=2082.27, samples=69

   iops        : min= 1570, max= 3638, avg=2447.70, stdev=574.29, samples=69

  lat (usec)   : 50=23.06%, 100=2.41%, 250=53.12%, 500=20.28%, 750=0.99%

  lat (usec)   : 1000=0.06%

  lat (msec)   : 2=0.02%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.02%

  lat (msec)   : 100=0.01%

  cpu          : usr=4.38%, sys=11.16%, ctx=171251, majf=0, minf=36

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued rwts: total=85804,85461,0,0 short=0,0,0,0 dropped=0,0,0,0

     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):

   READ: bw=14.9MiB/s (15.6MB/s), 14.9MiB/s-14.9MiB/s (15.6MB/s-15.6MB/s), io=518MiB (543MB), run=34776-34776msec

  WRITE: bw=14.6MiB/s (15.3MB/s), 14.6MiB/s-14.6MiB/s (15.3MB/s-15.3MB/s), io=506MiB (531MB), run=34776-34776msec

Disk stats (read/write):

  vda: ios=85360/85041, merge=0/18, ticks=7406/22572, in_queue=29978, util=99.75%

================================================================

write back

iometer: (g=0): rw=randrw, bs=(R) 512B-64.0KiB, (W) 512B-64.0KiB, (T) 512B-64.0KiB, ioengine=psync, iodepth=64

fio-3.7

Starting 1 process

Jobs: 1 (f=3): [m(1)][100.0%][r=29.4MiB/s,w=29.9MiB/s][r=6976,w=7009 IOPS][eta 00m:00s]

iometer: (groupid=0, jobs=1): err= 0: pid=1021: Sun Oct 24 22:05:06 2021

  Description  : [Emulation of Intel IOmeter File Server Access Pattern]

   read: IOPS=3871, BW=23.4MiB/s (24.5MB/s)(518MiB/22164msec)

    clat (usec): min=28, max=12350, avg=112.79, stdev=135.36

     lat (usec): min=30, max=12352, avg=114.83, stdev=135.55

    clat percentiles (usec):

     |  1.00th=[   35],  5.00th=[   37], 10.00th=[   38], 20.00th=[   41],

     | 30.00th=[   43], 40.00th=[   46], 50.00th=[   69], 60.00th=[  139],

     | 70.00th=[  157], 80.00th=[  178], 90.00th=[  212], 95.00th=[  262],

     | 99.00th=[  363], 99.50th=[  412], 99.90th=[  603], 99.95th=[  938],

     | 99.99th=[ 5342]

   bw (  KiB/s): min=10682, max=31639, per=100.00%, avg=23930.39, stdev=4126.10, samples=44

   iops        : min= 2212, max= 7366, avg=3861.02, stdev=1436.84, samples=44

  write: IOPS=3855, BW=22.8MiB/s (23.9MB/s)(506MiB/22164msec)

    clat (usec): min=36, max=203000, avg=126.42, stdev=930.79

     lat (usec): min=38, max=203002, avg=128.86, stdev=930.84

    clat percentiles (usec):

     |  1.00th=[   44],  5.00th=[   47], 10.00th=[   49], 20.00th=[   52],

     | 30.00th=[   55], 40.00th=[   58], 50.00th=[   62], 60.00th=[   97],

     | 70.00th=[  153], 80.00th=[  215], 90.00th=[  258], 95.00th=[  285],

     | 99.00th=[  388], 99.50th=[  441], 99.90th=[  693], 99.95th=[ 1221],

     | 99.99th=[15270]

   bw (  KiB/s): min=10302, max=32041, per=100.00%, avg=23399.55, stdev=4090.09, samples=44

   iops        : min= 2148, max= 7414, avg=3846.48, stdev=1417.26, samples=44

  lat (usec)   : 50=28.96%, 100=27.75%, 250=34.74%, 500=8.33%, 750=0.15%

  lat (usec)   : 1000=0.02%

  lat (msec)   : 2=0.02%, 4=0.01%, 10=0.01%, 20=0.01%, 50=0.01%

  lat (msec)   : 250=0.01%

  cpu          : usr=7.40%, sys=17.75%, ctx=171252, majf=0, minf=36

  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%

     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%

     issued rwts: total=85804,85461,0,0 short=0,0,0,0 dropped=0,0,0,0

     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):

   READ: bw=23.4MiB/s (24.5MB/s), 23.4MiB/s-23.4MiB/s (24.5MB/s-24.5MB/s), io=518MiB (543MB), run=22164-22164msec

  WRITE: bw=22.8MiB/s (23.9MB/s), 22.8MiB/s-22.8MiB/s (23.9MB/s-23.9MB/s), io=506MiB (531MB), run=22164-22164msec

Disk stats (read/write):

  vda: ios=85645/85340, merge=0/17, ticks=8026/10446, in_queue=18472, util=99.52%

  

  

前三個模式沒有太大的差異

但 write back 效能不管是讀或寫

都快了 1.5倍以上

做為參考

最近有個朋友問到如何用python走rest api 撈graylog的資料

因為一直以來都是用curl

沒用python

今天朋友說他試出來了 感謝他的分享

做個記錄

程式碼如下

import requests

user = 'admin'

pw = 'pwd'

send_format_date_from = '2021-10-21T16:00:00.000Z'

send_format_date_to = '2021-10-21T17:00:00.000Z'

str = 'search key word'

url='http://graylog_ip:9000/api/views/search/messages'

header = {'Accept':'text/csv,application/json', 'Content-Type':'application/json', 'X-Requested-By':'cli'}

#以下是使用絶對時間的語法 上方已定義區間

graylog_send_data={ "streams":["000000000000000000000001"], "timerange":[ "absolute",{ "from":send_format_date_from, "to":send_format_date_to } ], "query_string":{ "type":"elasticsearch", "query_string":str } }

#以下是使用相對時間的語法 range 是以秒為單位

graylog_send_data={ "streams":["000000000000000000000001"], "timerange":{ "type":"relative","range":60 }, "query_string":{ "type":"elasticsearch", "query_string":str } }

r = requests.post(url, auth=(user, pw), headers=header, json=graylog_send_data)

print(r.text)

相對時間或絶對時間擇一使用

升到 promox 7 後

有几台guest都出現了以下的問題

可是使用  badblocks xfs_repair  進行檢查

都沒有發現任何錯誤

而且看了一下nas各個HD的資訊

也沒有發現任何狀況

還在找原因

之前有一台是發生在swap 區

目前把swap 關掉

然後把 ram 從2G調到 4G

觀察到現在沒有異常

不知道是不是ram 的問題

2021/10/24 更新

發生狀況的有四台几器 共同的情況是這些guest的io都很大

分別處理如下

ntopng因為升版後 system id 變了 所以移至 LXC 後 重新要了新key

librenms 下載了新版的vm 把資料移轉到新几器上

https://docs.librenms.org/Support/FAQ/#how-do-i-move-my-librenms-install-to-another-server

剩下cacti 跟 syslog

從log來看是 write 時候的問題

目前所有的guest hd 預設都是使用 no cache

想說會不會是效能的問題

https://adminkk.blogspot.com/2016/05/wsus-proxmox-winmount-nfs-wsus-iscsi.html

於是把上面二台

一台調成 write back

一台調成 write through

 到目前跑了二天

持續觀察中

如何在synology的nas上憑証

在synology的nas裝好 pbs後

要更新出現以下的錯誤

Err:4 http://download.proxmox.com/debian/pbs bullseye InRelease

  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY DD4BA3917E23BF59

Reading package lists... Done

W: GPG error: http://download.proxmox.com/debian/pbs bullseye InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY DD4BA3917E23BF59

E: The repository 'http://download.proxmox.com/debian/pbs bullseye InRelease' is not signed.

N: Updating from such a repository can't be done securely, and is therefore disabled by default.

N: See apt-secure(8) manpage for repository creation and user configuration details.

應該是GPG key 沒有裝

以下指令安裝

wget https://enterprise.proxmox.com/debian/proxmox-release-bullseye.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bullseye.gpg 

再update就沒問題了

https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_11_Bullseye#Install_Proxmox_VE

今天有個user拿了一片DVD來

說要給其他看看

本來想說直接把檔案丟到google drive share出去就好

結果有保護

想說轉成MP4好了

試了很多軟体 最後  HandBrake 成功了

參考以下連結

https://ephrain.net/ubuntu-%E4%BD%BF%E7%94%A8-handbrake-%E5%B0%87-dvd-%E8%BD%89%E6%88%90-mp4/