原本一直以來都是使用flow-tools來收netflow的資料
http://freecode.com/projects/flow-tools
flow-capture
但最近的工作需要收集從fortiget傳來的netflow的資料
一開始也是使用flow-capture
但一直沒有資料
用tcpdump去看
是有資料進來的
但查了一些文件才知道fortiget送過來的是netflow v9
flow-capture不支援
於是改用nfdump 支援 v5 v7 v9
資料就正常收取了
http://nfdump.sourceforge.net/
nfcapd
2016/06/27
最近的新增需求是要起一個ftp server
可是不想要用vsftp或其他daemon
所以直接使用pyftpdlib
還滿方便的 有很多範例可以直接套用
但用了後發現沒有log功能
再查了一下資料
發現只要加紅色二行就搞定了
import logging
from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer
authorizer = DummyAuthorizer()
authorizer.add_user('user', '12345', '.', perm='elradfmwM')
handler = FTPHandler
handler.authorizer = authorizer
logging.basicConfig(filename='/var/log/pyftpd.log', level=logging.INFO)
server = FTPServer(('', 2121), handler)
server.serve_forever()
2016/06/18
在python中撈取mysql的資料
在cursor.execute後可以用cursor.fetchone()及cursor.fetchall()
如果回傳的只有一筆資料 可以使用cursor.fetchone() 可以少一點code
語法
cursor.execute(select count(*) from table)
result = cursor.fetchone()
print result[0]
如果回傳的資料有很多 則使用cursor.fetchall()
語法
cursor.execute(select * from table)
result = cursor.fetchall()
if result:
for record in result:
print record[0]
在cursor.execute後可以用cursor.fetchone()及cursor.fetchall()
如果回傳的只有一筆資料 可以使用cursor.fetchone() 可以少一點code
語法
cursor.execute(select count(*) from table)
result = cursor.fetchone()
print result[0]
如果回傳的資料有很多 則使用cursor.fetchall()
語法
cursor.execute(select * from table)
result = cursor.fetchall()
if result:
for record in result:
print record[0]
2016/06/17
今天找了一下有關mysql 日期時間的使用
大約記一下
分為
NOW() 2001-02-02 19:00:46
CURDATE() 2001-02-02
CURTIME() 19:00:46
使用加減功能
NOW() - INTERVAL 1 DAY 結果 2001-02-01 19:00:46
CURDATE()+ INTERVAL 1 DAY 結果 2001-02-03 00:00:00
未來 > NOW() < 過去
select * from table where enddate > NOW() 表示在NOW()之後的時間 表示未來 大未來
select * from table where enddate < NOW() 表示在NOW()之前的時間 表示過去 小過去
http://wen198599.pixnet.net/blog/post/22450019-mysql%E6%97%A5%E6%9C%9F%E5%92%8C%E6%99%82%E9%96%93%E5%87%BD%E6%95%B8%E4%B8%8D%E6%B1%82%E4%BA%BA
http://www.webasp.net/article/25/24538.htm
http://www.w3school.com.cn/sql/func_now.asp
http://www.mysqltutorial.org/mysql-now/
大約記一下
分為
NOW() 2001-02-02 19:00:46
CURDATE() 2001-02-02
CURTIME() 19:00:46
使用加減功能
NOW() - INTERVAL 1 DAY 結果 2001-02-01 19:00:46
CURDATE()
未來 > NOW() < 過去
select * from table where enddate > NOW() 表示在NOW()之後的時間 表示未來 大未來
select * from table where enddate < NOW() 表示在NOW()之前的時間 表示過去 小過去
http://wen198599.pixnet.net/blog/post/22450019-mysql%E6%97%A5%E6%9C%9F%E5%92%8C%E6%99%82%E9%96%93%E5%87%BD%E6%95%B8%E4%B8%8D%E6%B1%82%E4%BA%BA
http://www.webasp.net/article/25/24538.htm
http://www.w3school.com.cn/sql/func_now.asp
http://www.mysqltutorial.org/mysql-now/
2016/06/15
2016/06/11
今天使用python要把資料塞進mysql時一直出問題
明明就有執行
也沒錯誤訊訊息
但資料就是沒進去
本來以為是跳脫字元的問題
試了也沒有
後來找到一個方便的語法
也不用再考慮單引號 雙引號要使用跳脫字元的情況
語法如下
最後的 db.commit() 一定要下
就是因為這個沒下
才試了一下午
最後還是問了高手才知道的
sql="insert into table values(%s,%s,%s,)"
cursor.execute(sql,(now,i,88888))
db.commit()
now是現在時間
i是變數
88888是數字
對應到table
datetime
varchar(50)
bigint(20)
明明就有執行
也沒錯誤訊訊息
但資料就是沒進去
本來以為是跳脫字元的問題
試了也沒有
後來找到一個方便的語法
也不用再考慮單引號 雙引號要使用跳脫字元的情況
語法如下
最後的 db.commit() 一定要下
就是因為這個沒下
才試了一下午
最後還是問了高手才知道的
sql="insert into table values(%s,%s,%s,)"
cursor.execute(sql,(now,i,88888))
db.commit()
now是現在時間
i是變數
88888是數字
對應到table
datetime
varchar(50)
bigint(20)
2016/06/10
2016/05/31
fio 測試範例
# This job file tries to mimic the Intel IOMeter File Server Access Pattern
[global]
description=Emulation of Intel IOmeter File Server Access Pattern
[iometer]
bssplit=512/10:1k/5:2k/5:4k/60:8k/2:16k/4:32k/4:64k/10
filename=PhysicalDrive1:PhysicalDrive2:PhysicalDrive3
size=1G
rw=randrw
#set read 50% write 50%
rwmixread=50
direct=1
runtime=60
# IOMeter defines the server loads as the following:
# iodepth=1 Linear
# iodepth=4 Very Light
# iodepth=8 Light
# iodepth=64 Moderate
# iodepth=256 Heavy
iodepth=64
修改filename 及 size
其他參數視需要修改
以上存為 test.fio
執行fio test.fio
用shell 重複執行
#/bin/bash
rm /tmp/fio_test_log
while :
do
fio test.fio -output=/tmp/iolog
cat /tmp/iolog >> /tmp/fio_test_log
done
http://benjr.tw/93666
# This job file tries to mimic the Intel IOMeter File Server Access Pattern
[global]
description=Emulation of Intel IOmeter File Server Access Pattern
[iometer]
bssplit=512/10:1k/5:2k/5:4k/60:8k/2:16k/4:32k/4:64k/10
filename=PhysicalDrive1:PhysicalDrive2:PhysicalDrive3
size=1G
rw=randrw
#set read 50% write 50%
rwmixread=50
direct=1
runtime=60
# IOMeter defines the server loads as the following:
# iodepth=1 Linear
# iodepth=4 Very Light
# iodepth=8 Light
# iodepth=64 Moderate
# iodepth=256 Heavy
iodepth=64
修改filename 及 size
其他參數視需要修改
以上存為 test.fio
執行fio test.fio
用shell 重複執行
#/bin/bash
rm /tmp/fio_test_log
while :
do
fio test.fio -output=/tmp/iolog
cat /tmp/iolog >> /tmp/fio_test_log
done
http://benjr.tw/93666
2016/05/23
2016/05/20
已經使用很久的ipaudit有個scan report 的功能 如下圖
但使用curl加上html2text捉下來後ip就如圖所示會自動補0
找來找去終於找到一個方便的方法來處理
就是使用expr
整個程式如下
#!/bin/bash
/usr/bin/curl "http://10.123.123.123/~ipaudit/cgi-bin/ipahttp?30min/0traffic+current"|/usr/bin/html2text |sed -n '18,37p'|awk '$3 > 100 {print $1}' > /tmp/ipaudit_in_scan_ip_tmp
rm /tmp/ipaudit_in_scan_ip
for i in `cat /tmp/ipaudit_in_scan_ip_tmp`
do
a=`echo $i|cut -d '.' -f 1`
a1=`expr $a / 1`
a=`echo $i|cut -d '.' -f 2`
a2=`expr $a / 1`
a=`echo $i|cut -d '.' -f 3`
a3=`expr $a / 1`
a=`echo $i|cut -d '.' -f 4`
a4=`expr $a / 1`
echo $a1"."$a2"."$a3"."$a4 >> /tmp/ipaudit_in_scan_ip
done
但使用curl加上html2text捉下來後ip就如圖所示會自動補0
找來找去終於找到一個方便的方法來處理
就是使用expr
整個程式如下
#!/bin/bash
/usr/bin/curl "http://10.123.123.123/~ipaudit/cgi-bin/ipahttp?30min/0traffic+current"|/usr/bin/html2text |sed -n '18,37p'|awk '$3 > 100 {print $1}' > /tmp/ipaudit_in_scan_ip_tmp
rm /tmp/ipaudit_in_scan_ip
for i in `cat /tmp/ipaudit_in_scan_ip_tmp`
do
a=`echo $i|cut -d '.' -f 1`
a1=`expr $a / 1`
a=`echo $i|cut -d '.' -f 2`
a2=`expr $a / 1`
a=`echo $i|cut -d '.' -f 3`
a3=`expr $a / 1`
a=`echo $i|cut -d '.' -f 4`
a4=`expr $a / 1`
echo $a1"."$a2"."$a3"."$a4 >> /tmp/ipaudit_in_scan_ip
done
前几天把wsus換到了vm上
但換過去後卻一直出問題
三不五時服務就會死掉
log顯示如下 每次都是死在wsuspool
May 19 07:38:38 WSUS WAS: 5013: 服務應用程式集區 'WsusPool' 的處理序關閉時超過次數限制。處理序識別碼為 '5192'。
May 19 07:38:38 WSUS WAS: 5117: 服務應用程式集區 'WsusPool' 的背景工作處理序已要求回收,因為它已達到私用位元組記憶體限制。
May 19 08:08:27 WSUS WAS: 5117: 服務應用程式集區 'WsusPool' 的背景工作處理序已要求回收,因為它已達到私用位元組記憶體限制。
May 19 08:10:49 WSUS WAS: 5117: 服務應用程式集區 'WsusPool' 的背景工作處理序已要求回收,因為它已達到私用位元組記憶體限制。
May 19 08:12:19 WSUS WAS: 5138: 服務應用程式集區 'WsusPool' 的工作者處理序 '6184' 無法在指定的時間停止通訊協定 'http' 的接聽程式通道。資料欄位包含錯誤號碼。
May 19 08:12:19 WSUS WAS: 5013: 服務應用程式集區 'WsusPool' 的處理序關閉時超過次數限制。處理序識別碼為 '6184'。
May 19 08:12:19 WSUS WAS: 5117: 服務應用程式集區 'WsusPool' 的背景工作處理序已要求回收,因為它已達到私用位元組記憶體限制。
May 19 08:13:50 WSUS WAS: 5138: 服務應用程式集區 'WsusPool' 的工作者處理序 '5964' 無法在指定的時間停止通訊協定 'http' 的接聽程式通道。資料欄位包含錯誤號碼。
May 19 08:13:50 WSUS WAS: 5013: 服務應用程式集區 'WsusPool' 的處理序關閉時超過次數限制。處理序識別碼為 '5964'。
May 19 08:13:50 WSUS WAS: 5002: 正在自動停止應用程式集區 'WsusPool',因為服務該應用程式集區的處理序中發生一連串的失敗。
不過原來在實体机是windows server 2012 沒調過這個值也沒有問題
在vm上的是windows server 2012 R2
但換過去後卻一直出問題
三不五時服務就會死掉
log顯示如下 每次都是死在wsuspool
May 19 07:38:38 WSUS WAS: 5013: 服務應用程式集區 'WsusPool' 的處理序關閉時超過次數限制。處理序識別碼為 '5192'。
May 19 07:38:38 WSUS WAS: 5117: 服務應用程式集區 'WsusPool' 的背景工作處理序已要求回收,因為它已達到私用位元組記憶體限制。
May 19 08:08:27 WSUS WAS: 5117: 服務應用程式集區 'WsusPool' 的背景工作處理序已要求回收,因為它已達到私用位元組記憶體限制。
May 19 08:10:49 WSUS WAS: 5117: 服務應用程式集區 'WsusPool' 的背景工作處理序已要求回收,因為它已達到私用位元組記憶體限制。
May 19 08:12:19 WSUS WAS: 5138: 服務應用程式集區 'WsusPool' 的工作者處理序 '6184' 無法在指定的時間停止通訊協定 'http' 的接聽程式通道。資料欄位包含錯誤號碼。
May 19 08:12:19 WSUS WAS: 5013: 服務應用程式集區 'WsusPool' 的處理序關閉時超過次數限制。處理序識別碼為 '6184'。
May 19 08:12:19 WSUS WAS: 5117: 服務應用程式集區 'WsusPool' 的背景工作處理序已要求回收,因為它已達到私用位元組記憶體限制。
May 19 08:13:50 WSUS WAS: 5138: 服務應用程式集區 'WsusPool' 的工作者處理序 '5964' 無法在指定的時間停止通訊協定 'http' 的接聽程式通道。資料欄位包含錯誤號碼。
May 19 08:13:50 WSUS WAS: 5013: 服務應用程式集區 'WsusPool' 的處理序關閉時超過次數限制。處理序識別碼為 '5964'。
May 19 08:13:50 WSUS WAS: 5002: 正在自動停止應用程式集區 'WsusPool',因為服務該應用程式集區的處理序中發生一連串的失敗。
找了一下資料 請seek的工程師幫忙
調整了一下記憶体的值 (預設為1G 加大到4G)
過了一天 目前看來正常了
再觀察看看
設定流程如下圖所示
不過原來在實体机是windows server 2012 沒調過這個值也沒有問題
在vm上的是windows server 2012 R2
2016/05/19
一直以來很困擾在使用vi的編輯模式時按方向鍵會出現英文字的問題
找到解決方法
在家目錄底下,新增檔案「.vimrc」,在檔案中加入下列2行內容:
set nocompatible
set backspace=2
http://www.geego.com.tw/technical-discussion-forum/tech-tips-using-vi-editor-on-ubuntu-%E7%B7%A8%E8%BC%AF-%E6%96%B9%E5%90%91%E9%8D%B5-%E5%80%92%E9%80%80%E9%8D%B5-%E6%93%8D%E4%BD%9C
找到解決方法
在家目錄底下,新增檔案「.vimrc」,在檔案中加入下列2行內容:
set nocompatible
set backspace=2
http://www.geego.com.tw/technical-discussion-forum/tech-tips-using-vi-editor-on-ubuntu-%E7%B7%A8%E8%BC%AF-%E6%96%B9%E5%90%91%E9%8D%B5-%E5%80%92%E9%80%80%E9%8D%B5-%E6%93%8D%E4%BD%9C
2016/05/18
因為chrome在之前几個月開始就不再支援32bit的linux
所以家裡使用的電腦就想重灌為64bit
這次選了linux mint
安裝速度滿快的
而且直接捉到我的usb網卡 直是方便
但裝完後裝上chrome卻發現網頁上有些字的字型變的很怪
爬了一下文 發現是字型的問題
因為系統自動多裝了以下二套字型
sudo apt-get remove fonts-arphic-ukai
sudo apt-get remove fonts-arphic-uming
移除後再重開chrome就正常了
firefox並沒有這個問題
所以家裡使用的電腦就想重灌為64bit
這次選了linux mint
安裝速度滿快的
而且直接捉到我的usb網卡 直是方便
但裝完後裝上chrome卻發現網頁上有些字的字型變的很怪
爬了一下文 發現是字型的問題
因為系統自動多裝了以下二套字型
sudo apt-get remove fonts-arphic-ukai
sudo apt-get remove fonts-arphic-uming
移除後再重開chrome就正常了
firefox並沒有這個問題
2016/05/17
最近因為實體機的wsus出問題
因此把她移到proxmox上
本來是想在win上mount nfs來用
但看來wsus並不支援
後來改用iscsi
但在kvm的硬碟效能看來並不好
想調看看
依據下面的文件做了一下測試
https://pve.proxmox.com/wiki/Performance_Tweaks
使用fio
四個mode的結果分別如下
no cache (proxmox預設)
iometer: (groupid=0, jobs=1): err= 0: pid=4376: Tue May 17 01:14:11 2016 Descri
ption : [Emulation of Intel IOmeter File Server Access Pattern]
read : io=839580KB, bw=986.43KB/s, iops=187, runt=851139msec
slat (usec): min=33, max=12019, avg=135.28, stdev=127.09
clat (usec): min=863, max=45860K, avg=272726.04, stdev=690206.66
lat (msec): min=1, max=45860, avg=272.86, stdev=690.21
clat percentiles (msec):
| 1.00th=[ 14], 5.00th=[ 34], 10.00th=[ 52], 20.00th=[ 85],
| 30.00th=[ 119], 40.00th=[ 155], 50.00th=[ 198], 60.00th=[ 249],
| 70.00th=[ 314], 80.00th=[ 392], 90.00th=[ 498], 95.00th=[ 668],
| 99.00th=[ 1237], 99.50th=[ 1418], 99.90th=[ 1958], 99.95th=[10683],
| 99.99th=[16712]
bw (KB /s): min= 1, max= 3527, per=100.00%, avg=1031.92, stdev=539.62
write: io=209006KB, bw=251453B/s, iops=47, runt=851139msec
slat (usec): min=36, max=9979, avg=144.38, stdev=139.48
clat (msec): min=1, max=42109, avg=268.91, stdev=674.52
lat (msec): min=1, max=42109, avg=269.06, stdev=674.52
clat percentiles (msec):
| 1.00th=[ 13], 5.00th=[ 33], 10.00th=[ 52], 20.00th=[ 85],
| 30.00th=[ 117], 40.00th=[ 153], 50.00th=[ 194], 60.00th=[ 247],
| 70.00th=[ 310], 80.00th=[ 388], 90.00th=[ 490], 95.00th=[ 660],
| 99.00th=[ 1221], 99.50th=[ 1434], 99.90th=[ 2008], 99.95th=[ 8717],
| 99.99th=[16712]
bw (KB /s): min= 0, max= 976, per=100.00%, avg=257.99, stdev=155.70
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.07%, 10=0.50%, 20=1.59%, 50=7.30%
lat (msec) : 100=15.26%, 250=35.57%, 500=29.91%, 750=5.77%, 1000=1.93%
lat (msec) : 2000=1.99%, >=2000=0.10%
cpu : usr=1.06%, sys=4.46%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.3%, >=64=99.7%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.1%, 32=0.0%, 64=0.1%, >=64=0.0%
issued : total=r=159733/w=40419/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: io=839579KB, aggrb=986KB/s, minb=986KB/s, maxb=986KB/s, mint=851139msec
, maxt=851139msec
WRITE: io=209005KB, aggrb=245KB/s, minb=245KB/s, maxb=245KB/s, mint=851139msec
, maxt=851139msec
direct sync
iometer: (groupid=0, jobs=1): err= 0: pid=3960: Tue May 17 01:34:28 2016 Descri
ption : [Emulation of Intel IOmeter File Server Access Pattern]
read : io=839580KB, bw=990904B/s, iops=184, runt=867621msec
slat (usec): min=29, max=4295, avg=127.63, stdev=94.92
clat (msec): min=1, max=3990, avg=278.10, stdev=248.29
lat (msec): min=1, max=3990, avg=278.23, stdev=248.29
clat percentiles (msec):
| 1.00th=[ 15], 5.00th=[ 39], 10.00th=[ 61], 20.00th=[ 98],
| 30.00th=[ 135], 40.00th=[ 176], 50.00th=[ 219], 60.00th=[ 265],
| 70.00th=[ 326], 80.00th=[ 404], 90.00th=[ 529], 95.00th=[ 734],
| 99.00th=[ 1303], 99.50th=[ 1516], 99.90th=[ 1860], 99.95th=[ 2008],
| 99.99th=[ 2474]
bw (KB /s): min= 104, max= 3121, per=100.00%, avg=972.63, stdev=462.35
write: io=209006KB, bw=246676B/s, iops=46, runt=867621msec
slat (usec): min=33, max=3792, avg=137.96, stdev=101.21
clat (msec): min=1, max=3267, avg=273.82, stdev=245.31
lat (msec): min=1, max=3267, avg=273.95, stdev=245.31
clat percentiles (msec):
| 1.00th=[ 14], 5.00th=[ 37], 10.00th=[ 59], 20.00th=[ 95],
| 30.00th=[ 131], 40.00th=[ 172], 50.00th=[ 215], 60.00th=[ 262],
| 70.00th=[ 322], 80.00th=[ 396], 90.00th=[ 523], 95.00th=[ 734],
| 99.00th=[ 1270], 99.50th=[ 1483], 99.90th=[ 1795], 99.95th=[ 1926],
| 99.99th=[ 2376]
bw (KB /s): min= 1, max= 837, per=100.00%, avg=241.81, stdev=139.50
lat (msec) : 2=0.01%, 4=0.06%, 10=0.43%, 20=1.28%, 50=5.89%
lat (msec) : 100=13.20%, 250=36.27%, 500=31.53%, 750=6.52%, 1000=2.28%
lat (msec) : 2000=2.48%, >=2000=0.05%
cpu : usr=0.58%, sys=4.26%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=99.9%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued : total=r=159733/w=40419/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: io=839579KB, aggrb=967KB/s, minb=967KB/s, maxb=967KB/s, mint=867621msec
, maxt=867621msec
WRITE: io=209005KB, aggrb=240KB/s, minb=240KB/s, maxb=240KB/s, mint=867621msec
, maxt=867621msec
writethrough
iometer: (groupid=0, jobs=1): err= 0: pid=464: Tue May 17 01:44:14 2016 Descrip
tion : [Emulation of Intel IOmeter File Server Access Pattern]
read : io=839580KB, bw=2750.9KB/s, iops=523, runt=305208msec
slat (usec): min=33, max=3381, avg=148.74, stdev=118.02
clat (usec): min=525, max=3432.2K, avg=97497.21, stdev=190958.21
lat (usec): min=608, max=3432.2K, avg=97645.95, stdev=190948.63
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 8], 10.00th=[ 11], 20.00th=[ 17],
| 30.00th=[ 22], 40.00th=[ 28], 50.00th=[ 35], 60.00th=[ 42],
| 70.00th=[ 57], 80.00th=[ 96], 90.00th=[ 265], 95.00th=[ 465],
| 99.00th=[ 996], 99.50th=[ 1287], 99.90th=[ 1713], 99.95th=[ 1844],
| 99.99th=[ 2245]
bw (KB /s): min= 117, max=10905, per=100.00%, avg=2768.59, stdev=2650.18
write: io=209006KB, bw=701232B/s, iops=132, runt=305208msec
slat (usec): min=36, max=2704, avg=157.69, stdev=118.79
clat (usec): min=643, max=2868.8K, avg=96528.89, stdev=185218.15
lat (usec): min=730, max=2868.9K, avg=96686.58, stdev=185208.44
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 8], 10.00th=[ 11], 20.00th=[ 17],
| 30.00th=[ 22], 40.00th=[ 28], 50.00th=[ 35], 60.00th=[ 42],
| 70.00th=[ 57], 80.00th=[ 97], 90.00th=[ 265], 95.00th=[ 461],
| 99.00th=[ 963], 99.50th=[ 1221], 99.90th=[ 1631], 99.95th=[ 1778],
| 99.99th=[ 2008]
bw (KB /s): min= 1, max= 3243, per=100.00%, avg=688.92, stdev=677.96
lat (usec) : 750=0.01%, 1000=0.02%
lat (msec) : 2=0.21%, 4=1.13%, 10=8.32%, 20=17.18%, 50=40.00%
lat (msec) : 100=13.73%, 250=8.92%, 500=6.15%, 750=2.57%, 1000=0.79%
lat (msec) : 2000=0.96%, >=2000=0.02%
cpu : usr=2.95%, sys=11.47%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=22.4%, >=64=77.6%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.1%, 16=0.1%, 32=0.1%, 64=0.1%, >=64=0.0%
issued : total=r=159733/w=40419/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: io=839579KB, aggrb=2750KB/s, minb=2750KB/s, maxb=2750KB/s, mint=305208m
sec, maxt=305208msec
WRITE: io=209005KB, aggrb=684KB/s, minb=684KB/s, maxb=684KB/s, mint=305208msec
, maxt=305208msec
writeback
iometer: (groupid=0, jobs=1): err= 0: pid=1904: Tue May 17 01:56:37 2016 Descri
ption : [Emulation of Intel IOmeter File Server Access Pattern]
read : io=839580KB, bw=2514.4KB/s, iops=478, runt=333921msec
slat (usec): min=29, max=12023, avg=185.33, stdev=221.12
clat (usec): min=538, max=13009K, avg=106457.27, stdev=193275.54
lat (usec): min=670, max=13009K, avg=106642.60, stdev=193264.18
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 9], 10.00th=[ 13], 20.00th=[ 21],
| 30.00th=[ 29], 40.00th=[ 38], 50.00th=[ 48], 60.00th=[ 68],
| 70.00th=[ 93], 80.00th=[ 137], 90.00th=[ 265], 95.00th=[ 408],
| 99.00th=[ 783], 99.50th=[ 1156], 99.90th=[ 1795], 99.95th=[ 2024],
| 99.99th=[ 4490]
bw (KB /s): min= 70, max=10009, per=100.00%, avg=2528.25, stdev=1930.08
write: io=209006KB, bw=640934B/s, iops=121, runt=333921msec
slat (usec): min=32, max=12301, avg=195.26, stdev=240.65
clat (usec): min=637, max=10893K, avg=106044.70, stdev=199408.62
lat (usec): min=699, max=10894K, avg=106239.95, stdev=199397.25
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 9], 10.00th=[ 13], 20.00th=[ 21],
| 30.00th=[ 29], 40.00th=[ 38], 50.00th=[ 48], 60.00th=[ 68],
| 70.00th=[ 94], 80.00th=[ 137], 90.00th=[ 265], 95.00th=[ 404],
| 99.00th=[ 742], 99.50th=[ 1106], 99.90th=[ 1811], 99.95th=[ 2008],
| 99.99th=[ 7308]
bw (KB /s): min= 3, max= 2501, per=100.00%, avg=629.21, stdev=504.36
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.14%, 4=0.75%, 10=5.70%, 20=12.30%, 50=32.48%
lat (msec) : 100=20.56%, 250=17.35%, 500=7.78%, 750=1.87%, 1000=0.38%
lat (msec) : 2000=0.62%, >=2000=0.06%
cpu : usr=2.10%, sys=12.88%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=25.7%, >=64=74.3%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.8%, 8=0.2%, 16=0.1%, 32=0.0%, 64=0.1%, >=64=0.0%
issued : total=r=159733/w=40419/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: io=839579KB, aggrb=2514KB/s, minb=2514KB/s, maxb=2514KB/s, mint=333921m
sec, maxt=333921msec
WRITE: io=209005KB, aggrb=625KB/s, minb=625KB/s, maxb=625KB/s, mint=333921msec
, maxt=333921msec
單純看iops
writeback 和 writethrough 差不多
但文件上針對 writeback有這一句
Warn : you can loose datas in case of a powerfailure
所以看來 writethrough是比較好的選擇
因此把她移到proxmox上
本來是想在win上mount nfs來用
但看來wsus並不支援
後來改用iscsi
但在kvm的硬碟效能看來並不好
想調看看
依據下面的文件做了一下測試
https://pve.proxmox.com/wiki/Performance_Tweaks
使用fio
四個mode的結果分別如下
no cache (proxmox預設)
iometer: (groupid=0, jobs=1): err= 0: pid=4376: Tue May 17 01:14:11 2016 Descri
ption : [Emulation of Intel IOmeter File Server Access Pattern]
read : io=839580KB, bw=986.43KB/s, iops=187, runt=851139msec
slat (usec): min=33, max=12019, avg=135.28, stdev=127.09
clat (usec): min=863, max=45860K, avg=272726.04, stdev=690206.66
lat (msec): min=1, max=45860, avg=272.86, stdev=690.21
clat percentiles (msec):
| 1.00th=[ 14], 5.00th=[ 34], 10.00th=[ 52], 20.00th=[ 85],
| 30.00th=[ 119], 40.00th=[ 155], 50.00th=[ 198], 60.00th=[ 249],
| 70.00th=[ 314], 80.00th=[ 392], 90.00th=[ 498], 95.00th=[ 668],
| 99.00th=[ 1237], 99.50th=[ 1418], 99.90th=[ 1958], 99.95th=[10683],
| 99.99th=[16712]
bw (KB /s): min= 1, max= 3527, per=100.00%, avg=1031.92, stdev=539.62
write: io=209006KB, bw=251453B/s, iops=47, runt=851139msec
slat (usec): min=36, max=9979, avg=144.38, stdev=139.48
clat (msec): min=1, max=42109, avg=268.91, stdev=674.52
lat (msec): min=1, max=42109, avg=269.06, stdev=674.52
clat percentiles (msec):
| 1.00th=[ 13], 5.00th=[ 33], 10.00th=[ 52], 20.00th=[ 85],
| 30.00th=[ 117], 40.00th=[ 153], 50.00th=[ 194], 60.00th=[ 247],
| 70.00th=[ 310], 80.00th=[ 388], 90.00th=[ 490], 95.00th=[ 660],
| 99.00th=[ 1221], 99.50th=[ 1434], 99.90th=[ 2008], 99.95th=[ 8717],
| 99.99th=[16712]
bw (KB /s): min= 0, max= 976, per=100.00%, avg=257.99, stdev=155.70
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.07%, 10=0.50%, 20=1.59%, 50=7.30%
lat (msec) : 100=15.26%, 250=35.57%, 500=29.91%, 750=5.77%, 1000=1.93%
lat (msec) : 2000=1.99%, >=2000=0.10%
cpu : usr=1.06%, sys=4.46%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.3%, >=64=99.7%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.1%, 32=0.0%, 64=0.1%, >=64=0.0%
issued : total=r=159733/w=40419/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: io=839579KB, aggrb=986KB/s, minb=986KB/s, maxb=986KB/s, mint=851139msec
, maxt=851139msec
WRITE: io=209005KB, aggrb=245KB/s, minb=245KB/s, maxb=245KB/s, mint=851139msec
, maxt=851139msec
direct sync
iometer: (groupid=0, jobs=1): err= 0: pid=3960: Tue May 17 01:34:28 2016 Descri
ption : [Emulation of Intel IOmeter File Server Access Pattern]
read : io=839580KB, bw=990904B/s, iops=184, runt=867621msec
slat (usec): min=29, max=4295, avg=127.63, stdev=94.92
clat (msec): min=1, max=3990, avg=278.10, stdev=248.29
lat (msec): min=1, max=3990, avg=278.23, stdev=248.29
clat percentiles (msec):
| 1.00th=[ 15], 5.00th=[ 39], 10.00th=[ 61], 20.00th=[ 98],
| 30.00th=[ 135], 40.00th=[ 176], 50.00th=[ 219], 60.00th=[ 265],
| 70.00th=[ 326], 80.00th=[ 404], 90.00th=[ 529], 95.00th=[ 734],
| 99.00th=[ 1303], 99.50th=[ 1516], 99.90th=[ 1860], 99.95th=[ 2008],
| 99.99th=[ 2474]
bw (KB /s): min= 104, max= 3121, per=100.00%, avg=972.63, stdev=462.35
write: io=209006KB, bw=246676B/s, iops=46, runt=867621msec
slat (usec): min=33, max=3792, avg=137.96, stdev=101.21
clat (msec): min=1, max=3267, avg=273.82, stdev=245.31
lat (msec): min=1, max=3267, avg=273.95, stdev=245.31
clat percentiles (msec):
| 1.00th=[ 14], 5.00th=[ 37], 10.00th=[ 59], 20.00th=[ 95],
| 30.00th=[ 131], 40.00th=[ 172], 50.00th=[ 215], 60.00th=[ 262],
| 70.00th=[ 322], 80.00th=[ 396], 90.00th=[ 523], 95.00th=[ 734],
| 99.00th=[ 1270], 99.50th=[ 1483], 99.90th=[ 1795], 99.95th=[ 1926],
| 99.99th=[ 2376]
bw (KB /s): min= 1, max= 837, per=100.00%, avg=241.81, stdev=139.50
lat (msec) : 2=0.01%, 4=0.06%, 10=0.43%, 20=1.28%, 50=5.89%
lat (msec) : 100=13.20%, 250=36.27%, 500=31.53%, 750=6.52%, 1000=2.28%
lat (msec) : 2000=2.48%, >=2000=0.05%
cpu : usr=0.58%, sys=4.26%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=99.9%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
issued : total=r=159733/w=40419/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: io=839579KB, aggrb=967KB/s, minb=967KB/s, maxb=967KB/s, mint=867621msec
, maxt=867621msec
WRITE: io=209005KB, aggrb=240KB/s, minb=240KB/s, maxb=240KB/s, mint=867621msec
, maxt=867621msec
writethrough
iometer: (groupid=0, jobs=1): err= 0: pid=464: Tue May 17 01:44:14 2016 Descrip
tion : [Emulation of Intel IOmeter File Server Access Pattern]
read : io=839580KB, bw=2750.9KB/s, iops=523, runt=305208msec
slat (usec): min=33, max=3381, avg=148.74, stdev=118.02
clat (usec): min=525, max=3432.2K, avg=97497.21, stdev=190958.21
lat (usec): min=608, max=3432.2K, avg=97645.95, stdev=190948.63
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 8], 10.00th=[ 11], 20.00th=[ 17],
| 30.00th=[ 22], 40.00th=[ 28], 50.00th=[ 35], 60.00th=[ 42],
| 70.00th=[ 57], 80.00th=[ 96], 90.00th=[ 265], 95.00th=[ 465],
| 99.00th=[ 996], 99.50th=[ 1287], 99.90th=[ 1713], 99.95th=[ 1844],
| 99.99th=[ 2245]
bw (KB /s): min= 117, max=10905, per=100.00%, avg=2768.59, stdev=2650.18
write: io=209006KB, bw=701232B/s, iops=132, runt=305208msec
slat (usec): min=36, max=2704, avg=157.69, stdev=118.79
clat (usec): min=643, max=2868.8K, avg=96528.89, stdev=185218.15
lat (usec): min=730, max=2868.9K, avg=96686.58, stdev=185208.44
clat percentiles (msec):
| 1.00th=[ 4], 5.00th=[ 8], 10.00th=[ 11], 20.00th=[ 17],
| 30.00th=[ 22], 40.00th=[ 28], 50.00th=[ 35], 60.00th=[ 42],
| 70.00th=[ 57], 80.00th=[ 97], 90.00th=[ 265], 95.00th=[ 461],
| 99.00th=[ 963], 99.50th=[ 1221], 99.90th=[ 1631], 99.95th=[ 1778],
| 99.99th=[ 2008]
bw (KB /s): min= 1, max= 3243, per=100.00%, avg=688.92, stdev=677.96
lat (usec) : 750=0.01%, 1000=0.02%
lat (msec) : 2=0.21%, 4=1.13%, 10=8.32%, 20=17.18%, 50=40.00%
lat (msec) : 100=13.73%, 250=8.92%, 500=6.15%, 750=2.57%, 1000=0.79%
lat (msec) : 2000=0.96%, >=2000=0.02%
cpu : usr=2.95%, sys=11.47%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=22.4%, >=64=77.6%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.1%, 16=0.1%, 32=0.1%, 64=0.1%, >=64=0.0%
issued : total=r=159733/w=40419/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: io=839579KB, aggrb=2750KB/s, minb=2750KB/s, maxb=2750KB/s, mint=305208m
sec, maxt=305208msec
WRITE: io=209005KB, aggrb=684KB/s, minb=684KB/s, maxb=684KB/s, mint=305208msec
, maxt=305208msec
writeback
iometer: (groupid=0, jobs=1): err= 0: pid=1904: Tue May 17 01:56:37 2016 Descri
ption : [Emulation of Intel IOmeter File Server Access Pattern]
read : io=839580KB, bw=2514.4KB/s, iops=478, runt=333921msec
slat (usec): min=29, max=12023, avg=185.33, stdev=221.12
clat (usec): min=538, max=13009K, avg=106457.27, stdev=193275.54
lat (usec): min=670, max=13009K, avg=106642.60, stdev=193264.18
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 9], 10.00th=[ 13], 20.00th=[ 21],
| 30.00th=[ 29], 40.00th=[ 38], 50.00th=[ 48], 60.00th=[ 68],
| 70.00th=[ 93], 80.00th=[ 137], 90.00th=[ 265], 95.00th=[ 408],
| 99.00th=[ 783], 99.50th=[ 1156], 99.90th=[ 1795], 99.95th=[ 2024],
| 99.99th=[ 4490]
bw (KB /s): min= 70, max=10009, per=100.00%, avg=2528.25, stdev=1930.08
write: io=209006KB, bw=640934B/s, iops=121, runt=333921msec
slat (usec): min=32, max=12301, avg=195.26, stdev=240.65
clat (usec): min=637, max=10893K, avg=106044.70, stdev=199408.62
lat (usec): min=699, max=10894K, avg=106239.95, stdev=199397.25
clat percentiles (msec):
| 1.00th=[ 5], 5.00th=[ 9], 10.00th=[ 13], 20.00th=[ 21],
| 30.00th=[ 29], 40.00th=[ 38], 50.00th=[ 48], 60.00th=[ 68],
| 70.00th=[ 94], 80.00th=[ 137], 90.00th=[ 265], 95.00th=[ 404],
| 99.00th=[ 742], 99.50th=[ 1106], 99.90th=[ 1811], 99.95th=[ 2008],
| 99.99th=[ 7308]
bw (KB /s): min= 3, max= 2501, per=100.00%, avg=629.21, stdev=504.36
lat (usec) : 750=0.01%, 1000=0.01%
lat (msec) : 2=0.14%, 4=0.75%, 10=5.70%, 20=12.30%, 50=32.48%
lat (msec) : 100=20.56%, 250=17.35%, 500=7.78%, 750=1.87%, 1000=0.38%
lat (msec) : 2000=0.62%, >=2000=0.06%
cpu : usr=2.10%, sys=12.88%, ctx=0, majf=0, minf=0
IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=25.7%, >=64=74.3%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.8%, 8=0.2%, 16=0.1%, 32=0.0%, 64=0.1%, >=64=0.0%
issued : total=r=159733/w=40419/d=0, short=r=0/w=0/d=0, drop=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=64
Run status group 0 (all jobs):
READ: io=839579KB, aggrb=2514KB/s, minb=2514KB/s, maxb=2514KB/s, mint=333921m
sec, maxt=333921msec
WRITE: io=209005KB, aggrb=625KB/s, minb=625KB/s, maxb=625KB/s, mint=333921msec
, maxt=333921msec
單純看iops
writeback 和 writethrough 差不多
但文件上針對 writeback有這一句
Warn : you can loose datas in case of a powerfailure
所以看來 writethrough是比較好的選擇
2016/05/11
2016/04/17
因為ascenflow GG了
所以把認証移到forti上去
順便想看一下線上人數並記錄
但找了一下forti的mib
並沒有提供這個值
所以只好自行寫程式來撈了
使用python來撈目前online user的所有資料 並從中取出線上人數 在輸出的最後一行
直接使用shell也可以
#!/usr/bin/python
import os
import telnetlib
host="192.168.1.1"
user="admin"
passwd="password"
tn = telnetlib.Telnet(host)
tn.read_until("FG600C login: ")
tn.write(user + "\n")
tn.read_until("Password: ")
tn.write(passwd + "\n")
tn.read_until("FG600C # ")
tn.write("diagnose firewall auth filter group auth_user_group(radius_server)" + "\n")
tn.read_until("FG600C # ")
tn.write("diagnose firewall auth list" + "\n")
#tn.read_until("FG600C # ")
tn.write("exit" + "\n")
print tn.read_all()
tn.close()
2. 使用步驟1 的資料來新增data templates
3.使用步驟2 的資料來產生 graph templates
4. 在forti上新增步驟3所產生的graph template
等待一段時間後畫出的圖如下
發現有小數出現 雖然說不太會影響判讀 但感覺就是怪怪的
找了一下資料
發現把graph template的gprint type改成exact numbers就可以顯示整數了
改好後如下圖 看起來正常多了
所以把認証移到forti上去
順便想看一下線上人數並記錄
但找了一下forti的mib
並沒有提供這個值
所以只好自行寫程式來撈了
使用python來撈目前online user的所有資料 並從中取出線上人數 在輸出的最後一行
直接使用shell也可以
#!/usr/bin/python
import os
import telnetlib
host="192.168.1.1"
user="admin"
passwd="password"
tn = telnetlib.Telnet(host)
tn.read_until("FG600C login: ")
tn.write(user + "\n")
tn.read_until("Password: ")
tn.write(passwd + "\n")
tn.read_until("FG600C # ")
tn.write("diagnose firewall auth filter group auth_user_group(radius_server)" + "\n")
tn.read_until("FG600C # ")
tn.write("diagnose firewall auth list" + "\n")
#tn.read_until("FG600C # ")
tn.write("exit" + "\n")
print tn.read_all()
tn.close()
取出後把值吐給cacti來畫圖
步驟如下
1. 建立data input methods 使用自行寫好的程式
2. 使用步驟1 的資料來新增data templates
3.使用步驟2 的資料來產生 graph templates
4. 在forti上新增步驟3所產生的graph template
等待一段時間後畫出的圖如下
發現有小數出現 雖然說不太會影響判讀 但感覺就是怪怪的
找了一下資料
發現把graph template的gprint type改成exact numbers就可以顯示整數了
改好後如下圖 看起來正常多了
2016/04/06
如何使用python利用telnet指令到網路設備下指令並捉取回傳的資料
範例如下
#!/usr/bin/python
import os
import telnetlib
host="1.1.1.1"
user="abc"
passwd="password"
tn = telnetlib.Telnet(host)
tn.read_until("login: ")
tn.write(user + "\n")
tn.read_until("Password: ")
tn.write(passwd + "\n")
#read_until()到下完password 即可 接下來的指令必須一直下
#不再使用read_until() 否則最後的 print tn.read_all() 會沒有資料
tn.write("ls -al" + "\n")
tn.write("exit" + "\n")
print tn.read_all()
範例如下
#!/usr/bin/python
import os
import telnetlib
host="1.1.1.1"
user="abc"
passwd="password"
tn = telnetlib.Telnet(host)
tn.read_until("login: ")
tn.write(user + "\n")
tn.read_until("Password: ")
tn.write(passwd + "\n")
#read_until()到下完password 即可 接下來的指令必須一直下
#不再使用read_until() 否則最後的 print tn.read_all() 會沒有資料
tn.write("ls -al" + "\n")
tn.write("exit" + "\n")
print tn.read_all()
2016/04/02
有台ubuntu好久沒update了
今天要update時出現以下的訊息
W: GPG error: http://tw.archive.ubuntu.com trusty Release: The following signatures were invalid: BADSIG 40976EAF437D05B5 Ubuntu Archive Automatic Signing Key <ftpmaster@ubuntu.com>
E: Encountered a section with no Package: header
E: Problem with MergeList /var/lib/apt/lists/tw.archive.ubuntu.com_ubuntu_dists_trusty_main_i18n_Translation-en
E: The package lists or status file could not be parsed or opened.
看來好像是list有問題
找了下一下資料
用以下的方法處理後就ok了
cd /var/lib/apt
sudo mv lists lists.old
sudo mkdir -p lists/partial
sudo apt-get update
今天要update時出現以下的訊息
W: GPG error: http://tw.archive.ubuntu.com trusty Release: The following signatures were invalid: BADSIG 40976EAF437D05B5 Ubuntu Archive Automatic Signing Key <ftpmaster@ubuntu.com>
E: Encountered a section with no Package: header
E: Problem with MergeList /var/lib/apt/lists/tw.archive.ubuntu.com_ubuntu_dists_trusty_main_i18n_Translation-en
E: The package lists or status file could not be parsed or opened.
看來好像是list有問題
找了下一下資料
用以下的方法處理後就ok了
cd /var/lib/apt
sudo mv lists lists.old
sudo mkdir -p lists/partial
sudo apt-get update
訂閱:
文章 (Atom)












