[Veritas-vx] RAID5 full stripe writes

[Damjan Perenic]

Hello,

I created a fresh VxVM 6.0 RAID5 volume on four columns. I performed
some basic I/O operations and noticed that writes always end in
read-modify-write cycle. I haven't been able to trigger a full stripe
writes yet, regardless of the command I used to write to the
filesystem. I tried with 64k and with 128k stripe width but in both
cases, I cannot get writes aligned. Filesystem is VxFS.
Below are outputs of my simple tests and configuration of the RAID5
volume. Any hints where I got it wrong?


1. dd example
***@mlincek:~# dd if=/dev/zero of=/mlincek/zero.dd bs=384k

***@mlincek:~# iostat -zxn 2
extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
69.0 115.4 8827.8 14777.0 0.0 0.4 0.0 2.3 0 41 c0t2d0
69.0 115.4 8827.8 14777.0 0.0 0.3 0.0 1.4 0 25 c0t3d0
70.0 115.4 8955.8 14777.0 0.0 0.4 0.0 2.0 0 36 c0t5d0
69.5 115.9 8891.8 14841.0 0.0 0.4 0.0 2.2 0 41 c0t4d0

2. Copy file from another physical volume using cp command:

***@mlincek:~# cp testfile.tmp /mlincek/

extended device statistics
r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device
19.0 159.9 2250.5 20290.3 0.0 0.5 0.0 2.7 0 40 c0t2d0
17.5 158.4 2239.0 20278.9 0.0 0.3 0.0 1.5 0 16 c0t3d0
36.5 142.4 4489.5 18051.4 0.0 0.5 0.0 2.6 0 33 c0t5d0
37.0 143.4 4494.0 18119.8 0.0 0.5 0.0 3.0 0 36 c0t4d0


3. Volume config

-bash-4.1# vxprint -g mlincekdg -t
...
dg mlincekdg default default 28000 1333350408.27.mlincek
dm mlincekdg01 disk_1 auto 65536 1953459328 -
dm mlincekdg02 disk_2 auto 65536 1953459328 -
dm mlincekdg03 disk_3 auto 65536 1953459328 -
dm mlincekdg04 disk_4 auto 65536 1953459328 -
sd mlincekdg01-01 mlincek-01 mlincekdg01 0 1953458944 0/0 disk_1 ENA
sd mlincekdg02-01 mlincek-01 mlincekdg02 0 1953458944 1/0 disk_2 ENA
sd mlincekdg03-01 mlincek-01 mlincekdg03 0 1953458944 2/0 disk_3 ENA
sd mlincekdg04-01 mlincek-01 mlincekdg04 0 1953458944 3/0 disk_4 ENA
pl mlincek-01 mlincek ENABLED ACTIVE 5860376832 RAID 4/256 RW
v mlincek - ENABLED ACTIVE 5860376832 RAID - raid5

-bash-4.1# /opt/VRTSvxfs/sbin/vxtunefs -p /mlincek
Filesystem i/o parameters for /mlincek
read_pref_io = 131072
read_nstream = 4
read_unit_io = 131072
write_pref_io = 393216
write_nstream = 1
write_unit_io = 131072
...


Mount: /mlincek on
/dev/vx/dsk/mlincekdg/mlincekread/write/setuid/devices/rstchown/delaylog/largefiles/ioerror=mwdisable/xattr/dev=3606d60

Volume was created with the following command:
***@mlincek:~# vxassist -g mlincekdg -o ordered make mlincek
5860376576 layout=raid5 nlog=0 ncol=4 stripewidth=128k mlincekdg01
mlincekdg02 mlincekdg03 mlincekdg04