Tony Asleson
2015-03-05 20:42:47 UTC
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "libstoragemgmt".
The branch, master has been updated
via a8bfe32e8feb2e4af5a1fe68b06ad496ebc65eac (commit)
via d91b3befab8af044d86814d9b6474a664bd07dcd (commit)
via da411235e33a93aff59658752006302476ab3ee2 (commit)
via dd6760cde9346efa0d4eb6629cefc87c164c28b9 (commit)
via 902d057cff7e74de9af40e00c15f101bedf4ace6 (commit)
via 49b78aa3662fc61bf3b15c8ccc0a0923175f060f (commit)
via 1e3bd8e704dc132c73af7118df1e2816949824f6 (commit)
via 168c2d10d26a7ed6aeda2845ceb256cc1e639c42 (commit)
via b4d013e9a99bc442c0259c68338e9afeb2c17846 (commit)
via b635d0b993f90be68c8e7486252c73717f74de50 (commit)
via 1d311853b6f57e41f405245eebd5a4c7c1db3679 (commit)
via e9b69dc6516faeee4edc5a4c383e1bbcf10c549e (commit)
via c489a2c29542b1f5d884e7f5dfe77803eb831243 (commit)
via 01d800bd253c009cbed77da15505b62e6ecf124d (commit)
via c7d343ac0fa3d98d9672e8277f5a8b4a5e635956 (commit)
from a977d26c289acba25ae3e6460cb2022fdbcb16a1 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit a8bfe32e8feb2e4af5a1fe68b06ad496ebc65eac
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:51 2015 +0800
ONTAP Plugin: Add volume_raid_info() support.
* NetApp ONTAP strip size(minimum I/O size) is 4KiB, stripe size(
optimal I/O size) is 64KiB. Both are unchangeable.
* The extent count(disk count) is taken from aggregate 'disk-count'
property.
* Changed Filer.aggregates() to accept an optional argument 'aggr_name'
which query defined aggregate only.
* Uncommented and updated the old code for converting NetApp RAID level to
libstoragemgmt RAID level.
* Tested on ONTAP simulator 8.1.1 7-mode and real ONTAP 8.0.2 7-mode.
Changes in V5(No changes in V2, V3, V4):
* Sync API changes for argument name.
Signed-off-by: Gris Ge <***@redhat.com>
commit d91b3befab8af044d86814d9b6474a664bd07dcd
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:50 2015 +0800
MegaRAID Plugin: Add volume_raid_info() support.
* Use 'storcli /c0/v1 show all' command line output to determine
RAID type, strip size and disk count.
* Calculate optimal I/O size by strip size multiple with RAID
data(not mirrot, not parity) disks count.
* Tested query on RAID 0, 1, 5, 10, 50.
* Tested the optimal I/O size on RAID 5:
[***@storageqe-08 ~]# lsmenv mega lsmcli vri --vol SV03403550:VD1
Device alias: mega
URI: megaraid://
lsmcli vri --vol SV03403550:VD1
Volume ID | RAID Type | Strip Size | Extent Count | Minimum I/O Size | Optimal I/O Size
--------------------------------------------------------------------------------------------
SV03403550:VD1 | RAID5 | 131072 | 5 | 131072 | 524288
Time: 0:00.29
[***@storageqe-08 ~]# dd if=/dev/urandom of=test.img bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 153.174 s, 6.8 MB/s
[***@storageqe-08 ~]# dd if=./test.img of=/dev/sdb bs=131072 oflag=direct
8000+0 records in
8000+0 records out
1048576000 bytes (1.0 GB) copied, 58.9573 s, 17.8 MB/s
[***@storageqe-08 ~]# dd if=./test.img of=/dev/sdb bs=524288 oflag=direct
2000+0 records in
2000+0 records out
1048576000 bytes (1.0 GB) copied, 37.7282 s, 27.8 MB/s
[***@storageqe-08 ~]# dd if=./test.img of=/dev/sdb bs=524288 oflag=direct
2000+0 records in
2000+0 records out
1048576000 bytes (1.0 GB) copied, 35.3351 s, 29.7 MB/s
[***@storageqe-08 ~]# dd if=./test.img of=/dev/sdb bs=131072 oflag=direct
8000+0 records in
8000+0 records out
1048576000 bytes (1.0 GB) copied, 70.0779 s, 15.0 MB/s
Signed-off-by: Gris Ge <***@redhat.com>
commit da411235e33a93aff59658752006302476ab3ee2
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:49 2015 +0800
MegaRAID Plugin: Add Disk.STATUS_RECONSTRUCT support.
* In storcli of MegaRAID, 'Rbld' of disk status indicate this disk
in using for reconstructing pool data.
Signed-off-by: Gris Ge <***@redhat.com>
commit dd6760cde9346efa0d4eb6629cefc87c164c28b9
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:48 2015 +0800
MegaRAID Plugin: Add volumes() support.
* Treating MegaRAID VD as LSM Volume.
* Using 'storcli /c0/vall show all' to query all volumes.
* Add new capability: Capabilities.VOLUMES
Signed-off-by: Gris Ge <***@redhat.com>
commit 902d057cff7e74de9af40e00c15f101bedf4ace6
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:47 2015 +0800
MegaRAID plugin: Add pools() method support.
* Treating each MegaRAID DG(disk group) as LSM pool.
* Based on storcli output of:
storcli /c0/dall show all J
Signed-off-by: Gris Ge <***@redhat.com>
commit 49b78aa3662fc61bf3b15c8ccc0a0923175f060f
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:46 2015 +0800
Simulator C Plugin: Add lsm_volume_raid_info() support.
* Simply set XXX_UNKNOWN on output parameter.
V2:
- Add call to register volume_raid_info
- Add LSM_CAP_VOLUMERAID_INFO to capabilties
Changes in V3:
* Use lsm_register_plugin_v1_2() to register lsm_volume_raid_info() support.
Changes in V5(no change in V4):
* Sync API changes:
* argument name 'disk_count'
* argument type 'uint32_t'.
* constant name 'LSM_VOLUME_DISK_COUNT_UNKNOWN'.
Signed-off-by: Gris Ge <***@redhat.com>
Signed-off-by: Tony Asleson <***@redhat.com>
commit 1e3bd8e704dc132c73af7118df1e2816949824f6
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:45 2015 +0800
Simulator Plugin: Add volume_raid_info() support
* Introduced full support of volume_raid_info().
* For sub-pool, use raid info from parent pool.
* For RAID 1 and JBOD, set strip_size, min_io_size, and opt_io_size
as block size(512).
* For other RAID, calculate out opt_io_size with data disk count.
* For RAID_TYPE_MIXED, raise PLUGIN_BUG LsmError.
* Replaced PoolRAID.RAID_TYPE_RAID_XXX with Volume.RAID_TYPE_RAID_XXX.
* Replaced PoolRAID.RAID_TYPE_NOT_APPLICABLE with Volume.RAID_TYPE_OTHER.
* Bumped simulator data version to 3.1 due to previous Volume.RAID_TYPE_OTHER
change.
Changes in V4(No changes in V2 and V3):
* Removed unneeded import 'Volume' in simulator.py.
Changes in V5:
* Sync changes of API changes for return argument name 'disk_count' and
constant name Volume.DISK_COUNT_UNKNOWN.
Signed-off-by: Gris Ge <***@redhat.com>
commit 168c2d10d26a7ed6aeda2845ceb256cc1e639c42
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:44 2015 +0800
Constant Test: Fix missing constant with number in it.
* Allowing check_const.pl to check constants with number in it.
Example:
LSM_VOLUME_RAID_TYPE_RAID1
Signed-off-by: Gris Ge <***@redhat.com>
commit b4d013e9a99bc442c0259c68338e9afeb2c17846
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:43 2015 +0800
C Unit Test: Add test for lsm_volume_raid_info() method
* Simply invoke lsm_volume_raid_info() with no additional test.
Changes in V5(No change in V2, V3, V4):
* Sync API changes for argument name('extent_count' to 'disk_count') and data
type.
Signed-off-by: Gris Ge <***@redhat.com>
commit b635d0b993f90be68c8e7486252c73717f74de50
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:42 2015 +0800
lsmcli Test: Add test for volume-raid-info command.
* Simply run that command and check the volume ID of output.
Signed-off-by: Gris Ge <***@redhat.com>
commit 1d311853b6f57e41f405245eebd5a4c7c1db3679
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:41 2015 +0800
lsmcli: Add volume_raid_info() support.
* New command:
lsmcli volume-raid-info --vol <VOL_ID>
* New alias:
lsmcli vri == lsmcli volume-raid-info
Changes in V2:
* Fix output format when volume not found passed to _get_item() in
volume_raid_info()
Changes in V5(No changes in V3, V4):
* Sync API changes of name 'extent_count' to 'disk_count', column name
changed to 'Disk Count'.
Signed-off-by: Gris Ge <***@redhat.com>
commit e9b69dc6516faeee4edc5a4c383e1bbcf10c549e
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:40 2015 +0800
C Library: New method lsm_volume_raid_info()
* Please check python API document for detail about lsm_volume_raid_info()
method. Quick info:
Retrieves the pool id that the volume is derived from.
@param[in] c Valid connection
@param[in] v Volume ptr.
@param[out] raid_type Enum of lsm_volume_raid_type
@param[out] strip_size Size of the strip on disk or other storage extent.
@param[out] extent_count Count of disks or other storage extents in this
RAID group.
@param[out] min_io_size Minimum I/O size, also the preferred I/O size
of random I/O.
@param[out] opt_io_size Optimal I/O size, also the preferred I/O size
of sequential I/O.
@param[in] flags Reserved, set to 0
@return LSM_ERR_OK on success else error reason.
* New plugin interface: lsm_plug_volume_raid_info
* New enum type: lsm_volume_raid_type
* New capability:
LSM_CAP_VOLUME_RAID_INFO
* New constants:
LSM_VOLUME_RAID_TYPE_UNKNOWN = -1,
/**^ Unknown */
LSM_VOLUME_RAID_TYPE_RAID0 = 0,
/**^ Stripe */
LSM_VOLUME_RAID_TYPE_RAID1 = 1,
/**^ Mirror between two disks. For 4 disks or more, they are RAID10.*/
LSM_VOLUME_RAID_TYPE_RAID3 = 3,
/**^ Byte-level striping with dedicated parity */
LSM_VOLUME_RAID_TYPE_RAID4 = 4,
/**^ Block-level striping with dedicated parity */
/**^ Block-level striping with dedicated parity */
LSM_VOLUME_RAID_TYPE_RAID5 = 5,
/**^ Block-level striping with distributed parity */
LSM_VOLUME_RAID_TYPE_RAID6 = 6,
/**^ Block-level striping with two distributed parities, aka, RAID-DP */
LSM_VOLUME_RAID_TYPE_RAID10 = 10,
/**^ Stripe of mirrors */
LSM_VOLUME_RAID_TYPE_RAID15 = 15,
/**^ Parity of mirrors */
LSM_VOLUME_RAID_TYPE_RAID16 = 16,
/**^ Dual parity of mirrors */
LSM_VOLUME_RAID_TYPE_RAID50 = 50,
/**^ Stripe of parities */
LSM_VOLUME_RAID_TYPE_RAID60 = 60,
/**^ Stripe of dual parities */
LSM_VOLUME_RAID_TYPE_RAID51 = 51,
/**^ Mirror of parities */
LSM_VOLUME_RAID_TYPE_RAID61 = 61,
/**^ Mirror of dual parities */
LSM_VOLUME_RAID_TYPE_JBOD = 20,
/**^ Just bunch of disks, no parity, no striping. */
LSM_VOLUME_RAID_TYPE_MIXED = 21,
/**^ This volume contains multiple RAID settings. */
LSM_VOLUME_RAID_TYPE_OTHER = 22,
/**^ Vendor specific RAID type */
LSM_VOLUME_STRIP_SIZE_UNKNOWN
LSM_VOLUME_EXTENT_COUNT_UNKNOWN
LSM_VOLUME_MIN_IO_SIZE_UNKNOWN
LSM_VOLUME_OPT_IO_SIZE_UNKNOWN
V2: Change call back registration
Changes in V3:
* New implementation for adding new methods:
* New struct lsm_ops_v1_2:
Free to change during version 1.2 development phase.
Will frozen it once 1.2 released.
* New plugin register method: lsm_register_plugin_v1_2()
It takes all arguments required by old lsm_register_plugin_v1()
addition to struct lsm_ops_v1_2 pointer.
* Once version 1.2 released, we could work on struct lsm_ops_v1_3 and
lsm_register_plugin_v1_3().
* Add 'New in version 1.2' comment of lsm_volume_raid_info() function.
Changes in V4:
* Add LSM_FLAG_UNUSED_CHECK in public lsm_volume_raid_info() function.
* Changed the value of these constants from -1 to 0 to align with
libblkid/sysfs:
* LSM_VOLUME_STRIP_SIZE_UNKNOWN
* LSM_VOLUME_EXTENT_COUNT_UNKNOWN
* LSM_VOLUME_MIN_IO_SIZE_UNKNOWN
* LSM_VOLUME_OPT_IO_SIZE_UNKNOWN
Changes in V5:
* Rename output argument 'extent_count' to 'disk_count'.
* Change constant name from LSM_VOLUME_EXTENT_COUNT_UNKNOWN to
LSM_VOLUME_DISK_COUNT_UNKNOWN.
* Change data type from 'int32_t' to 'uint32_t' for these output arugments:
* strip_size
* disk_count
* min_io_size
* opt_io_size
Signed-off-by: Gris Ge <***@redhat.com>
Signed-off-by: Tony Asleson <***@redhat.com>
commit c489a2c29542b1f5d884e7f5dfe77803eb831243
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:39 2015 +0800
Python Library: New method volume_raid_info()
* The docstring of lsm.Client.volume_raid_info() contains full detail
about this new method. Quick info:
Usage:
volume_raid_info(self, volume, flags=0)
Returns:
[raid_type, strip_size, extent_count, min_io_size, opt_io_size]
# strip_size is the size of strip on each disk/extent
# extent_count is the disk/extent count.
# min_io_size is minimum I/O size. Also the preferred I/O size
# of random I/O.
# opt_io_size is optimal I/O size. Also the preferred I/O size
# of sequential I/O.
* Why not use 'pool_raid_info' instead?
Some RAID systems(EMC VMAX/DMX and LVM RAID) are not implementing RAID
at pool level but at volume level.
* Why use 'extent_count' instead of 'disk_count'?
Some RAID systems(EMC VMAX/DMX and LVM RAID) are not using disk
directly to assemble RAID group.
* Why we need 'min_io_size' and 'opt_io_size' when we have 'extent_count'
and 'strip_size'?
Normally, min_io_size is strip_size, opt_io_size could be calculated by
raid_type, strip_size and extent_count. But on NetApp, I/O test[1]
indicate their optimal I/O size is 64KiB no matter how many disks in
the RAID group. It might[2] because NetApp created a WAFL filesystem on
RAID group which changed the optimal I/O size.
In general, the optimal I/O size or min_io_size of some RAID system
might not base on strip size and RAID disk/extent count.
We'd better expose those information directly instead forcing user
to guess from strip size and disk/extent count.
* New constants:
Volume.RAID_TYPE_UNKNOWN
# The plugin failed to detect the volume's RAID type.
Volume.RAID_TYPE_RAID0
# Stripe
Volume.RAID_TYPE_RAID1
# Mirror for two disks. For 4 disks or more, they are RAID10.
Volume.RAID_TYPE_RAID3
# Byte-level striping with dedicated parity
Volume.RAID_TYPE_RAID4
# Block-level striping with dedicated parity
Volume.RAID_TYPE_RAID5
# Block-level striping with distributed parity
Volume.RAID_TYPE_RAID6
# Block-level striping with two distributed parities, aka, RAID-DP
Volume.RAID_TYPE_RAID10
# Stripe of mirrors
Volume.RAID_TYPE_RAID15
# Parity of mirrors
Volume.RAID_TYPE_RAID16
# Dual parity of mirrors
Volume.RAID_TYPE_RAID50
# Stripe of parities
Volume.RAID_TYPE_RAID60
# Stripe of dual parities
Volume.RAID_TYPE_RAID51
# Mirror of parities
Volume.RAID_TYPE_RAID61
# Mirror of dual parities
Volume.RAID_TYPE_JBOD
# Just bunch of disks, no parity, no striping.
Volume.RAID_TYPE_MIXED
# This volume contains multiple RAID settings.
Volume.RAID_TYPE_OTHER
# Vendor specific RAID type
Volume.STRIP_SIZE_UNKNOWN
Volume.EXTENT_COUNT_UNKNOWN
Volume.MIN_IO_SIZE_UNKNOWN
Volume.OPT_IO_SIZE_UNKNOWN
* New Capability:
lsm.Volume.VOLUME_RAID_INFO
[1] On a 24 disks RAID6(RAID-DP), 4KiB strip size(not changeable):
* With I/O size 90112(4096 * 22), write speed is 73.4 MB/s
* With I/O size 65536, write speed is 86.9 MB/s
# the optimal_io_size exposed via sysfs from SCSI BLOCK LIMITS(0xB0) VPD
[2] No NetApp official document confirm or deny it. Waiting NetApp's reply.
Changes in V2:
* Add 'New in 1.2' docstring.
Changes in V4(No change in V3):
* Change the value of these constants from -1 to 0 to align with
libblkid/sysfs:
Volume.STRIP_SIZE_UNKNOWN
Volume.MIN_IO_SIZE_UNKNOWN
Volume.OPT_IO_SIZE_UNKNOWN
Volume.EXTENT_COUNT_UNKNOWN
Changes in V5:
* Rename the return value 'extent_count' to 'disk_count' as it fit the
most common expectation on RAID.
For LVM RAID, EMC VMAX and other specific disk slice based RAID,
the 'disk_count' here means the slice count.
The constant renamed from EXTENT_COUNT_UNKNOWN to DISK_COUNT_UNKNOWN.
* Updated docstring to explain the 'disk_count' return value when facing
RAID system using disk slice.
Signed-off-by: Gris Ge <***@redhat.com>
commit 01d800bd253c009cbed77da15505b62e6ecf124d
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:38 2015 +0800
Python Library: Fix decorator problem with docstrings
* With decorator, the docstring of original method will be deleted.
* Use functools.wraps() to keep the docstring of original method.
Check
http://stackoverflow.com/questions/1782843/python-decorator-problem-with-docstrings
for detail.
* With this fix user can check method help message in interactive python with
command:
help(lsm.Client.volume_create)
Signed-off-by: Gris Ge <***@redhat.com>
commit c7d343ac0fa3d98d9672e8277f5a8b4a5e635956
Author: Tony Asleson <***@redhat.com>
Date: Thu Mar 5 16:28:37 2015 +0800
lsm_plugin_ipc.cpp: Bug Fix, using san_ops for fs_ops
In a number of places in the fs handlers were were checking
if san_ops was valid before dereferencing a fs_ops pointer.
This is clearly wrong and will result is a seg fault if a
plugin implements fs operations and not block operations.
Signed-off-by: Tony Asleson <***@redhat.com>
Signed-off-by: Gris Ge <***@redhat.com>
-----------------------------------------------------------------------
Summary of changes:
c_binding/include/libstoragemgmt/libstoragemgmt.h | 20 ++
.../libstoragemgmt/libstoragemgmt_capabilities.h | 3 +
.../libstoragemgmt/libstoragemgmt_plug_interface.h | 55 +++++
.../include/libstoragemgmt/libstoragemgmt_types.h | 43 ++++
c_binding/lsm_datatypes.hpp | 1 +
c_binding/lsm_mgmt.cpp | 48 +++++
c_binding/lsm_plugin_ipc.cpp | 86 +++++++--
plugin/megaraid/megaraid.py | 220 +++++++++++++++++++-
plugin/ontap/na.py | 8 +-
plugin/ontap/ontap.py | 55 ++++--
plugin/sim/simarray.py | 149 +++++++++-----
plugin/sim/simulator.py | 3 +
plugin/simc/simc_lsmplugin.c | 33 +++-
python_binding/lsm/_client.py | 103 +++++++++
python_binding/lsm/_common.py | 1 +
python_binding/lsm/_data.py | 42 ++++
test/cmdtest.py | 21 ++
test/tester.c | 30 +++
tools/lsmcli/cmdline.py | 18 ++-
tools/lsmcli/data_display.py | 58 +++++
tools/utility/check_const.pl | 6 +-
21 files changed, 912 insertions(+), 91 deletions(-)
hooks/post-receive
generated because a ref change was pushed to the repository containing
the project "libstoragemgmt".
The branch, master has been updated
via a8bfe32e8feb2e4af5a1fe68b06ad496ebc65eac (commit)
via d91b3befab8af044d86814d9b6474a664bd07dcd (commit)
via da411235e33a93aff59658752006302476ab3ee2 (commit)
via dd6760cde9346efa0d4eb6629cefc87c164c28b9 (commit)
via 902d057cff7e74de9af40e00c15f101bedf4ace6 (commit)
via 49b78aa3662fc61bf3b15c8ccc0a0923175f060f (commit)
via 1e3bd8e704dc132c73af7118df1e2816949824f6 (commit)
via 168c2d10d26a7ed6aeda2845ceb256cc1e639c42 (commit)
via b4d013e9a99bc442c0259c68338e9afeb2c17846 (commit)
via b635d0b993f90be68c8e7486252c73717f74de50 (commit)
via 1d311853b6f57e41f405245eebd5a4c7c1db3679 (commit)
via e9b69dc6516faeee4edc5a4c383e1bbcf10c549e (commit)
via c489a2c29542b1f5d884e7f5dfe77803eb831243 (commit)
via 01d800bd253c009cbed77da15505b62e6ecf124d (commit)
via c7d343ac0fa3d98d9672e8277f5a8b4a5e635956 (commit)
from a977d26c289acba25ae3e6460cb2022fdbcb16a1 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit a8bfe32e8feb2e4af5a1fe68b06ad496ebc65eac
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:51 2015 +0800
ONTAP Plugin: Add volume_raid_info() support.
* NetApp ONTAP strip size(minimum I/O size) is 4KiB, stripe size(
optimal I/O size) is 64KiB. Both are unchangeable.
* The extent count(disk count) is taken from aggregate 'disk-count'
property.
* Changed Filer.aggregates() to accept an optional argument 'aggr_name'
which query defined aggregate only.
* Uncommented and updated the old code for converting NetApp RAID level to
libstoragemgmt RAID level.
* Tested on ONTAP simulator 8.1.1 7-mode and real ONTAP 8.0.2 7-mode.
Changes in V5(No changes in V2, V3, V4):
* Sync API changes for argument name.
Signed-off-by: Gris Ge <***@redhat.com>
commit d91b3befab8af044d86814d9b6474a664bd07dcd
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:50 2015 +0800
MegaRAID Plugin: Add volume_raid_info() support.
* Use 'storcli /c0/v1 show all' command line output to determine
RAID type, strip size and disk count.
* Calculate optimal I/O size by strip size multiple with RAID
data(not mirrot, not parity) disks count.
* Tested query on RAID 0, 1, 5, 10, 50.
* Tested the optimal I/O size on RAID 5:
[***@storageqe-08 ~]# lsmenv mega lsmcli vri --vol SV03403550:VD1
Device alias: mega
URI: megaraid://
lsmcli vri --vol SV03403550:VD1
Volume ID | RAID Type | Strip Size | Extent Count | Minimum I/O Size | Optimal I/O Size
--------------------------------------------------------------------------------------------
SV03403550:VD1 | RAID5 | 131072 | 5 | 131072 | 524288
Time: 0:00.29
[***@storageqe-08 ~]# dd if=/dev/urandom of=test.img bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 153.174 s, 6.8 MB/s
[***@storageqe-08 ~]# dd if=./test.img of=/dev/sdb bs=131072 oflag=direct
8000+0 records in
8000+0 records out
1048576000 bytes (1.0 GB) copied, 58.9573 s, 17.8 MB/s
[***@storageqe-08 ~]# dd if=./test.img of=/dev/sdb bs=524288 oflag=direct
2000+0 records in
2000+0 records out
1048576000 bytes (1.0 GB) copied, 37.7282 s, 27.8 MB/s
[***@storageqe-08 ~]# dd if=./test.img of=/dev/sdb bs=524288 oflag=direct
2000+0 records in
2000+0 records out
1048576000 bytes (1.0 GB) copied, 35.3351 s, 29.7 MB/s
[***@storageqe-08 ~]# dd if=./test.img of=/dev/sdb bs=131072 oflag=direct
8000+0 records in
8000+0 records out
1048576000 bytes (1.0 GB) copied, 70.0779 s, 15.0 MB/s
Signed-off-by: Gris Ge <***@redhat.com>
commit da411235e33a93aff59658752006302476ab3ee2
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:49 2015 +0800
MegaRAID Plugin: Add Disk.STATUS_RECONSTRUCT support.
* In storcli of MegaRAID, 'Rbld' of disk status indicate this disk
in using for reconstructing pool data.
Signed-off-by: Gris Ge <***@redhat.com>
commit dd6760cde9346efa0d4eb6629cefc87c164c28b9
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:48 2015 +0800
MegaRAID Plugin: Add volumes() support.
* Treating MegaRAID VD as LSM Volume.
* Using 'storcli /c0/vall show all' to query all volumes.
* Add new capability: Capabilities.VOLUMES
Signed-off-by: Gris Ge <***@redhat.com>
commit 902d057cff7e74de9af40e00c15f101bedf4ace6
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:47 2015 +0800
MegaRAID plugin: Add pools() method support.
* Treating each MegaRAID DG(disk group) as LSM pool.
* Based on storcli output of:
storcli /c0/dall show all J
Signed-off-by: Gris Ge <***@redhat.com>
commit 49b78aa3662fc61bf3b15c8ccc0a0923175f060f
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:46 2015 +0800
Simulator C Plugin: Add lsm_volume_raid_info() support.
* Simply set XXX_UNKNOWN on output parameter.
V2:
- Add call to register volume_raid_info
- Add LSM_CAP_VOLUMERAID_INFO to capabilties
Changes in V3:
* Use lsm_register_plugin_v1_2() to register lsm_volume_raid_info() support.
Changes in V5(no change in V4):
* Sync API changes:
* argument name 'disk_count'
* argument type 'uint32_t'.
* constant name 'LSM_VOLUME_DISK_COUNT_UNKNOWN'.
Signed-off-by: Gris Ge <***@redhat.com>
Signed-off-by: Tony Asleson <***@redhat.com>
commit 1e3bd8e704dc132c73af7118df1e2816949824f6
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:45 2015 +0800
Simulator Plugin: Add volume_raid_info() support
* Introduced full support of volume_raid_info().
* For sub-pool, use raid info from parent pool.
* For RAID 1 and JBOD, set strip_size, min_io_size, and opt_io_size
as block size(512).
* For other RAID, calculate out opt_io_size with data disk count.
* For RAID_TYPE_MIXED, raise PLUGIN_BUG LsmError.
* Replaced PoolRAID.RAID_TYPE_RAID_XXX with Volume.RAID_TYPE_RAID_XXX.
* Replaced PoolRAID.RAID_TYPE_NOT_APPLICABLE with Volume.RAID_TYPE_OTHER.
* Bumped simulator data version to 3.1 due to previous Volume.RAID_TYPE_OTHER
change.
Changes in V4(No changes in V2 and V3):
* Removed unneeded import 'Volume' in simulator.py.
Changes in V5:
* Sync changes of API changes for return argument name 'disk_count' and
constant name Volume.DISK_COUNT_UNKNOWN.
Signed-off-by: Gris Ge <***@redhat.com>
commit 168c2d10d26a7ed6aeda2845ceb256cc1e639c42
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:44 2015 +0800
Constant Test: Fix missing constant with number in it.
* Allowing check_const.pl to check constants with number in it.
Example:
LSM_VOLUME_RAID_TYPE_RAID1
Signed-off-by: Gris Ge <***@redhat.com>
commit b4d013e9a99bc442c0259c68338e9afeb2c17846
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:43 2015 +0800
C Unit Test: Add test for lsm_volume_raid_info() method
* Simply invoke lsm_volume_raid_info() with no additional test.
Changes in V5(No change in V2, V3, V4):
* Sync API changes for argument name('extent_count' to 'disk_count') and data
type.
Signed-off-by: Gris Ge <***@redhat.com>
commit b635d0b993f90be68c8e7486252c73717f74de50
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:42 2015 +0800
lsmcli Test: Add test for volume-raid-info command.
* Simply run that command and check the volume ID of output.
Signed-off-by: Gris Ge <***@redhat.com>
commit 1d311853b6f57e41f405245eebd5a4c7c1db3679
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:41 2015 +0800
lsmcli: Add volume_raid_info() support.
* New command:
lsmcli volume-raid-info --vol <VOL_ID>
* New alias:
lsmcli vri == lsmcli volume-raid-info
Changes in V2:
* Fix output format when volume not found passed to _get_item() in
volume_raid_info()
Changes in V5(No changes in V3, V4):
* Sync API changes of name 'extent_count' to 'disk_count', column name
changed to 'Disk Count'.
Signed-off-by: Gris Ge <***@redhat.com>
commit e9b69dc6516faeee4edc5a4c383e1bbcf10c549e
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:40 2015 +0800
C Library: New method lsm_volume_raid_info()
* Please check python API document for detail about lsm_volume_raid_info()
method. Quick info:
Retrieves the pool id that the volume is derived from.
@param[in] c Valid connection
@param[in] v Volume ptr.
@param[out] raid_type Enum of lsm_volume_raid_type
@param[out] strip_size Size of the strip on disk or other storage extent.
@param[out] extent_count Count of disks or other storage extents in this
RAID group.
@param[out] min_io_size Minimum I/O size, also the preferred I/O size
of random I/O.
@param[out] opt_io_size Optimal I/O size, also the preferred I/O size
of sequential I/O.
@param[in] flags Reserved, set to 0
@return LSM_ERR_OK on success else error reason.
* New plugin interface: lsm_plug_volume_raid_info
* New enum type: lsm_volume_raid_type
* New capability:
LSM_CAP_VOLUME_RAID_INFO
* New constants:
LSM_VOLUME_RAID_TYPE_UNKNOWN = -1,
/**^ Unknown */
LSM_VOLUME_RAID_TYPE_RAID0 = 0,
/**^ Stripe */
LSM_VOLUME_RAID_TYPE_RAID1 = 1,
/**^ Mirror between two disks. For 4 disks or more, they are RAID10.*/
LSM_VOLUME_RAID_TYPE_RAID3 = 3,
/**^ Byte-level striping with dedicated parity */
LSM_VOLUME_RAID_TYPE_RAID4 = 4,
/**^ Block-level striping with dedicated parity */
/**^ Block-level striping with dedicated parity */
LSM_VOLUME_RAID_TYPE_RAID5 = 5,
/**^ Block-level striping with distributed parity */
LSM_VOLUME_RAID_TYPE_RAID6 = 6,
/**^ Block-level striping with two distributed parities, aka, RAID-DP */
LSM_VOLUME_RAID_TYPE_RAID10 = 10,
/**^ Stripe of mirrors */
LSM_VOLUME_RAID_TYPE_RAID15 = 15,
/**^ Parity of mirrors */
LSM_VOLUME_RAID_TYPE_RAID16 = 16,
/**^ Dual parity of mirrors */
LSM_VOLUME_RAID_TYPE_RAID50 = 50,
/**^ Stripe of parities */
LSM_VOLUME_RAID_TYPE_RAID60 = 60,
/**^ Stripe of dual parities */
LSM_VOLUME_RAID_TYPE_RAID51 = 51,
/**^ Mirror of parities */
LSM_VOLUME_RAID_TYPE_RAID61 = 61,
/**^ Mirror of dual parities */
LSM_VOLUME_RAID_TYPE_JBOD = 20,
/**^ Just bunch of disks, no parity, no striping. */
LSM_VOLUME_RAID_TYPE_MIXED = 21,
/**^ This volume contains multiple RAID settings. */
LSM_VOLUME_RAID_TYPE_OTHER = 22,
/**^ Vendor specific RAID type */
LSM_VOLUME_STRIP_SIZE_UNKNOWN
LSM_VOLUME_EXTENT_COUNT_UNKNOWN
LSM_VOLUME_MIN_IO_SIZE_UNKNOWN
LSM_VOLUME_OPT_IO_SIZE_UNKNOWN
V2: Change call back registration
Changes in V3:
* New implementation for adding new methods:
* New struct lsm_ops_v1_2:
Free to change during version 1.2 development phase.
Will frozen it once 1.2 released.
* New plugin register method: lsm_register_plugin_v1_2()
It takes all arguments required by old lsm_register_plugin_v1()
addition to struct lsm_ops_v1_2 pointer.
* Once version 1.2 released, we could work on struct lsm_ops_v1_3 and
lsm_register_plugin_v1_3().
* Add 'New in version 1.2' comment of lsm_volume_raid_info() function.
Changes in V4:
* Add LSM_FLAG_UNUSED_CHECK in public lsm_volume_raid_info() function.
* Changed the value of these constants from -1 to 0 to align with
libblkid/sysfs:
* LSM_VOLUME_STRIP_SIZE_UNKNOWN
* LSM_VOLUME_EXTENT_COUNT_UNKNOWN
* LSM_VOLUME_MIN_IO_SIZE_UNKNOWN
* LSM_VOLUME_OPT_IO_SIZE_UNKNOWN
Changes in V5:
* Rename output argument 'extent_count' to 'disk_count'.
* Change constant name from LSM_VOLUME_EXTENT_COUNT_UNKNOWN to
LSM_VOLUME_DISK_COUNT_UNKNOWN.
* Change data type from 'int32_t' to 'uint32_t' for these output arugments:
* strip_size
* disk_count
* min_io_size
* opt_io_size
Signed-off-by: Gris Ge <***@redhat.com>
Signed-off-by: Tony Asleson <***@redhat.com>
commit c489a2c29542b1f5d884e7f5dfe77803eb831243
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:39 2015 +0800
Python Library: New method volume_raid_info()
* The docstring of lsm.Client.volume_raid_info() contains full detail
about this new method. Quick info:
Usage:
volume_raid_info(self, volume, flags=0)
Returns:
[raid_type, strip_size, extent_count, min_io_size, opt_io_size]
# strip_size is the size of strip on each disk/extent
# extent_count is the disk/extent count.
# min_io_size is minimum I/O size. Also the preferred I/O size
# of random I/O.
# opt_io_size is optimal I/O size. Also the preferred I/O size
# of sequential I/O.
* Why not use 'pool_raid_info' instead?
Some RAID systems(EMC VMAX/DMX and LVM RAID) are not implementing RAID
at pool level but at volume level.
* Why use 'extent_count' instead of 'disk_count'?
Some RAID systems(EMC VMAX/DMX and LVM RAID) are not using disk
directly to assemble RAID group.
* Why we need 'min_io_size' and 'opt_io_size' when we have 'extent_count'
and 'strip_size'?
Normally, min_io_size is strip_size, opt_io_size could be calculated by
raid_type, strip_size and extent_count. But on NetApp, I/O test[1]
indicate their optimal I/O size is 64KiB no matter how many disks in
the RAID group. It might[2] because NetApp created a WAFL filesystem on
RAID group which changed the optimal I/O size.
In general, the optimal I/O size or min_io_size of some RAID system
might not base on strip size and RAID disk/extent count.
We'd better expose those information directly instead forcing user
to guess from strip size and disk/extent count.
* New constants:
Volume.RAID_TYPE_UNKNOWN
# The plugin failed to detect the volume's RAID type.
Volume.RAID_TYPE_RAID0
# Stripe
Volume.RAID_TYPE_RAID1
# Mirror for two disks. For 4 disks or more, they are RAID10.
Volume.RAID_TYPE_RAID3
# Byte-level striping with dedicated parity
Volume.RAID_TYPE_RAID4
# Block-level striping with dedicated parity
Volume.RAID_TYPE_RAID5
# Block-level striping with distributed parity
Volume.RAID_TYPE_RAID6
# Block-level striping with two distributed parities, aka, RAID-DP
Volume.RAID_TYPE_RAID10
# Stripe of mirrors
Volume.RAID_TYPE_RAID15
# Parity of mirrors
Volume.RAID_TYPE_RAID16
# Dual parity of mirrors
Volume.RAID_TYPE_RAID50
# Stripe of parities
Volume.RAID_TYPE_RAID60
# Stripe of dual parities
Volume.RAID_TYPE_RAID51
# Mirror of parities
Volume.RAID_TYPE_RAID61
# Mirror of dual parities
Volume.RAID_TYPE_JBOD
# Just bunch of disks, no parity, no striping.
Volume.RAID_TYPE_MIXED
# This volume contains multiple RAID settings.
Volume.RAID_TYPE_OTHER
# Vendor specific RAID type
Volume.STRIP_SIZE_UNKNOWN
Volume.EXTENT_COUNT_UNKNOWN
Volume.MIN_IO_SIZE_UNKNOWN
Volume.OPT_IO_SIZE_UNKNOWN
* New Capability:
lsm.Volume.VOLUME_RAID_INFO
[1] On a 24 disks RAID6(RAID-DP), 4KiB strip size(not changeable):
* With I/O size 90112(4096 * 22), write speed is 73.4 MB/s
* With I/O size 65536, write speed is 86.9 MB/s
# the optimal_io_size exposed via sysfs from SCSI BLOCK LIMITS(0xB0) VPD
[2] No NetApp official document confirm or deny it. Waiting NetApp's reply.
Changes in V2:
* Add 'New in 1.2' docstring.
Changes in V4(No change in V3):
* Change the value of these constants from -1 to 0 to align with
libblkid/sysfs:
Volume.STRIP_SIZE_UNKNOWN
Volume.MIN_IO_SIZE_UNKNOWN
Volume.OPT_IO_SIZE_UNKNOWN
Volume.EXTENT_COUNT_UNKNOWN
Changes in V5:
* Rename the return value 'extent_count' to 'disk_count' as it fit the
most common expectation on RAID.
For LVM RAID, EMC VMAX and other specific disk slice based RAID,
the 'disk_count' here means the slice count.
The constant renamed from EXTENT_COUNT_UNKNOWN to DISK_COUNT_UNKNOWN.
* Updated docstring to explain the 'disk_count' return value when facing
RAID system using disk slice.
Signed-off-by: Gris Ge <***@redhat.com>
commit 01d800bd253c009cbed77da15505b62e6ecf124d
Author: Gris Ge <***@redhat.com>
Date: Thu Mar 5 16:28:38 2015 +0800
Python Library: Fix decorator problem with docstrings
* With decorator, the docstring of original method will be deleted.
* Use functools.wraps() to keep the docstring of original method.
Check
http://stackoverflow.com/questions/1782843/python-decorator-problem-with-docstrings
for detail.
* With this fix user can check method help message in interactive python with
command:
help(lsm.Client.volume_create)
Signed-off-by: Gris Ge <***@redhat.com>
commit c7d343ac0fa3d98d9672e8277f5a8b4a5e635956
Author: Tony Asleson <***@redhat.com>
Date: Thu Mar 5 16:28:37 2015 +0800
lsm_plugin_ipc.cpp: Bug Fix, using san_ops for fs_ops
In a number of places in the fs handlers were were checking
if san_ops was valid before dereferencing a fs_ops pointer.
This is clearly wrong and will result is a seg fault if a
plugin implements fs operations and not block operations.
Signed-off-by: Tony Asleson <***@redhat.com>
Signed-off-by: Gris Ge <***@redhat.com>
-----------------------------------------------------------------------
Summary of changes:
c_binding/include/libstoragemgmt/libstoragemgmt.h | 20 ++
.../libstoragemgmt/libstoragemgmt_capabilities.h | 3 +
.../libstoragemgmt/libstoragemgmt_plug_interface.h | 55 +++++
.../include/libstoragemgmt/libstoragemgmt_types.h | 43 ++++
c_binding/lsm_datatypes.hpp | 1 +
c_binding/lsm_mgmt.cpp | 48 +++++
c_binding/lsm_plugin_ipc.cpp | 86 +++++++--
plugin/megaraid/megaraid.py | 220 +++++++++++++++++++-
plugin/ontap/na.py | 8 +-
plugin/ontap/ontap.py | 55 ++++--
plugin/sim/simarray.py | 149 +++++++++-----
plugin/sim/simulator.py | 3 +
plugin/simc/simc_lsmplugin.c | 33 +++-
python_binding/lsm/_client.py | 103 +++++++++
python_binding/lsm/_common.py | 1 +
python_binding/lsm/_data.py | 42 ++++
test/cmdtest.py | 21 ++
test/tester.c | 30 +++
tools/lsmcli/cmdline.py | 18 ++-
tools/lsmcli/data_display.py | 58 +++++
tools/utility/check_const.pl | 6 +-
21 files changed, 912 insertions(+), 91 deletions(-)
hooks/post-receive
--
libstoragemgmt
libstoragemgmt