Discussion:
[Libstoragemgmt-devel] [PATCH 0/6] Add new disk status: STATUS_FREE
Gris Ge
2015-03-05 14:07:20 UTC
Permalink
* This patch set introduced new disk status -- STATUS_FREE to indicate
disk could be used for pool or assign as dedicated spare disk.

* Patch set is based on:
[RESEND PATCH V5 00/15] New method: volume_raid_info()

Gris Ge (6):
MegaRAID: Fix incorrect system status query.
API: New disk status DISK.STATUS_FREE/LSM_DISK_STATUS_FREE
lsmcli: Add support of Disk.STATUS_FREE
Simulator Plugin: Support Disk.STATUS_FREE
ONTAP Plugin: Add support of Disk.STATUS_FREE
MegaRAID Plugin: Add support of Disk.STATUS_FREE

.../include/libstoragemgmt/libstoragemgmt_types.h | 10 +++
plugin/megaraid/megaraid.py | 90 ++++++++++++----------
plugin/ontap/ontap.py | 10 +++
plugin/sim/simarray.py | 9 ++-
python_binding/lsm/_data.py | 8 ++
tools/lsmcli/data_display.py | 1 +
6 files changed, 87 insertions(+), 41 deletions(-)
--
1.8.3.1
Gris Ge
2015-03-05 14:07:23 UTC
Permalink
* Show Disk.STATUS_FREE as 'Free' in output.

Signed-off-by: Gris Ge <***@redhat.com>
---
tools/lsmcli/data_display.py | 1 +
1 file changed, 1 insertion(+)

diff --git a/tools/lsmcli/data_display.py b/tools/lsmcli/data_display.py
index e0524c8..19f7114 100644
--- a/tools/lsmcli/data_display.py
+++ b/tools/lsmcli/data_display.py
@@ -201,6 +201,7 @@ _DISK_STATUS_CONV = {
Disk.STATUS_MAINTENANCE_MODE: 'Maintenance',
Disk.STATUS_SPARE_DISK: 'Spare',
Disk.STATUS_RECONSTRUCT: 'Reconstruct',
+ Disk.STATUS_FREE: 'Free',
}
--
1.8.3.1
Gris Ge
2015-03-05 14:07:22 UTC
Permalink
* New status constant:
Disk.STATUS_FREE
LSM_DISK_STATUS_FREE

* In code document(Both in C and Python code):
New in version 1.2, indicate the whole disk is not holding any data or
acting as a dedicate spare disk.
This disk could be assigned as a dedicated spare disk or used for
creating pool.
If any spare disk(like those on NetApp ONTAP) does not require
any explicit action when assigning to pool, it should be treated as free
disk and marked as STATUS_FREE|STATUS_SPARE_DISK.

* Future design consideration for sliced RAID system(LVM RAID and EMC VMAX):
1. Introduce Disk.STATUS_FREE_PARTIAL
2. Introduce new class:
DiskSlice
properties:
id
owner_disk_id # Disk.id
block_size
block_count
start_block
status # FREE or SPARE or/and OK

Signed-off-by: Gris Ge <***@redhat.com>
---
c_binding/include/libstoragemgmt/libstoragemgmt_types.h | 10 ++++++++++
python_binding/lsm/_data.py | 8 ++++++++
2 files changed, 18 insertions(+)

diff --git a/c_binding/include/libstoragemgmt/libstoragemgmt_types.h b/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
index 562fcff..c5607c1 100644
--- a/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
+++ b/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
@@ -237,6 +237,16 @@ typedef enum {
#define LSM_DISK_STATUS_MAINTENANCE_MODE 0x0000000000000400
#define LSM_DISK_STATUS_SPARE_DISK 0x0000000000000800
#define LSM_DISK_STATUS_RECONSTRUCT 0x0000000000001000
+#define LSM_DISK_STATUS_FREE 0x0000000000002000
+/**^
+ * New in version 1.2, New in version 1.2, indicate the whole disk is not
+ * holding any data or acting as a dedicate spare disk.
+ * This disk could be assigned as a dedicated spare disk or used for creating
+ * pool.
+ * If any spare disk(like those on NetApp ONTAP) does not require any explicit
+ * action when assigning to pool, it should be treated as free disk and marked
+ * as LSM_DISK_STATUS_FREE|LSM_DISK_STATUS_SPARE_DISK.
+ * */

#define LSM_DISK_BLOCK_SIZE_NOT_FOUND -1
#define LSM_DISK_BLOCK_COUNT_NOT_FOUND -1
diff --git a/python_binding/lsm/_data.py b/python_binding/lsm/_data.py
index 6fb2325..23681dd 100644
--- a/python_binding/lsm/_data.py
+++ b/python_binding/lsm/_data.py
@@ -199,6 +199,14 @@ class Disk(IData):
# Indicate disk is a spare disk.
STATUS_RECONSTRUCT = 1 << 12
# Indicate disk is reconstructing data.
+ STATUS_FREE = 1 << 13
+ # New in version 1.2, indicate the whole disk is not holding any data or
+ # acting as a dedicate spare disk.
+ # This disk could be assigned as a dedicated spare disk or used for
+ # creating pool.
+ # If any spare disk(like those on NetApp ONTAP) does not require
+ # any explicit action when assigning to pool, it should be treated as
+ # free disk and marked as STATUS_FREE|STATUS_SPARE_DISK.

def __init__(self, _id, _name, _disk_type, _block_size, _num_of_blocks,
_status, _system_id, _plugin_data=None):
--
1.8.3.1
Gris Ge
2015-03-05 14:07:26 UTC
Permalink
* Treat 'UGood' and 'UBad' as free disk.

* Update algorithm of disk status query to support show spare disk with
sector error as SPARE|ERROR

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/megaraid/megaraid.py | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/plugin/megaraid/megaraid.py b/plugin/megaraid/megaraid.py
index 188ba58..df25eb3 100644
--- a/plugin/megaraid/megaraid.py
+++ b/plugin/megaraid/megaraid.py
@@ -91,29 +91,36 @@ _DISK_STATE_MAP = {
'Offln': Disk.STATUS_ERROR,
'GHS': Disk.STATUS_SPARE_DISK | Disk.STATUS_OK,
'DHS': Disk.STATUS_SPARE_DISK | Disk.STATUS_OK,
- 'UGood': Disk.STATUS_STOPPED | Disk.STATUS_OK,
- 'UBad': Disk.STATUS_STOPPED | Disk.STATUS_ERROR,
+ 'UGood': Disk.STATUS_FREE | Disk.STATUS_OK,
+ 'UBad': Disk.STATUS_FREE | Disk.STATUS_ERROR,
'Rbld': Disk.STATUS_RECONSTRUCT,
}


def _disk_status_of(disk_show_basic_dict, disk_show_stat_dict):
+ disk_status = _DISK_STATE_MAP.get(
+ disk_show_basic_dict['State'], 0)
+
if disk_show_stat_dict['Media Error Count'] or \
disk_show_stat_dict['Other Error Count'] or \
disk_show_stat_dict['S.M.A.R.T alert flagged by drive'] != 'No':
- return Disk.STATUS_ERROR
+ disk_status -= Disk.STATUS_OK
+ disk_status |= Disk.STATUS_ERROR

- if disk_show_stat_dict['Predictive Failure Count']:
- return Disk.STATUS_PREDICTIVE_FAILURE
+ elif disk_show_stat_dict['Predictive Failure Count']:
+ disk_status -= Disk.STATUS_OK
+ disk_status |= Disk.STATUS_PREDICTIVE_FAILURE

if disk_show_basic_dict['Sp'] == 'D':
- return Disk.STATUS_STOPPED
+ disk_status |= Disk.STATUS_STOPPED

if disk_show_basic_dict['Sp'] == 'F':
- return Disk.STATUS_OTHER
+ disk_status |= Disk.STATUS_OTHER
+
+ if disk_status == 0:
+ disk_status == Disk.STATUS_UNKNOWN

- return _DISK_STATE_MAP.get(
- disk_show_basic_dict['State'], Disk.STATUS_UNKNOWN)
+ return disk_status


def _mega_size_to_lsm(mega_size):
--
1.8.3.1
Gris Ge
2015-03-05 14:07:25 UTC
Permalink
* Any disk not have 'aggregate' property is considered as free disk.

* Added comments indicate how spare and free disk works on ONTAP:
1. Spare == free disk
2. Zeroed spare disk == OK | SPARE | FREE
3. Non-zeroed spare disk == STOPPED | SPARE | FREE

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/ontap/ontap.py | 10 ++++++++++
1 file changed, 10 insertions(+)

diff --git a/plugin/ontap/ontap.py b/plugin/ontap/ontap.py
index b6358a8..16115a4 100644
--- a/plugin/ontap/ontap.py
+++ b/plugin/ontap/ontap.py
@@ -265,6 +265,10 @@ class Ontap(IStorageAreaNetwork, INfs):
if 'is-zeroed' in na_disk and na_disk['is-zeroed'] == 'true':
status |= Disk.STATUS_OK | Disk.STATUS_SPARE_DISK
else:
+ # If spare disk is not zerored, it will be automaticlly
+ # zeroed before assigned to aggregate.
+ # Hence we consider non-zeroed spare disks as stopped
+ # spare disks.
status |= Disk.STATUS_STOPPED | Disk.STATUS_SPARE_DISK
elif rs == 'present':
status |= Disk.STATUS_OK
@@ -279,6 +283,12 @@ class Ontap(IStorageAreaNetwork, INfs):
if 'is-offline' in na_disk and na_disk['is-offline'] == 'true':
status |= Disk.STATUS_ERROR

+ if 'aggregate' not in na_disk:
+ # All free disks are automatically marked as spare disks. They
+ # could easily convert to data or parity disk without any
+ # explicit command.
+ status |= Disk.STATUS_FREE
+
if status == 0:
status = Disk.STATUS_UNKNOWN
--
1.8.3.1
Gris Ge
2015-03-05 14:07:24 UTC
Permalink
* Treating sim_disk['role'] == None as Free disk.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/sim/simarray.py | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/plugin/sim/simarray.py b/plugin/sim/simarray.py
index 3fa7782..c3d2051 100644
--- a/plugin/sim/simarray.py
+++ b/plugin/sim/simarray.py
@@ -194,7 +194,7 @@ class BackStore(object):

DISK_KEY_LIST = [
'id', 'name', 'total_space', 'disk_type', 'status',
- 'owner_pool_id']
+ 'owner_pool_id', 'role']

VOL_KEY_LIST = [
'id', 'vpd83', 'name', 'total_space', 'consumed_size',
@@ -1762,12 +1762,17 @@ class SimArray(object):

@staticmethod
def _sim_disk_2_lsm(sim_disk):
+ print sim_disk.items()
+ disk_status = Disk.STATUS_OK
+ if sim_disk['role'] is None:
+ disk_status |= Disk.STATUS_FREE
+
return Disk(
SimArray._sim_id_to_lsm_id(sim_disk['id'], 'DISK'),
sim_disk['name'],
sim_disk['disk_type'], BackStore.BLK_SIZE,
int(sim_disk['total_space'] / BackStore.BLK_SIZE),
- Disk.STATUS_OK, BackStore.SYS_ID)
+ disk_status, BackStore.SYS_ID)

@_handle_errors
def disks(self):
--
1.8.3.1
Gris Ge
2015-03-05 14:07:21 UTC
Permalink
* The storcli-1.14.12-1.noarch only support 'show health' command for
Nytro MegaRAID card. This patch change system status query using
'show all' command.

* Use single command 'storcli /c0 show all J' to get all information
to generate lsm.System.

* Status information is extracted from 'Status' section of 'show all' output.

* Append firmware version(firmware package build version) in System.name.
User could compare this version string with LSI webpage for upgrade.

* Tested on Intel and IBM rebranded MegaRAID cards.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/megaraid/megaraid.py | 65 ++++++++++++++++++++++++---------------------
1 file changed, 35 insertions(+), 30 deletions(-)

diff --git a/plugin/megaraid/megaraid.py b/plugin/megaraid/megaraid.py
index e754cd8..188ba58 100644
--- a/plugin/megaraid/megaraid.py
+++ b/plugin/megaraid/megaraid.py
@@ -299,43 +299,48 @@ class MegaRAID(IPlugin):
return self._storcli_exec(
["show", "ctrlcount"]).get("Controller Count")

- def _lsm_status_of_ctrl(self, ctrl_num):
- ctrl_health_output = self._storcli_exec(
- ["/c%d" % ctrl_num, 'show', 'health', 'all'])
- health_info = ctrl_health_output['Controller Health Info']
+ def _lsm_status_of_ctrl(self, ctrl_show_all_output):
+ lsi_status_info = ctrl_show_all_output['Status']
+ status_info = ''
+ status = System.STATUS_UNKNOWN
+ if lsi_status_info['Controller Status'] == 'Optimal':
+ status = System.STATUS_OK
+ else:
# TODO(Gris Ge): Try pull a disk off to check whether this change.
- if health_info['Overall Health'] == 'GOOD':
- return System.STATUS_OK, ''
-
- return System.STATUS_UNKNOWN, "%s reason code %s" % (
- health_info['Overall Health'],
- health_info['Reason Code'])
-
- def _sys_id_of_ctrl_num(self, ctrl_num, ctrl_show_output=None):
- if ctrl_show_output is None:
- ctrl_show_output = self._storcli_exec(["/c%d" % ctrl_num, "show"])
- sys_id = ctrl_show_output['Serial Number']
- if not sys_id:
- raise LsmError(
- ErrorNumber.PLUGIN_BUG,
- "_sys_id_of_ctrl_num(): Fail to get system id: %s" %
- ctrl_show_output.items())
+ status_info = "%s: " % lsi_status_info['Controller Status']
+ for key_name in lsi_status_info.keys():
+ if key_name == 'Controller Status':
+ continue
+ if lsi_status_info[key_name] != 0 and \
+ lsi_status_info[key_name] != 'No' and \
+ lsi_status_info[key_name] != 'NA':
+ status_info += " %s:%s" % (
+ key_name, lsi_status_info[key_name])
+
+ return status, status_info
+
+ def _sys_id_of_ctrl_num(self, ctrl_num, ctrl_show_all_output=None):
+ if ctrl_show_all_output is None:
+ return self._storcli_exec(
+ ["/c%d" % ctrl_num, "show"])['Serial Number']
else:
- return sys_id
+ return ctrl_show_all_output['Basics']['Serial Number']

@_handle_errors
def systems(self, flags=0):
rc_lsm_syss = []
for ctrl_num in range(self._ctrl_count()):
- ctrl_show_output = self._storcli_exec(["/c%d" % ctrl_num, "show"])
- sys_id = self._sys_id_of_ctrl_num(ctrl_num, ctrl_show_output)
- sys_name = "%s %s %s:%s:%s" % (
- ctrl_show_output['Product Name'],
- ctrl_show_output['Host Interface'],
- format(ctrl_show_output['Bus Number'], '02x'),
- format(ctrl_show_output['Device Number'], '02x'),
- format(ctrl_show_output['Function Number'], '02x'))
- (status, status_info) = self._lsm_status_of_ctrl(ctrl_num)
+ ctrl_show_all_output = self._storcli_exec(
+ ["/c%d" % ctrl_num, "show", "all"])
+ sys_id = self._sys_id_of_ctrl_num(ctrl_num, ctrl_show_all_output)
+ sys_name = "%s %s %s ver: %s" % (
+ ctrl_show_all_output['Basics']['Model'],
+ ctrl_show_all_output['Bus']['Host Interface'],
+ ctrl_show_all_output['Basics']['PCI Address'],
+ ctrl_show_all_output['Version']['Firmware Package Build'],
+ )
+ (status, status_info) = self._lsm_status_of_ctrl(
+ ctrl_show_all_output)
plugin_data = "/c%d"
# Since PCI slot sequence might change.
# This string just stored for quick system verification.
--
1.8.3.1
Tony Asleson
2015-03-05 20:54:46 UTC
Permalink
Hi Gris,

Just saw one thing:

megaraid.py

Line: 121
disk_status == Disk.STATUS_UNKNOWN

should be:
disk_status = Disk.STATUS_UNKNOWN

Support for this in SMI-S, guessing ugly to impossible?

Thanks!

Regards,
Tony
Post by Gris Ge
* This patch set introduced new disk status -- STATUS_FREE to indicate
disk could be used for pool or assign as dedicated spare disk.
[RESEND PATCH V5 00/15] New method: volume_raid_info()
MegaRAID: Fix incorrect system status query.
API: New disk status DISK.STATUS_FREE/LSM_DISK_STATUS_FREE
lsmcli: Add support of Disk.STATUS_FREE
Simulator Plugin: Support Disk.STATUS_FREE
ONTAP Plugin: Add support of Disk.STATUS_FREE
MegaRAID Plugin: Add support of Disk.STATUS_FREE
.../include/libstoragemgmt/libstoragemgmt_types.h | 10 +++
plugin/megaraid/megaraid.py | 90 ++++++++++++----------
plugin/ontap/ontap.py | 10 +++
plugin/sim/simarray.py | 9 ++-
python_binding/lsm/_data.py | 8 ++
tools/lsmcli/data_display.py | 1 +
6 files changed, 87 insertions(+), 41 deletions(-)
Gris Ge
2015-03-06 14:28:52 UTC
Permalink
Post by Tony Asleson
Hi Gris,
Hi Tony,
Post by Tony Asleson
megaraid.py
Line: 121
disk_status == Disk.STATUS_UNKNOWN
disk_status = Disk.STATUS_UNKNOWN
Thanks for the detailed review.
Post by Tony Asleson
Support for this in SMI-S, guessing ugly to impossible?
The official way is ugly by using 'Extent Composition Subprofile'
profile to walk through all StorageExtents recursively.

But we do have very good vendor specific way:
* EMC, they have 'EMCInUse' property in CIM_DiskDrive.
* LSI MegaRAID, they have 'StoragePool_InstanceID' in primordial
CIM_StorageExtent(directly associated with CIM_DiskDrive).

I emailed to SNIA for this issue on 14 May 2014, only EMC replied with
their way, seems other vendor just ignored this request.

Let me add support for these two products in V2 patch.

Best regards.
Post by Tony Asleson
Thanks!
Regards,
Tony
--
Gris Ge
Gris Ge
2015-03-06 15:34:26 UTC
Permalink
* The storcli-1.14.12-1.noarch only support 'show health' command for
Nytro MegaRAID card. This patch change system status query using
'show all' command.

* Use single command 'storcli /c0 show all J' to get all information
to generate lsm.System.

* Status information is extracted from 'Status' section of 'show all' output.

* Append firmware version(firmware package build version) in System.name.
User could compare this version string with LSI webpage for upgrade.

* Tested on Intel and IBM rebranded MegaRAID cards.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/megaraid/megaraid.py | 65 ++++++++++++++++++++++++---------------------
1 file changed, 35 insertions(+), 30 deletions(-)

diff --git a/plugin/megaraid/megaraid.py b/plugin/megaraid/megaraid.py
index e754cd8..188ba58 100644
--- a/plugin/megaraid/megaraid.py
+++ b/plugin/megaraid/megaraid.py
@@ -299,43 +299,48 @@ class MegaRAID(IPlugin):
return self._storcli_exec(
["show", "ctrlcount"]).get("Controller Count")

- def _lsm_status_of_ctrl(self, ctrl_num):
- ctrl_health_output = self._storcli_exec(
- ["/c%d" % ctrl_num, 'show', 'health', 'all'])
- health_info = ctrl_health_output['Controller Health Info']
+ def _lsm_status_of_ctrl(self, ctrl_show_all_output):
+ lsi_status_info = ctrl_show_all_output['Status']
+ status_info = ''
+ status = System.STATUS_UNKNOWN
+ if lsi_status_info['Controller Status'] == 'Optimal':
+ status = System.STATUS_OK
+ else:
# TODO(Gris Ge): Try pull a disk off to check whether this change.
- if health_info['Overall Health'] == 'GOOD':
- return System.STATUS_OK, ''
-
- return System.STATUS_UNKNOWN, "%s reason code %s" % (
- health_info['Overall Health'],
- health_info['Reason Code'])
-
- def _sys_id_of_ctrl_num(self, ctrl_num, ctrl_show_output=None):
- if ctrl_show_output is None:
- ctrl_show_output = self._storcli_exec(["/c%d" % ctrl_num, "show"])
- sys_id = ctrl_show_output['Serial Number']
- if not sys_id:
- raise LsmError(
- ErrorNumber.PLUGIN_BUG,
- "_sys_id_of_ctrl_num(): Fail to get system id: %s" %
- ctrl_show_output.items())
+ status_info = "%s: " % lsi_status_info['Controller Status']
+ for key_name in lsi_status_info.keys():
+ if key_name == 'Controller Status':
+ continue
+ if lsi_status_info[key_name] != 0 and \
+ lsi_status_info[key_name] != 'No' and \
+ lsi_status_info[key_name] != 'NA':
+ status_info += " %s:%s" % (
+ key_name, lsi_status_info[key_name])
+
+ return status, status_info
+
+ def _sys_id_of_ctrl_num(self, ctrl_num, ctrl_show_all_output=None):
+ if ctrl_show_all_output is None:
+ return self._storcli_exec(
+ ["/c%d" % ctrl_num, "show"])['Serial Number']
else:
- return sys_id
+ return ctrl_show_all_output['Basics']['Serial Number']

@_handle_errors
def systems(self, flags=0):
rc_lsm_syss = []
for ctrl_num in range(self._ctrl_count()):
- ctrl_show_output = self._storcli_exec(["/c%d" % ctrl_num, "show"])
- sys_id = self._sys_id_of_ctrl_num(ctrl_num, ctrl_show_output)
- sys_name = "%s %s %s:%s:%s" % (
- ctrl_show_output['Product Name'],
- ctrl_show_output['Host Interface'],
- format(ctrl_show_output['Bus Number'], '02x'),
- format(ctrl_show_output['Device Number'], '02x'),
- format(ctrl_show_output['Function Number'], '02x'))
- (status, status_info) = self._lsm_status_of_ctrl(ctrl_num)
+ ctrl_show_all_output = self._storcli_exec(
+ ["/c%d" % ctrl_num, "show", "all"])
+ sys_id = self._sys_id_of_ctrl_num(ctrl_num, ctrl_show_all_output)
+ sys_name = "%s %s %s ver: %s" % (
+ ctrl_show_all_output['Basics']['Model'],
+ ctrl_show_all_output['Bus']['Host Interface'],
+ ctrl_show_all_output['Basics']['PCI Address'],
+ ctrl_show_all_output['Version']['Firmware Package Build'],
+ )
+ (status, status_info) = self._lsm_status_of_ctrl(
+ ctrl_show_all_output)
plugin_data = "/c%d"
# Since PCI slot sequence might change.
# This string just stored for quick system verification.
--
1.8.3.1
Gris Ge
2015-03-06 15:34:27 UTC
Permalink
* New status constant:
Disk.STATUS_FREE
LSM_DISK_STATUS_FREE

* In code document(Both in C and Python code):
New in version 1.2, indicate the whole disk is not holding any data or
acting as a dedicate spare disk.
This disk could be assigned as a dedicated spare disk or used for
creating pool.
If any spare disk(like those on NetApp ONTAP) does not require
any explicit action when assigning to pool, it should be treated as free
disk and marked as STATUS_FREE|STATUS_SPARE_DISK.

* Future design consideration for sliced RAID system(LVM RAID and EMC VMAX):
1. Introduce Disk.STATUS_FREE_PARTIAL
2. Introduce new class:
DiskSlice
properties:
id
owner_disk_id # Disk.id
block_size
block_count
start_block
status # FREE or SPARE or/and OK

Signed-off-by: Gris Ge <***@redhat.com>
---
c_binding/include/libstoragemgmt/libstoragemgmt_types.h | 10 ++++++++++
python_binding/lsm/_data.py | 8 ++++++++
2 files changed, 18 insertions(+)

diff --git a/c_binding/include/libstoragemgmt/libstoragemgmt_types.h b/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
index 562fcff..c5607c1 100644
--- a/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
+++ b/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
@@ -237,6 +237,16 @@ typedef enum {
#define LSM_DISK_STATUS_MAINTENANCE_MODE 0x0000000000000400
#define LSM_DISK_STATUS_SPARE_DISK 0x0000000000000800
#define LSM_DISK_STATUS_RECONSTRUCT 0x0000000000001000
+#define LSM_DISK_STATUS_FREE 0x0000000000002000
+/**^
+ * New in version 1.2, New in version 1.2, indicate the whole disk is not
+ * holding any data or acting as a dedicate spare disk.
+ * This disk could be assigned as a dedicated spare disk or used for creating
+ * pool.
+ * If any spare disk(like those on NetApp ONTAP) does not require any explicit
+ * action when assigning to pool, it should be treated as free disk and marked
+ * as LSM_DISK_STATUS_FREE|LSM_DISK_STATUS_SPARE_DISK.
+ * */

#define LSM_DISK_BLOCK_SIZE_NOT_FOUND -1
#define LSM_DISK_BLOCK_COUNT_NOT_FOUND -1
diff --git a/python_binding/lsm/_data.py b/python_binding/lsm/_data.py
index 6fb2325..23681dd 100644
--- a/python_binding/lsm/_data.py
+++ b/python_binding/lsm/_data.py
@@ -199,6 +199,14 @@ class Disk(IData):
# Indicate disk is a spare disk.
STATUS_RECONSTRUCT = 1 << 12
# Indicate disk is reconstructing data.
+ STATUS_FREE = 1 << 13
+ # New in version 1.2, indicate the whole disk is not holding any data or
+ # acting as a dedicate spare disk.
+ # This disk could be assigned as a dedicated spare disk or used for
+ # creating pool.
+ # If any spare disk(like those on NetApp ONTAP) does not require
+ # any explicit action when assigning to pool, it should be treated as
+ # free disk and marked as STATUS_FREE|STATUS_SPARE_DISK.

def __init__(self, _id, _name, _disk_type, _block_size, _num_of_blocks,
_status, _system_id, _plugin_data=None):
--
1.8.3.1
Gris Ge
2015-03-06 15:34:29 UTC
Permalink
* Treating sim_disk['role'] == None as Free disk.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/sim/simarray.py | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/plugin/sim/simarray.py b/plugin/sim/simarray.py
index 3fa7782..c3d2051 100644
--- a/plugin/sim/simarray.py
+++ b/plugin/sim/simarray.py
@@ -194,7 +194,7 @@ class BackStore(object):

DISK_KEY_LIST = [
'id', 'name', 'total_space', 'disk_type', 'status',
- 'owner_pool_id']
+ 'owner_pool_id', 'role']

VOL_KEY_LIST = [
'id', 'vpd83', 'name', 'total_space', 'consumed_size',
@@ -1762,12 +1762,17 @@ class SimArray(object):

@staticmethod
def _sim_disk_2_lsm(sim_disk):
+ print sim_disk.items()
+ disk_status = Disk.STATUS_OK
+ if sim_disk['role'] is None:
+ disk_status |= Disk.STATUS_FREE
+
return Disk(
SimArray._sim_id_to_lsm_id(sim_disk['id'], 'DISK'),
sim_disk['name'],
sim_disk['disk_type'], BackStore.BLK_SIZE,
int(sim_disk['total_space'] / BackStore.BLK_SIZE),
- Disk.STATUS_OK, BackStore.SYS_ID)
+ disk_status, BackStore.SYS_ID)

@_handle_errors
def disks(self):
--
1.8.3.1
Gris Ge
2015-03-06 15:34:30 UTC
Permalink
* Any disk not have 'aggregate' property is considered as free disk.

* Added comments indicate how spare and free disk works on ONTAP:
1. Spare == free disk
2. Zeroed spare disk == OK | SPARE | FREE
3. Non-zeroed spare disk == STOPPED | SPARE | FREE

Changes in V2:
* Should not mark partner filer's disks as free.
# Don't check status of partner's disks, simply mark as STATUS_OTHER.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/ontap/ontap.py | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/plugin/ontap/ontap.py b/plugin/ontap/ontap.py
index b6358a8..570833a 100644
--- a/plugin/ontap/ontap.py
+++ b/plugin/ontap/ontap.py
@@ -265,13 +265,17 @@ class Ontap(IStorageAreaNetwork, INfs):
if 'is-zeroed' in na_disk and na_disk['is-zeroed'] == 'true':
status |= Disk.STATUS_OK | Disk.STATUS_SPARE_DISK
else:
+ # If spare disk is not zerored, it will be automaticlly
+ # zeroed before assigned to aggregate.
+ # Hence we consider non-zeroed spare disks as stopped
+ # spare disks.
status |= Disk.STATUS_STOPPED | Disk.STATUS_SPARE_DISK
elif rs == 'present':
status |= Disk.STATUS_OK
elif rs == 'partner':
# Before we have good way to connect two controller,
# we have to mark partner disk as OTHER
- status |= Disk.STATUS_OTHER
+ return Disk.STATUS_OTHER

if 'is-prefailed' in na_disk and na_disk['is-prefailed'] == 'true':
status |= Disk.STATUS_STOPPING
@@ -279,6 +283,12 @@ class Ontap(IStorageAreaNetwork, INfs):
if 'is-offline' in na_disk and na_disk['is-offline'] == 'true':
status |= Disk.STATUS_ERROR

+ if 'aggregate' not in na_disk:
+ # All free disks are automatically marked as spare disks. They
+ # could easily convert to data or parity disk without any
+ # explicit command.
+ status |= Disk.STATUS_FREE
+
if status == 0:
status = Disk.STATUS_UNKNOWN
--
1.8.3.1
Gris Ge
2015-03-06 15:34:28 UTC
Permalink
* Show Disk.STATUS_FREE as 'Free' in output.

Signed-off-by: Gris Ge <***@redhat.com>
---
tools/lsmcli/data_display.py | 1 +
1 file changed, 1 insertion(+)

diff --git a/tools/lsmcli/data_display.py b/tools/lsmcli/data_display.py
index e0524c8..19f7114 100644
--- a/tools/lsmcli/data_display.py
+++ b/tools/lsmcli/data_display.py
@@ -201,6 +201,7 @@ _DISK_STATUS_CONV = {
Disk.STATUS_MAINTENANCE_MODE: 'Maintenance',
Disk.STATUS_SPARE_DISK: 'Spare',
Disk.STATUS_RECONSTRUCT: 'Reconstruct',
+ Disk.STATUS_FREE: 'Free',
}
--
1.8.3.1
Gris Ge
2015-03-06 15:34:31 UTC
Permalink
* Treat 'UGood' and 'UBad' as free disk.

* Update algorithm of disk status query to support show spare disk with
sector error as SPARE|ERROR

Changes in V2:

* Fix the incorrect data assign:
disk_status == Disk.STATUS_UNKNOWN
Should be:
disk_status = Disk.STATUS_UNKNOWN

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/megaraid/megaraid.py | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/plugin/megaraid/megaraid.py b/plugin/megaraid/megaraid.py
index 188ba58..a2d1440 100644
--- a/plugin/megaraid/megaraid.py
+++ b/plugin/megaraid/megaraid.py
@@ -91,29 +91,36 @@ _DISK_STATE_MAP = {
'Offln': Disk.STATUS_ERROR,
'GHS': Disk.STATUS_SPARE_DISK | Disk.STATUS_OK,
'DHS': Disk.STATUS_SPARE_DISK | Disk.STATUS_OK,
- 'UGood': Disk.STATUS_STOPPED | Disk.STATUS_OK,
- 'UBad': Disk.STATUS_STOPPED | Disk.STATUS_ERROR,
+ 'UGood': Disk.STATUS_FREE | Disk.STATUS_OK,
+ 'UBad': Disk.STATUS_FREE | Disk.STATUS_ERROR,
'Rbld': Disk.STATUS_RECONSTRUCT,
}


def _disk_status_of(disk_show_basic_dict, disk_show_stat_dict):
+ disk_status = _DISK_STATE_MAP.get(
+ disk_show_basic_dict['State'], 0)
+
if disk_show_stat_dict['Media Error Count'] or \
disk_show_stat_dict['Other Error Count'] or \
disk_show_stat_dict['S.M.A.R.T alert flagged by drive'] != 'No':
- return Disk.STATUS_ERROR
+ disk_status -= Disk.STATUS_OK
+ disk_status |= Disk.STATUS_ERROR

- if disk_show_stat_dict['Predictive Failure Count']:
- return Disk.STATUS_PREDICTIVE_FAILURE
+ elif disk_show_stat_dict['Predictive Failure Count']:
+ disk_status -= Disk.STATUS_OK
+ disk_status |= Disk.STATUS_PREDICTIVE_FAILURE

if disk_show_basic_dict['Sp'] == 'D':
- return Disk.STATUS_STOPPED
+ disk_status |= Disk.STATUS_STOPPED

if disk_show_basic_dict['Sp'] == 'F':
- return Disk.STATUS_OTHER
+ disk_status |= Disk.STATUS_OTHER
+
+ if disk_status == 0:
+ disk_status = Disk.STATUS_UNKNOWN

- return _DISK_STATE_MAP.get(
- disk_show_basic_dict['State'], Disk.STATUS_UNKNOWN)
+ return disk_status


def _mega_size_to_lsm(mega_size):
--
1.8.3.1
Gris Ge
2015-03-06 15:34:32 UTC
Permalink
* Using SNIA SMI-S 1.4 'Disk Sparing' profile to set Disk.STATUS_SPARE_DISK.
# Works well on EMC VMAX, Fujitsu ETERNUS, LSI MegaRAID.
# They are only array have spare disks I got access.

* Use vendor specific way to set Disk.STATUS_FREE:
1. EMC:
'EMCInUse' property of CIM_DiskDrive. False == Free.
# Tested on EMC VNX.

2. LSI MegaRAID:
StoragePool_InstanceID of CIM_StorageExtent,
if not contain 'Pool' keyword, it could be free disk or spare disk.
Then filter out spare disk, we got free disks.
# Tested on LSI MegaRAID.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/smispy/smis_disk.py | 28 ++++++++++++++++++++++++++--
1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/plugin/smispy/smis_disk.py b/plugin/smispy/smis_disk.py
index 6b476f1..0321bd1 100644
--- a/plugin/smispy/smis_disk.py
+++ b/plugin/smispy/smis_disk.py
@@ -77,10 +77,11 @@ def cim_disk_pros():
"""
Return all CIM_DiskDrive Properties needed to create a Disk object.
The 'Type' and 'MediaType' is only for MegaRAID.
+ The 'EMCInUse' is only for EMC.
"""
return ['OperationalStatus', 'Name', 'SystemName',
'Caption', 'InterconnectType', 'DiskType', 'DeviceID',
- 'Type', 'MediaType']
+ 'Type', 'MediaType', 'EMCInUse']


def sys_id_of_cim_disk(cim_disk):
@@ -173,11 +174,34 @@ def cim_disk_to_lsm_disk(smis_common, cim_disk):
"""
# CIM_DiskDrive does not have disk size information.
# We have to find out the Primordial CIM_StorageExtent for that.
+ # The 'StoragePool_InstanceID' is for LSI MegaRAID only.
cim_ext = _pri_cim_ext_of_cim_disk(
smis_common, cim_disk.path,
- property_list=['BlockSize', 'NumberOfBlocks'])
+ property_list=[
+ 'BlockSize', 'NumberOfBlocks', 'StoragePool_InstanceID'])

status = _disk_status_of_cim_disk(cim_disk)
+ # Check spare disk, LSI MegaRAID use dedicate spare disk.
+ cim_srss = smis_common.AssociatorNames(
+ cim_ext.path, AssocClass='CIM_IsSpare',
+ ResultClass='CIM_StorageRedundancySet')
+ if len(cim_srss) >= 1:
+ status |= Disk.STATUS_SPARE_DISK
+
+ if smis_common.is_megaraid():
+ # Dirty hack:
+ # On LSI MegaRAID, free or spare disks will assigned to a internal
+ # storage pool with instance ID like '500605B002884430_SASHDD'
+ # In stread of find out all valid storage pool, we just
+ # assume valid pool of LSI holds 'Pool' keywords in DeviceID.
+ if 'Pool' not in cim_ext['StoragePool_InstanceID']:
+ # Check spare disk, LSI MegaRAID use dedicate spare disk.
+ if not status & Disk.STATUS_SPARE_DISK:
+ status |= Disk.STATUS_FREE
+
+ elif 'EMCInUse' in cim_disk.keys() and cim_disk['EMCInUse'] is False:
+ status |= Disk.STATUS_FREE
+
name = ''
block_size = Disk.BLOCK_SIZE_NOT_FOUND
num_of_block = Disk.BLOCK_COUNT_NOT_FOUND
--
1.8.3.1
Tony Asleson
2015-03-10 22:39:21 UTC
Permalink
Hi Gris,

Some comments:

- simarray.py as a print statement in it.
- When we add support for making a pool from part of a disk as some
arrays allow, will be adding another new flag to indicate that the disk
contains data, but can be used in a pool create?

Thanks,
Tony
Post by Gris Ge
* This patch set introduced new disk status -- STATUS_FREE to indicate
disk could be used for pool or assign as dedicated spare disk.
* Patch 6/7: Fix incorrect data assignment in MegaRAID plugin.
* Patch 5/7: Don't mark partner's disks as free disk in ONTAP plugin..
* Patch 7/7: Include SMI-S plugin support for STATUS_FREE using
vendor specific ways. Also come with STATUS_SPARE_DISK support.
MegaRAID: Fix incorrect system status query.
API: New disk status DISK.STATUS_FREE/LSM_DISK_STATUS_FREE
lsmcli: Add support of Disk.STATUS_FREE
Simulator Plugin: Support Disk.STATUS_FREE
ONTAP Plugin: Add support of Disk.STATUS_FREE
MegaRAID Plugin: Add support of Disk.STATUS_FREE
SMI-S Plugin: Add Disk.STATUS_SPARE_DISK and Disk.STATUS_FREE
.../include/libstoragemgmt/libstoragemgmt_types.h | 10 +++
plugin/megaraid/megaraid.py | 90 ++++++++++++----------
plugin/ontap/ontap.py | 12 ++-
plugin/sim/simarray.py | 9 ++-
plugin/smispy/smis_disk.py | 28 ++++++-
python_binding/lsm/_data.py | 8 ++
tools/lsmcli/data_display.py | 1 +
7 files changed, 114 insertions(+), 44 deletions(-)
Gris Ge
2015-03-11 07:42:28 UTC
Permalink
Post by Tony Asleson
Hi Gris,
Hi Tony,
Post by Tony Asleson
- simarray.py as a print statement in it.
Thanks for the review.
Will fix it in V3 patchset including hpsa support.
Post by Tony Asleson
- When we add support for making a pool from part of a disk as some
arrays allow, will be adding another new flag to indicate that the disk
contains data, but can be used in a pool create?
It would be Disk.STATUS_FREE_PARTIAL, I would suggest we add it when
we really have a plugin supporting disk slice RAID in stead of adding
it now.
Post by Tony Asleson
Thanks,
Tony
--
Gris Ge
Gris Ge
2015-03-14 13:03:58 UTC
Permalink
* This patch set introduced new disk status -- STATUS_FREE to indicate
disk could be used for pool or assign as dedicated spare disk.

Changes in V2:

* Patch 6/7: Fix incorrect data assignment in MegaRAID plugin.
* Patch 5/7: Don't mark partner's disks as free disk in ONTAP plugin..
* Patch 7/7: Include SMI-S plugin support for STATUS_FREE using
vendor specific ways. Also come with STATUS_SPARE_DISK support.

Changes in V3:
* Patch 6/9: Remove the debug print line in simulator plugin.
* Patch 1/9: New patch. Change currnt git tree version as 1.2.
* Patch 2/9: New patch. Fix missing VOLUME_RAID_INFO capability of megaraid
plugin.

Gris Ge (9):
Versioning: Change current git tree to version 1.2
MegaRAID: Add missing capability VOLUME_RAID_INFO
MegaRAID: Fix incorrect system status query.
API: New disk status DISK.STATUS_FREE/LSM_DISK_STATUS_FREE
lsmcli: Add support of Disk.STATUS_FREE
Simulator Plugin: Support Disk.STATUS_FREE
ONTAP Plugin: Add support of Disk.STATUS_FREE
MegaRAID Plugin: Add support of Disk.STATUS_FREE
SMI-S Plugin: Add Disk.STATUS_SPARE_DISK and Disk.STATUS_FREE

.../include/libstoragemgmt/libstoragemgmt_types.h | 10 +++
configure.ac | 2 +-
plugin/megaraid/megaraid.py | 91 ++++++++++++----------
plugin/ontap/ontap.py | 12 ++-
plugin/sim/simarray.py | 8 +-
plugin/smispy/smis_disk.py | 28 ++++++-
python_binding/lsm/_data.py | 8 ++
tools/lsmcli/data_display.py | 1 +
8 files changed, 115 insertions(+), 45 deletions(-)
--
1.8.3.1
Gris Ge
2015-03-14 13:03:59 UTC
Permalink
* This make sure 'make rpm' generated rpms with newer version number
compare to stable rpms provided by OS.

Signed-off-by: Gris Ge <***@redhat.com>
---
configure.ac | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 74e6811..db657bb 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,7 @@ dnl Process this file with autoconf to produce a configure script.
dnl Copyright (C) 2011 Red Hat, Inc.
dnl See COPYING.LIB for the License of this software

-AC_INIT([libstoragemgmt], [1.1.0], [libstoragemgmt-***@lists.sourceforge.net], [], [http://sourceforge.net/projects/libstoragemgmt/])
+AC_INIT([libstoragemgmt], [1.2.0], [libstoragemgmt-***@lists.sourceforge.net], [], [http://sourceforge.net/projects/libstoragemgmt/])
AC_CONFIG_SRCDIR([configure.ac])
AC_CONFIG_AUX_DIR([build-aux])
AC_CONFIG_HEADERS([config.h])
--
1.8.3.1
Gris Ge
2015-03-14 13:04:00 UTC
Permalink
* Add missing lsm.Capabilities.VOLUME_RAID_INFO.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/megaraid/megaraid.py | 1 +
1 file changed, 1 insertion(+)

diff --git a/plugin/megaraid/megaraid.py b/plugin/megaraid/megaraid.py
index e754cd8..beb68e8 100644
--- a/plugin/megaraid/megaraid.py
+++ b/plugin/megaraid/megaraid.py
@@ -255,6 +255,7 @@ def capabilities(self, system, flags=_FLAG_RSVD):
cap = Capabilities()
cap.set(Capabilities.DISKS)
cap.set(Capabilities.VOLUMES)
+ cap.set(Capabilities.VOLUME_RAID_INFO)
return cap

def _storcli_exec(self, storcli_cmds, flag_json=True):
--
1.8.3.1
Gris Ge
2015-03-14 13:04:01 UTC
Permalink
* The storcli-1.14.12-1.noarch only support 'show health' command for
Nytro MegaRAID card. This patch change system status query using
'show all' command.

* Use single command 'storcli /c0 show all J' to get all information
to generate lsm.System.

* Status information is extracted from 'Status' section of 'show all' output.

* Append firmware version(firmware package build version) in System.name.
User could compare this version string with LSI webpage for upgrade.

* Tested on Intel and IBM rebranded MegaRAID cards.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/megaraid/megaraid.py | 65 ++++++++++++++++++++++++---------------------
1 file changed, 35 insertions(+), 30 deletions(-)

diff --git a/plugin/megaraid/megaraid.py b/plugin/megaraid/megaraid.py
index beb68e8..111e223 100644
--- a/plugin/megaraid/megaraid.py
+++ b/plugin/megaraid/megaraid.py
@@ -300,43 +300,48 @@ def _ctrl_count(self):
return self._storcli_exec(
["show", "ctrlcount"]).get("Controller Count")

- def _lsm_status_of_ctrl(self, ctrl_num):
- ctrl_health_output = self._storcli_exec(
- ["/c%d" % ctrl_num, 'show', 'health', 'all'])
- health_info = ctrl_health_output['Controller Health Info']
+ def _lsm_status_of_ctrl(self, ctrl_show_all_output):
+ lsi_status_info = ctrl_show_all_output['Status']
+ status_info = ''
+ status = System.STATUS_UNKNOWN
+ if lsi_status_info['Controller Status'] == 'Optimal':
+ status = System.STATUS_OK
+ else:
# TODO(Gris Ge): Try pull a disk off to check whether this change.
- if health_info['Overall Health'] == 'GOOD':
- return System.STATUS_OK, ''
-
- return System.STATUS_UNKNOWN, "%s reason code %s" % (
- health_info['Overall Health'],
- health_info['Reason Code'])
-
- def _sys_id_of_ctrl_num(self, ctrl_num, ctrl_show_output=None):
- if ctrl_show_output is None:
- ctrl_show_output = self._storcli_exec(["/c%d" % ctrl_num, "show"])
- sys_id = ctrl_show_output['Serial Number']
- if not sys_id:
- raise LsmError(
- ErrorNumber.PLUGIN_BUG,
- "_sys_id_of_ctrl_num(): Fail to get system id: %s" %
- ctrl_show_output.items())
+ status_info = "%s: " % lsi_status_info['Controller Status']
+ for key_name in lsi_status_info.keys():
+ if key_name == 'Controller Status':
+ continue
+ if lsi_status_info[key_name] != 0 and \
+ lsi_status_info[key_name] != 'No' and \
+ lsi_status_info[key_name] != 'NA':
+ status_info += " %s:%s" % (
+ key_name, lsi_status_info[key_name])
+
+ return status, status_info
+
+ def _sys_id_of_ctrl_num(self, ctrl_num, ctrl_show_all_output=None):
+ if ctrl_show_all_output is None:
+ return self._storcli_exec(
+ ["/c%d" % ctrl_num, "show"])['Serial Number']
else:
- return sys_id
+ return ctrl_show_all_output['Basics']['Serial Number']

@_handle_errors
def systems(self, flags=0):
rc_lsm_syss = []
for ctrl_num in range(self._ctrl_count()):
- ctrl_show_output = self._storcli_exec(["/c%d" % ctrl_num, "show"])
- sys_id = self._sys_id_of_ctrl_num(ctrl_num, ctrl_show_output)
- sys_name = "%s %s %s:%s:%s" % (
- ctrl_show_output['Product Name'],
- ctrl_show_output['Host Interface'],
- format(ctrl_show_output['Bus Number'], '02x'),
- format(ctrl_show_output['Device Number'], '02x'),
- format(ctrl_show_output['Function Number'], '02x'))
- (status, status_info) = self._lsm_status_of_ctrl(ctrl_num)
+ ctrl_show_all_output = self._storcli_exec(
+ ["/c%d" % ctrl_num, "show", "all"])
+ sys_id = self._sys_id_of_ctrl_num(ctrl_num, ctrl_show_all_output)
+ sys_name = "%s %s %s ver: %s" % (
+ ctrl_show_all_output['Basics']['Model'],
+ ctrl_show_all_output['Bus']['Host Interface'],
+ ctrl_show_all_output['Basics']['PCI Address'],
+ ctrl_show_all_output['Version']['Firmware Package Build'],
+ )
+ (status, status_info) = self._lsm_status_of_ctrl(
+ ctrl_show_all_output)
plugin_data = "/c%d"
# Since PCI slot sequence might change.
# This string just stored for quick system verification.
--
1.8.3.1
Gris Ge
2015-03-14 13:04:02 UTC
Permalink
* New status constant:
Disk.STATUS_FREE
LSM_DISK_STATUS_FREE

* In code document(Both in C and Python code):
New in version 1.2, indicate the whole disk is not holding any data or
acting as a dedicate spare disk.
This disk could be assigned as a dedicated spare disk or used for
creating pool.
If any spare disk(like those on NetApp ONTAP) does not require
any explicit action when assigning to pool, it should be treated as free
disk and marked as STATUS_FREE|STATUS_SPARE_DISK.

* Future design consideration for sliced RAID system(LVM RAID and EMC VMAX):
1. Introduce Disk.STATUS_FREE_PARTIAL
2. Introduce new class:
DiskSlice
properties:
id
owner_disk_id # Disk.id
block_size
block_count
start_block
status # FREE or SPARE or/and OK

Signed-off-by: Gris Ge <***@redhat.com>
---
c_binding/include/libstoragemgmt/libstoragemgmt_types.h | 10 ++++++++++
python_binding/lsm/_data.py | 8 ++++++++
2 files changed, 18 insertions(+)

diff --git a/c_binding/include/libstoragemgmt/libstoragemgmt_types.h b/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
index 562fcff..c5607c1 100644
--- a/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
+++ b/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
@@ -237,6 +237,16 @@ typedef enum {
#define LSM_DISK_STATUS_MAINTENANCE_MODE 0x0000000000000400
#define LSM_DISK_STATUS_SPARE_DISK 0x0000000000000800
#define LSM_DISK_STATUS_RECONSTRUCT 0x0000000000001000
+#define LSM_DISK_STATUS_FREE 0x0000000000002000
+/**^
+ * New in version 1.2, New in version 1.2, indicate the whole disk is not
+ * holding any data or acting as a dedicate spare disk.
+ * This disk could be assigned as a dedicated spare disk or used for creating
+ * pool.
+ * If any spare disk(like those on NetApp ONTAP) does not require any explicit
+ * action when assigning to pool, it should be treated as free disk and marked
+ * as LSM_DISK_STATUS_FREE|LSM_DISK_STATUS_SPARE_DISK.
+ * */

#define LSM_DISK_BLOCK_SIZE_NOT_FOUND -1
#define LSM_DISK_BLOCK_COUNT_NOT_FOUND -1
diff --git a/python_binding/lsm/_data.py b/python_binding/lsm/_data.py
index 6fb2325..23681dd 100644
--- a/python_binding/lsm/_data.py
+++ b/python_binding/lsm/_data.py
@@ -199,6 +199,14 @@ class Disk(IData):
# Indicate disk is a spare disk.
STATUS_RECONSTRUCT = 1 << 12
# Indicate disk is reconstructing data.
+ STATUS_FREE = 1 << 13
+ # New in version 1.2, indicate the whole disk is not holding any data or
+ # acting as a dedicate spare disk.
+ # This disk could be assigned as a dedicated spare disk or used for
+ # creating pool.
+ # If any spare disk(like those on NetApp ONTAP) does not require
+ # any explicit action when assigning to pool, it should be treated as
+ # free disk and marked as STATUS_FREE|STATUS_SPARE_DISK.

def __init__(self, _id, _name, _disk_type, _block_size, _num_of_blocks,
_status, _system_id, _plugin_data=None):
--
1.8.3.1
Gris Ge
2015-03-14 13:04:03 UTC
Permalink
* Show Disk.STATUS_FREE as 'Free' in output.

Signed-off-by: Gris Ge <***@redhat.com>
---
tools/lsmcli/data_display.py | 1 +
1 file changed, 1 insertion(+)

diff --git a/tools/lsmcli/data_display.py b/tools/lsmcli/data_display.py
index e0524c8..19f7114 100644
--- a/tools/lsmcli/data_display.py
+++ b/tools/lsmcli/data_display.py
@@ -201,6 +201,7 @@ def disk_type_to_str(disk_type):
Disk.STATUS_MAINTENANCE_MODE: 'Maintenance',
Disk.STATUS_SPARE_DISK: 'Spare',
Disk.STATUS_RECONSTRUCT: 'Reconstruct',
+ Disk.STATUS_FREE: 'Free',
}
--
1.8.3.1
Gris Ge
2015-03-14 13:04:04 UTC
Permalink
* Treating sim_disk['role'] == None as Free disk.

Changes in V3(No changes in V2):

* Remove debug code of 'print sim_disk.items()'.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/sim/simarray.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/plugin/sim/simarray.py b/plugin/sim/simarray.py
index 3fa7782..ae14848 100644
--- a/plugin/sim/simarray.py
+++ b/plugin/sim/simarray.py
@@ -194,7 +194,7 @@ class BackStore(object):

DISK_KEY_LIST = [
'id', 'name', 'total_space', 'disk_type', 'status',
- 'owner_pool_id']
+ 'owner_pool_id', 'role']

VOL_KEY_LIST = [
'id', 'vpd83', 'name', 'total_space', 'consumed_size',
@@ -1762,12 +1762,16 @@ def pools(self, flags=0):

@staticmethod
def _sim_disk_2_lsm(sim_disk):
+ disk_status = Disk.STATUS_OK
+ if sim_disk['role'] is None:
+ disk_status |= Disk.STATUS_FREE
+
return Disk(
SimArray._sim_id_to_lsm_id(sim_disk['id'], 'DISK'),
sim_disk['name'],
sim_disk['disk_type'], BackStore.BLK_SIZE,
int(sim_disk['total_space'] / BackStore.BLK_SIZE),
- Disk.STATUS_OK, BackStore.SYS_ID)
+ disk_status, BackStore.SYS_ID)

@_handle_errors
def disks(self):
--
1.8.3.1
Gris Ge
2015-03-14 13:04:05 UTC
Permalink
* Any disk not have 'aggregate' property is considered as free disk.

* Added comments indicate how spare and free disk works on ONTAP:
1. Spare == free disk
2. Zeroed spare disk == OK | SPARE | FREE
3. Non-zeroed spare disk == STOPPED | SPARE | FREE

Changes in V2:
* Should not mark partner filer's disks as free.
# Don't check status of partner's disks, simply mark as STATUS_OTHER.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/ontap/ontap.py | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/plugin/ontap/ontap.py b/plugin/ontap/ontap.py
index b6358a8..570833a 100644
--- a/plugin/ontap/ontap.py
+++ b/plugin/ontap/ontap.py
@@ -265,13 +265,17 @@ def _status_of_na_disk(na_disk):
if 'is-zeroed' in na_disk and na_disk['is-zeroed'] == 'true':
status |= Disk.STATUS_OK | Disk.STATUS_SPARE_DISK
else:
+ # If spare disk is not zerored, it will be automaticlly
+ # zeroed before assigned to aggregate.
+ # Hence we consider non-zeroed spare disks as stopped
+ # spare disks.
status |= Disk.STATUS_STOPPED | Disk.STATUS_SPARE_DISK
elif rs == 'present':
status |= Disk.STATUS_OK
elif rs == 'partner':
# Before we have good way to connect two controller,
# we have to mark partner disk as OTHER
- status |= Disk.STATUS_OTHER
+ return Disk.STATUS_OTHER

if 'is-prefailed' in na_disk and na_disk['is-prefailed'] == 'true':
status |= Disk.STATUS_STOPPING
@@ -279,6 +283,12 @@ def _status_of_na_disk(na_disk):
if 'is-offline' in na_disk and na_disk['is-offline'] == 'true':
status |= Disk.STATUS_ERROR

+ if 'aggregate' not in na_disk:
+ # All free disks are automatically marked as spare disks. They
+ # could easily convert to data or parity disk without any
+ # explicit command.
+ status |= Disk.STATUS_FREE
+
if status == 0:
status = Disk.STATUS_UNKNOWN
--
1.8.3.1
Gris Ge
2015-03-14 13:04:06 UTC
Permalink
* Treat 'UGood' and 'UBad' as free disk.

* Update algorithm of disk status query to support show spare disk with
sector error as SPARE|ERROR

Changes in V2:

* Fix the incorrect data assign:
disk_status == Disk.STATUS_UNKNOWN
Should be:
disk_status = Disk.STATUS_UNKNOWN

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/megaraid/megaraid.py | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/plugin/megaraid/megaraid.py b/plugin/megaraid/megaraid.py
index 111e223..30d48dc 100644
--- a/plugin/megaraid/megaraid.py
+++ b/plugin/megaraid/megaraid.py
@@ -91,29 +91,36 @@ def _disk_type_of(disk_show_basic_dict):
'Offln': Disk.STATUS_ERROR,
'GHS': Disk.STATUS_SPARE_DISK | Disk.STATUS_OK,
'DHS': Disk.STATUS_SPARE_DISK | Disk.STATUS_OK,
- 'UGood': Disk.STATUS_STOPPED | Disk.STATUS_OK,
- 'UBad': Disk.STATUS_STOPPED | Disk.STATUS_ERROR,
+ 'UGood': Disk.STATUS_FREE | Disk.STATUS_OK,
+ 'UBad': Disk.STATUS_FREE | Disk.STATUS_ERROR,
'Rbld': Disk.STATUS_RECONSTRUCT,
}


def _disk_status_of(disk_show_basic_dict, disk_show_stat_dict):
+ disk_status = _DISK_STATE_MAP.get(
+ disk_show_basic_dict['State'], 0)
+
if disk_show_stat_dict['Media Error Count'] or \
disk_show_stat_dict['Other Error Count'] or \
disk_show_stat_dict['S.M.A.R.T alert flagged by drive'] != 'No':
- return Disk.STATUS_ERROR
+ disk_status -= Disk.STATUS_OK
+ disk_status |= Disk.STATUS_ERROR

- if disk_show_stat_dict['Predictive Failure Count']:
- return Disk.STATUS_PREDICTIVE_FAILURE
+ elif disk_show_stat_dict['Predictive Failure Count']:
+ disk_status -= Disk.STATUS_OK
+ disk_status |= Disk.STATUS_PREDICTIVE_FAILURE

if disk_show_basic_dict['Sp'] == 'D':
- return Disk.STATUS_STOPPED
+ disk_status |= Disk.STATUS_STOPPED

if disk_show_basic_dict['Sp'] == 'F':
- return Disk.STATUS_OTHER
+ disk_status |= Disk.STATUS_OTHER
+
+ if disk_status == 0:
+ disk_status = Disk.STATUS_UNKNOWN

- return _DISK_STATE_MAP.get(
- disk_show_basic_dict['State'], Disk.STATUS_UNKNOWN)
+ return disk_status


def _mega_size_to_lsm(mega_size):
--
1.8.3.1
Gris Ge
2015-03-14 13:04:07 UTC
Permalink
* Using SNIA SMI-S 1.4 'Disk Sparing' profile to set Disk.STATUS_SPARE_DISK.
# Works well on EMC VMAX, Fujitsu ETERNUS, LSI MegaRAID.
# They are only array have spare disks I got access.

* Use vendor specific way to set Disk.STATUS_FREE:
1. EMC:
'EMCInUse' property of CIM_DiskDrive. False == Free.
# Tested on EMC VNX.

2. LSI MegaRAID:
StoragePool_InstanceID of CIM_StorageExtent,
if not contain 'Pool' keyword, it could be free disk or spare disk.
Then filter out spare disk, we got free disks.
# Tested on LSI MegaRAID.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/smispy/smis_disk.py | 28 ++++++++++++++++++++++++++--
1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/plugin/smispy/smis_disk.py b/plugin/smispy/smis_disk.py
index 6b476f1..0321bd1 100644
--- a/plugin/smispy/smis_disk.py
+++ b/plugin/smispy/smis_disk.py
@@ -77,10 +77,11 @@ def cim_disk_pros():
"""
Return all CIM_DiskDrive Properties needed to create a Disk object.
The 'Type' and 'MediaType' is only for MegaRAID.
+ The 'EMCInUse' is only for EMC.
"""
return ['OperationalStatus', 'Name', 'SystemName',
'Caption', 'InterconnectType', 'DiskType', 'DeviceID',
- 'Type', 'MediaType']
+ 'Type', 'MediaType', 'EMCInUse']


def sys_id_of_cim_disk(cim_disk):
@@ -173,11 +174,34 @@ def cim_disk_to_lsm_disk(smis_common, cim_disk):
"""
# CIM_DiskDrive does not have disk size information.
# We have to find out the Primordial CIM_StorageExtent for that.
+ # The 'StoragePool_InstanceID' is for LSI MegaRAID only.
cim_ext = _pri_cim_ext_of_cim_disk(
smis_common, cim_disk.path,
- property_list=['BlockSize', 'NumberOfBlocks'])
+ property_list=[
+ 'BlockSize', 'NumberOfBlocks', 'StoragePool_InstanceID'])

status = _disk_status_of_cim_disk(cim_disk)
+ # Check spare disk, LSI MegaRAID use dedicate spare disk.
+ cim_srss = smis_common.AssociatorNames(
+ cim_ext.path, AssocClass='CIM_IsSpare',
+ ResultClass='CIM_StorageRedundancySet')
+ if len(cim_srss) >= 1:
+ status |= Disk.STATUS_SPARE_DISK
+
+ if smis_common.is_megaraid():
+ # Dirty hack:
+ # On LSI MegaRAID, free or spare disks will assigned to a internal
+ # storage pool with instance ID like '500605B002884430_SASHDD'
+ # In stread of find out all valid storage pool, we just
+ # assume valid pool of LSI holds 'Pool' keywords in DeviceID.
+ if 'Pool' not in cim_ext['StoragePool_InstanceID']:
+ # Check spare disk, LSI MegaRAID use dedicate spare disk.
+ if not status & Disk.STATUS_SPARE_DISK:
+ status |= Disk.STATUS_FREE
+
+ elif 'EMCInUse' in cim_disk.keys() and cim_disk['EMCInUse'] is False:
+ status |= Disk.STATUS_FREE
+
name = ''
block_size = Disk.BLOCK_SIZE_NOT_FOUND
num_of_block = Disk.BLOCK_COUNT_NOT_FOUND
--
1.8.3.1
Tony Asleson
2015-03-16 20:25:44 UTC
Permalink
Hi Gris,

Sorry, I forgot to ask this before, but why are you making changes to
support megaraid SMI-S provider when we now have a specific plugin for them?

Thanks,
Tony
Post by Gris Ge
* Using SNIA SMI-S 1.4 'Disk Sparing' profile to set Disk.STATUS_SPARE_DISK.
# Works well on EMC VMAX, Fujitsu ETERNUS, LSI MegaRAID.
# They are only array have spare disks I got access.
'EMCInUse' property of CIM_DiskDrive. False == Free.
# Tested on EMC VNX.
StoragePool_InstanceID of CIM_StorageExtent,
if not contain 'Pool' keyword, it could be free disk or spare disk.
Then filter out spare disk, we got free disks.
# Tested on LSI MegaRAID.
---
plugin/smispy/smis_disk.py | 28 ++++++++++++++++++++++++++--
1 file changed, 26 insertions(+), 2 deletions(-)
diff --git a/plugin/smispy/smis_disk.py b/plugin/smispy/smis_disk.py
index 6b476f1..0321bd1 100644
--- a/plugin/smispy/smis_disk.py
+++ b/plugin/smispy/smis_disk.py
"""
Return all CIM_DiskDrive Properties needed to create a Disk object.
The 'Type' and 'MediaType' is only for MegaRAID.
+ The 'EMCInUse' is only for EMC.
"""
return ['OperationalStatus', 'Name', 'SystemName',
'Caption', 'InterconnectType', 'DiskType', 'DeviceID',
- 'Type', 'MediaType']
+ 'Type', 'MediaType', 'EMCInUse']
"""
# CIM_DiskDrive does not have disk size information.
# We have to find out the Primordial CIM_StorageExtent for that.
+ # The 'StoragePool_InstanceID' is for LSI MegaRAID only.
cim_ext = _pri_cim_ext_of_cim_disk(
smis_common, cim_disk.path,
- property_list=['BlockSize', 'NumberOfBlocks'])
+ property_list=[
+ 'BlockSize', 'NumberOfBlocks', 'StoragePool_InstanceID'])
status = _disk_status_of_cim_disk(cim_disk)
+ # Check spare disk, LSI MegaRAID use dedicate spare disk.
+ cim_srss = smis_common.AssociatorNames(
+ cim_ext.path, AssocClass='CIM_IsSpare',
+ ResultClass='CIM_StorageRedundancySet')
+ status |= Disk.STATUS_SPARE_DISK
+
+ # On LSI MegaRAID, free or spare disks will assigned to a internal
+ # storage pool with instance ID like '500605B002884430_SASHDD'
+ # In stread of find out all valid storage pool, we just
+ # assume valid pool of LSI holds 'Pool' keywords in DeviceID.
+ # Check spare disk, LSI MegaRAID use dedicate spare disk.
+ status |= Disk.STATUS_FREE
+
+ status |= Disk.STATUS_FREE
+
name = ''
block_size = Disk.BLOCK_SIZE_NOT_FOUND
num_of_block = Disk.BLOCK_COUNT_NOT_FOUND
Gris Ge
2015-03-17 07:54:08 UTC
Permalink
Post by Tony Asleson
Hi Gris,
Sorry, I forgot to ask this before, but why are you making changes to
support megaraid SMI-S provider when we now have a specific plugin for them?
Thanks,
Tony
Hi Tony,

Just in case some user want to use LSI SMI-S provider for remote monitoring.
As long as that's not a big workload, I am willing to keep maintaining
LSI SMI-S provider.

Considering we offer limited support of LSI SMI-S provider, I am OK if
you want me to drop it or put it into maintenance only mode.

Thanks.
--
Gris Ge
Gris Ge
2015-03-19 08:08:42 UTC
Permalink
* This patch set introduced new disk status -- STATUS_FREE to indicate
disk could be used for pool or assign as dedicated spare disk.

Changes in V2:

* Patch 6/7: Fix incorrect data assignment in MegaRAID plugin.
* Patch 5/7: Don't mark partner's disks as free disk in ONTAP plugin..
* Patch 7/7: Include SMI-S plugin support for STATUS_FREE using
vendor specific ways. Also come with STATUS_SPARE_DISK support.

Changes in V3:
* Patch 6/9: Remove the debug print line in simulator plugin.
* Patch 1/9: New patch. Change currnt git tree version as 1.2.
* Patch 2/9: New patch. Fix missing VOLUME_RAID_INFO capability of megaraid
plugin.

Changes in V4:
* Removed unrelated patches and submitted them in separate patch sets.
* Removed LSI MegaRAID SMI-S Disk.STATUS_FREE code as we have dedicate
MegaRAID plugin now.

Gris Ge (6):
API: New disk status DISK.STATUS_FREE/LSM_DISK_STATUS_FREE
lsmcli: Add support of Disk.STATUS_FREE
Simulator Plugin: Support Disk.STATUS_FREE
ONTAP Plugin: Add support of Disk.STATUS_FREE
MegaRAID Plugin: Add support of Disk.STATUS_FREE
SMI-S Plugin: Add Disk.STATUS_SPARE_DISK and Disk.STATUS_FREE

.../include/libstoragemgmt/libstoragemgmt_types.h | 10 +++++++++
plugin/megaraid/megaraid.py | 25 ++++++++++++++--------
plugin/ontap/ontap.py | 12 ++++++++++-
plugin/sim/simarray.py | 8 +++++--
plugin/smispy/smis_disk.py | 12 ++++++++++-
python_binding/lsm/_data.py | 8 +++++++
tools/lsmcli/data_display.py | 1 +
7 files changed, 63 insertions(+), 13 deletions(-)
--
1.8.3.1
Gris Ge
2015-03-19 08:08:44 UTC
Permalink
* Show Disk.STATUS_FREE as 'Free' in output.

Signed-off-by: Gris Ge <***@redhat.com>
---
tools/lsmcli/data_display.py | 1 +
1 file changed, 1 insertion(+)

diff --git a/tools/lsmcli/data_display.py b/tools/lsmcli/data_display.py
index e0524c8..19f7114 100644
--- a/tools/lsmcli/data_display.py
+++ b/tools/lsmcli/data_display.py
@@ -201,6 +201,7 @@ def disk_type_to_str(disk_type):
Disk.STATUS_MAINTENANCE_MODE: 'Maintenance',
Disk.STATUS_SPARE_DISK: 'Spare',
Disk.STATUS_RECONSTRUCT: 'Reconstruct',
+ Disk.STATUS_FREE: 'Free',
}
--
1.8.3.1
Gris Ge
2015-03-19 08:08:43 UTC
Permalink
* New status constant:
Disk.STATUS_FREE
LSM_DISK_STATUS_FREE

* In code document(Both in C and Python code):
New in version 1.2, indicate the whole disk is not holding any data or
acting as a dedicate spare disk.
This disk could be assigned as a dedicated spare disk or used for
creating pool.
If any spare disk(like those on NetApp ONTAP) does not require
any explicit action when assigning to pool, it should be treated as free
disk and marked as STATUS_FREE|STATUS_SPARE_DISK.

* Future design consideration for sliced RAID system(LVM RAID and EMC VMAX):
1. Introduce Disk.STATUS_FREE_PARTIAL
2. Introduce new class:
DiskSlice
properties:
id
owner_disk_id # Disk.id
block_size
block_count
start_block
status # FREE or SPARE or/and OK

Signed-off-by: Gris Ge <***@redhat.com>
---
c_binding/include/libstoragemgmt/libstoragemgmt_types.h | 10 ++++++++++
python_binding/lsm/_data.py | 8 ++++++++
2 files changed, 18 insertions(+)

diff --git a/c_binding/include/libstoragemgmt/libstoragemgmt_types.h b/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
index 562fcff..c5607c1 100644
--- a/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
+++ b/c_binding/include/libstoragemgmt/libstoragemgmt_types.h
@@ -237,6 +237,16 @@ typedef enum {
#define LSM_DISK_STATUS_MAINTENANCE_MODE 0x0000000000000400
#define LSM_DISK_STATUS_SPARE_DISK 0x0000000000000800
#define LSM_DISK_STATUS_RECONSTRUCT 0x0000000000001000
+#define LSM_DISK_STATUS_FREE 0x0000000000002000
+/**^
+ * New in version 1.2, New in version 1.2, indicate the whole disk is not
+ * holding any data or acting as a dedicate spare disk.
+ * This disk could be assigned as a dedicated spare disk or used for creating
+ * pool.
+ * If any spare disk(like those on NetApp ONTAP) does not require any explicit
+ * action when assigning to pool, it should be treated as free disk and marked
+ * as LSM_DISK_STATUS_FREE|LSM_DISK_STATUS_SPARE_DISK.
+ * */

#define LSM_DISK_BLOCK_SIZE_NOT_FOUND -1
#define LSM_DISK_BLOCK_COUNT_NOT_FOUND -1
diff --git a/python_binding/lsm/_data.py b/python_binding/lsm/_data.py
index 6fb2325..23681dd 100644
--- a/python_binding/lsm/_data.py
+++ b/python_binding/lsm/_data.py
@@ -199,6 +199,14 @@ class Disk(IData):
# Indicate disk is a spare disk.
STATUS_RECONSTRUCT = 1 << 12
# Indicate disk is reconstructing data.
+ STATUS_FREE = 1 << 13
+ # New in version 1.2, indicate the whole disk is not holding any data or
+ # acting as a dedicate spare disk.
+ # This disk could be assigned as a dedicated spare disk or used for
+ # creating pool.
+ # If any spare disk(like those on NetApp ONTAP) does not require
+ # any explicit action when assigning to pool, it should be treated as
+ # free disk and marked as STATUS_FREE|STATUS_SPARE_DISK.

def __init__(self, _id, _name, _disk_type, _block_size, _num_of_blocks,
_status, _system_id, _plugin_data=None):
--
1.8.3.1
Gris Ge
2015-03-19 08:08:45 UTC
Permalink
* Treating sim_disk['role'] == None as Free disk.

Changes in V3(No changes in V2):

* Remove debug code of 'print sim_disk.items()'.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/sim/simarray.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/plugin/sim/simarray.py b/plugin/sim/simarray.py
index 3fa7782..ae14848 100644
--- a/plugin/sim/simarray.py
+++ b/plugin/sim/simarray.py
@@ -194,7 +194,7 @@ class BackStore(object):

DISK_KEY_LIST = [
'id', 'name', 'total_space', 'disk_type', 'status',
- 'owner_pool_id']
+ 'owner_pool_id', 'role']

VOL_KEY_LIST = [
'id', 'vpd83', 'name', 'total_space', 'consumed_size',
@@ -1762,12 +1762,16 @@ def pools(self, flags=0):

@staticmethod
def _sim_disk_2_lsm(sim_disk):
+ disk_status = Disk.STATUS_OK
+ if sim_disk['role'] is None:
+ disk_status |= Disk.STATUS_FREE
+
return Disk(
SimArray._sim_id_to_lsm_id(sim_disk['id'], 'DISK'),
sim_disk['name'],
sim_disk['disk_type'], BackStore.BLK_SIZE,
int(sim_disk['total_space'] / BackStore.BLK_SIZE),
- Disk.STATUS_OK, BackStore.SYS_ID)
+ disk_status, BackStore.SYS_ID)

@_handle_errors
def disks(self):
--
1.8.3.1
Gris Ge
2015-03-19 08:08:46 UTC
Permalink
* Any disk not have 'aggregate' property is considered as free disk.

* Added comments indicate how spare and free disk works on ONTAP:
1. Spare == free disk
2. Zeroed spare disk == OK | SPARE | FREE
3. Non-zeroed spare disk == STOPPED | SPARE | FREE

Changes in V2:
* Should not mark partner filer's disks as free.
# Don't check status of partner's disks, simply mark as STATUS_OTHER.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/ontap/ontap.py | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/plugin/ontap/ontap.py b/plugin/ontap/ontap.py
index 5e50a44..f9f1f56 100644
--- a/plugin/ontap/ontap.py
+++ b/plugin/ontap/ontap.py
@@ -265,13 +265,17 @@ def _status_of_na_disk(na_disk):
if 'is-zeroed' in na_disk and na_disk['is-zeroed'] == 'true':
status |= Disk.STATUS_OK | Disk.STATUS_SPARE_DISK
else:
+ # If spare disk is not zerored, it will be automaticlly
+ # zeroed before assigned to aggregate.
+ # Hence we consider non-zeroed spare disks as stopped
+ # spare disks.
status |= Disk.STATUS_STOPPED | Disk.STATUS_SPARE_DISK
elif rs == 'present':
status |= Disk.STATUS_OK
elif rs == 'partner':
# Before we have good way to connect two controller,
# we have to mark partner disk as OTHER
- status |= Disk.STATUS_OTHER
+ return Disk.STATUS_OTHER

if 'is-prefailed' in na_disk and na_disk['is-prefailed'] == 'true':
status |= Disk.STATUS_STOPPING
@@ -279,6 +283,12 @@ def _status_of_na_disk(na_disk):
if 'is-offline' in na_disk and na_disk['is-offline'] == 'true':
status |= Disk.STATUS_ERROR

+ if 'aggregate' not in na_disk:
+ # All free disks are automatically marked as spare disks. They
+ # could easily convert to data or parity disk without any
+ # explicit command.
+ status |= Disk.STATUS_FREE
+
if status == 0:
status = Disk.STATUS_UNKNOWN
--
1.8.3.1
Gris Ge
2015-03-19 08:08:47 UTC
Permalink
* Treat 'UGood' and 'UBad' as free disk.

* Update algorithm of disk status query to support show spare disk with
sector error as SPARE|ERROR

Changes in V2:

* Fix the incorrect data assign:
disk_status == Disk.STATUS_UNKNOWN
Should be:
disk_status = Disk.STATUS_UNKNOWN

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/megaraid/megaraid.py | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/plugin/megaraid/megaraid.py b/plugin/megaraid/megaraid.py
index cb775f3..78603fc 100644
--- a/plugin/megaraid/megaraid.py
+++ b/plugin/megaraid/megaraid.py
@@ -87,29 +87,36 @@ def _disk_type_of(disk_show_basic_dict):
'Offln': Disk.STATUS_ERROR,
'GHS': Disk.STATUS_SPARE_DISK | Disk.STATUS_OK,
'DHS': Disk.STATUS_SPARE_DISK | Disk.STATUS_OK,
- 'UGood': Disk.STATUS_STOPPED | Disk.STATUS_OK,
- 'UBad': Disk.STATUS_STOPPED | Disk.STATUS_ERROR,
+ 'UGood': Disk.STATUS_FREE | Disk.STATUS_OK,
+ 'UBad': Disk.STATUS_FREE | Disk.STATUS_ERROR,
'Rbld': Disk.STATUS_RECONSTRUCT,
}


def _disk_status_of(disk_show_basic_dict, disk_show_stat_dict):
+ disk_status = _DISK_STATE_MAP.get(
+ disk_show_basic_dict['State'], 0)
+
if disk_show_stat_dict['Media Error Count'] or \
disk_show_stat_dict['Other Error Count'] or \
disk_show_stat_dict['S.M.A.R.T alert flagged by drive'] != 'No':
- return Disk.STATUS_ERROR
+ disk_status -= Disk.STATUS_OK
+ disk_status |= Disk.STATUS_ERROR

- if disk_show_stat_dict['Predictive Failure Count']:
- return Disk.STATUS_PREDICTIVE_FAILURE
+ elif disk_show_stat_dict['Predictive Failure Count']:
+ disk_status -= Disk.STATUS_OK
+ disk_status |= Disk.STATUS_PREDICTIVE_FAILURE

if disk_show_basic_dict['Sp'] == 'D':
- return Disk.STATUS_STOPPED
+ disk_status |= Disk.STATUS_STOPPED

if disk_show_basic_dict['Sp'] == 'F':
- return Disk.STATUS_OTHER
+ disk_status |= Disk.STATUS_OTHER
+
+ if disk_status == 0:
+ disk_status = Disk.STATUS_UNKNOWN

- return _DISK_STATE_MAP.get(
- disk_show_basic_dict['State'], Disk.STATUS_UNKNOWN)
+ return disk_status


def _mega_size_to_lsm(mega_size):
--
1.8.3.1
Gris Ge
2015-03-19 08:08:48 UTC
Permalink
* Using SNIA SMI-S 1.4 'Disk Sparing' profile to set Disk.STATUS_SPARE_DISK.
# Works well on EMC VMAX, Fujitsu ETERNUS, LSI MegaRAID.
# They are only array have spare disks I got access.

* Use vendor specific way to set Disk.STATUS_FREE:
1. EMC:
'EMCInUse' property of CIM_DiskDrive. False == Free.
# Tested on EMC VNX.

2. LSI MegaRAID:
StoragePool_InstanceID of CIM_StorageExtent,
if not contain 'Pool' keyword, it could be free disk or spare disk.
Then filter out spare disk, we got free disks.
# Tested on LSI MegaRAID.

Changes in V4(No changes in V2, V3):

* Remove LSI MegaRAID free disk code as we have dedicated plugin for
MegaRAID.

Signed-off-by: Gris Ge <***@redhat.com>
---
plugin/smispy/smis_disk.py | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/plugin/smispy/smis_disk.py b/plugin/smispy/smis_disk.py
index 6b476f1..e5bd5ea 100644
--- a/plugin/smispy/smis_disk.py
+++ b/plugin/smispy/smis_disk.py
@@ -77,10 +77,11 @@ def cim_disk_pros():
"""
Return all CIM_DiskDrive Properties needed to create a Disk object.
The 'Type' and 'MediaType' is only for MegaRAID.
+ The 'EMCInUse' is only for EMC.
"""
return ['OperationalStatus', 'Name', 'SystemName',
'Caption', 'InterconnectType', 'DiskType', 'DeviceID',
- 'Type', 'MediaType']
+ 'Type', 'MediaType', 'EMCInUse']


def sys_id_of_cim_disk(cim_disk):
@@ -178,6 +179,15 @@ def cim_disk_to_lsm_disk(smis_common, cim_disk):
property_list=['BlockSize', 'NumberOfBlocks'])

status = _disk_status_of_cim_disk(cim_disk)
+ cim_srss = smis_common.AssociatorNames(
+ cim_ext.path, AssocClass='CIM_IsSpare',
+ ResultClass='CIM_StorageRedundancySet')
+ if len(cim_srss) >= 1:
+ status |= Disk.STATUS_SPARE_DISK
+
+ if 'EMCInUse' in cim_disk.keys() and cim_disk['EMCInUse'] is False:
+ status |= Disk.STATUS_FREE
+
name = ''
block_size = Disk.BLOCK_SIZE_NOT_FOUND
num_of_block = Disk.BLOCK_COUNT_NOT_FOUND
--
1.8.3.1
Tony Asleson
2015-03-19 15:19:36 UTC
Permalink
Looks good!, branch merged!

Thanks!
Tony
Post by Gris Ge
* This patch set introduced new disk status -- STATUS_FREE to indicate
disk could be used for pool or assign as dedicated spare disk.
* Patch 6/7: Fix incorrect data assignment in MegaRAID plugin.
* Patch 5/7: Don't mark partner's disks as free disk in ONTAP plugin..
* Patch 7/7: Include SMI-S plugin support for STATUS_FREE using
vendor specific ways. Also come with STATUS_SPARE_DISK support.
* Patch 6/9: Remove the debug print line in simulator plugin.
* Patch 1/9: New patch. Change currnt git tree version as 1.2.
* Patch 2/9: New patch. Fix missing VOLUME_RAID_INFO capability of megaraid
plugin.
* Removed unrelated patches and submitted them in separate patch sets.
* Removed LSI MegaRAID SMI-S Disk.STATUS_FREE code as we have dedicate
MegaRAID plugin now.
API: New disk status DISK.STATUS_FREE/LSM_DISK_STATUS_FREE
lsmcli: Add support of Disk.STATUS_FREE
Simulator Plugin: Support Disk.STATUS_FREE
ONTAP Plugin: Add support of Disk.STATUS_FREE
MegaRAID Plugin: Add support of Disk.STATUS_FREE
SMI-S Plugin: Add Disk.STATUS_SPARE_DISK and Disk.STATUS_FREE
.../include/libstoragemgmt/libstoragemgmt_types.h | 10 +++++++++
plugin/megaraid/megaraid.py | 25 ++++++++++++++--------
plugin/ontap/ontap.py | 12 ++++++++++-
plugin/sim/simarray.py | 8 +++++--
plugin/smispy/smis_disk.py | 12 ++++++++++-
python_binding/lsm/_data.py | 8 +++++++
tools/lsmcli/data_display.py | 1 +
7 files changed, 63 insertions(+), 13 deletions(-)
Continue reading on narkive:
Loading...