[Libstoragemgmt-devel] [PATCH 0/5] Pool create API changes.

[Gris Ge]

When I coding simulator plugin to support the pool create and delete,
I realized how silly my design is for pool creating API. It's almost a
nightmare to handle such long list of optional parameters in plugin.
Hence we change it again. :(

In this patch set:
* Changed pool create API by split the old one into:
pool_create() # for normally user who don't care RAID
# member. Just care about size, maybe
# RAID type or disk type.
pool_create_from_disks() # for advanced user
pool_create_from_volumes() # for advanced user
pool_create_from_pool() # for advanced user
Please check the patch #1 for more detail.

* Updated lsmcli and its manpage for this change.
* Enable pool create/delete support in simulator plugin.

Any comments or correction will be appreciated.

Gris Ge (5):
client.py: change pool create/delete python API
data.py: add disk type into member type of Pool, more pool
capabilities
common.py: add pool create related error
lsmcli: add support of new pool create/delete API
simulator.py: add support of pool create/delete

doc/man/lsmcli.1.in | 121 ++++--
lsm/lsm/client.py | 209 +++++-----
lsm/lsm/cmdline.py | 258 ++++++++----
lsm/lsm/common.py | 3 +
lsm/lsm/data.py | 136 ++++--
lsm/lsm/simarray.py | 1122 ++++++++++++++++++++++++++++++++++++++++++++------
lsm/lsm/simulator.py | 33 +-
7 files changed, 1521 insertions(+), 361 deletions(-)

[Gris Ge]

* Add two ErrorNumber constants for pool creation related error:
ErrorNumber.DISK_BUSY
ErrorNumber.VOLUME_BUSY

Signed-off-by: Gris Ge <***@redhat.com>
---
lsm/lsm/common.py | 3 +++
1 file changed, 3 insertions(+)

diff --git a/lsm/lsm/common.py b/lsm/lsm/common.py
index f0467c1..e8c2298 100644
--- a/lsm/lsm/common.py
+++ b/lsm/lsm/common.py
@@ -469,6 +469,9 @@ class ErrorNumber(object):
UNSUPPORTED_PROVISIONING = 451
UNSUPPORTED_REPLICATION_TYPE = 452

+ DISK_BUSY = 500
+ VOLUME_BUSY = 501
+

class JobStatus(object):
INPROGRESS = 1

[Gris Ge]

* Changed the old pool_create() into:

1. pool_create(self, system_id, pool_name, size_bytes,
raid_type=Pool.RAID_TYPE_UNKNOWN,
member_type=Pool.MEMBER_TYPE_UNKNOWN)
Most user just care about these things when creating a pool:
A. Capacity
B. Availability
C. Performance
To honor that, the pool_create() accept:
size_bytes => Capacity
raid_type => Availability + Performance
member_type => Performance
# For example use MEMBER_TYPE_DISK_SSD for a SSD pool.

2. pool_create_from_disks(), pool_create_from_volumes(),
pool_create_from_pool()
These methods are dedicated to advanced user who want to define what
members to assemble the pool and how.

* Require (only documented) pool_create() and pool_create_from_xxx() to
return full properties(including optional data). [2]

* Removed thin provisioning support from creating pool.
Most array(except EMC VNX[1]) only create pool either THICK or THIN.
No need to bother user for one more parameters.

[1] We will create THIN pool on EMC VNX, as THICK pool (aka EMC RAID Group in
CLARiiON and VNX)is unmaintained and lacking lun resize feature.

[2] Planed to change lsmcli display codes to determine what to display by
itself rather than follow data.py to fit the 80 text width requirements.

Signed-off-by: Gris Ge <***@redhat.com>
---
lsm/lsm/client.py | 209 +++++++++++++++++++++++++++---------------------------
1 file changed, 105 insertions(+), 104 deletions(-)

diff --git a/lsm/lsm/client.py b/lsm/lsm/client.py
index a422037..bbca00e 100644
--- a/lsm/lsm/client.py
+++ b/lsm/lsm/client.py
@@ -288,122 +288,123 @@ class Client(INetworkAttachedStorage):
"""
return self._tp.rpc('pools', del_self(locals()))

+ ## Create new pool in user friendly way. Depending on this capability:
+ ## Capabilities.POOL_CREATE
+ ## For plugin developer: this method require complex codes to chose
+ ## pool members, please refer to SimData.pool_create() in simarray.py for
+ ## sample codes.
## Return the newly created pool object.
- ## When creating a pool, user are normally concern about these things:
- ## * Size
- ## * RAID level
- ## * Disk count # this determine the IOPS.
- ## * Disk type # SAS/SATA/SSD, this also determine the IOPS.
- ## The pool_create() will create a pool with any/all of these info
- ## provided and let the plugin/array guess out the rest.
- ##
- ## These are Capabilities indicate the support status of this method:
- ## * Capabilities.POOL_CREATE
- ## Capabilities.POOL_CREATE is the priority 1 capability for
- ## pool_create(), without it, pool_create() is not supported at all.
- ## This call is mandatory:
- ## pool_create(system_id='system_id',
- ## pool_name='pool_name',
- ## size_bytes='size_bytes')
- ## Creates a pool with size bigger or equal to requested
- ## 'size_bytes'.
- ##
- ## * Capabilities.POOL_CREATE_RAID_TYPE
- ## This capability indicate user add define RAID type/level when
- ## creating pool.
- ## Capabilities.POOL_CREATE_RAID_XX is for certain RAID type/level.
- ## This call is mandatory:
- ## pool_create(system_id='system_id',
- ## pool_name='pool_name',
- ## size_bytes='size_bytes',
- ## raid_type='raid_type' )
- ## Creates a pool with requested RAID level and
- ## 'size_bytes'.
- ## The new pool should holding size bigger or equal to
- ## requested 'size_bytes'.
- ##
- ## * Capabilities.POOL_CREATE_VIA_MEMBER_COUNT
- ## This capability require Capabilities.POOL_CREATE_RAID_TYPE.
- ## This capability allow user to define how many disk/volume/pool
- ## should be used to create new pool.
- ## This call is mandatory:
- ## pool_create(system_id='system_id',
- ## pool_name='pool_name',
- ## raid_type='raid_type' ,
- ## member_count='member_count')
- ## Create a pool with requested more or equal count of
- ## disk/volume/pool as requested.
- ## For example, user can request a 8 disks RAID5 pool via
- ## pool_create(system_id='abc', pool_name='pool1',
- ## member_count=8, raid_type=Pool.RAID_TYPE_RAID5)
- ##
- ## * POOL_CREATE_MEMBER_TYPE_DISK
- ## This capability require Capabilities.POOL_CREATE_RAID_TYPE
- ## and Capabilities.POOL_CREATE_VIA_MEMBER_COUNT
- ## This capability indicate user can define which type of Disk in
- ## could be used to create a pool in 'member_type'.
- ## These calls are mandatory:
- ## pool_create(system_id='system_id',
- ## pool_name='pool_name',
- ## raid_type='raid_type',
- ## member_type='member_type',
- ## member_count='member_count')
- ##
- ## pool_create(system_id='system_id',
- ## pool_name='pool_name',
- ## size_bytes='size_bytes',
- ## member_type='member_type')
- ## * POOL_CREATE_VIA_MEMBER_IDS
- ## This capability require Capabilities.POOL_CREATE_RAID_TYPE
- ## This capability is used for advanced user who want to control
- ## which disks/volumes/pools will be used to create new pool.
- ## This call is mandatory:
- ## pool_create(system_id='system_id',
- ## pool_name='pool name',
- ## raid_type='# enumerated_type',
- ## member_type='member_type',
- ## member_ids=(member_ids))
- ## * Capabilities.POOL_CREATE_THIN or POOL_CREATE_THICK
- ## These capabilities indicate user can define thinp_type in
- ## pool_create().
- # @param self The this pointer
- # @param system_id ID of the system to create pool on.
- # @param pool_name The name of requested new pool.
- # @param raid_type The RAID type applied to members. Should be
- # Data.Extent.RAID_TYPE_XXXX
- # @param member_type Identify the type of member. Should be
- # Data.Extent.MEMBER_TYPE_XXXX
- # @param member_ids The array of IDs for pool members.
- # Could be ID of disks/pools/volumes
- # @param member_count How many disk/volume/pool should plugin to chose
- # if member_ids is not enough or not defined.
- # If len(member_ids) is larger than member_count,
- # will raise LsmError ErrorNumber.INVALID_ARGUMENT
- # @param size_bytes Required pool size to create.
- # @param thinp_type Pool.THINP_TYPE_UNKNOWN, or THINP_TYPE_THICK,
- # or THINP_TYPE_THIN
- # @param flags Reserved for future use, must be zero.
- # @returns the newly created pool object.
- def pool_create(self,
- system_id,
- pool_name='',
+ # @param self The this pointer
+ # @param system_id The id of system where new pool should reside.
+ # @param pool_name The name for new pool. Will not fail if created
+ # pool_name is not the same as requested.
+ # @param size_bytes The size in bytes for new pool.
+ # New pool can have equal or larger size than
+ # requested, but not less. Should larger than 0.
+ # @param raid_type Optional. If defined, new pool should using
+ # defined RAID type.
+ # When member_type was set to Pool.MEMBER_TYPE_POOL,
+ # only allowed raid_type is RAID_TYPE_UNKNOWN or
+ # RAID_TYPE_NOT_APPLICABLE
+ # @param member_type Optional. If defined, new pool will be assembled
+ # by defined member types. For example;
+ # when member_type == Pool.MEMBER_TYPE_DISK_SAS,
+ # new pool will be created from SAS disks only.
+ # @param flags Reserved for future use.
+ # @returns A tuple (job_id, new_pool), when one is None the other is
+ # valid.
+ @return_requires(unicode, Pool)
+ def pool_create(self, system_id, pool_name, size_bytes,
raid_type=Pool.RAID_TYPE_UNKNOWN,
- member_type=Pool.MEMBER_TYPE_UNKNOWN,
- member_ids=None,
- member_count=0,
- size_bytes=0,
- thinp_type=Pool.THINP_TYPE_UNKNOWN,
- flags=0):
+ member_type=Pool.MEMBER_TYPE_UNKNOWN, flags=0):
"""
Returns the created new pool object.
"""
+ # TODO: check size_bytes larger than 0 and raise error.
return self._tp.rpc('pool_create', del_self(locals()))

+ ## Create new pool in the hard way by defined what exactly disks should
+ ## be used. Depending on these capabilities:
+ ## Capabilities.POOL_CREATE_FROM_DISKS
+ ## Return the newly created pool object with all supported optional data.
+ # @param self The this pointer
+ # @param system_id The id of system where new pool should reside.
+ # @param pool_name The name for new pool. Will not fail if created
+ # pool_name is not the same as requested.
+ # @param member_ids The ids of disks to create new pool.
+ # The new pool could contain more disks than
+ # requested due to internal needs, but if possible,
+ # new pool should only contain requested disks.
+ # @param raid_type The RAID level for new pool.
+ # Capabilities.POOL_CREATE_DISK_RAID_XXX will
+ # indicate the supported RAID level.
+ # @param flags Reserved for future use.
+ # @returns A tuple (job_id, new_pool), when one is None the other is
+ # valid.
+ @return_requires(unicode, Pool)
+ def pool_create_from_disks(self, system_id, pool_name, member_ids,
+ raid_type, flags=0):
+ """
+ Creates pool from disks.
+ Returns the created new pool object.
+ """
+ return self._tp.rpc('pool_create_from_disks', del_self(locals()))
+
+ ## Create new pool in the hard way by defined what exactly volumes should
+ ## be used. Depending on these capabilities:
+ ## Capabilities.POOL_CREATE_FROM_VOLUMES
+ ## Return the newly created pool object with all supported optional data.
+ # @param self The this pointer
+ # @param system_id The id of system where new pool should reside.
+ # @param pool_name The name for new pool. Will not fail if created
+ # pool_name is not the same as requested.
+ # @param member_ids The ids of volumes to create new pool.
+ # The new pool could contain more volumes than
+ # requested due to internal needs, but if possible,
+ # new pool should only contain requested volumes.
+ # @param raid_type The RAID level for new pool.
+ # Capabilities.POOL_CREATE_VOLUME_RAID_XXX will
+ # indicate the supported RAID level.
+ # @param flags Reserved for future use.
+ # @returns A tuple (job_id, new_pool), when one is None the other is
+ # valid.
+ @return_requires(unicode, Pool)
+ def pool_create_from_volumes(self, system_id, pool_name, member_ids,
+ raid_type, flags=0):
+ """
+ Creates pool from volumes.
+ Returns the created new pool object.
+ """
+ return self._tp.rpc('pool_create_from_volumes', del_self(locals()))
+
+ ## Create new pool in the hard way by defined what exactly pool should
+ ## be allocate space from. Depending on this capability:
+ ## Capabilities.POOL_CREATE_FROM_POOL
+ ## Return the newly created pool object with all supported optional data.
+ # @param self The this pointer
+ # @param system_id The id of system where new pool should reside.
+ # @param pool_name The name for new pool. Will not fail if created
+ # pool_name is not the same as requested.
+ # @param member_id The id of pool to allocate space from for new pool.
+ # @param size_bytes The size of the new pool.
+ # @param flags Reserved for future use.
+ # @returns A tuple (job_id, new_pool), when one is None the other is
+ # valid.
+ @return_requires(unicode, Pool)
+ def pool_create_from_pool(self, system_id, pool_name, member_id,
+ size_bytes, flags=0):
+ """
+ Creates pool from volumes.
+ Returns the created new pool object.
+ """
+ return self._tp.rpc('pool_create_from_pool', del_self(locals()))
+
## Remove a pool. This method depend on Capabilities.POOL_DELETE
# @param self The this pointer
# @param pool The pool object
# @param flags Reserved for future use, must be zero.
# @returns None on success, else job id. Raises LsmError on errors.
+ @return_requires(unicode)
def pool_delete(self, pool, flags=0):
"""
Return None on success, else job id. Raises LsmError on errors.

[Gris Ge]

* Letting Pool.member_type holding disk type and introduce these methods:
Pool.member_type_is_disk()
Pool.member_type_to_disk_type()
Pool.disk_type_to_member_type()

* Add capability entries for pool creating.

Signed-off-by: Gris Ge <***@redhat.com>
---
lsm/lsm/data.py | 136 ++++++++++++++++++++++++++++++++++++++++++--------------
1 file changed, 103 insertions(+), 33 deletions(-)

diff --git a/lsm/lsm/data.py b/lsm/lsm/data.py
index 2e63e0a..ad6455d 100644
--- a/lsm/lsm/data.py
+++ b/lsm/lsm/data.py
@@ -234,8 +234,8 @@ class IData(object):
return self.value_convert(key_name, value, human, enum_as_number,
list_convert)

- elif hasattr(self, '_optional_data') and \
- key_name in opt_pros_header.keys():
+ elif (hasattr(self, '_optional_data') and
+ key_name in opt_pros_header.keys()):
if key_name not in self._optional_data.list():
return None

@@ -752,6 +752,7 @@ class Pool(IData):
RAID_TYPE_JBOD = 20
RAID_TYPE_UNKNOWN = 21
RAID_TYPE_NOT_APPLICABLE = 22
+ # NOT_APPLICABLE indicate current pool only has one member.
RAID_TYPE_MIXED = 23

# The string of each RAID_TYPE is for CIM_StorageSetting['ElementName']
@@ -819,23 +820,80 @@ class Pool(IData):
MEMBER_TYPE_UNKNOWN = 0
MEMBER_TYPE_DISK = 1
MEMBER_TYPE_DISK_MIX = 10
- MEMBER_TYPE_DISK_SAS = 11
+ MEMBER_TYPE_DISK_ATA = 11
MEMBER_TYPE_DISK_SATA = 12
- MEMBER_TYPE_DISK_SCSI = 13
- MEMBER_TYPE_DISK_SSD = 14
- MEMBER_TYPE_DISK_NL_SAS = 15
+ MEMBER_TYPE_DISK_SAS = 13
+ MEMBER_TYPE_DISK_FC = 14
+ MEMBER_TYPE_DISK_SOP = 15
+ MEMBER_TYPE_DISK_SCSI = 16
+ MEMBER_TYPE_DISK_NL_SAS = 17
+ MEMBER_TYPE_DISK_HDD = 18
+ MEMBER_TYPE_DISK_SSD = 19
+ MEMBER_TYPE_DISK_HYBRID = 110
+
MEMBER_TYPE_POOL = 2
MEMBER_TYPE_VOLUME = 3

+ _MEMBER_TYPE_2_DISK_TYPE = {
+ MEMBER_TYPE_DISK: Disk.DISK_TYPE_UNKNOWN,
+ MEMBER_TYPE_DISK_MIX: Disk.DISK_TYPE_UNKNOWN,
+ MEMBER_TYPE_DISK_ATA: Disk.DISK_TYPE_ATA,
+ MEMBER_TYPE_DISK_SATA: Disk.DISK_TYPE_SATA,
+ MEMBER_TYPE_DISK_SAS: Disk.DISK_TYPE_SAS,
+ MEMBER_TYPE_DISK_FC: Disk.DISK_TYPE_FC,
+ MEMBER_TYPE_DISK_SOP: Disk.DISK_TYPE_SOP,
+ MEMBER_TYPE_DISK_SCSI: Disk.DISK_TYPE_SCSI,
+ MEMBER_TYPE_DISK_NL_SAS: Disk.DISK_TYPE_NL_SAS,
+ MEMBER_TYPE_DISK_HDD: Disk.DISK_TYPE_HDD,
+ MEMBER_TYPE_DISK_SSD: Disk.DISK_TYPE_SSD,
+ MEMBER_TYPE_DISK_HYBRID: Disk.DISK_TYPE_HYBRID,
+ }
+
+ @staticmethod
+ def member_type_is_disk(member_type):
+ """
+ Returns True if defined 'member_type' is disk.
+ False when else.
+ """
+ if member_type in Pool._MEMBER_TYPE_2_DISK_TYPE.keys():
+ return True
+ return False
+
+ @staticmethod
+ def member_type_to_disk_type(member_type):
+ """
+ Convert member_type to disk_type.
+ For non-disk member, we return Disk.DISK_TYPE_NOT_APPLICABLE
+ """
+ if member_type in Pool._MEMBER_TYPE_2_DISK_TYPE.keys():
+ return Pool._MEMBER_TYPE_2_DISK_TYPE[member_type]
+ return Disk.DISK_TYPE_NOT_APPLICABLE
+
+ @staticmethod
+ def disk_type_to_member_type(disk_type):
+ """
+ Convert disk_type to Pool.MEMBER_TYPE_DISK_XXXX
+ Will return Pool.MEMBER_TYPE_DISK as failback.
+ """
+ key = get_key(Pool._MEMBER_TYPE_2_DISK_TYPE, disk_type)
+ if key or key == 0:
+ return key
+ return Pool.MEMBER_TYPE_DISK
+
_MEMBER_TYPE = {
MEMBER_TYPE_UNKNOWN: 'UNKNOWN',
MEMBER_TYPE_DISK: 'DISK', # Pool was created from Disk(s).
MEMBER_TYPE_DISK_MIX: 'DISK_MIX', # Has two or more types of disks.
- MEMBER_TYPE_DISK_SAS: 'DISK_SAS',
+ MEMBER_TYPE_DISK_ATA: 'DISK_ATA',
MEMBER_TYPE_DISK_SATA: 'DISK_SATA',
+ MEMBER_TYPE_DISK_SAS: 'DISK_SAS',
+ MEMBER_TYPE_DISK_FC: 'DISK_FC',
+ MEMBER_TYPE_DISK_SOP: 'DISK_SOP',
MEMBER_TYPE_DISK_SCSI: 'DISK_SCSI',
- MEMBER_TYPE_DISK_SSD: 'DISK_SSD',
MEMBER_TYPE_DISK_NL_SAS: 'DISK_NL_SAS',
+ MEMBER_TYPE_DISK_HDD: 'DISK_HDD',
+ MEMBER_TYPE_DISK_SSD: 'DISK_SSD',
+ MEMBER_TYPE_DISK_HYBRID: 'DISK_HYBRID',
MEMBER_TYPE_POOL: 'POOL', # Pool was created from other Pool(s).
MEMBER_TYPE_VOLUME: 'VOLUME', # Pool was created from Volume(s).
}
@@ -1425,31 +1483,43 @@ class Capabilities(IData):

#Pool
POOL_CREATE = 130
- POOL_CREATE_THIN = 131
- POOL_CREATE_THICK = 132
- POOL_CREATE_RAID_TYPE = 133
- POOL_CREATE_VIA_MEMBER_COUNT = 134
- POOL_CREATE_VIA_MEMBER_IDS = 135
- POOL_CREATE_MEMBER_TYPE_DISK = 136
- POOL_CREATE_MEMBER_TYPE_POOL = 137
- POOL_CREATE_MEMBER_TYPE_VOL = 138
-
- POOL_CREATE_RAID_0 = 140
- POOL_CREATE_RAID_1 = 141
- POOL_CREATE_RAID_JBOD = 142
- POOL_CREATE_RAID_3 = 143
- POOL_CREATE_RAID_4 = 144
- POOL_CREATE_RAID_5 = 145
- POOL_CREATE_RAID_6 = 146
- POOL_CREATE_RAID_10 = 147
- POOL_CREATE_RAID_50 = 148
- POOL_CREATE_RAID_51 = 149
- POOL_CREATE_RAID_60 = 150
- POOL_CREATE_RAID_61 = 151
- POOL_CREATE_RAID_15 = 152
- POOL_CREATE_RAID_16 = 153
-
- POOL_DELETE = 160
+ POOL_CREATE_FROM_DISKS = 131
+ POOL_CREATE_FROM_VOLUMES = 132
+ POOL_CREATE_FROM_POOL = 133
+
+ POOL_CREATE_DISK_RAID_0 = 140
+ POOL_CREATE_DISK_RAID_1 = 141
+ POOL_CREATE_DISK_RAID_JBOD = 142
+ POOL_CREATE_DISK_RAID_3 = 143
+ POOL_CREATE_DISK_RAID_4 = 144
+ POOL_CREATE_DISK_RAID_5 = 145
+ POOL_CREATE_DISK_RAID_6 = 146
+ POOL_CREATE_DISK_RAID_10 = 147
+ POOL_CREATE_DISK_RAID_50 = 148
+ POOL_CREATE_DISK_RAID_51 = 149
+ POOL_CREATE_DISK_RAID_60 = 150
+ POOL_CREATE_DISK_RAID_61 = 151
+ POOL_CREATE_DISK_RAID_15 = 152
+ POOL_CREATE_DISK_RAID_16 = 153
+ POOL_CREATE_DISK_RAID_NOT_APPLICABLE = 154
+
+ POOL_CREATE_VOLUME_RAID_0 = 160
+ POOL_CREATE_VOLUME_RAID_1 = 161
+ POOL_CREATE_VOLUME_RAID_JBOD = 162
+ POOL_CREATE_VOLUME_RAID_3 = 163
+ POOL_CREATE_VOLUME_RAID_4 = 164
+ POOL_CREATE_VOLUME_RAID_5 = 165
+ POOL_CREATE_VOLUME_RAID_6 = 166
+ POOL_CREATE_VOLUME_RAID_10 = 167
+ POOL_CREATE_VOLUME_RAID_50 = 168
+ POOL_CREATE_VOLUME_RAID_51 = 169
+ POOL_CREATE_VOLUME_RAID_60 = 170
+ POOL_CREATE_VOLUME_RAID_61 = 171
+ POOL_CREATE_VOLUME_RAID_15 = 172
+ POOL_CREATE_VOLUME_RAID_16 = 173
+ POOL_CREATE_VOLUME_RAID_NOT_APPLICABLE = 174
+
+ POOL_DELETE = 200

def to_dict(self):
rc = {'class': self.__class__.__name__,

[Gris Ge]

* Add support of new pool create API.
* Manpage updated.
* Allowing "--optional" switch in pool create commands.

Signed-off-by: Gris Ge <***@redhat.com>
---
doc/man/lsmcli.1.in | 121 ++++++++++++++++++------
lsm/lsm/cmdline.py | 258 ++++++++++++++++++++++++++++++++++++----------------
2 files changed, 277 insertions(+), 102 deletions(-)

diff --git a/doc/man/lsmcli.1.in b/doc/man/lsmcli.1.in
index 058b596..ff11472 100644
--- a/doc/man/lsmcli.1.in
+++ b/doc/man/lsmcli.1.in
@@ -1,4 +1,4 @@
-.TH LSMCLI "1" "January 2014" "lsmcli @VERSION@" "libStorageMgmt"
+.TH LSMCLI "1" "March 2014" "lsmcli @VERSION@" "libStorageMgmt"
.SH NAME
lsmcli - LibStorageMgmt command line interface

@@ -44,7 +44,8 @@ Quick examples(please refer to "LibStorageMgmt User Guide" for detail):
\fBontap+ssl://***@host\fR
.RE 1
.RS 12
-* Storage Array support SMI-S(like, EMC CX/VNX, HDS AMS, IBM SVC/DS, and etc):
+* Storage Array support SMI-S(like, EMC CX/VNX, HDS AMS, IBM SVC/DS, LSI
+MegaRAID and etc):
.RS 4
\fBsmis://***@host:<port>?namespace=<namespace>\fR
.br
@@ -573,44 +574,112 @@ For two or more files/paths:
'\fB--src fileA --src fileB --dst old_fileA --dst old_fileB\fR'.

.SS pool-create
-Creates a storage pool.
+Creates a storage pool. LibStorageMgmt will automatically choose the correct
+pool members to assemble new pool. This require POOL_CREATE capability.
.TP 15
\fB--name\fR \fI<POOL_NAME>\fR
Required. Human friendly name for new pool.
.TP
\fB--size\fR \fI<POOL_SIZE>\fR
-Optional. The size of new pool. Due to data alignment and other issue, the
-size of new pool might largger than requested.
+Required. The size of new pool. Due to data alignment or other issue, the
+size of new pool might larger than requested.
See \fBSIZE OPTION\fR for allowed formats.
.TP
-\fB--member-id\fR \fI<MEM_ID>\fR
-Optional. Repeatable. Pool member ID, could be ID of Disk/Pool/Volume. Should
-be used with \fB--member-type\fR. For two or more members: '\fB--member-id
-DISK_A --member DISK_B --raid-type RAID1 --member-type DISK\fR'
+\fB--sys\fR \fI<SYS_ID>\fR
+Required. ID of the system to create new pool.
+.TP
+\fB--raid-type\fR \fI<RAID_TYPE>\fR
+Optional. The RAID type of new pool. Valid values are:
+\fBNOT_APPLICABLE\fR, \fBJBOD\fR, \fBRAID0\fR, \fBRAID1\fR, \fBRAID5\fR,
+\fBRAID6\fR, \fBRAID10\fR, \fBRAID50\fR, \fBRAID51\fR, \fBRAID60\fR,
+\fBRAID61\fR.
+.br
+The \fBNOT_APPLICABLE\fR means pool only contain 1 member.
+If not defined, will let array to determine the RAID type.
+.br
+When using with \fB--member-type POOL\fR, \fB--raid-type\fR should be unset or
+defined as \fBNOT_APPLICABLE\fR.
.TP
\fB--member-type\fR \fI<MEM_TYPE>\fR
-Optional. Valid values are: \fBDISK\fR, \fBPOOL\fR, \fBVOLUME\fR.
+Optional. The type of resource to create new pool. Valid values are:
+\fBDISK\fR, \fBVOLUME\fR, \fBPOOL\fR, \fBDISK_ATA\fR, \fBDISK_SATA\fR,
+\fBDISK_SAS\fR, \fBDISK_FC\fR, \fBDISK_SOP\fR \fBDISK_SCSI\fR, \fBDISK_NL_SAS,
+\fBDISK_HDD\fR, \fBDISK_SSD\fR, \fBDISK_HYBRID\fR.
.br
-The 'DISK' member type means pool is created from disk(s), might has RAID.
+The \fBDISK\fR member type means creating pool from disk(s). For \fBDISK_\fR
+prefixed types, they are used to request new pool creating from certain type
+of DISK.
+\fBDISK_SOP\fR indicate SCSI over PCI-E, normally a PCI-E based SSD.
+\fBDISK_HYBRID\fR indicate HDD and SSD hybrid(mixed) disk.
.br
-The 'VOLUME' member type means pool is created from volume(s), might has RAID.
+The \fBVOLUME\fR member type means creating pool from volume(s).
.br
-The 'POOL' member type means pool is allocated from other pool(s).
+The \fBPOOL\fR member type means create sub-pool from another pool.
+
+.SS pool-create-from-disks
+Create a new pool by specifying which disks to use with which RAID type.
+This require POOL_CREATE_FROM_DISKS capability.
+.TP 15
+\fB--name\fR \fI<POOL_NAME>\fR
+Required. Human friendly name for new pool.
.TP
\fB--raid-type\fR \fI<RAID_TYPE>\fR
-Optional. The RAID type of new pool. Valid values are:
-\fBJBOD\fR, \fBRAID0\fR, \fBRAID1\fR, \fBRAID5\fR, \fBRAID6\fR, \fBRAID10\fR,
-\fBRAID50\fR, \fBRAID51\fR, \fBRAID60\fR, \fBRAID61\fR.
-.TP
-\fB--thinp-type\fR \fI<THINP_TYPE>\fR
-Optional. The Thin Provisioning type of new pool. Valid values are:
-\fBTHIN\fR, \fBTHICK\fR. 'THIN' indicates pool can create Thin Provisioning
-volume or filesystem. 'THICK' indicates pool can only create fully allocated
-volume or filesystem.
-.TP
-\fB--member-count\fR \fI<MEM_COUNT>\fR
-Optional. The count of members. If '\fR--member-id\fR' is not defined or less
-than '\fB--member-count\fR', plugin/array will automatically choose the rest.
+Required. The RAID type of new pool. Valid values are:
+\fBNOT_APPLICABLE\fR, \fBJBOD\fR, \fBRAID0\fR, \fBRAID1\fR, \fBRAID5\fR,
+\fBRAID6\fR, \fBRAID10\fR, \fBRAID50\fR, \fBRAID51\fR, \fBRAID60\fR,
+\fBRAID61\fR.
+.br
+The \fBNOT_APPLICABLE\fR means pool only contain 1 disks.
+For supported RAID types of certain array, please use \fBcapabilities\fR
+command for POOL_CREATE_DISK_RAID_XXX entries.
+.TP
+\fB--member-id\fR \fI<DISK_ID>\fR
+Required. Repeatable. The ID of disk to create new pool.
+For two or more members: '\fB--member-id DISK_ID_A --member DISK_ID_B\fR'.
+.TP
+\fB--sys\fR \fI<SYS_ID>\fR
+Required. ID of the system to create new pool.
+
+.SS pool-create-from-volumes
+Create a new pool by specifying which volumes to use with which RAID type.
+This require POOL_CREATE_FROM_VOLUMES capability.
+.TP 15
+\fB--name\fR \fI<POOL_NAME>\fR
+Required. Human friendly name for new pool.
+.TP
+\fB--raid-type\fR \fI<RAID_TYPE>\fR
+Required. The RAID type of new pool. Valid values are:
+\fBNOT_APPLICABLE\fR, \fBJBOD\fR, \fBRAID0\fR, \fBRAID1\fR, \fBRAID5\fR,
+\fBRAID6\fR, \fBRAID10\fR, \fBRAID50\fR, \fBRAID51\fR, \fBRAID60\fR,
+\fBRAID61\fR.
+.br
+The \fBNOT_APPLICABLE\fR means pool only contain 1 volume.
+For supported RAID types of certain array, please use \fBcapabilities\fR
+command for POOL_CREATE_VOLUME_RAID_XXX entries.
+.TP
+\fB--member-id\fR \fI<VOL_ID>\fR
+Required. Repeatable. The ID of volume to create new pool.
+For two or more members: '\fB--member-id VOL_ID_A --member VOL_ID_B\fR'.
+.TP
+\fB--sys\fR \fI<SYS_ID>\fR
+Required. ID of the system to create new pool.
+
+.SS pool-create-from-pool
+Create a new sub-pool from specified pool. This require POOL_CREATE_FROM_POOLS
+capability.
+.TP 15
+\fB--name\fR \fI<POOL_NAME>\fR
+Required. Human friendly name for new pool.
+.TP
+\fB--size\fR \fI<POOL_SIZE>\fR
+Required. The spaces of new pool.
+See \fBSIZE OPTION\fR for allowed formats.
+.TP
+\fB--member-id\fR \fI<POOL_ID>\fR
+Required. The ID of pool to create new pool from.
+.TP
+\fB--sys\fR \fI<SYS_ID>\fR
+Required. ID of the system to create new pool.

.SS pool-delete
Deletes a storage pool.
diff --git a/lsm/lsm/cmdline.py b/lsm/lsm/cmdline.py
index 64d44a1..183c5b6 100644
--- a/lsm/lsm/cmdline.py
+++ b/lsm/lsm/cmdline.py
@@ -109,11 +109,29 @@ replicate_help = "replication type: " + ", ".join(replicate_types)

size_help = 'Can use B, KiB, MiB, GiB, TiB, PiB postfix (IEC sizing)'

-member_types = ('DISK', 'VOLUME', 'POOL')
-member_help = "Member type: " + ", ".join(member_types)
+member_types = ('DISK', 'VOLUME', 'POOL', 'DISK_ATA', 'DISK_SATA',
+ 'DISK_SAS', 'DISK_FC', 'DISK_SOP', 'DISK_SCSI', 'DISK_NL_SAS',
+ 'DISK_HDD', 'DISK_SSD', 'DISK_HYBRID')

-raid_types = ('JBOD', 'RAID1', 'RAID3', 'RAID4', 'RAID5', 'RAID6')
-raid_help = "RAID type: " + ", ".join(raid_types)
+member_types_formated = ''
+for i in range(0, len(member_types), 4):
+ member_types_formated += "\n "
+ for member_type_str in member_types[i:i+4]:
+ member_types_formated += "%-15s" % member_type_str
+
+member_help = "Valid member type: " + member_types_formated
+
+raid_types = ('JBOD', 'RAID0', 'RAID1', 'RAID3', 'RAID4', 'RAID5', 'RAID6',
+ 'RAID10', 'RAID50', 'RAID60', 'RAID51', 'RAID61',
+ 'NOT_APPLICABLE')
+
+raid_types_formated = ''
+for i in range(0, len(raid_types), 4):
+ raid_types_formated += "\n "
+ for raid_type_str in raid_types[i:i+4]:
+ raid_types_formated += "%-15s" % raid_type_str
+
+raid_help = "Valid RAID type:" + raid_types_formated

sys_id_opt = dict(name='--sys', metavar='<SYS_ID>', help='System ID')
pool_id_opt = dict(name='--pool', metavar='<POOL_ID>', help='Pool ID')
@@ -593,30 +611,92 @@ cmds = (
name='pool-create',
help='Creates a storage pool',
args=[
+ dict(sys_id_opt),
dict(name="--name", metavar="<POOL_NAME>",
help="Human friendly name for new pool"),
- dict(sys_id_opt),
+ dict(size_opt),
],
optional=[
- dict(name="--member-id", metavar='<MEMBER_ID>',
- help='Pool member ID, could be ID of Disk/Pool/Volume.\n'
- 'This is a repeatable argument',
- action='append'),
+ dict(name="--raid-type", metavar='<RAID_TYPE>',
+ help=raid_help,
+ choices=raid_types,
+ type=str.upper),
dict(name="--member-type", metavar='<MEMBER_TYPE>',
help=member_help,
choices=member_types),
+ dict(name=('-o', '--optional'),
+ help='Retrieve additional optional info if available',
+ default=False,
+ action='store_true'),
+ ],
+ ),
+
+ dict(
+ name='pool-create-from-disks',
+ help='Creates a storage pool from disks',
+ args=[
+ dict(sys_id_opt),
+ dict(name="--name", metavar="<POOL_NAME>",
+ help="Human friendly name for new pool"),
+ dict(name="--member-id", metavar='<MEMBER_ID>',
+ help='The ID of disks to create new pool\n'
+ 'This is a repeatable argument',
+ action='append'),
+ dict(name="--raid-type", metavar='<RAID_TYPE>',
+ help=raid_help,
+ choices=raid_types,
+ type=str.upper),
+ ],
+ optional=[
+ dict(name=('-o', '--optional'),
+ help='Retrieve additional optional info if available',
+ default=False,
+ action='store_true'),
+ ],
+ ),
+
+ dict(
+ name='pool-create-from-volumes',
+ help='Creates a storage pool from volumes',
+ args=[
+ dict(sys_id_opt),
+ dict(name="--name", metavar="<POOL_NAME>",
+ help="Human friendly name for new pool"),
+ dict(name="--member-id", metavar='<MEMBER_ID>',
+ help='The ID of volumes to create new pool\n'
+ 'This is a repeatable argument',
+ action='append'),
dict(name="--raid-type", metavar='<RAID_TYPE>',
help=raid_help,
choices=raid_types,
type=str.upper),
- dict(size_opt),
- dict(name="--thinp-type",
- metavar='<THINP_TYPE>',
- help="Thin Provisioning type, THIN, THICK",
- choices=['THIN', 'THICK']),
- dict(name="--member-count",
- help="Pool member count, 1 or greater. \n"
- "If bigger than defined --member-id, will auto choose"),
+ ],
+ optional=[
+ dict(name=('-o', '--optional'),
+ help='Retrieve additional optional info if available',
+ default=False,
+ action='store_true'),
+ ],
+ ),
+
+ dict(
+ name='pool-create-from-pool',
+ help='Creates a sub-pool from another storage pool',
+ args=[
+ dict(sys_id_opt),
+ dict(name="--name", metavar="<POOL_NAME>",
+ help="Human friendly name for new pool"),
+ dict(name="--member-id", metavar='<POOL_ID>',
+ help='The ID of pool to create new pool from\n',
+ action='append'),
+ dict(name="--size", metavar='<SIZE>',
+ help='The size of new pool'),
+ ],
+ optional=[
+ dict(name=('-o', '--optional'),
+ help='Retrieve additional optional info if available',
+ default=False,
+ action='store_true'),
],
),

@@ -1553,82 +1633,108 @@ class CmdLine:
def pool_create(self, args):
pool_name = args.name
raid_type = data.Pool.RAID_TYPE_UNKNOWN
- member_ids = []
member_type = data.Pool.MEMBER_TYPE_UNKNOWN
- member_count = 0
- thinp_type = data.Pool.THINP_TYPE_UNKNOWN
- size_bytes = 0
- prov_type = data.Pool.THINP_TYPE_UNKNOWN
+ size_bytes = self._size(self.args.size)

if args.raid_type:
raid_type = data.Pool.raid_type_str_to_type(
- self.args.opt_raid_type_str)
- if raid_type == data.Pool.RAID_TYPE_UNKNOWN or \
- raid_type == data.Pool.RAID_TYPE_NOT_APPLICABLE:
+ self.args.raid_type)
+ if raid_type == data.Pool.RAID_TYPE_UNKNOWN:
raise ArgError("Unknown RAID type specified: %s" %
- self.args.opt_raid_type_str)
-
- if len(args.member_id) >= 1:
- member_ids = args.member_id
-
- if args.size:
- size_bytes = self._size(self.args.opt_size)
+ args.raid_type)

if args.member_type:
member_type = data.Pool.member_type_str_to_type(
args.member_type)
-
- if member_ids and member_type != data.Pool.MEMBER_TYPE_UNKNOWN:
- if (member_type == data.Pool.MEMBER_TYPE_DISK):
- disks = self.c.disks()
- for member_id in member_ids:
- for disk in disks:
- if disk.id == member_id:
- break
- else:
- raise ArgError("Invalid Disk ID specified in " +
- "--member-id %s " % member_id)
- elif (member_type == data.Pool.MEMBER_TYPE_VOLUME):
- volumes = self.c.volumes()
- for member_id in member_ids:
- for volume in volumes:
- if volume.id == member_id:
- break
- else:
- raise ArgError("Invalid Volume ID specified in " +
- "--member-ids %s " % member_id)
- elif (member_type == data.Pool.MEMBER_TYPE_POOL):
- if not self.args.opt_size:
- raise ArgError("--size is mandatory when creating Pool " +
- "against another Pool")
- pools = self.c.pools()
- for member_id in member_ids:
- for pool in pools:
- if pool.id == member_id:
- break
- else:
- raise ArgError("Invalid Pool ID specified in " +
- "--member-ids %s " % member_id)
- else:
- raise ArgError("Unknown pool member-type %s, should be %s" %
- (args.member_type, '[DISK/VOLUME/POOL]'))
-
- if args.provisioning:
- provisioning = args.provisioning
- prov_type = data.Pool.thinp_type_str_to_type(provisioning)
+ if member_type == data.Pool.MEMBER_TYPE_UNKNOWN:
+ raise ArgError("Unkonwn member type specified: %s" %
+ args.member_type)

pool = self._wait_for_it("pool-create",
- *self.c.pool_create(self.args.opt_system,
+ *self.c.pool_create(self.args.sys,
pool_name,
+ size_bytes,
raid_type,
member_type,
- member_ids,
- member_count,
- size_bytes,
- prov_type,
0))
self.display_data([pool])

+ def pool_create_from_disks(self, args):
+ if len(args.member_id) <= 0:
+ raise ArgError("No disk ID was provided for new pool")
+
+ member_ids = args.member_id
+ disks = self.c.disks()
+ disk_ids = [d.id for d in disks]
+ for member_id in member_ids:
+ if member_id not in disk_ids:
+ raise ArgError("Invalid Disk ID specified in " +
+ "--member-id %s " % member_id)
+
+ raid_type = data.Pool.raid_type_str_to_type(
+ self.args.raid_type)
+ if raid_type == data.Pool.RAID_TYPE_UNKNOWN:
+ raise ArgError("Unknown RAID type specified: %s" %
+ self.args.raid_type)
+
+ pool_name = args.name
+ pool = self._wait_for_it(
+ "pool-create-from-disks",
+ *self.c.pool_create_from_disks(
+ self.args.sys, pool_name, member_ids, raid_type, 0))
+ self.display_data([pool])
+
+ def pool_create_from_volumes(self, args):
+ if len(args.member_id) <= 0:
+ raise ArgError("No volume ID was provided for new pool")
+
+ member_ids = args.member_id
+ volumes = self.c.volumes()
+ vol_ids = [v.id for v in volumes]
+ for member_id in member_ids:
+ if member_id not in vol_ids:
+ raise ArgError("Invalid volumes ID specified in " +
+ "--member-id %s " % member_id)
+
+ raid_type = data.Pool.raid_type_str_to_type(
+ self.args.raid_type)
+ if raid_type == data.Pool.RAID_TYPE_UNKNOWN:
+ raise ArgError("Unknown RAID type specified: %s" %
+ self.args.raid_type)
+
+ pool_name = args.name
+ pool = self._wait_for_it(
+ "pool-create-from-volumes",
+ *self.c.pool_create_from_volumes(
+ self.args.sys, pool_name, member_ids, raid_type, 0))
+ self.display_data([pool])
+
+ def pool_create_from_pool(self, args):
+ if len(args.member_id) <= 0:
+ raise ArgError("No volume ID was provided for new pool")
+
+ member_ids = args.member_id
+ if len(member_ids) > 1:
+ raise ArgError("Two or more member defined, but creating pool " +
+ "from pool only allow one member pool")
+
+ member_id = member_ids[0]
+
+ pools = self.c.pools()
+ pool_ids = [p.id for p in pools]
+ if member_id not in pool_ids:
+ raise ArgError("Invalid pools ID specified in " +
+ "--member-id %s " % member_id)
+
+ size_bytes = self._size(self.args.size)
+
+ pool_name = args.name
+ pool = self._wait_for_it(
+ "pool-create-from-pool",
+ *self.c.pool_create_from_pool(
+ self.args.sys, pool_name, member_id, size_bytes, 0))
+ self.display_data([pool])
+
def _read_configfile(self):
"""
Set uri from config file. Will be overridden by cmdline option or

[Gris Ge]

* Simulator version number bumped to 2.1.
User has to remove their old state file.
* Pool/Volume/Snapshot/NfsExport ID will not generated randomly, but
sequential. So that newly create item will be shown as the last one of
output.
* Added more disks for user to play on pool creation.
* Add support of all pool create/delete methods support:
1. Auto chose disk/volume/pool # pretty complex work flow.
2. Better pool/raid size calculation method.
* PEP8 passed.
* "make check" passed. # no pool create/delete tests there yet.

Signed-off-by: Gris Ge <***@redhat.com>
---
lsm/lsm/simarray.py | 1122 ++++++++++++++++++++++++++++++++++++++++++++------
lsm/lsm/simulator.py | 33 +-
2 files changed, 1033 insertions(+), 122 deletions(-)

diff --git a/lsm/lsm/simarray.py b/lsm/lsm/simarray.py
index 4074788..5d11133 100644
--- a/lsm/lsm/simarray.py
+++ b/lsm/lsm/simarray.py
@@ -26,9 +26,10 @@ import os
import time

from common import md5, LsmError, ErrorNumber, size_human_2_size_bytes, \
- JobStatus
+ JobStatus, size_bytes_2_size_human
from data import System, Volume, Disk, Pool, FileSystem, AccessGroup, \
- Initiator, Snapshot, NfsExport
+ Initiator, Snapshot, NfsExport, OptionalData
+

class SimJob(object):
"""
@@ -84,6 +85,11 @@ class SimArray(object):
"simulator, please move or delete %s" %
dump_file)

+ @staticmethod
+ def _sort_by_id(resources):
+ # isupper() just make sure the 'lsm_test_aggr' come as first one.
+ return sorted(resources, key=lambda k: (k.id.isupper(), k.id))
+
def __init__(self, dump_file=None):
if dump_file is None:
self.dump_file = SimArray.SIM_DATA_FILE
@@ -138,17 +144,66 @@ class SimArray(object):

def volumes(self):
sim_vols = self.data.volumes()
- return [SimArray._sim_vol_2_lsm(v) for v in sim_vols]
-
- def pools(self):
+ return SimArray._sort_by_id(
+ [SimArray._sim_vol_2_lsm(v) for v in sim_vols])
+
+ def _sim_pool_2_lsm(self, sim_pool, flags=0):
+ pool_id = sim_pool['pool_id']
+ name = sim_pool['name']
+ total_space = self.data._pool_total_space(pool_id)
+ free_space = self.data._pool_free_space(pool_id)
+ status = sim_pool['status']
+ sys_id = sim_pool['sys_id']
+ opt_data = OptionalData()
+ if flags == Pool.RETRIEVE_FULL_INFO:
+ opt_data.set('raid_type', sim_pool['raid_type'])
+ opt_data.set('member_type', sim_pool['member_type'])
+ opt_data.set('member_ids', sim_pool['member_ids'])
+ opt_data.set('thinp_type', Pool.THINP_TYPE_THIN)
+ opt_data.set('status_info', '')
+ opt_data.set('element_type', sim_pool['element_type'])
+
+ return Pool(pool_id, name, total_space, free_space, status, sys_id,
+ opt_data)
+
+ def pools(self, flags=0):
rc = []
sim_pools = self.data.pools()
for sim_pool in sim_pools:
- pool = Pool(sim_pool['pool_id'], sim_pool['name'],
- sim_pool['total_space'], sim_pool['free_space'],
- sim_pool['status'], sim_pool['sys_id'])
- rc.extend([pool])
- return rc
+ rc.extend([self._sim_pool_2_lsm(sim_pool, flags)])
+ return SimArray._sort_by_id(rc)
+
+ def pool_create(self, sys_id, pool_name, size_bytes,
+ raid_type=Pool.RAID_TYPE_UNKNOWN,
+ member_type=Pool.MEMBER_TYPE_UNKNOWN, flags=0):
+ sim_pool = self.data.pool_create(
+ sys_id, pool_name, size_bytes, raid_type, member_type, flags)
+ return self.data.job_create(
+ self._sim_pool_2_lsm(sim_pool, Pool.RETRIEVE_FULL_INFO))
+
+ def pool_create_from_disks(self, sys_id, pool_name, member_ids, raid_type,
+ flags=0):
+ sim_pool = self.data.pool_create_from_disks(
+ sys_id, pool_name, member_ids, raid_type, flags)
+ return self.data.job_create(
+ self._sim_pool_2_lsm(sim_pool, Pool.RETRIEVE_FULL_INFO))
+
+ def pool_create_from_volumes(self, sys_id, pool_name, member_ids,
+ raid_type, flags=0):
+ sim_pool = self.data.pool_create_from_volumes(
+ sys_id, pool_name, member_ids, raid_type, flags)
+ return self.data.job_create(
+ self._sim_pool_2_lsm(sim_pool, Pool.RETRIEVE_FULL_INFO))
+
+ def pool_create_from_pool(self, sys_id, pool_name, member_id, size_bytes,
+ flags=0):
+ sim_pool = self.data.pool_create_from_pool(
+ sys_id, pool_name, member_id, size_bytes, flags)
+ return self.data.job_create(
+ self._sim_pool_2_lsm(sim_pool, Pool.RETRIEVE_FULL_INFO))
+
+ def pool_delete(self, pool_id, flags=0):
+ return self.data.job_create(self.data.pool_delete(pool_id, flags))[0]

def disks(self):
rc = []
@@ -159,7 +214,7 @@ class SimArray(object):
int(sim_disk['total_space']/SimData.SIM_DATA_BLK_SIZE),
Disk.STATUS_OK, sim_disk['sys_id'])
rc.extend([disk])
- return rc
+ return SimArray._sort_by_id(rc)

def volume_create(self, pool_id, vol_name, size_bytes, thinp, flags=0):
sim_vol = self.data.volume_create(
@@ -210,7 +265,8 @@ class SimArray(object):

def fs(self):
sim_fss = self.data.fs()
- return [SimArray._sim_fs_2_lsm(f) for f in sim_fss]
+ return SimArray._sort_by_id(
+ [SimArray._sim_fs_2_lsm(f) for f in sim_fss])

def fs_create(self, pool_id, fs_name, size_bytes, flags=0):
sim_fs = self.data.fs_create(pool_id, fs_name, size_bytes, flags)
@@ -275,7 +331,8 @@ class SimArray(object):

def exports(self, flags=0):
sim_exps = self.data.exports(flags)
- return [SimArray._sim_exp_2_lsm(e) for e in sim_exps]
+ return SimArray._sort_by_id(
+ [SimArray._sim_exp_2_lsm(e) for e in sim_exps])

def fs_export(self, fs_id, exp_path, root_hosts, rw_hosts, ro_hosts,
anon_uid, anon_gid, auth_type, options, flags=0):
@@ -366,7 +423,7 @@ class SimData(object):
}

sim_vol = {
- 'vol_id': "VOL_ID_%s" % SimData._random_vpd(4),
+ 'vol_id': self._next_vol_id()
'vpd83': SimData._random_vpd(),
'name': vol_name,
'total_space': size_bytes,
@@ -408,14 +465,14 @@ class SimData(object):
'init_ids': [init_id,],
'sys_id': SimData.SIM_DATA_SYS_ID,
'name': name,
- 'ag_id': "AG_ID_%s" % SimData._random_vpd(4),
+ 'ag_id': self._next_ag_id()
}

self.fs_dict = {
FileSystem.id = sim_fs,
}
sim_fs = {
- 'fs_id': "FS_ID_%s" % SimData._random_vpd(4),
+ 'fs_id': self._next_fs_id(),
'name': fs_name,
'total_space': size_bytes,
'free_space': size_bytes,
@@ -434,7 +491,7 @@ class SimData(object):
Snapshot.id: sim_snap,
}
sim_snap = {
- 'snap_id': "SNAP_ID_%s" % SimData._random_vpd(4),
+ 'snap_id': self._next_snap_id(),
'name': snap_name,
'fs_id': fs_id,
'files': [file_path, ],
@@ -444,7 +501,7 @@ class SimData(object):
Export.id: sim_exp,
}
sim_exp = {
- 'exp_id': "EXP_ID_%s" % SimData._random_vpd(4),
+ 'exp_id': self._next_exp_id(),
'fs_id': fs_id,
'exp_path': exp_path,
'auth_type': auth_type,
@@ -455,12 +512,68 @@ class SimData(object):
'anon_gid': anon_gid,
'options': [option, ],
}
+
+ self.pool_dict = {
+ Pool.id: sim_pool,
+ }
+ sim_pool = {
+ 'name': pool_name,
+ 'pool_id': Pool.id,
+ 'raid_type': Pool.RAID_TYPE_XXXX,
+ 'member_ids': [ disk_id or pool_id or volume_id ],
+ 'member_type': Pool.MEMBER_TYPE_XXXX,
+ 'member_size': size_bytes # space allocated from each member pool.
+ # only for MEMBER_TYPE_POOL
+ 'status': SIM_DATA_POOL_STATUS,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ 'element_type': SimData.SIM_DATA_POOL_ELEMENT_TYPE,
+ }
"""
SIM_DATA_BLK_SIZE = 512
- SIM_DATA_VERSION = "2.0"
+ SIM_DATA_VERSION = "2.1"
SIM_DATA_SYS_ID = 'sim-01'
SIM_DATA_INIT_NAME = 'NULL'
- SIM_DATA_TMO = 30000 # ms
+ SIM_DATA_TMO = 30000 # ms
+ SIM_DATA_POOL_STATUS = Pool.STATUS_OK
+ SIM_DATA_DISK_DEFAULT_RAID = Pool.RAID_TYPE_RAID0
+ SIM_DATA_VOLUME_DEFAULT_RAID = Pool.RAID_TYPE_RAID0
+ SIM_DATA_POOL_ELEMENT_TYPE = Pool.ELEMENT_TYPE_FS \
+ | Pool.ELEMENT_TYPE_POOL \
+ | Pool.ELEMENT_TYPE_VOLUME
+
+ SIM_DATA_SYS_POOL_ELEMENT_TYPE = SIM_DATA_POOL_ELEMENT_TYPE \
+ | Pool.ELEMENT_TYPE_SYS_RESERVED
+
+ SIM_DATA_CUR_VOL_ID = 0
+ SIM_DATA_CUR_POOL_ID = 0
+ SIM_DATA_CUR_FS_ID = 0
+ SIM_DATA_CUR_AG_ID = 0
+ SIM_DATA_CUR_SNAP_ID = 0
+ SIM_DATA_CUR_EXP_ID = 0
+
+ def _next_pool_id(self):
+ self.SIM_DATA_CUR_POOL_ID += 1
+ return "POOL_ID_%08d" % self.SIM_DATA_CUR_POOL_ID
+
+ def _next_vol_id(self):
+ self.SIM_DATA_CUR_VOL_ID += 1
+ return "VOL_ID_%08d" % self.SIM_DATA_CUR_VOL_ID
+
+ def _next_fs_id(self):
+ self.SIM_DATA_CUR_FS_ID += 1
+ return "FS_ID_%08d" % self.SIM_DATA_CUR_FS_ID
+
+ def _next_ag_id(self):
+ self.SIM_DATA_CUR_AG_ID += 1
+ return "AG_ID_%08d" % self.SIM_DATA_CUR_AG_ID
+
+ def _next_snap_id(self):
+ self.SIM_DATA_CUR_SNAP_ID += 1
+ return "SNAP_ID_%08d" % self.SIM_DATA_CUR_SNAP_ID
+
+ def _next_exp_id(self):
+ self.SIM_DATA_CUR_EXP_ID += 1
+ return "EXP_ID_%08d" % self.SIM_DATA_CUR_EXP_ID

@staticmethod
def _state_signature():
@@ -479,35 +592,38 @@ class SimData(object):
System.STATUS_OK)]
pool_size_200g = size_human_2_size_bytes('200GiB')
self.pool_dict = {
- 'POO1': {
- 'pool_id': 'POO1',
- 'name': 'Pool 1',
- 'member_type': Pool.MEMBER_TYPE_DISK,
- 'member_ids': ['DISK_ID_000', 'DISK_ID_001'],
- 'raid_type': Pool.RAID_TYPE_RAID1,
- 'status': Pool.STATUS_OK,
- 'sys_id': SimData.SIM_DATA_SYS_ID,
- },
- 'POO2': {
- 'pool_id': 'POO2',
- 'name': 'Pool 2',
- 'total_space': pool_size_200g,
- 'member_type': Pool.MEMBER_TYPE_POOL,
- 'member_ids': ['POO1'],
- 'raid_type': Pool.RAID_TYPE_NOT_APPLICABLE,
- 'status': Pool.STATUS_OK,
- 'sys_id': SimData.SIM_DATA_SYS_ID,
- },
- # lsm_test_aggr pool is requred by test/runtest.sh
- 'lsm_test_aggr': {
- 'pool_id': 'lsm_test_aggr',
- 'name': 'lsm_test_aggr',
- 'member_type': Pool.MEMBER_TYPE_DISK,
- 'member_ids': ['DISK_ID_002', 'DISK_ID_003'],
- 'raid_type': Pool.RAID_TYPE_RAID0,
- 'status': Pool.STATUS_OK,
- 'sys_id': SimData.SIM_DATA_SYS_ID,
- },
+ 'POO1': {
+ 'pool_id': 'POO1',
+ 'name': 'Pool 1',
+ 'member_type': Pool.MEMBER_TYPE_DISK_SATA,
+ 'member_ids': ['DISK_ID_000', 'DISK_ID_001'],
+ 'raid_type': Pool.RAID_TYPE_RAID1,
+ 'status': SimData.SIM_DATA_POOL_STATUS,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ 'element_type': SimData.SIM_DATA_SYS_POOL_ELEMENT_TYPE,
+ },
+ 'POO2': {
+ 'pool_id': 'POO2',
+ 'name': 'Pool 2',
+ 'member_type': Pool.MEMBER_TYPE_POOL,
+ 'member_ids': ['POO1'],
+ 'member_size': pool_size_200g,
+ 'raid_type': Pool.RAID_TYPE_NOT_APPLICABLE,
+ 'status': Pool.STATUS_OK,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ 'element_type': SimData.SIM_DATA_POOL_ELEMENT_TYPE,
+ },
+ # lsm_test_aggr pool is requred by test/runtest.sh
+ 'lsm_test_aggr': {
+ 'pool_id': 'lsm_test_aggr',
+ 'name': 'lsm_test_aggr',
+ 'member_type': Pool.MEMBER_TYPE_DISK_SAS,
+ 'member_ids': ['DISK_ID_002', 'DISK_ID_003'],
+ 'raid_type': Pool.RAID_TYPE_RAID0,
+ 'status': Pool.STATUS_OK,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ 'element_type': SimData.SIM_DATA_POOL_ELEMENT_TYPE,
+ },
}
self.vol_dict = {
}
@@ -518,6 +634,7 @@ class SimData(object):
self.exp_dict = {
}
disk_size_2t = size_human_2_size_bytes('2TiB')
+ disk_size_512g = size_human_2_size_bytes('512GiB')
self.disk_dict = {
'DISK_ID_000': {
'disk_id': 'DISK_ID_000',
@@ -547,6 +664,133 @@ class SimData(object):
'disk_type': Disk.DISK_TYPE_SAS,
'sys_id': SimData.SIM_DATA_SYS_ID,
},
+ # More free disks for user to create pools on.
+ 'DISK_ID_004': {
+ 'disk_id': 'DISK_ID_004',
+ 'name': 'SAS Disk 004',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SAS,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_005': {
+ 'disk_id': 'DISK_ID_005',
+ 'name': 'SAS Disk 005',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SAS,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_006': {
+ 'disk_id': 'DISK_ID_006',
+ 'name': 'SAS Disk 006',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SAS,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_007': {
+ 'disk_id': 'DISK_ID_007',
+ 'name': 'SAS Disk 007',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SAS,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_008': {
+ 'disk_id': 'DISK_ID_008',
+ 'name': 'SAS Disk 008',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SAS,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_009': {
+ 'disk_id': 'DISK_ID_009',
+ 'name': 'SSD Disk 009',
+ 'total_space': disk_size_512g,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_010': {
+ 'disk_id': 'DISK_ID_010',
+ 'name': 'SSD Disk 010',
+ 'total_space': disk_size_512g,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_011': {
+ 'disk_id': 'DISK_ID_011',
+ 'name': 'SSD Disk 011',
+ 'total_space': disk_size_512g,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_012': {
+ 'disk_id': 'DISK_ID_012',
+ 'name': 'SSD Disk 012',
+ 'total_space': disk_size_512g,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_013': {
+ 'disk_id': 'DISK_ID_013',
+ 'name': 'SSD Disk 013',
+ 'total_space': disk_size_512g,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_014': {
+ 'disk_id': 'DISK_ID_014',
+ 'name': 'SSD Disk 014',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_015': {
+ 'disk_id': 'DISK_ID_015',
+ 'name': 'SSD Disk 015',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_016': {
+ 'disk_id': 'DISK_ID_016',
+ 'name': 'SSD Disk 016',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_017': {
+ 'disk_id': 'DISK_ID_017',
+ 'name': 'SSD Disk 017',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_018': {
+ 'disk_id': 'DISK_ID_018',
+ 'name': 'SSD Disk 018',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_019': {
+ 'disk_id': 'DISK_ID_019',
+ 'name': 'SSD Disk 019',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_020': {
+ 'disk_id': 'DISK_ID_020',
+ 'name': 'SSD Disk 020',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
+ 'DISK_ID_021': {
+ 'disk_id': 'DISK_ID_021',
+ 'name': 'SSD Disk 021',
+ 'total_space': disk_size_2t,
+ 'disk_type': Disk.DISK_TYPE_SSD,
+ 'sys_id': SimData.SIM_DATA_SYS_ID,
+ },
}
self.ag_dict = {
}
@@ -560,7 +804,7 @@ class SimData(object):
'POO1', 'Volume 001', size_human_2_size_bytes('200GiB'),
Volume.PROVISION_DEFAULT)

- self.pool_dict['POO3']= {
+ self.pool_dict['POO3'] = {
'pool_id': 'POO3',
'name': 'Pool 3',
'member_type': Pool.MEMBER_TYPE_VOLUME,
@@ -571,6 +815,7 @@ class SimData(object):
'raid_type': Pool.RAID_TYPE_RAID1,
'status': Pool.STATUS_OK,
'sys_id': SimData.SIM_DATA_SYS_ID,
+ 'element_type': SimData.SIM_DATA_POOL_ELEMENT_TYPE,
}

return
@@ -592,6 +837,11 @@ class SimData(object):
if free_space <= sim_fs['consume_size']:
return 0
free_space -= sim_fs['consume_size']
+ for sim_pool in self.pool_dict.values():
+ if sim_pool['member_type'] != Pool.MEMBER_TYPE_POOL:
+ continue
+ if pool_id in sim_pool['member_ids']:
+ free_space -= sim_pool['member_size']
return free_space

@staticmethod
@@ -604,54 +854,91 @@ class SimData(object):
vpd.append(str('%02X' % (random.randint(0, 255))))
return "".join(vpd)

- def _pool_total_space(self, pool_id):
- """
- Find out the correct size of RAID pool
- """
- member_type = self.pool_dict[pool_id]['member_type']
- if member_type == Pool.MEMBER_TYPE_POOL:
- return self.pool_dict[pool_id]['total_space']
-
- all_size = 0
- item_size = 0 # disk size, used by RAID 3/4/5/6
- member_ids = self.pool_dict[pool_id]['member_ids']
- raid_type = self.pool_dict[pool_id]['raid_type']
- member_count = len(member_ids)
-
- if member_type == Pool.MEMBER_TYPE_DISK:
+ def _size_of_raid(self, member_type, member_ids, raid_type,
+ pool_each_size=0):
+ member_sizes = []
+ if Pool.member_type_is_disk(member_type):
for member_id in member_ids:
- all_size += self.disk_dict[member_id]['total_space']
- item_size = self.disk_dict[member_id]['total_space']
+ member_sizes.extend([self.disk_dict[member_id]['total_space']])

elif member_type == Pool.MEMBER_TYPE_VOLUME:
for member_id in member_ids:
- all_size += self.vol_dict[member_id]['total_space']
- item_size = self.vol_dict[member_id]['total_space']
+ member_sizes.extend([self.vol_dict[member_id]['total_space']])

- if raid_type == Pool.RAID_TYPE_JBOD:
- return int(all_size)
- elif raid_type == Pool.RAID_TYPE_RAID0:
+ elif member_type == Pool.MEMBER_TYPE_POOL:
+ for member_id in member_ids:
+ member_sizes.extend([pool_each_size])
+
+ else:
+ raise LsmError(ErrorNumber.INTERNAL_ERROR,
+ "Got unsupported member_type in _size_of_raid()" +
+ ": %d" % member_type)
+ all_size = 0
+ member_size = 0
+ member_count = len(member_ids)
+ for member_size in member_sizes:
+ all_size += member_size
+
+ if raid_type == Pool.RAID_TYPE_JBOD or \
+ raid_type == Pool.RAID_TYPE_NOT_APPLICABLE or \
+ raid_type == Pool.RAID_TYPE_RAID0:
return int(all_size)
- elif raid_type == Pool.RAID_TYPE_RAID1 or \
- raid_type == Pool.RAID_TYPE_RAID10:
+ elif (raid_type == Pool.RAID_TYPE_RAID1 or
+ raid_type == Pool.RAID_TYPE_RAID10):
+ if member_count % 2 == 1:
+ return 0
return int(all_size/2)
- elif raid_type == Pool.RAID_TYPE_RAID3 or \
- raid_type == Pool.RAID_TYPE_RAID4 or \
- raid_type == Pool.RAID_TYPE_RAID5 or \
- raid_type == Pool.RAID_TYPE_RAID50:
- return int(all_size - item_size)
- elif raid_type == Pool.RAID_TYPE_RAID6 or \
- raid_type == Pool.RAID_TYPE_RAID60:
- return int(all_size - item_size - item_size)
+ elif (raid_type == Pool.RAID_TYPE_RAID3 or
+ raid_type == Pool.RAID_TYPE_RAID4 or
+ raid_type == Pool.RAID_TYPE_RAID5):
+ if member_count < 3:
+ return 0
+ return int(all_size - member_size)
+ elif raid_type == Pool.RAID_TYPE_RAID50:
+ if member_count < 6 or member_count % 2 == 1:
+ return 0
+ return int(all_size - member_size*2)
+ elif raid_type == Pool.RAID_TYPE_RAID6:
+ if member_count < 4:
+ return 0
+ return int(all_size - member_size * 2)
+ elif raid_type == Pool.RAID_TYPE_RAID60:
+ if member_count < 8 or member_count % 2 == 1:
+ return 0
+ return int(all_size - member_size*4)
elif raid_type == Pool.RAID_TYPE_RAID51:
- return int((all_size - item_size)/2)
+ if member_count < 6 or member_count % 2 == 1:
+ return 0
+ return int(all_size/2 - member_size)
elif raid_type == Pool.RAID_TYPE_RAID61:
- return int((all_size - item_size - item_size)/2)
- return 0
+ if member_count < 8 or member_count % 2 == 1:
+ return 0
+ print "%s" % size_bytes_2_size_human(all_size)
+ print "%s" % size_bytes_2_size_human(member_size)
+ return int(all_size/2 - member_size*2)
+ raise LsmError(ErrorNumber.INTERNAL_ERROR,
+ "_size_of_raid() got invalid raid type: " +
+ "%s(%d)" % (Pool.raid_type_to_str(raid_type),
+ raid_type))
+
+ def _pool_total_space(self, pool_id):
+ """
+ Find out the correct size of RAID pool
+ """
+ sim_pool = self.pool_dict[pool_id]
+ each_pool_size_bytes = 0
+ member_type = sim_pool['member_type']
+ if sim_pool['member_type'] == Pool.MEMBER_TYPE_POOL:
+ each_pool_size_bytes = sim_pool['member_size']
+
+ return self._size_of_raid(
+ member_type, sim_pool['member_ids'], sim_pool['raid_type'],
+ each_pool_size_bytes)

@staticmethod
def _block_rounding(size_bytes):
- return (size_bytes / SimData.SIM_DATA_BLK_SIZE + 1) * \
+ return (size_bytes + SimData.SIM_DATA_BLK_SIZE - 1) / \
+ SimData.SIM_DATA_BLK_SIZE * \
SimData.SIM_DATA_BLK_SIZE

def job_create(self, returned_item):
@@ -688,14 +975,7 @@ class SimData(object):
return self.syss

def pools(self):
- rc = []
- for sim_pool in self.pool_dict.values():
- sim_pool['total_space'] = \
- self._pool_total_space(sim_pool['pool_id'])
- sim_pool['free_space'] = \
- self._pool_free_space(sim_pool['pool_id'])
- rc.extend([sim_pool])
- return rc
+ return self.pool_dict.values()

def volumes(self):
return self.vol_dict.values()
@@ -714,8 +994,7 @@ class SimData(object):
raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
"Insufficient space in pool")
sim_vol = dict()
- vol_id = "VOL_ID_%s" % SimData._random_vpd(4)
- sim_vol['vol_id'] = vol_id
+ sim_vol['vol_id'] = self._next_vol_id()
sim_vol['vpd83'] = SimData._random_vpd()
sim_vol['name'] = vol_name
sim_vol['total_space'] = size_bytes
@@ -723,7 +1002,7 @@ class SimData(object):
sim_vol['sys_id'] = SimData.SIM_DATA_SYS_ID
sim_vol['pool_id'] = pool_id
sim_vol['consume_size'] = size_bytes
- self.vol_dict[vol_id] = sim_vol
+ self.vol_dict[sim_vol['vol_id']] = sim_vol
return sim_vol

def volume_delete(self, vol_id, flags=0):
@@ -761,17 +1040,16 @@ class SimData(object):
raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
"Insufficient space in pool")
sim_vol = dict()
- vol_id = "VOL_ID_%s" % SimData._random_vpd(4)
- sim_vol['vol_id'] = vol_id
+ sim_vol['vol_id'] = self._next_vol_id()
sim_vol['vpd83'] = SimData._random_vpd()
sim_vol['name'] = new_vol_name
sim_vol['total_space'] = size_bytes
sim_vol['sys_id'] = SimData.SIM_DATA_SYS_ID
sim_vol['pool_id'] = dst_pool_id
sim_vol['consume_size'] = size_bytes
- self.vol_dict[vol_id] = sim_vol
+ self.vol_dict[sim_vol['vol_id']] = sim_vol

- dst_vol_id = vol_id
+ dst_vol_id = sim_vol['vol_id']
if 'replicate' not in self.vol_dict[src_vol_id].keys():
self.vol_dict[src_vol_id]['replicate'] = dict()

@@ -886,7 +1164,7 @@ class SimData(object):
sim_ag['init_ids'] = [init_id]
sim_ag['sys_id'] = SimData.SIM_DATA_SYS_ID
sim_ag['name'] = name
- sim_ag['ag_id'] = "AG_ID_%s" % SimData._random_vpd(4)
+ sim_ag['ag_id'] = self._next_ag_id()
self.ag_dict[sim_ag['ag_id']] = sim_ag
return sim_ag

@@ -1083,7 +1361,7 @@ class SimData(object):
raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
"Insufficient space in pool")
sim_fs = dict()
- fs_id = "FS_ID_%s" % SimData._random_vpd(4)
+ fs_id = self._next_fs_id()
sim_fs['fs_id'] = fs_id
sim_fs['name'] = fs_name
sim_fs['total_space'] = size_bytes
@@ -1161,7 +1439,7 @@ class SimData(object):
if 'snaps' not in self.fs_dict[fs_id].keys():
self.fs_dict[fs_id]['snaps'] = []

- snap_id = "SNAP_ID_%s" % SimData._random_vpd(4)
+ snap_id = self._next_snap_id()
sim_snap = dict()
sim_snap['snap_id'] = snap_id
sim_snap['name'] = snap_name
@@ -1273,7 +1551,7 @@ class SimData(object):
raise LsmError(ErrorNumber.INVALID_INIT,
"No such File System: %s" % fs_id)
sim_exp = dict()
- sim_exp['exp_id'] = "EXP_ID_%s" % SimData._random_vpd(4)
+ sim_exp['exp_id'] = self._next_exp_id()
sim_exp['fs_id'] = fs_id
if exp_path is None:
sim_exp['exp_path'] = "/%s" % sim_exp['exp_id']
@@ -1296,18 +1574,628 @@ class SimData(object):
del self.exp_dict[exp_id]
return None

- def pool_create(self,
- system_id,
- pool_name='',
- raid_type=Pool.RAID_TYPE_UNKNOWN,
- member_type=Pool.MEMBER_TYPE_UNKNOWN,
- member_ids=None,
- member_count=0,
- size_bytes=0,
- thinp_type=Pool.THINP_TYPE_UNKNOWN,
- flags=0):
+ def _free_disks_list(self, disk_type=Disk.DISK_TYPE_UNKNOWN):
+ """
+ Return a list of free sim_disk.
+ Return [] if no free disk found.
+ """
+ free_sim_disks = []
+ for sim_disk in self.disk_dict.values():
+ if disk_type != Disk.DISK_TYPE_UNKNOWN and \
+ sim_disk['disk_type'] != disk_type:
+ continue
+ flag_free = True
+ for sim_pool in self.pool_dict.values():
+ if Pool.member_type_is_disk(sim_pool['member_type']) and \
+ sim_disk['disk_id'] in sim_pool['member_ids']:
+ flag_free = False
+ break
+ if flag_free is True:
+ free_sim_disks.extend([sim_disk])
+ return sorted(free_sim_disks, key=lambda k: (k['disk_id']))
+
+ def _free_disks(self, disk_type=Disk.DISK_TYPE_UNKNOWN):
+ """
+ Return a dictionary like this:
+ {
+ Disk.DISK_TYPE_XXX: {
+ Disk.total_space: [sim_disk, ]
+ }
+ }
+ Return None if not free.
+ """
+ free_sim_disks = self._free_disks_list()
+ rc = dict()
+ for sim_disk in free_sim_disks:
+ if disk_type != Disk.DISK_TYPE_UNKNOWN and \
+ sim_disk['disk_type'] != disk_type:
+ continue
+
+ cur_type = sim_disk['disk_type']
+ cur_size = sim_disk['total_space']
+
+ if cur_type not in rc.keys():
+ rc[cur_type] = dict()
+
+ if cur_size not in rc[cur_type]:
+ rc[cur_type][cur_size] = []
+
+ rc[cur_type][cur_size].extend([sim_disk])
+
+ return rc
+
+ def _free_volumes_list(self):
+ rc = []
+ for sim_vol in self.vol_dict.values():
+ flag_free = True
+ for sim_pool in self.pool_dict.values():
+ if sim_pool['member_type'] == Pool.MEMBER_TYPE_VOLUME and \
+ sim_vol['vol_id'] in sim_pool['member_ids']:
+ flag_free = False
+ break
+ if flag_free:
+ rc.extend([sim_vol])
+ return sorted(rc, key=lambda k: (k['vol_id']))
+
+ def _free_volumes(self, size_bytes=0):
+ """
+ We group sim_vol based on theri total_space because RAID (except
+ JBOD) require member in the same spaces.
+ Return a dictionary like this:
+ {
+ sim_vol['total_space']: [sim_vol, ]
+ }
+ """
+ free_sim_vols = self._free_volumes_list()
+ if len(free_sim_vols) <= 0:
+ return dict()
+ rc = dict()
+ for sim_vol in free_sim_vols:
+ if size_bytes != 0 and sim_vol['total_space'] != size_bytes:
+ continue
+ # TODO: one day we will introduce free_size of Volume.
+ # in that case we will check whether
+ # total_space == vol_free_size(sim_vol['vol_id'])
+
+ if sim_vol['total_space'] not in rc.keys():
+ rc[sim_vol['total_space']] = []
+ rc[sim_vol['total_space']].extend([sim_vol])
+ return rc
+
+ def _free_pools_list(self):
+ """
+ Return a list of sim_pool or []
+ """
+ free_sim_pools = []
+ for sim_pool in self.pool_dict.values():
+ # TODO: one day we will introduce free_size of Volume.
+ # in that case we will check whether
+ # total_space == pool_free_size(sim_pool['pool_id'])
+ pool_id = sim_pool['pool_id']
+ if self._pool_free_space(pool_id) > 0:
+ free_sim_pools.extend([sim_pool])
+ return sorted(
+ free_sim_pools,
+ key=lambda k: (k['pool_id'].isupper(), k['pool_id']))
+
+ def _pool_create_from_disks(self, pool_name, member_ids, raid_type,
+ raise_error=False):
+ # Check:
+ # 1. The disk_id is valid
+ # 2. All disks are the same disk type.
+ # 3. All disks are free.
+ # 4. All disks' total space is the same.
+ if len(member_ids) <= 0:
+ if raise_error:
+ raise LsmError(ErrorNumber.INVALID_DISK,
+ "No disk ID defined")
+ else:
+ return None
+
+ if raid_type == Pool.RAID_TYPE_NOT_APPLICABLE and \
+ len(member_ids) >= 2:
+ if raise_error:
+ raise LsmError(ErrorNumber.INVALID_ARGUMENT,
+ "Pool.RAID_TYPE_NOT_APPLICABLE means only 1 " +
+ "member, but got 2 or more: %s" %
+ ', '.join(member_ids))
+ else:
+ return None
+
+ current_disk_type = None
+ current_total_space = None
+ for disk_id in member_ids:
+ if disk_id not in self.disk_dict.keys():
+ if raise_error:
+ raise LsmError(ErrorNumber.INVALID_DISK,
+ "The disk ID %s does not exist" % disk_id)
+ else:
+ return None
+ sim_disk = self.disk_dict[disk_id]
+ if current_disk_type is None:
+ current_disk_type = sim_disk['disk_type']
+ elif current_disk_type != sim_disk['disk_type']:
+ if raise_error:
+ raise LsmError(
+ ErrorNumber.NO_SUPPORT,
+ "Mixing disk types in one pool " +
+ "is not supported: %s and %s" %
+ (Disk.disk_type_to_str(current_disk_type),
+ Disk.disk_type_to_str(sim_disk['disk_type'])))
+ else:
+ return None
+ if current_total_space is None:
+ current_total_space = sim_disk['total_space']
+ elif current_total_space != sim_disk['total_space']:
+ if raise_error:
+ raise LsmError(ErrorNumber.NO_SUPPORT,
+ "Mixing different size of disks is not " +
+ "supported")
+ else:
+ return None
+
+ all_free_disks = self._free_disks_list()
+ if all_free_disks is None:
+ if raise_error:
+ raise LsmError(ErrorNumber.DISK_BUSY,
+ "No free disk to create new pool")
+ else:
+ return None
+ all_free_disk_ids = [d['disk_id'] for d in all_free_disks]
+ for disk_id in member_ids:
+ if disk_id not in all_free_disk_ids:
+ if raise_error:
+ raise LsmError(ErrorNumber.DISK_BUSY,
+ "Disk %s is used by other pool" % disk_id)
+ else:
+ return None
+
+ if raid_type == Pool.RAID_TYPE_UNKNOWN or \
+ raid_type == Pool.RAID_TYPE_MIXED:
+ if raise_error:
+ raise LsmError(ErrorNumber.NO_SUPPORT,
+ "RAID type %s(%d) is not supported" %
+ (Pool.raid_type_to_str(raid_type), raid_type))
+ else:
+ return None
+
+ pool_id = self._next_pool_id()
if pool_name == '':
pool_name = 'POOL %s' % SimData._random_vpd(4)

- ## Coding
- return
+ sim_pool = dict()
+ sim_pool['name'] = pool_name
+ sim_pool['pool_id'] = pool_id
+ if len(member_ids) == 1:
+ sim_pool['raid_type'] = Pool.RAID_TYPE_NOT_APPLICABLE
+ else:
+ sim_pool['raid_type'] = raid_type
+ sim_pool['member_ids'] = member_ids
+ sim_pool['member_type'] = \
+ Pool.disk_type_to_member_type(current_disk_type)
+ sim_pool['sys_id'] = SimData.SIM_DATA_SYS_ID
+ sim_pool['element_type'] = SimData.SIM_DATA_POOL_ELEMENT_TYPE
+ sim_pool['status'] = SimData.SIM_DATA_POOL_STATUS
+ self.pool_dict[pool_id] = sim_pool
+ return sim_pool
+
+ def pool_create_from_disks(self, sys_id, pool_name, member_ids, raid_type,
+ flags=0):
+ """
+ return newly create sim_pool or None.
+ """
+ if sys_id != SimData.SIM_DATA_SYS_ID:
+ raise LsmError(ErrorNumber.INVALID_SYSTEM,
+ "No such system: %s" % sys_id)
+
+ return self._pool_create_from_disks(pool_name, member_ids, raid_type,
+ raise_error=True)
+
+ def _pool_create_from_volumes(self, pool_name, member_ids, raid_type,
+ raise_error=False):
+ # Check:
+ # 1. The vol_id is valid
+ # 3. All volumes are free.
+ if len(member_ids) == 0:
+ if raise_error:
+ raise LsmError(ErrorNumber.INVALID_VOLUME,
+ "No volume ID defined")
+ else:
+ return None
+
+ if raid_type == Pool.RAID_TYPE_NOT_APPLICABLE and \
+ len(member_ids) >= 2:
+ if raise_error:
+ raise LsmError(ErrorNumber.INVALID_ARGUMENT,
+ "Pool.RAID_TYPE_NOT_APPLICABLE means only 1 " +
+ "member, but got 2 or more: %s" %
+ ', '.join(member_ids))
+ else:
+ return None
+
+ current_vol_size = None
+ for vol_id in member_ids:
+ if vol_id not in self.vol_dict.keys() and raise_error:
+ if raise_error:
+ raise LsmError(ErrorNumber.INVALID_DISK,
+ "The vol ID %s does not exist" % vol_id)
+ else:
+ return None
+ sim_vol = self.vol_dict[vol_id]
+ if current_vol_size is None:
+ current_vol_size = sim_vol['total_space']
+ elif current_vol_size != sim_vol['total_space']:
+ if raise_error:
+ raise LsmError(ErrorNumber.NO_SUPPORT,
+ "Mixing volume size in one pool " +
+ "is not supported: %d and %d" %
+ (current_vol_size, sim_vol['total_space']))
+ else:
+ return None
+
+ all_free_vols = self._free_volumes_list()
+ if all_free_vols is None:
+ if raise_error:
+ raise LsmError(ErrorNumber.VOLUME_BUSY,
+ "No free volume to create new pool")
+ else:
+ return None
+ all_free_vol_ids = [v['vol_id'] for v in all_free_vols]
+ for vol_id in member_ids:
+ if vol_id not in all_free_vol_ids:
+ if raise_error:
+ raise LsmError(ErrorNumber.VOLUME_BUSY,
+ "Volume %s is used by other pool" % vol_id)
+ else:
+ return None
+
+ if raid_type == Pool.RAID_TYPE_UNKNOWN or \
+ raid_type == Pool.RAID_TYPE_MIXED:
+ if raise_error:
+ raise LsmError(ErrorNumber.NO_SUPPORT,
+ "RAID type %s(%d) is not supported" %
+ (Pool.raid_type_to_str(raid_type), raid_type))
+ else:
+ return None
+
+ pool_id = self._next_pool_id()
+ if pool_name == '':
+ pool_name = 'POOL %s' % SimData._random_vpd(4)
+ sim_pool = dict()
+ sim_pool['name'] = pool_name
+ sim_pool['pool_id'] = pool_id
+ if len(member_ids) == 1:
+ sim_pool['raid_type'] = Pool.RAID_TYPE_NOT_APPLICABLE
+ else:
+ sim_pool['raid_type'] = raid_type
+ sim_pool['member_ids'] = member_ids
+ sim_pool['member_type'] = Pool.MEMBER_TYPE_VOLUME
+ sim_pool['sys_id'] = SimData.SIM_DATA_SYS_ID
+ sim_pool['element_type'] = SimData.SIM_DATA_POOL_ELEMENT_TYPE
+ sim_pool['status'] = SimData.SIM_DATA_POOL_STATUS
+ self.pool_dict[pool_id] = sim_pool
+ return sim_pool
+
+ def pool_create_from_volumes(self, sys_id, pool_name, member_ids,
+ raid_type, flags=0):
+ if sys_id != SimData.SIM_DATA_SYS_ID:
+ raise LsmError(ErrorNumber.INVALID_SYSTEM,
+ "No such system: %s" % sys_id)
+ return self._pool_create_from_volumes(pool_name, member_ids, raid_type,
+ raise_error=True)
+
+ def _pool_create_from_pool(self, pool_name, member_id,
+ size_bytes, raise_error=False):
+
+ size_bytes = SimData._block_rounding(size_bytes)
+ free_sim_pools = self._free_pools_list()
+ free_sim_pool_ids = [p['pool_id'] for p in free_sim_pools]
+ if len(free_sim_pool_ids) == 0 or \
+ member_id not in free_sim_pool_ids:
+ if raise_error:
+ raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
+ "Pool %s " % member_id +
+ "is full, no space to create new pool")
+ else:
+ return None
+
+ free_size = self._pool_free_space(member_id)
+ if free_size < size_bytes:
+ raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
+ "Pool %s does not have requested free" %
+ member_id + "to create new pool")
+
+ pool_id = self._next_pool_id()
+ if pool_name == '':
+ pool_name = 'POOL %s' % SimData._random_vpd(4)
+ sim_pool = dict()
+ sim_pool['name'] = pool_name
+ sim_pool['pool_id'] = pool_id
+ sim_pool['raid_type'] = Pool.RAID_TYPE_NOT_APPLICABLE
+ sim_pool['member_ids'] = [member_id]
+ sim_pool['member_type'] = Pool.MEMBER_TYPE_POOL
+ sim_pool['member_size'] = size_bytes
+ sim_pool['sys_id'] = SimData.SIM_DATA_SYS_ID
+ sim_pool['element_type'] = SimData.SIM_DATA_POOL_ELEMENT_TYPE
+ sim_pool['status'] = SimData.SIM_DATA_POOL_STATUS
+ self.pool_dict[pool_id] = sim_pool
+ return sim_pool
+
+ def pool_create_from_pool(self, sys_id, pool_name, member_id, size_bytes,
+ flags=0):
+ if sys_id != SimData.SIM_DATA_SYS_ID:
+ raise LsmError(ErrorNumber.INVALID_SYSTEM,
+ "No such system: %s" % sys_id)
+ return self._pool_create_from_pool(pool_name, member_id, size_bytes,
+ raise_error=True)
+
+ def _auto_choose_disk(self, size_bytes, raid_type, disk_type,
+ raise_error=False):
+ """
+ Return a list of member ids suitable for creating RAID pool with
+ required size_bytes.
+ Return [] if nothing found.
+ if raise_error is True, raise error if not found
+ """
+ disk_type_str = "disk"
+ if disk_type != Disk.DISK_TYPE_UNKNOWN:
+ disk_type_str = "disk(type: %s)" % Disk.disk_type_to_str(disk_type)
+
+ if raid_type == Pool.RAID_TYPE_NOT_APPLICABLE:
+ # NOT_APPLICABLE means pool will only contain one disk.
+ sim_disks = self._free_disks_list(disk_type)
+ if len(sim_disks) == 0:
+ if raise_error:
+ raise LsmError(ErrorNumber.DISK_BUSY,
+ "No free %s found" % disk_type_str)
+ else:
+ return []
+
+ for sim_disk in sim_disks:
+ if sim_disk['total_space'] >= size_bytes:
+ return [sim_disk]
+ if raise_error:
+ raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
+ "No %s is bigger than " % disk_type_str +
+ "expected size: %s(%d)" %
+ (size_bytes_2_size_human(size_bytes),
+ size_bytes))
+ else:
+ return []
+
+ if raid_type == Pool.RAID_TYPE_JBOD:
+ # JBOD does not require all disks in the same size or the same type.
+ sim_disks = self._free_disks_list(disk_type)
+ if len(sim_disks) == 0:
+ if raise_error:
+ raise LsmError(ErrorNumber.DISK_BUSY,
+ "No free %s found" % disk_type_str)
+ else:
+ return []
+
+ chose_sim_disks = []
+ all_free_size = 0
+ for sim_disk in sim_disks:
+ chose_sim_disks.extend([sim_disk])
+ all_free_size += sim_disk['total_space']
+ if all_free_size >= size_bytes:
+ return chose_sim_disks
+ if raise_error:
+ raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
+ "No enough %s to provide size %s(%d)" %
+ (disk_type_str,
+ size_bytes_2_size_human(size_bytes),
+ size_bytes))
+ else:
+ return []
+
+ # All rest RAID type require member are in the same size and same
+ # type.
+ sim_disks_struct = self._free_disks(disk_type)
+ for cur_disk_type in sim_disks_struct.keys():
+ for cur_disk_size in sim_disks_struct[cur_disk_type].keys():
+ cur_sim_disks = sim_disks_struct[cur_disk_type][cur_disk_size]
+ if len(cur_sim_disks) == 0:
+ continue
+ chose_sim_disks = []
+ for member_count in range(1, len(cur_sim_disks) + 1):
+ partial_sim_disks = cur_sim_disks[0:member_count]
+ member_ids = [x['disk_id'] for x in partial_sim_disks]
+ raid_actual_size = self._size_of_raid(
+ Pool.MEMBER_TYPE_DISK, member_ids, raid_type)
+ if size_bytes <= raid_actual_size:
+ return cur_sim_disks[0:member_count]
+
+ if raise_error:
+ raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
+ "No enough %s " % disk_type_str +
+ "to create %s providing size: %s(%d)" %
+ (Pool.raid_type_to_str(raid_type),
+ size_bytes_2_size_human(size_bytes),
+ size_bytes))
+ else:
+ return []
+
+ def _auto_choose_vol(self, size_bytes, raid_type, raise_error=False):
+ """
+ Return a list of member ids suitable for creating RAID pool with
+ required size_bytes.
+ Return [] if nothing found.
+ Raise LsmError if raise_error is True
+ """
+ if raid_type == Pool.RAID_TYPE_NOT_APPLICABLE:
+ # NOT_APPLICABLE means pool will only contain one volume.
+ sim_vols = self._free_volumes_list()
+ if len(sim_vols) == 0:
+ if raise_error:
+ raise LsmError(ErrorNumber.VOLUME_BUSY,
+ "No free volume found")
+ else:
+ return []
+ for sim_vol in sim_vols:
+ if sim_vol['total_space'] >= size_bytes:
+ return [sim_vol]
+
+ if raise_error:
+ raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
+ "No volume is bigger than expected size: " +
+ "%s(%d)" %
+ (size_bytes_2_size_human(size_bytes),
+ size_bytes))
+ else:
+ return []
+
+ if raid_type == Pool.RAID_TYPE_JBOD:
+ # JBOD does not require all vols in the same size.
+ sim_vols = self._free_volumes_list()
+ if len(sim_vols) == 0:
+ if raise_error:
+ raise LsmError(ErrorNumber.VOLUME_BUSY,
+ "No free volume found")
+ else:
+ return []
+
+ chose_sim_vols = []
+ all_free_size = 0
+ for sim_vol in sim_vols:
+ chose_sim_vols.extend([sim_vol])
+ all_free_size += sim_vol['total_space']
+ if all_free_size >= size_bytes:
+ return chose_sim_vols
+
+ if raise_error:
+ raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
+ "No enough volumes to provide size %s(%d)" %
+ (size_bytes_2_size_human(size_bytes),
+ size_bytes))
+ else:
+ return []
+
+ # Rest RAID types require volume to be the same size.
+ sim_vols_dict = self._free_volumes()
+ for vol_size in sim_vols_dict.keys():
+ sim_vols = sim_vols_dict[vol_size]
+ if len(sim_vols) == 0:
+ continue
+ for member_count in range(1, len(sim_vols) + 1):
+ partial_sim_vols = sim_vols[0:member_count]
+ member_ids = [v['vol_id'] for v in partial_sim_vols]
+ raid_actual_size = self._size_of_raid(
+ Pool.MEMBER_TYPE_VOLUME, member_ids, raid_type)
+ if size_bytes <= raid_actual_size:
+ return sim_vols[0:member_count]
+
+ if raise_error:
+ raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
+ "No enough volumes to create "
+ "%s providing size: %s(%d)" %
+ (Pool.raid_type_to_str(raid_type),
+ size_bytes_2_size_human(size_bytes),
+ size_bytes))
+ else:
+ return []
+
+ def _auto_choose_pool(self, size_bytes, raise_error=False):
+ """
+ Return a sim_pool.
+ Return None if not found.
+ """
+ sim_pools = self._free_pools_list()
+ if len(sim_pools) >= 1:
+ for sim_pool in sim_pools:
+ pool_id = sim_pool['pool_id']
+ free_size = self._pool_free_space(pool_id)
+ if free_size >= size_bytes:
+ return sim_pool
+
+ if raise_error:
+ raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
+ "No pool is bigger than expected size: " +
+ "%s(%d)" %
+ (size_bytes_2_size_human(size_bytes),
+ size_bytes))
+ else:
+ return None
+
+ def pool_create(self, sys_id, pool_name, size_bytes,
+ raid_type=Pool.RAID_TYPE_UNKNOWN,
+ member_type=Pool.MEMBER_TYPE_UNKNOWN, flags=0):
+ if sys_id != SimData.SIM_DATA_SYS_ID:
+ raise LsmError(ErrorNumber.INVALID_SYSTEM,
+ "No such system: %s" % sys_id)
+
+ size_bytes = SimData._block_rounding(size_bytes)
+
+ raise_error = False
+ if member_type != Pool.MEMBER_TYPE_UNKNOWN:
+ raise_error = True
+
+ if member_type == Pool.MEMBER_TYPE_UNKNOWN or \
+ Pool.member_type_is_disk(member_type):
+ disk_raid_type = raid_type
+ if raid_type == Pool.RAID_TYPE_UNKNOWN:
+ disk_raid_type = SimData.SIM_DATA_DISK_DEFAULT_RAID
+ if member_type == Pool.MEMBER_TYPE_UNKNOWN:
+ disk_type = Disk.DISK_TYPE_UNKNOWN
+ else:
+ disk_type = Pool.member_type_to_disk_type(member_type)
+ sim_disks = self._auto_choose_disk(
+ size_bytes, disk_raid_type, disk_type, raise_error)
+ if len(sim_disks) >= 1:
+ member_ids = [d['disk_id'] for d in sim_disks]
+ sim_pool = self._pool_create_from_disks(
+ pool_name, member_ids, disk_raid_type, raise_error)
+ if sim_pool:
+ return sim_pool
+
+ if member_type == Pool.MEMBER_TYPE_UNKNOWN or \
+ member_type == Pool.MEMBER_TYPE_VOLUME:
+ vol_raid_type = raid_type
+ if raid_type == Pool.RAID_TYPE_UNKNOWN:
+ vol_raid_type = SimData.SIM_DATA_VOLUME_DEFAULT_RAID
+ sim_vols = self._auto_choose_vol(
+ size_bytes, vol_raid_type, raise_error)
+ if len(sim_vols) >= 1:
+ member_ids = [v['vol_id'] for v in sim_vols]
+ sim_pool = self._pool_create_from_volumes(
+ pool_name, member_ids, disk_raid_type, raise_error)
+ if sim_pool:
+ return sim_pool
+
+ if member_type == Pool.MEMBER_TYPE_POOL:
+ if raid_type != Pool.RAID_TYPE_UNKNOWN and \
+ raid_type != Pool.RAID_TYPE_NOT_APPLICABLE:
+ raise LsmError(ErrorNumber.NO_SUPPORT,
+ "Pool based pool does not support " +
+ "raid_type: %s(%d)" %
+ (Pool.raid_type_to_str(raid_type),
+ raid_type))
+
+ if member_type == Pool.MEMBER_TYPE_UNKNOWN:
+ if raid_type != Pool.RAID_TYPE_UNKNOWN and \
+ raid_type != Pool.RAID_TYPE_NOT_APPLICABLE:
+ raise LsmError(ErrorNumber.NO_SUPPORT,
+ "No enough free disk or volume spaces " +
+ "to create new pool. And pool based " +
+ "pool does not support raid_type: %s" %
+ Pool.raid_type_to_str(raid_type))
+
+ member_sim_pool = self._auto_choose_pool(size_bytes, raise_error)
+ if member_sim_pool:
+ member_id = member_sim_pool['pool_id']
+ sim_pool = self._pool_create_from_pool(
+ pool_name, member_id, size_bytes, raise_error)
+ if sim_pool:
+ return sim_pool
+
+ # only member_type == Pool.MEMBER_TYPE_UNKNOWN can reach here.
+ raise LsmError(ErrorNumber.SIZE_INSUFFICIENT_SPACE,
+ "No enough free spaces to create new pool")
+
+ def pool_delete(self, pool_id, flags=0):
+ if pool_id not in self.pool_dict.keys():
+ raise LsmError(ErrorNumber.INVALID_POOL,
+ "Pool not found: %s" % pool_id)
+
+ del(self.pool_dict[pool_id])
+ return None
diff --git a/lsm/lsm/simulator.py b/lsm/lsm/simulator.py
index 0ca3582..17d3027 100644
--- a/lsm/lsm/simulator.py
+++ b/lsm/lsm/simulator.py
@@ -16,7 +16,7 @@
# Author: tasleson

from common import uri_parse
-from data import Capabilities
+from data import Capabilities, Pool
from iplugin import INfs, IStorageAreaNetwork
from version import VERSION
from simarray import SimArray
@@ -86,9 +86,33 @@ class SimPlugin(INfs, IStorageAreaNetwork):
return [SimPlugin._sim_data_2_lsm(s) for s in sim_syss]

def pools(self, flags=0):
- sim_pools = self.sim_array.pools()
+ sim_pools = self.sim_array.pools(flags)
return [SimPlugin._sim_data_2_lsm(p) for p in sim_pools]

+ def pool_create(self, system_id, pool_name, size_bytes,
+ raid_type=Pool.RAID_TYPE_UNKNOWN,
+ member_type=Pool.MEMBER_TYPE_UNKNOWN, flags=0):
+ return self.sim_array.pool_create(
+ system_id, pool_name, size_bytes, raid_type, member_type, flags)
+
+ def pool_create_from_disks(self, system_id, pool_name, member_ids,
+ raid_type, flags=0):
+ return self.sim_array.pool_create_from_disks(
+ system_id, pool_name, member_ids, raid_type, flags)
+
+ def pool_create_from_volumes(self, system_id, pool_name, member_ids,
+ raid_type, flags=0):
+ return self.sim_array.pool_create_from_volumes(
+ system_id, pool_name, member_ids, raid_type, flags)
+
+ def pool_create_from_pool(self, system_id, pool_name, member_id,
+ size_bytes, flags=0):
+ return self.sim_array.pool_create_from_pool(
+ system_id, pool_name, member_id, size_bytes, flags)
+
+ def pool_delete(self, pool, flags=0):
+ return self.sim_array.pool_delete(pool.id, flags)
+
def volumes(self, flags=0):
sim_vols = self.sim_array.volumes()
return [SimPlugin._sim_data_2_lsm(v) for v in sim_vols]
@@ -142,8 +166,8 @@ class SimPlugin(INfs, IStorageAreaNetwork):

def access_group_create(self, name, initiator_id, id_type, system_id,
flags=0):
- sim_ag = self.sim_array.access_group_create(name, initiator_id,
- id_type, system_id, flags)
+ sim_ag = self.sim_array.access_group_create(
+ name, initiator_id, id_type, system_id, flags)
return SimPlugin._sim_data_2_lsm(sim_ag)

def access_group_del(self, group, flags=0):
@@ -281,4 +305,3 @@ class SimPlugin(INfs, IStorageAreaNetwork):

def export_remove(self, export, flags=0):
return self.sim_array.fs_unexport(export.id, flags)
-

[Tony Asleson]

Patch set committed, thanks!

Regards,
Tony

Post by Gris Ge
When I coding simulator plugin to support the pool create and delete,
I realized how silly my design is for pool creating API. It's almost a
nightmare to handle such long list of optional parameters in plugin.
Hence we change it again. :(
pool_create() # for normally user who don't care RAID
# member. Just care about size, maybe
# RAID type or disk type.
pool_create_from_disks() # for advanced user
pool_create_from_volumes() # for advanced user
pool_create_from_pool() # for advanced user
Please check the patch #1 for more detail.
* Updated lsmcli and its manpage for this change.
* Enable pool create/delete support in simulator plugin.
Any comments or correction will be appreciated.
client.py: change pool create/delete python API
data.py: add disk type into member type of Pool, more pool
capabilities
common.py: add pool create related error
lsmcli: add support of new pool create/delete API
simulator.py: add support of pool create/delete
doc/man/lsmcli.1.in | 121 ++++--
lsm/lsm/client.py | 209 +++++-----
lsm/lsm/cmdline.py | 258 ++++++++----
lsm/lsm/common.py | 3 +
lsm/lsm/data.py | 136 ++++--
lsm/lsm/simarray.py | 1122 ++++++++++++++++++++++++++++++++++++++++++++------
lsm/lsm/simulator.py | 33 +-
7 files changed, 1521 insertions(+), 361 deletions(-)