Gris Ge
2014-04-16 15:01:13 UTC
# Time line is based on GMT+8
[Wednesday 09 April 2014] [23:01:49] <fge> tasleson: Can we start now?
[Wednesday 09 April 2014] [23:02:04] <tasleson> Sure
[Wednesday 09 April 2014] [23:02:51] <fge> tasleson: Just one thing from me: Can we reorgnized the libstoramgmt/lsm/lsm folder?
[Wednesday 09 April 2014] [23:03:00] <fge> into 3 part:
[Wednesday 09 April 2014] [23:03:09] <fge> 1. lsm internal use.
[Wednesday 09 April 2014] [23:03:12] <fge> 2. plugin
[Wednesday 09 April 2014] [23:03:14] <fge> 3. lsmcli
[Wednesday 09 April 2014] [23:06:19] <tasleson> Trying to think if we can do that and maintain the same name spaces we have?
[Wednesday 09 April 2014] [23:06:56] <fge> I was thinking that 3 part is in-dependent with each other.
[Wednesday 09 April 2014] [23:07:20] <fge> in the lsmcli patch I just send, lsmcli is just a user of lsm API.
[Wednesday 09 April 2014] [23:08:41] <fge> no private methods or constants will be in lsmcli or plugin eventually.
[Wednesday 09 April 2014] [23:09:13] <tasleson> I understand your reasoning, I'm trying to remember the packaging & the module rules for python etc.
[Wednesday 09 April 2014] [23:10:00] <fge> It seems cinder seperate plugin and intenral codes.
[Wednesday 09 April 2014] [23:10:03] <fge> Checking.
[Wednesday 09 April 2014] [23:11:06] <fge> ok. Cinder does not have Makefile.am or similar.
[Wednesday 09 April 2014] [23:12:08] <fge> If you are agree on this folder reorgnize, I can take the autoconf/etc works.
[Wednesday 09 April 2014] [23:13:09] <tasleson> It isn't the autoconf stuff I was concerned with it's how the python is organized. I would like to keep the same name space, eg. lsm.Client() etc
[Wednesday 09 April 2014] [23:13:21] <tasleson> Go ahead and give it a try.
[Wednesday 09 April 2014] [23:13:58] <fge> Then I am done today.
[Wednesday 09 April 2014] [23:14:28] <tasleson> I wanted to discuss the replication API stuff
[Wednesday 09 April 2014] [23:14:39] <fge> Sure.
[Wednesday 09 April 2014] [23:14:44] <tasleson> and capabilities
[Wednesday 09 April 2014] [23:14:54] <tasleson> Lets do the simple one first, capabilitites
[Wednesday 09 April 2014] [23:15:18] <tasleson> My intention was that all plugins implement the methods in the IPlugin interface
[Wednesday 09 April 2014] [23:16:24] <tasleson> So job control, capabilities, plugin_info, pools & systems would always be available.
[Wednesday 09 April 2014] [23:17:16] <tasleson> However, your documents refer to querying to see if 'pools' was available and Ma Shimiao had posted a patch adding pools as a capability
[Wednesday 09 April 2014] [23:17:52] <tasleson> My question is do we want to only make capabilities a required method and everything else optional?
[Wednesday 09 April 2014] [23:18:40] <tasleson> If you don't support listing pools the plugin is not as useful, but I guess some operations could still be done
[Wednesday 09 April 2014] [23:20:01] <fge> How about cinder needed methods mandatory and else optional guided by capacity.
[Wednesday 09 April 2014] [23:21:05] <fge> Creating a list of methods at https://etherpad.mozilla.org/2owbIECBN5
[Wednesday 09 April 2014] [23:23:05] <fge> how's that look like?
[Wednesday 09 April 2014] [23:23:32] <tasleson> We can't make the library be exactly what one user needs, we should also target blivet, libvirt etc. as potential users.
[Wednesday 09 April 2014] [23:24:13] <tasleson> Some arrays may be file only, no block.
[Wednesday 09 April 2014] [23:25:07] <fge> ok. I missed the file part.
[Wednesday 09 April 2014] [23:26:42] <tasleson> It really comes down to what is the bare essentials a plugin should provide otherwise it isn't really even viable
[Wednesday 09 April 2014] [23:29:21] <tasleson> We have specific capabilities for each of the ones you listed
[Wednesday 09 April 2014] [23:29:39] <tasleson> A user can query if the plugin supports volume create/delete
[Wednesday 09 April 2014] [23:30:08] <tasleson> Do we want listing pools as an optional thing?
[Wednesday 09 April 2014] [23:30:35] <tasleson> If you take that away, what is left...
[Wednesday 09 April 2014] [23:31:10] <fge> so, you are suggesting systems(), pools() are mandatory methods for plugin.
[Wednesday 09 April 2014] [23:31:17] <fge> no capabilities check needed.
[Wednesday 09 April 2014] [23:31:48] <tasleson> Yes, that was my intention from day 1.
[Wednesday 09 April 2014] [23:32:09] <tasleson> IPlugin doc: Plug-in interface that all plug-ins must implement for basic operation.
[Wednesday 09 April 2014] [23:33:24] <tasleson> There are other methods in the base interface class too, but they are easy to fake if needed
[Wednesday 09 April 2014] [23:33:41] <tasleson> time outs (get/set) job status/job free
[Wednesday 09 April 2014] [23:33:55] <tasleson> If you don't support jobs don't return a job id ever
[Wednesday 09 April 2014] [23:34:10] <tasleson> But then your operations better not take too long too
[Wednesday 09 April 2014] [23:34:46] <fge> that does not works on SMI-S.
[Wednesday 09 April 2014] [23:35:12] <tasleson> What doesn't work on SMI-S?
[Wednesday 09 April 2014] [23:35:16] <fge> IBM, LSI does not provide job support, but will take a while on pool_creating.
[Wednesday 09 April 2014] [23:35:27] <fge> or volume creating.
[Wednesday 09 April 2014] [23:36:06] <fge> And that seems only change for a while.
[Wednesday 09 April 2014] [23:36:16] <fge> s/only/won't/
[Wednesday 09 April 2014] [23:37:22] <fge> if we are going to require 'job or quick', lsm has to implement a way for them.
[Wednesday 09 April 2014] [23:37:34] <tasleson> Well in those isolated cases users will just have a bad experience then... We can't fix when vendors don't support things
[Wednesday 09 April 2014] [23:37:46] <tasleson> We aren't going to fix vendor issues
[Wednesday 09 April 2014] [23:38:11] <tasleson> I will talk with James @ snia about jobs and CTP
[Wednesday 09 April 2014] [23:38:34] <tasleson> It sucks if you tie up the API for potentially long periods of time
[Wednesday 09 April 2014] [23:39:03] <fge> try mention, 'Repliation' profile for remote replication require proxy SMI-S provider connecting two arrays.
[Wednesday 09 April 2014] [23:39:29] <fge> I talked with IBM and Huawei who does not support job control.
[Wednesday 09 April 2014] [23:40:03] <fge> Their main reason was using embemed smi-s provider cannot provide enough CPU/Member resource for job control.
[Wednesday 09 April 2014] [23:40:22] <tasleson> Wow
[Wednesday 09 April 2014] [23:40:53] <fge> and without job control, InvokeMethod cannot provide user friendly error info.
[Wednesday 09 April 2014] [23:41:02] <fge> but a vendor specific error code.
[Wednesday 09 April 2014] [23:41:17] <tasleson> Well I'm not going to worry about crappy smi-s implementations
[Wednesday 09 April 2014] [23:41:50] <fge> I think with that reason, CTP should 'FAIL' when no job control found.
[Wednesday 09 April 2014] [23:42:01] <tasleson> Agreed!
[Wednesday 09 April 2014] [23:42:16] <tasleson> Ok, lets move on to replication stuff
[Wednesday 09 April 2014] [23:42:42] <fge> I will update the wiki for capacities changes you suggested.
[Wednesday 09 April 2014] [23:42:54] <fge> yes. Replication.
[Wednesday 09 April 2014] [23:43:16] <tasleson> volume_replicate and volume_replicate_create ... seem like duplicated functionality?
[Wednesday 09 April 2014] [23:44:04] <tasleson> In my opinion your VolumeReplicate class is the relationship between one or more volumes.
[Wednesday 09 April 2014] [23:44:17] <fge> yes.
[Wednesday 09 April 2014] [23:44:44] <tasleson> If we have a volume_replication_create it should take as parameters 2 volumes and then create a relation ship between them
[Wednesday 09 April 2014] [23:45:24] <fge> that will not works on snapshot(UNSYNC_DELTA_RO, UNSYNC_DELTA_RW)
[Wednesday 09 April 2014] [23:46:20] <tasleson> I know that EMC requires that you have an existing source and target before you try to mirror them
[Wednesday 09 April 2014] [23:47:08] <tasleson> The DELTA RO/RW would be an unsupported relationship to create I guess...
[Wednesday 09 April 2014] [23:47:33] <fge> then, we are talking about SNIA new term:
[Wednesday 09 April 2014] [23:47:41] <fge> snapsho, clone, mirror.
[Wednesday 09 April 2014] [23:48:10] <fge> we create different methods and classes for these three.
[Wednesday 09 April 2014] [23:48:21] <fge> snapshot: UNSYNC_DELTA_RO, UNSYNC_DELTA_RW
[Wednesday 09 April 2014] [23:48:37] <fge> clone: UNSYNC_FULL_LOCAL, UNSYNC_FULL_REMOTE
[Wednesday 09 April 2014] [23:48:54] <fge> mirror: SYNC_LOCAL, ASYNC_LOCAL, and remote one.
[Wednesday 09 April 2014] [23:49:26] <fge> actually, there types has their own characters.
[Wednesday 09 April 2014] [23:50:07] <fge> It make things complex when merging them into single class -- VolumeReplication
[Wednesday 09 April 2014] [23:50:10] <tasleson> But in all cases there is a relationship between the two or more volumes
[Wednesday 09 April 2014] [23:50:38] <fge> Yes. A relationship.
[Wednesday 09 April 2014] [23:51:24] <fge> so, you suggest volume_replication_create() only handle clone and mirror after target volume created.
[Wednesday 09 April 2014] [23:51:43] <fge> and for snapshot, we just use another method?
[Wednesday 09 April 2014] [23:52:13] <tasleson> I think we should change the name that represents the relationship so that is doesn't match the method volume_repliate
[Wednesday 09 April 2014] [23:52:31] <tasleson> And then add your new method to create the relationship
[Wednesday 09 April 2014] [23:53:20] <tasleson> I would like the API to include the ability to take two existing volumes and mirror them for example
[Wednesday 09 April 2014] [23:54:01] <tasleson> Or take two volumes and re-copy or sync one to the other
[Wednesday 09 April 2014] [23:54:26] <tasleson> There are real use cases in those
[Wednesday 09 April 2014] [23:55:16] <fge> ok. I get your idea. I thinking about how we present the relationship.
[Wednesday 09 April 2014] [23:55:51] <tasleson> We need the ability to express the relationships between them and manage those relationships. However, it would be great if the API intuitively didn't allow you to do unsupported things
[Wednesday 09 April 2014] [23:56:42] <tasleson> Like the example we just went though, it doesn't work to take two volumes and create a DELTA RW between them. Well you could delete the target and re-create?
[Wednesday 09 April 2014] [23:57:06] <tasleson> hmm, maybe that is how we solve a problem like that?
[Wednesday 09 April 2014] [23:57:52] <tasleson> In cases where it really isn't supported we blitz the target and re-create?
[Wednesday 09 April 2014] [23:57:57] <fge> I believe no storage admin or user understand the reason to create target volume before snapshot(DELLTA_RO/RW)
[Wednesday 09 April 2014] [23:58:34] <tasleson> Yes, I agree, but if you actually did that in the API what could we possibly do?
[Wednesday 09 April 2014] [23:58:46] <tasleson> Would it be bad to do as I'm suggesting?
[Wednesday 09 April 2014] [23:58:59] <tasleson> They know that the destination will match the source.
[Wednesday 09 April 2014] [23:59:17] <tasleson> So if the destination has any valuable data it will be destroyed
[Thursday 10 April 2014] [00:00:08] <tasleson> We could use the destination meta data, (name etc) for the new one.
[Thursday 10 April 2014] [00:00:23] <fge> wait a second, just make sure we are on the same page.
[Thursday 10 April 2014] [00:00:29] <tasleson> Sure
[Thursday 10 April 2014] [00:00:53] <fge> 1. Change VolumeReplication to some name better fit volume "relationship" term.
[Thursday 10 April 2014] [00:01:30] <tasleson> Yes, relationship/association etc.
[Thursday 10 April 2014] [00:01:37] <fge> 2. volume_replication_create() take already created source volume and target volume.
[Thursday 10 April 2014] [00:02:00] <tasleson> Yes, but rename to match #1
[Thursday 10 April 2014] [00:02:24] <fge> 3. for DELLTA_RO/RW, target volume will be deleted and re-create as DELLTA_RO/RW requires.
[Thursday 10 April 2014] [00:02:45] <fge> is that you are suggesting?
[Thursday 10 April 2014] [00:02:59] <tasleson> Yes, but the preferred way for those would be volume_replicate like we have today
[Thursday 10 April 2014] [00:03:52] <tasleson> Does that work?
[Thursday 10 April 2014] [00:04:25] <fge> confused, current volume_replicate() take 'pool' as target pool.
[Thursday 10 April 2014] [00:04:33] <fge> where target volume will be created from.
[Thursday 10 April 2014] [00:04:54] <fge> it does not match #2.
[Thursday 10 April 2014] [00:06:48] <tasleson> I was advocating we keep the volume_replicate we have today which simply creates a copy of a source with the specified relationship. After that the user can query that relationship and modify it if needed. In the case where a user has existing volumes and would like to create a relationship they would use the new method to create that relationship
[Thursday 10 April 2014] [00:07:29] <tasleson> Obviously there are requirements when creating a relationship, like volumes should be same size?
[Thursday 10 April 2014] [00:08:23] <tasleson> Maybe we should step back and come up with some use cases and then craft an API to accommodate?
[Thursday 10 April 2014] [00:08:49] <fge> volume_replicate() == volume_create() + volume_relationship_create()
[Thursday 10 April 2014] [00:09:25] <tasleson> Yes, but the relationship create part is automatic right?
[Thursday 10 April 2014] [00:09:26] <fge> is that correct expression of your idea?
[Thursday 10 April 2014] [00:10:15] <tasleson> Well I guess in some implementation it might not be
[Thursday 10 April 2014] [00:10:35] <tasleson> For EMC create SYNC_MIRROR would be exactly that
[Thursday 10 April 2014] [00:11:26] <tasleson> Which causes another problem...
[Thursday 10 April 2014] [00:11:45] <tasleson> Each of those operations could be a separate job
[Thursday 10 April 2014] [00:12:17] <fge> ok. I am saying that method will invoke those two methods.
[Thursday 10 April 2014] [00:12:19] <fge> I mean:
[Thursday 10 April 2014] [00:12:50] <fge> volume_replicate(): Create replication for given source volume with new volume in given pool.
[Thursday 10 April 2014] [00:13:16] <fge> volume_relation_create(): Create relation for given source and target volume.
[Thursday 10 April 2014] [00:13:28] <fge> volume_replicate() works for snapshot/mirror/clone.
[Thursday 10 April 2014] [00:13:37] <fge> volume_relation_create() works for mirror/clone.
[Thursday 10 April 2014] [00:14:13] <fge> Then volume_relation_resync(), volume_relation_split() for snapshot/mirror/clone.
[Thursday 10 April 2014] [00:14:31] <fge> and volume_relation_failover(), giveback() for mirror.
[Thursday 10 April 2014] [00:15:25] <tasleson> That all looks good. However, like I mentioned before, we could volume_relation_create() work for all cases too.
[Thursday 10 April 2014] [00:16:54] <tasleson> This meeting is running late, I think we made some good progress. Why don't we both think about what we have so far and see if it works with all the use cases we can come up with?
[Thursday 10 April 2014] [00:17:28] <fge> some array just only accept snapshot creating in special pool.
[Thursday 10 April 2014] [00:17:40] <fge> where normal volume is not acceptable in that pool.
[Thursday 10 April 2014] [00:17:46] <tasleson> Yeah, there are a bunch of restrictions for different vendors
[Thursday 10 April 2014] [00:17:54] <fge> in that case, you "delete-create" will not works.
[Thursday 10 April 2014] [00:18:37] <tasleson> OK, well maybe the pool parameter is a suggestion :-)
[Thursday 10 April 2014] [00:18:56] <tasleson> Oh, wait wrong method
[Thursday 10 April 2014] [00:19:45] <fge> How about this
[Thursday 10 April 2014] [00:20:26] <fge> Can you draft down your ideas about replication in similar form like the wiki page does?
[Thursday 10 April 2014] [00:20:51] <tasleson> Sure, I will do that today
[Thursday 10 April 2014] [00:21:39] <fge> that's would be easier than we discuss IRC guessing how the API designs.
[Thursday 10 April 2014] [00:22:13] <tasleson> I will put down use cases and a possible API to accommodate
[Thursday 10 April 2014] [00:22:26] <tasleson> May not be as pretty as yours though :-)
[Thursday 10 April 2014] [00:22:45] <tasleson> Formatting wise...
[Thursday 10 April 2014] [00:23:02] <fge> actually, I think my VolumeReplication design sucks now.
[Thursday 10 April 2014] [00:23:10] <fge> I should split them into three classes.
[Thursday 10 April 2014] [00:23:24] <fge> where less document/explaination needed.
[Thursday 10 April 2014] [00:23:55] <fge> anyway, I will think through this in my dream.
[Thursday 10 April 2014] [00:24:07] <tasleson> OK, have a good evening!
[Thursday 10 April 2014] [00:24:26] <fge> You have a nice day.
[Wednesday 09 April 2014] [23:01:49] <fge> tasleson: Can we start now?
[Wednesday 09 April 2014] [23:02:04] <tasleson> Sure
[Wednesday 09 April 2014] [23:02:51] <fge> tasleson: Just one thing from me: Can we reorgnized the libstoramgmt/lsm/lsm folder?
[Wednesday 09 April 2014] [23:03:00] <fge> into 3 part:
[Wednesday 09 April 2014] [23:03:09] <fge> 1. lsm internal use.
[Wednesday 09 April 2014] [23:03:12] <fge> 2. plugin
[Wednesday 09 April 2014] [23:03:14] <fge> 3. lsmcli
[Wednesday 09 April 2014] [23:06:19] <tasleson> Trying to think if we can do that and maintain the same name spaces we have?
[Wednesday 09 April 2014] [23:06:56] <fge> I was thinking that 3 part is in-dependent with each other.
[Wednesday 09 April 2014] [23:07:20] <fge> in the lsmcli patch I just send, lsmcli is just a user of lsm API.
[Wednesday 09 April 2014] [23:08:41] <fge> no private methods or constants will be in lsmcli or plugin eventually.
[Wednesday 09 April 2014] [23:09:13] <tasleson> I understand your reasoning, I'm trying to remember the packaging & the module rules for python etc.
[Wednesday 09 April 2014] [23:10:00] <fge> It seems cinder seperate plugin and intenral codes.
[Wednesday 09 April 2014] [23:10:03] <fge> Checking.
[Wednesday 09 April 2014] [23:11:06] <fge> ok. Cinder does not have Makefile.am or similar.
[Wednesday 09 April 2014] [23:12:08] <fge> If you are agree on this folder reorgnize, I can take the autoconf/etc works.
[Wednesday 09 April 2014] [23:13:09] <tasleson> It isn't the autoconf stuff I was concerned with it's how the python is organized. I would like to keep the same name space, eg. lsm.Client() etc
[Wednesday 09 April 2014] [23:13:21] <tasleson> Go ahead and give it a try.
[Wednesday 09 April 2014] [23:13:58] <fge> Then I am done today.
[Wednesday 09 April 2014] [23:14:28] <tasleson> I wanted to discuss the replication API stuff
[Wednesday 09 April 2014] [23:14:39] <fge> Sure.
[Wednesday 09 April 2014] [23:14:44] <tasleson> and capabilities
[Wednesday 09 April 2014] [23:14:54] <tasleson> Lets do the simple one first, capabilitites
[Wednesday 09 April 2014] [23:15:18] <tasleson> My intention was that all plugins implement the methods in the IPlugin interface
[Wednesday 09 April 2014] [23:16:24] <tasleson> So job control, capabilities, plugin_info, pools & systems would always be available.
[Wednesday 09 April 2014] [23:17:16] <tasleson> However, your documents refer to querying to see if 'pools' was available and Ma Shimiao had posted a patch adding pools as a capability
[Wednesday 09 April 2014] [23:17:52] <tasleson> My question is do we want to only make capabilities a required method and everything else optional?
[Wednesday 09 April 2014] [23:18:40] <tasleson> If you don't support listing pools the plugin is not as useful, but I guess some operations could still be done
[Wednesday 09 April 2014] [23:20:01] <fge> How about cinder needed methods mandatory and else optional guided by capacity.
[Wednesday 09 April 2014] [23:21:05] <fge> Creating a list of methods at https://etherpad.mozilla.org/2owbIECBN5
[Wednesday 09 April 2014] [23:23:05] <fge> how's that look like?
[Wednesday 09 April 2014] [23:23:32] <tasleson> We can't make the library be exactly what one user needs, we should also target blivet, libvirt etc. as potential users.
[Wednesday 09 April 2014] [23:24:13] <tasleson> Some arrays may be file only, no block.
[Wednesday 09 April 2014] [23:25:07] <fge> ok. I missed the file part.
[Wednesday 09 April 2014] [23:26:42] <tasleson> It really comes down to what is the bare essentials a plugin should provide otherwise it isn't really even viable
[Wednesday 09 April 2014] [23:29:21] <tasleson> We have specific capabilities for each of the ones you listed
[Wednesday 09 April 2014] [23:29:39] <tasleson> A user can query if the plugin supports volume create/delete
[Wednesday 09 April 2014] [23:30:08] <tasleson> Do we want listing pools as an optional thing?
[Wednesday 09 April 2014] [23:30:35] <tasleson> If you take that away, what is left...
[Wednesday 09 April 2014] [23:31:10] <fge> so, you are suggesting systems(), pools() are mandatory methods for plugin.
[Wednesday 09 April 2014] [23:31:17] <fge> no capabilities check needed.
[Wednesday 09 April 2014] [23:31:48] <tasleson> Yes, that was my intention from day 1.
[Wednesday 09 April 2014] [23:32:09] <tasleson> IPlugin doc: Plug-in interface that all plug-ins must implement for basic operation.
[Wednesday 09 April 2014] [23:33:24] <tasleson> There are other methods in the base interface class too, but they are easy to fake if needed
[Wednesday 09 April 2014] [23:33:41] <tasleson> time outs (get/set) job status/job free
[Wednesday 09 April 2014] [23:33:55] <tasleson> If you don't support jobs don't return a job id ever
[Wednesday 09 April 2014] [23:34:10] <tasleson> But then your operations better not take too long too
[Wednesday 09 April 2014] [23:34:46] <fge> that does not works on SMI-S.
[Wednesday 09 April 2014] [23:35:12] <tasleson> What doesn't work on SMI-S?
[Wednesday 09 April 2014] [23:35:16] <fge> IBM, LSI does not provide job support, but will take a while on pool_creating.
[Wednesday 09 April 2014] [23:35:27] <fge> or volume creating.
[Wednesday 09 April 2014] [23:36:06] <fge> And that seems only change for a while.
[Wednesday 09 April 2014] [23:36:16] <fge> s/only/won't/
[Wednesday 09 April 2014] [23:37:22] <fge> if we are going to require 'job or quick', lsm has to implement a way for them.
[Wednesday 09 April 2014] [23:37:34] <tasleson> Well in those isolated cases users will just have a bad experience then... We can't fix when vendors don't support things
[Wednesday 09 April 2014] [23:37:46] <tasleson> We aren't going to fix vendor issues
[Wednesday 09 April 2014] [23:38:11] <tasleson> I will talk with James @ snia about jobs and CTP
[Wednesday 09 April 2014] [23:38:34] <tasleson> It sucks if you tie up the API for potentially long periods of time
[Wednesday 09 April 2014] [23:39:03] <fge> try mention, 'Repliation' profile for remote replication require proxy SMI-S provider connecting two arrays.
[Wednesday 09 April 2014] [23:39:29] <fge> I talked with IBM and Huawei who does not support job control.
[Wednesday 09 April 2014] [23:40:03] <fge> Their main reason was using embemed smi-s provider cannot provide enough CPU/Member resource for job control.
[Wednesday 09 April 2014] [23:40:22] <tasleson> Wow
[Wednesday 09 April 2014] [23:40:53] <fge> and without job control, InvokeMethod cannot provide user friendly error info.
[Wednesday 09 April 2014] [23:41:02] <fge> but a vendor specific error code.
[Wednesday 09 April 2014] [23:41:17] <tasleson> Well I'm not going to worry about crappy smi-s implementations
[Wednesday 09 April 2014] [23:41:50] <fge> I think with that reason, CTP should 'FAIL' when no job control found.
[Wednesday 09 April 2014] [23:42:01] <tasleson> Agreed!
[Wednesday 09 April 2014] [23:42:16] <tasleson> Ok, lets move on to replication stuff
[Wednesday 09 April 2014] [23:42:42] <fge> I will update the wiki for capacities changes you suggested.
[Wednesday 09 April 2014] [23:42:54] <fge> yes. Replication.
[Wednesday 09 April 2014] [23:43:16] <tasleson> volume_replicate and volume_replicate_create ... seem like duplicated functionality?
[Wednesday 09 April 2014] [23:44:04] <tasleson> In my opinion your VolumeReplicate class is the relationship between one or more volumes.
[Wednesday 09 April 2014] [23:44:17] <fge> yes.
[Wednesday 09 April 2014] [23:44:44] <tasleson> If we have a volume_replication_create it should take as parameters 2 volumes and then create a relation ship between them
[Wednesday 09 April 2014] [23:45:24] <fge> that will not works on snapshot(UNSYNC_DELTA_RO, UNSYNC_DELTA_RW)
[Wednesday 09 April 2014] [23:46:20] <tasleson> I know that EMC requires that you have an existing source and target before you try to mirror them
[Wednesday 09 April 2014] [23:47:08] <tasleson> The DELTA RO/RW would be an unsupported relationship to create I guess...
[Wednesday 09 April 2014] [23:47:33] <fge> then, we are talking about SNIA new term:
[Wednesday 09 April 2014] [23:47:41] <fge> snapsho, clone, mirror.
[Wednesday 09 April 2014] [23:48:10] <fge> we create different methods and classes for these three.
[Wednesday 09 April 2014] [23:48:21] <fge> snapshot: UNSYNC_DELTA_RO, UNSYNC_DELTA_RW
[Wednesday 09 April 2014] [23:48:37] <fge> clone: UNSYNC_FULL_LOCAL, UNSYNC_FULL_REMOTE
[Wednesday 09 April 2014] [23:48:54] <fge> mirror: SYNC_LOCAL, ASYNC_LOCAL, and remote one.
[Wednesday 09 April 2014] [23:49:26] <fge> actually, there types has their own characters.
[Wednesday 09 April 2014] [23:50:07] <fge> It make things complex when merging them into single class -- VolumeReplication
[Wednesday 09 April 2014] [23:50:10] <tasleson> But in all cases there is a relationship between the two or more volumes
[Wednesday 09 April 2014] [23:50:38] <fge> Yes. A relationship.
[Wednesday 09 April 2014] [23:51:24] <fge> so, you suggest volume_replication_create() only handle clone and mirror after target volume created.
[Wednesday 09 April 2014] [23:51:43] <fge> and for snapshot, we just use another method?
[Wednesday 09 April 2014] [23:52:13] <tasleson> I think we should change the name that represents the relationship so that is doesn't match the method volume_repliate
[Wednesday 09 April 2014] [23:52:31] <tasleson> And then add your new method to create the relationship
[Wednesday 09 April 2014] [23:53:20] <tasleson> I would like the API to include the ability to take two existing volumes and mirror them for example
[Wednesday 09 April 2014] [23:54:01] <tasleson> Or take two volumes and re-copy or sync one to the other
[Wednesday 09 April 2014] [23:54:26] <tasleson> There are real use cases in those
[Wednesday 09 April 2014] [23:55:16] <fge> ok. I get your idea. I thinking about how we present the relationship.
[Wednesday 09 April 2014] [23:55:51] <tasleson> We need the ability to express the relationships between them and manage those relationships. However, it would be great if the API intuitively didn't allow you to do unsupported things
[Wednesday 09 April 2014] [23:56:42] <tasleson> Like the example we just went though, it doesn't work to take two volumes and create a DELTA RW between them. Well you could delete the target and re-create?
[Wednesday 09 April 2014] [23:57:06] <tasleson> hmm, maybe that is how we solve a problem like that?
[Wednesday 09 April 2014] [23:57:52] <tasleson> In cases where it really isn't supported we blitz the target and re-create?
[Wednesday 09 April 2014] [23:57:57] <fge> I believe no storage admin or user understand the reason to create target volume before snapshot(DELLTA_RO/RW)
[Wednesday 09 April 2014] [23:58:34] <tasleson> Yes, I agree, but if you actually did that in the API what could we possibly do?
[Wednesday 09 April 2014] [23:58:46] <tasleson> Would it be bad to do as I'm suggesting?
[Wednesday 09 April 2014] [23:58:59] <tasleson> They know that the destination will match the source.
[Wednesday 09 April 2014] [23:59:17] <tasleson> So if the destination has any valuable data it will be destroyed
[Thursday 10 April 2014] [00:00:08] <tasleson> We could use the destination meta data, (name etc) for the new one.
[Thursday 10 April 2014] [00:00:23] <fge> wait a second, just make sure we are on the same page.
[Thursday 10 April 2014] [00:00:29] <tasleson> Sure
[Thursday 10 April 2014] [00:00:53] <fge> 1. Change VolumeReplication to some name better fit volume "relationship" term.
[Thursday 10 April 2014] [00:01:30] <tasleson> Yes, relationship/association etc.
[Thursday 10 April 2014] [00:01:37] <fge> 2. volume_replication_create() take already created source volume and target volume.
[Thursday 10 April 2014] [00:02:00] <tasleson> Yes, but rename to match #1
[Thursday 10 April 2014] [00:02:24] <fge> 3. for DELLTA_RO/RW, target volume will be deleted and re-create as DELLTA_RO/RW requires.
[Thursday 10 April 2014] [00:02:45] <fge> is that you are suggesting?
[Thursday 10 April 2014] [00:02:59] <tasleson> Yes, but the preferred way for those would be volume_replicate like we have today
[Thursday 10 April 2014] [00:03:52] <tasleson> Does that work?
[Thursday 10 April 2014] [00:04:25] <fge> confused, current volume_replicate() take 'pool' as target pool.
[Thursday 10 April 2014] [00:04:33] <fge> where target volume will be created from.
[Thursday 10 April 2014] [00:04:54] <fge> it does not match #2.
[Thursday 10 April 2014] [00:06:48] <tasleson> I was advocating we keep the volume_replicate we have today which simply creates a copy of a source with the specified relationship. After that the user can query that relationship and modify it if needed. In the case where a user has existing volumes and would like to create a relationship they would use the new method to create that relationship
[Thursday 10 April 2014] [00:07:29] <tasleson> Obviously there are requirements when creating a relationship, like volumes should be same size?
[Thursday 10 April 2014] [00:08:23] <tasleson> Maybe we should step back and come up with some use cases and then craft an API to accommodate?
[Thursday 10 April 2014] [00:08:49] <fge> volume_replicate() == volume_create() + volume_relationship_create()
[Thursday 10 April 2014] [00:09:25] <tasleson> Yes, but the relationship create part is automatic right?
[Thursday 10 April 2014] [00:09:26] <fge> is that correct expression of your idea?
[Thursday 10 April 2014] [00:10:15] <tasleson> Well I guess in some implementation it might not be
[Thursday 10 April 2014] [00:10:35] <tasleson> For EMC create SYNC_MIRROR would be exactly that
[Thursday 10 April 2014] [00:11:26] <tasleson> Which causes another problem...
[Thursday 10 April 2014] [00:11:45] <tasleson> Each of those operations could be a separate job
[Thursday 10 April 2014] [00:12:17] <fge> ok. I am saying that method will invoke those two methods.
[Thursday 10 April 2014] [00:12:19] <fge> I mean:
[Thursday 10 April 2014] [00:12:50] <fge> volume_replicate(): Create replication for given source volume with new volume in given pool.
[Thursday 10 April 2014] [00:13:16] <fge> volume_relation_create(): Create relation for given source and target volume.
[Thursday 10 April 2014] [00:13:28] <fge> volume_replicate() works for snapshot/mirror/clone.
[Thursday 10 April 2014] [00:13:37] <fge> volume_relation_create() works for mirror/clone.
[Thursday 10 April 2014] [00:14:13] <fge> Then volume_relation_resync(), volume_relation_split() for snapshot/mirror/clone.
[Thursday 10 April 2014] [00:14:31] <fge> and volume_relation_failover(), giveback() for mirror.
[Thursday 10 April 2014] [00:15:25] <tasleson> That all looks good. However, like I mentioned before, we could volume_relation_create() work for all cases too.
[Thursday 10 April 2014] [00:16:54] <tasleson> This meeting is running late, I think we made some good progress. Why don't we both think about what we have so far and see if it works with all the use cases we can come up with?
[Thursday 10 April 2014] [00:17:28] <fge> some array just only accept snapshot creating in special pool.
[Thursday 10 April 2014] [00:17:40] <fge> where normal volume is not acceptable in that pool.
[Thursday 10 April 2014] [00:17:46] <tasleson> Yeah, there are a bunch of restrictions for different vendors
[Thursday 10 April 2014] [00:17:54] <fge> in that case, you "delete-create" will not works.
[Thursday 10 April 2014] [00:18:37] <tasleson> OK, well maybe the pool parameter is a suggestion :-)
[Thursday 10 April 2014] [00:18:56] <tasleson> Oh, wait wrong method
[Thursday 10 April 2014] [00:19:45] <fge> How about this
[Thursday 10 April 2014] [00:20:26] <fge> Can you draft down your ideas about replication in similar form like the wiki page does?
[Thursday 10 April 2014] [00:20:51] <tasleson> Sure, I will do that today
[Thursday 10 April 2014] [00:21:39] <fge> that's would be easier than we discuss IRC guessing how the API designs.
[Thursday 10 April 2014] [00:22:13] <tasleson> I will put down use cases and a possible API to accommodate
[Thursday 10 April 2014] [00:22:26] <tasleson> May not be as pretty as yours though :-)
[Thursday 10 April 2014] [00:22:45] <tasleson> Formatting wise...
[Thursday 10 April 2014] [00:23:02] <fge> actually, I think my VolumeReplication design sucks now.
[Thursday 10 April 2014] [00:23:10] <fge> I should split them into three classes.
[Thursday 10 April 2014] [00:23:24] <fge> where less document/explaination needed.
[Thursday 10 April 2014] [00:23:55] <fge> anyway, I will think through this in my dream.
[Thursday 10 April 2014] [00:24:07] <tasleson> OK, have a good evening!
[Thursday 10 April 2014] [00:24:26] <fge> You have a nice day.
--
Gris Ge
Gris Ge