[Libstoragemgmt-devel] Using object id instead of object

Discussion:

Tony Asleson

2014-03-24 21:27:58 UTC

Gris has brought this up a couple of times. The fact that the majority
of the API passes objects instead of object IDs to functions/methods.

In normal operation this doesn't seem to be much of an issue. However,
today I did some simulated testing with 50K volumes. When listing them
all it takes about ~25 seconds. When you delete one of them from the
command line it takes about 22 seconds. When using the library it's
instant as you already have the object to use. This to me is the main
disadvantage. The command line retrieves all the objects of the given
type, makes sure that the one you passed is present and then uses the
object to perform the operation.

If we switched this to just using the IDs then we would pass it to the
plug-in and let it look the object up and perform the operation. Either
that or we would need to add to the API, to retrieve the information
about a specific object using its ID.

The main benefit would be for CLI users as library users won't see much
difference.

Thoughts/Comments?

Regards,
Tony

m***@cn.fujitsu.com

2014-03-25 03:02:56 UTC

Permalink

Post by Tony Asleson
Gris has brought this up a couple of times. The fact that the majority
of the API passes objects instead of object IDs to functions/methods.
In normal operation this doesn't seem to be much of an issue. However,
today I did some simulated testing with 50K volumes. When listing them
all it takes about ~25 seconds. When you delete one of them from the
command line it takes about 22 seconds. When using the library it's
instant as you already have the object to use. This to me is the main
disadvantage. The command line retrieves all the objects of the given
type, makes sure that the one you passed is present and then uses the
object to perform the operation.
If we switched this to just using the IDs then we would pass it to the
plug-in and let it look the object up and perform the operation. Either
that or we would need to add to the API, to retrieve the information
about a specific object using its ID.
The main benefit would be for CLI users as library users won't see much
difference.
Thoughts/Comments?

I think what affects the speed is there are a lot of retrieve.
If we use IDs instead of objects, can we lessen retrieve?
If not, I think there will be nothing improved.

Regards,
Ma

Post by Tony Asleson
Regards,
Tony
------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their
applications. Written by three acclaimed leaders in the field,
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/13534_NeoTech
_______________________________________________
Libstoragemgmt-devel mailing list
https://lists.sourceforge.net/lists/listinfo/libstoragemgmt-d

Gris Ge

2014-03-25 04:51:13 UTC

Permalink

Post by m***@cn.fujitsu.com
I think what affects the speed is there are a lot of retrieve.
If we use IDs instead of objects, can we lessen retrieve?
If not, I think there will be nothing improved.
Regards,
Ma

Let's take volume_delete() against SMI-S plugin as an example.
For lsmcli, progress A is:
1. Query all volumes via volumes() call.
2. Find out the volume object to delete by checking volume.id.
3. Send to volume_delete()
4. Inside of plugin, smis.py using volume.id to find out the
CIM_Instance. [1]
5. Call out volume delete method to array.

For library user, progress B is:
1. Already have volume object. If not, follow A.1 and A.2 to get
one.
2. The same as A.3, A.4, A.5

For REST API (not support yet, just plan), progress C is:
1. Get DELETE http method call against /volumes/<volume_id>
2. The same as progress A.

So, for lsmcli and future REST API, yes, we will have performance improve.
For library user, we might have performance improve and surly not
performance regression.

The bad side of using object ID:
* Plugin have to handle ID validation check.

[1] smis.py needs some care on getting cim_xxx for given ID. Currently,
we enumerate all and find the given one.

--
Gris Ge

Tony Asleson

2014-03-25 16:09:02 UTC

Permalink

Post by Gris Ge

Let's take volume_delete() against SMI-S plugin as an example.
1. Query all volumes via volumes() call.
2. Find out the volume object to delete by checking volume.id.
3. Send to volume_delete()
4. Inside of plugin, smis.py using volume.id to find out the
CIM_Instance. [1]
5. Call out volume delete method to array.
1. Already have volume object. If not, follow A.1 and A.2 to get
one.
2. The same as A.3, A.4, A.5
1. Get DELETE http method call against /volumes/<volume_id>
2. The same as progress A.
So, for lsmcli and future REST API, yes, we will have performance improve.
For library user, we might have performance improve and surly not
performance regression.

It is possible that for some plug-in implementations it would result in
a performance issue. For example it is possible that a plug-in,
supplied only with the id would need to query the array for the rest of
the information, something that is might not had to have done before,
which would then cause the operation to take more time.

Regards,
Tony

m***@cn.fujitsu.com

2014-03-26 04:20:47 UTC

Permalink

Hi, Gris and Tony

Post by Tony Asleson

Post by Gris Ge

For SMI-S, it seems that will be a performance improve.

Post by Tony Asleson

Post by Gris Ge
1. Already have volume object. If not, follow A.1 and A.2 to get
one.
2. The same as A.3, A.4, A.5
1. Get DELETE http method call against /volumes/<volume_id>
2. The same as progress A.
So, for lsmcli and future REST API, yes, we will have performance improve.
For library user, we might have performance improve and surly not
performance regression.

That's also what I want to say.
Different arrays have different implementations.
If using IDs can't satisfy every plug-ins, we should stay with objects before
we find a good way to solve this problem.
That's my opinion.

Regards,
Ma

Gris Ge

2014-03-26 07:13:58 UTC

Permalink

Post by m***@cn.fujitsu.com

Post by Tony Asleson
It is possible that for some plug-in implementations it would result in
a performance issue. For example it is possible that a plug-in,
supplied only with the id would need to query the array for the rest of
the information, something that is might not had to have done before,
which would then cause the operation to take more time.

Post by Tony Asleson
Regards,
Tony

I will walk through all the parameters, and then we talk on
details, not on possibility.

Thanks.

--
Gris Ge