Using VIATRA transformations efficiently on a subgraph of an EMF model

Question

Hi,
I have EMF models describing program codes and I am using manually-implemented transformations to simplify these models. I'd like to introduce some new, expression-simplification rules using VIATRA without rewriting all the old transformations (for the moment). I use batch transformations in VIATRA to avoid any interference. While describing the new transformations using VIATRA's language was convenient and quicker than implementing them manually, the performance of the VIATRA-based reductions is much worse, even if the old ones are not really optimized. (About the models: the whole program code model can contain up to 1..10 million EObjects. The expression subtrees I'm trying to reduce are typically small, but in some cases they can contain 1k..100k elements -- these are the situations when model transformations could help. I was able to reduce the size of expression from about 1000 to 10 using VIATRA transformations, now I need to make these reductions fast.)
The old reductions are applied in a loop, until any of them can modify the model. If I apply the new, VIATRA-based reductions at the beginning to the whole model, it makes the old reductions slow. If in each loop I create a new BatchTransformation, fire the rules and dispose the engine right after, the speed of the old reductions is OK, but the initialization of the BatchTransformation takes a long time. Ok, it's not a big surprise.
However, my VIATRA-based reductions are only used to simplify complex expression subtrees, so it would be enough to fire them when a huge expression subtree is created (it is known when does it happen), and only on this expression subtree. But:
1) If I create a BatchTransformation with EMFScope(<root of the expression subtree>), no rules can fire. The expression subtrees can contain VariableReferences whose reference to a variable is not a containment reference. My rules have constraitns on the referenced variables, but they are out of the scope of the transformation.
2) If I create a new ResourceSet, containing the expression to be reduced and the variable declarations, the rules work, but I mess up my original model as I remove some elements from the original resource.
3) If I use a BatchTransformation with EMFScope(<root of the whole model>, options) and the options object has an IBaseIndexObjectFilter which filters out all expression subtrees except the one I try to reduce, the performance seems to be worse (or at least not any better) than if I don't use any filtering at all.
I know that most probably I am misusing VIATRA, but is there any way to efficiently restrict the transformations for a subgraph of the EMF model? Or I just shouldn't use VIATRA in this case? Any helpful suggestions are welcome.
Cheers,
Daniel

Answer 1

Hi,
Daniel Darvas wrote on Mon, 27 July 2015 05:38
While describing the new transformations using VIATRA's language was convenient and quicker than implementing them manually, the performance of the VIATRA-based reductions is much worse, even if the old ones are not really optimized.
Just to make it clear: these kinds of issues are caused by your configuration of the incremental engine of EMF-IncQuery (which is typically used by VIATRA rules) and not VIATRA itself.
Quote:
If I apply the new, VIATRA-based reductions at the beginning to the whole model, it makes the old reductions slow.
To explain: this is because an IncQuery engine is initialized on your model and it processes all your changes, even if they are made outside of VIATRA.
Quote:
If in each loop I create a new BatchTransformation, fire the rules and dispose the engine right after, the speed of the old reductions is OK, but the initialization of the BatchTransformation takes a long time. Ok, it's not a big surprise.
... indeed, as in each loop you initialize a new IncQuery engine from scratch, which is an expensive operation in your case.
Now on to my advice which will hopefully help:
Quote:
3) If I use a BatchTransformation with EMFScope(<root of the whole model>, options) and the options object has an IBaseIndexObjectFilter which filters out all expression subtrees except the one I try to reduce, the performance seems to be worse (or at least not any better) than if I don't use any filtering at all.
There is a recently fixed issue (https://bugs.eclipse.org/bugs/show_bug.cgi?id=473841) which will hopefully remedy your problem. Could you please check with a snapshot or CI build from today whether this is the case?
Quote:
I know that most probably I am misusing VIATRA, but is there any way to efficiently restrict the transformations for a subgraph of the EMF model? Or I just shouldn't use VIATRA in this case? Any helpful suggestions are welcome.
For your usecase (assuming you are interested in "batch" transformations only), perhaps a better fit is the new local search-based pattern matching engine available from IncQuery 1.0.0. More details on how to use it can be found at
https://wiki.eclipse.org/EMFIncQuery/UserDocumentation/API/Advanced#Using_alternative_search_algorithms
An a few concrete examples:
https://github.com/FTSRG/trainbenchmark-ttc/blob/master/hu.bme.mit.trainbenchmark.ttc.benchmark.emfincquery/src/main/java/hu/bme/mit/trainbenchmark/ttc/benchmark/emfincquery/benchmarkcases/EMFIncQueryBenchmarkCase.java#L85
https://github.com/FTSRG/trainbenchmark-ttc/blob/master/hu.bme.mit.trainbenchmark.ttc.benchmark.emfincquery/src/main/java/hu/bme/mit/trainbenchmark/ttc/benchmark/emfincquery/benchmarkcases/EMFIncQueryPosLength.java#L42
However, it is important to note that the search-based pattern matcher does not support updates which are required for EVM rules for VIATRA. So in order to make use of this feature, unfortunately you need to rewrite your transformation according to the "Batch IncQuery" style, as described in https://github.com/IncQueryLabs/incquery-examples-cps/wiki/Alternative-transformation-methods#batch and https://github.com/IncQueryLabs/incquery-examples-cps/wiki/Simple-Xtend-and-IncQuery-M2M-transformation
I hope this helps.
Istvan