Process Modeling - Quality of Models

Quality of Models

Earliest process models reflected the dynamics of the process with a practical process obtained by instantiation in terms of relevant concepts, available technologies, specific implementation environments, process constraints and so on.

Enormous number of research has been done on quality of models but less focus has been shifted towards the quality of process models. Quality issues of process models cannot be evaluated exhaustively however there are four main guidelines and frameworks in practice for such. These are: top-down quality frameworks, bottom-up metrics related to quality aspects, empirical surveys related to modeling techniques, and pragmatic guidelines.

Hommes quoted Wang et al. (1994) that all the main characteristic of quality of models can all be grouped under 2 groups namely correctness and usefulness of a model, correctness ranges from the model correspondence to the phenomenon that is modeled to its correspondence to syntactical rules of the modeling and also it is independent of the purpose to which the model is used.

Whereas the usefulness can be seen as the model being helpful for the specific purpose at hand for which the model is constructed at first place. Hommes also makes a further distinction between internal correctness (empirical, syntactical and semantic quality) and external correctness (validity).

A common starting point for defining the quality of conceptual model is to look at the linguistic properties of the modeling language of which syntax and semantics are most often applied.

Also the broader approach is to be based on semiotics rather than linguistic as was done by Krogstie using the top-down quality framework known as SEQUAL. It defines several quality aspects based on relationships between a model, knowledge Externalisation, domain, a modeling language, and the activities of learning, taking action, and modeling.

The framework does not however provide ways to determine various degrees of quality but has been used extensively for business process modeling in empirical tests carried out According to previous research done by Moody et al. with use of conceptual model quality framework proposed by Lindland et al. (1994) to evaluate quality of process model, three levels of quality were identified:

• Syntactic quality: Assesses extent to which the model conforms to the grammar rules of modeling language being used.
• Semantic quality: whether the model accurately represents user requirements
• Pragmatic quality: whether the model can be understood sufficiently by all relevant stakeholders in the modeling process. That is the model should enable its interpreters to make use of it for fulfilling their need.

From the research it was noticed that the quality framework was found to be both easy to use and useful in evaluating the quality of process models however it had limitations in regards to reliability and difficult to identify defects. These limitations led to refinement of the framework through subsequent research done by Krogstie. This framework is called SEQUEL framework by Krogstie et al. 1995 (Refined further by Krogstie & Jørgensen, 2002) which included three more quality aspects.

• Physical quality: whether the externalized model externalized model is persistent and available for the audience to make sense of it.
• Empirical quality: whether the model is modeled according to the established regulations regarding a given language.
• Social quality: This regards the agreement between the stakeholders in the modeling domain.

Dimensions of Conceptual Quality framework Modeling Domain is the set of all statements that are relevant and correct for describing a problem domain, Language Extension is the set of all statements that are possible given the grammar and vocabulary of the modeling languages used. Model Externalization is the conceptual representation of the problem domain.

It is defined as the set of statements about the problem domain that are actually made. Social Actor Interpretation and Technical Actor Interpretation are the sets of statements that actors both human model users and the tools that interact with the model, respectively ‘think’ the conceptual representation of the problem domain contains.

Finally, Participant Knowledge is the set of statements that human actors, who are involved in the modeling process, believe should be made to represent the problem domain. These quality dimensions were later divided into two groups that deal with physical and social aspects of the model.

In later work, Krogstie et al. stated that while the extension of the SEQUAL framework has fixed some of the limitation of the initial framework, however other limitation remain . In particular, the framework is too static in its view upon semantic quality, mainly considering models, not modeling activities, and comparing these models to a static domain rather than seeing the model as a facilitator for changing the domain.

Also, the framework’s definition of pragmatic quality is quite narrow, focusing on understanding, in line with the semiotics of Morris, while newer research in linguistics and semiotics has focused beyond mere understanding, on how the model is used and impact its interpreters.

The need for a more dynamic view in the semiotic quality framework is particularly evident when considering process models, which themselves often prescribe or even enact actions in the problem domain, hence a change to the model may also change the problem domain directly. This paper discusses the quality framework in relation to active process models and suggests a revised framework based on this.

Further work by Krogstie et al. (2006) to revise SEQUAL framework to be more appropriate for active process models by redefining physical quality with a more narrow interpretation than previous research.

The other framework in use is Guidelines of Modeling (GoM) based on general accounting principles include the six principles: Correctness, Clarity deals with the comprehensibility and explicitness (System description) of model systems. Comprehensibility relates to graphical arrangement of the information objects and, therefore, supports the understand ability of a model. Relevance relates to the model and the situation being presented. Comparability involves the ability to compare models that is semantic comparison between two models, Economic efficiency; the produced cost of the design process need at least to be covered by the proposed use of cost cuttings and revenue increases.

Since the purpose of organizations in most cases is the maximization of profit, the principle defines the borderline for the modeling process. The last principle is Systematic design defines that there should be an accepted differentiation between diverse views within modeling. Correctness, relevance and economic efficiency are prerequisites in the quality of models and must be fulfilled while the remaining guidelines are optional but necessary.

The two frameworks SEQUAL and GOM have a limitation of use in that they cannot be used by people who are not competent with modeling. They provide major quality metrics but are not easily applicable by non-experts.

The use of bottom-up metrics related to quality aspects of process models is trying to bridge the gap of use of the other two frameworks by non-experts in modeling but it is mostly theoretical and no empirical tests have been carried out to support their use.

Most experiments carried out relate to the relationship between metrics and quality aspects and these works have been done individually by different authors: Canfora et al. study the connection mainly between count metrics (for example, the number of tasks or splits -and maintainability of software process models; Cardoso validates the correlation between control flow complexity and perceived complexity; and Mendling et al. use metrics to predict control flow errors such as deadlocks in process models.

The results reveal that an increase in size of a model appears to have a negative impact on quality and their comprehensibility. Further work by Mendling et al. investigates the connection between metrics and understanding and While some metrics are confirmed regarding their impact, also personal factors of the modeler – like competence – are revealed as important for understanding about the models.

Several empirical surveys carried out still do not give clear guidelines or ways of evaluating the quality of process models but it is necessary to have clear set of guidelines to guide modelers in this task. Pragmatic guidelines have been proposed by different practitioners even though it is difficult to provide an exhaustive account of such guidelines from practice. In, 10 tips for process modeling are summarized, many technical definitions and rules are provided, but it does not teach how to create process models that are effective in their primary mission - maximizing shared understanding of the as-is or to-be process. Most of the guidelines are not easily put to practice but “label activities verb–noun” rule has been suggested by other practitioners before and analyzed empirically. From the research. value of process models is not only dependent on the choice of graphical constructs but also on their annotation with textual labels which need to be analyzed. It was found that it results in better models in terms of understanding than alternative labelling styles.

From the earlier research and ways to evaluate process model quality it has been seen that the process model's size, structure, expertise of the modeler and modularity have an impact on its overall understandability. Based on these a set of guidelines was presented 7 Process Modeling Guidelines (7PMG). This guideline uses the verb-object style, as well as guidelines on the number of elements in a model, the application of structured modeling, and the decomposition of a process model. The guidelines are as follows:

• G1 Minimize the number of elements in a model
• G2 Minimize the routing paths per element
• G3 Use one start and one end event
• G4 Model as structured as possible
• G5 Avoid OR routing elements
• G6 Use verb-object activity labels
• G7 Decompose a model with more than 50 elements

7PMG still though has limitations with its use: Validity problem 7PMG does not relate to the content of a process model, but only to the way this content is organized and represented. It does suggest ways of organizing different structures of the process model while the content is kept intact but the pragmatic issue of what must be included in the model is still left out. The second limitation relates to the prioritizing guideline the derived ranking has a small empirical basis as it relies on the involvement of 21 process modelers only.

This could be seen on the one hand as a need for a wider involvement of process modelers’ experience, but it also rises the question what alternative approaches may be available to arrive at a prioritizing guideline.