Ranking and selecting the best performance appraisal method using the MULTIMOORA approach integrated Shannon’s entropy

Abstract

The selection process of appropriate Performance Appraisal (PA) methods for organizations in today’s dynamic and agile environments along with its funding scales is a complex problem. Performance appraisal in modern organizations has become a part of the strategic approach toward integrating business policies and human resource activities. The existence of multiple criteria in the decision-making procedure makes finding the optimal PA method more challenging. The current study tackles a PA method assessment by applying a multiple criteria decision analysis method i.e., MULTIMOORA integrated Shannon’s entropy significance coefficient. A case study on the optimal PA method selection is analyzed by identifying the criteria and alternatives based on the literature and expert comments of the case-study employing two approaches, that is, MULTIMOORA and Entropy MULTIMOORA. The final rankings of the suggested methods are compared to TOPSIS and TOPSIS integrated Shannon’s entropy methods utilizing correlation coefficients of the final ranks. Eventually, by identifying the optimal PA approach i.e., 360-degree feedback, the selected optimal method employed in the case study and results are demonstrated and described with a comprehensive example.

Introduction

In the 21st century, at the beginning of the post-industrial era, with the development of global trade and rapid rise in economic transactions, complex and competitive environments are formed (Dobbs 2014). By identifying opportunities and threats, organizations can improve their reactions in these competitive environments. One of the structural factors in any organization is Human Resources (HR). Employees are one of the critical assets for organizations to sustain their competitive advantages by utilizing specific knowledge and skills (Ahmed et al. 2013). Performance appraisal (PA) is a formal management procedure which provides an evaluation of the individual’s performance quality in an organization (Macwan and Sajja 2013). In the past few years, PA approaches have attracted considerable attention. Many quantitative and evaluative methods have developed and evolved, and there has been a considerable amount of research work in the PA field over the past few years (Ahmed et al. 2013; Prowse and Prowse 2010; Shaout and Yousif 2014).

Selecting the most appropriate PA approach for organizations is a challenging job. There are advantages and disadvantages to every method which makes them more or less precise. There are different criteria for selecting the best PA method for an organization. Whenever there is a problem with multiple criteria and multiple alternatives, it is a multiple criteria decision making (MCDM) problem. There have been a few studies which have analysed PA approaches with MCDM methods (Carlucci 2010; Jafari et al. 2009; Pereira 2016; Shaout and Yousif 2014) but to the best of the authors’ knowledge, there are no studies regarding the analysis of a case-based PA method selection based on the MULTIMOORA approach. In the current study, the first step was identifying appropriate criteria and PA approaches from the literature review and expert opinion on which to base the research’s case study. Then, the best PA approach was chosen by utilizing a MULTIMOORA approach based on integrated Shannon’s entropy significant coefficient. The MULTIMOORA method is an updated form of multi-objective optimization on the basis of ratio analysis (MOORA), which is an efficient and straightforward multi-attribute decision making (MADM) technique (Brauers and Zavadskas 2010b). Then, a comprehensive employees’ performance evaluation was obtained by applying the optimal PA approach which was ranked first in the decision making process.

This paper is structured as follow; Section “The applications and developments of the MULTIMOORA approach” briefly reviews applications of the MULTIMOORA method; Section “Developments in performance appraisal methods” conducts a short survey of PA approaches; Section “Research gaps and contributions of the current study” reviews research gaps; Section “MULTIMOORA approach” and “MULTIMOORA approach based on integrated Shannon’s entropy” give a short explanation of the MULTIMOORA approach and Shannon’s entropy combined with the MULTIMOORA method respectively; Section “Findings and results” presents the applications of proposed method in a real-world case study for a PA selection problem conducted in a cross-industrial company in Iran; and Section “Conclusion” offers conclusions and recommendations for future researchers.

Literature review

The applications and developments of the MULTIMOORA approach

The multi-objective optimization on the basis of ratio analysis (MOORA) technique extended by Brauers and Zavadskas (2006) to the MULTIMOORA approach, is one of the most efficient and straightforward multiple attribute decision-making methods (MADM). The MULTIMOORA approach is an improved and comprehensive form of the MOORA technique; due to the particular procedure of the MULTIMOORA method which integrates three subordinate ranks, the results can be more robust and accurate than traditional MADM methods and its previous form i.e., the MOORA approach (Brauers and Ginevičius 2010). Brauers and Zavadskas (2010b) amended the MOORA technique into the standard MULTIMOORA form by applying the proposed technique to project management and testing the robustness of the MULTIMOORA approach. Stankevičienė et al. (2014) proposed investigations and calculations of rankings for country risk and sustainability which optimized results by implementing MOORA and MULTIMOORA methods. Baležentis and Baležentis (2014) reviewed the MULTIMOORA method and discussed the extensions of MULTIMOORA with other data structures along with a survey of applications of the MULTIMOORA and MOORA methods. Liu et al. (2014a) proposed an extended version of the MULTIMOORA approach based on interval 2-tuple linguistic variables which is called ITL-MULTIMOORA for evaluating and selecting HCW treatment technologies. Liu et al. (2014b) suggested a novel risk priority model for evaluating the risk of failure modes based on fuzzy set theory and the MULTIMOORA method. Liu et al. (2015) presented a novel hybrid MCDM model by integrating the 2-tuple DEMATEL (Decision-Making Trial and Evaluation Laboratory) technique and fuzzy MULTIMOORA method for the selection of health-care waste (HCW) treatment alternatives. Zavadskas et al. (2015) proposed an IVIF-MULTIMOORA for group decision making in real-world civil engineering problems. Hafezalkotob and Hafezalkotob (2015a) utilized the MULTIMOORA approach with target-based attributes in a materials selection in a biomedical application.

Ceballos et al. (2016) compared rankings obtained by fuzzy MULTIMOORA, fuzzy TOPSIS (The Technique for Order of Preference by Similarity to Ideal Solution), fuzzy VIKOR (VIseKriterijumska Optimizacija I Kompromisno Resenje), and fuzzy WASPAS (Weighted Aggregated Sum-Product Assessment) to answer the question in every MCDM problem, that is “Which method should be used to solve it?”. Although some efforts have been made, the question is still open. Dai et al. (2016) proposed a new multi-attribute group decision-making method based on triangular fuzzy data structure with a MULTIMOORA approach to rank the best investment scenario from four alternatives. Zhao et al. (2016) suggested a novel approach toward Failure mode and effect analysis (FMEA) based on interval-valued intuitionistic fuzzy sets (IVIFSs) and a MULTIMOORA approach to handle the uncertainty and vagueness in a FMEA process and to achieve a more accurate ranking of failure modes identified by FMEA. Hafezalkotob and Hafezalkotob (2015b, 2016a) extended the MULTIMOORA approach based on Shannon’s entropy with crisp and fuzzy data in a material selection problem. Hafezalkotob et al. (2016) integrated the MULTIMOORA approach with interval numbers in an application for a material selection problem. Sahu et al. (2016) modified the MULTIMOORA approach considering generalized interval-valued trapezoidal fuzzy numbers ordered weighted geometric average in order to evaluate CNC machine tools. Tian et al. (2016) proposed several simplified neutrosophic linguistic distance measures by employing a distance-based method to determine criterion weights along with an improved MULTIMOORA approach based on a neutrosophic linguistic normalized weighted Bonferroni mean and simplified neutrosophic linguistic normalized geometric weighted Bonferroni mean operators as well as a simplified neutrosophic linguistic distance measure.

By developing a hesitant fuzzy linguistic term set (HFLTS) into novel concepts named double hierarchy linguistic term set (DHLTS) and double hierarchy hesitant fuzzy linguistic term set (DHHFLTS) and integrating these new concepts with the MULTIMOORA approach which results in the DHHFL-MULTIMOORA method, Gou et al. (2017) applied the proposed method to select the optimal city in China by evaluating the implementation status of haze controlling measures. Stanujkic and Zavadskas (2017) proposed a new extension to the MULTIMOORA approach by using single-valued neutrosophic sets which result in more efficiency in solving complex problems where solving requires assessment and prediction. Awasthi and Baležentis (2017) presented a hybrid approach based on benefits, costs, opportunities and risks (BOCR) and a fuzzy MULTIMOORA approach for the selection of a logistics service provider along with a Monte Carlo simulation based sensitivity analysis to determine the robustness of MULTIMOORA with variation in criterion and decision maker weights. Zavadskas et al. (2017) proposed a hybrid approach by combining the Step-Wise Weight Assessment Ratio Analysis (SWARA) technique with a single-valued neutrosophic set MULTIMOORA approach to create a decision support method for residential house construction materials selection. An approach that integrates fuzzy MULTIMOORA and multi-choice conic goal programming was presented by Deliktas and Ustun (2017) to consider criteria in choosing the best students and define the optimum assignments among predefined programs. Tian et al. (2018) suggested a hybrid QFD-based (Quality Function Deployment) fuzzy MCDM approach based on a fuzzy maximizing deviation method (MDM) and the BWM approach along with the MULTIMOORA approach in Changsha, China in order to increases understand about a smart Bike-Sharing Program (BSP).

Developments in performance appraisal methods

Performance appraisal (PA) is a term refering to “a basic process involving superior annual reports on subordinate’s performance of the organization center or managers” (Fletcher 2001). However, nowadays, there is a vast amount of research and studies analyzing different methods of PA, factors affecting the PA process, and PA methods. PA approaches are one of the evaluation processes for continuous improvement and one of the effective tools used in organizational performance management (DeNisi and Murphy 2017). There have been a few studies on PA approaches including the following. Levy and Williams (2004) conducted a systematic literature review of over 300 articles on PA and found that the focus of recent PA studies is changing from theoretical development and enhancements to practical applications. Caruth and Humphreys (2008) demonstrated the need for a more aligned and integrated framework for PA to enhance effective strategic control. Wei and Bi (2008) applied a performance evaluation based on knowledge management and evaluated the criteria by using the ANP (Analytic Network Process) method. Fan and Tang (2009) proposed a PA method based on fuzzy integrals and analyzed the performance of Industry-University-Research cooperative innovation centers in China. Jafari et al. (2009) proposed a framework based on SAW (Simple Additive Weighting) for the selection of the optimal PA method and compared some PA methods to facilitate the selection process for organizations.

Espinilla et al. (2010) developed a web-based evaluation system by using integral PA based on previous PA and web-based models. Suriyakumari and Kathiravan (2013) proposed a Domain Driven Data Mining (D3M) and opinion mining approach for performance evaluation to evaluate the performance of employees in virtual organizations. Espinilla et al. (2013) proposed a PA modeling based on a heterogeneous framework for a 360-degree feedback approach and implemented it in a multinational clothing company.

Ahmed et al. (2013) developed a PA criteria-based system by using fuzzy logic. There are a few more studies which are similar to the research conducted by Ahmed et al. (2013) (Chen 2015; Monsur and Akkas 2015; Ozkan et al. 2014). Later, Shaout and Yousif (2014) developed the same decision matrix used by Jafari et al. (2009) and proposed AHP (Analytic Hierarchy Process) and FTOPSIS method to select the optimal PA approach. On the other hand, Ishizaka and Pereira (2016) presented a PA method based on PROMTHEE (Preference Ranking Organization Method for Enrichment of Evaluations) and ANP by incorporating the visual techniques GAIA and stacked bar chart. Zhou et al. (2016) examined the roles and methods of PA in hospitals from a strategic management perspective. Ikramullah et al. (2016) developed a conceptual framework for analyzing the processes and procedures involved in PA systems to use a more efficient PA method. Komissarova and Zenin (2016) provided a comparative legal analysis of fundamentals of effective PAs, “concerning matters raised in determining universal core concepts and principles of PA for implementation of PA in various national jurisdictions.” DeNisi and Murphy (2017) examined 100 years of research on PA and performance management and presented a comprehensive overview.

Research gaps and contributions of the current study

To the best of the authors’ knowledge, there is not a single study that uses a MULTIMOORA approach for the selection of an optimal PA method. Therefore, this paper presents a new application for the MULTIMOORA method. Furthermore, the reason that the MULTIMOORA approach might be useful for PA method selection is, as mentioned in the literature review, the MULTIMOORA approach has demonstrated that it is an uncomplicated and fast algorithm that has resulted in optimal rankings in other sectors. In order to show that the proposed approach i.e., MULTIMOORA could also provide an optimal ranking in the selection of PA methods, this study utilizes the MULTIMOORA algorithm which has been compared to the TOPSIS method, in order to present the accuracy of the MULTIMOORA and Entropy MULTIMOORA.

As noted in Section “Developments in performance appraisal methods”, only three studies have investigated the selection of optimal PA methods using MCDM, i.e., selecting the best PA method by employing a novel framework based on weights and multiple linear regression (Jafari et al. 2009), choosing the best performance evaluation method by using Analytic Hierarchy Process (AHP) and fuzzy AHP, and TOPSIS and FTOPSIS methods (Shaout and Yousif 2014), presenting a new PA system based on MCDM methods (i.e., PROMETHEE and ANP) (Ishizaka and Pereira 2016). None of those studies considered a real-world case study for the selection of a PA method. Hence, this study is more realistic, and the assumptions given in this MCDM problem are much closer to what is happening in organizations facing a selection of the appropriate PA method in real-world situations.

The focus of this paper is to identify the best criteria based on the validity of the literature and the practicality of real-world applications for analyzing an optimal PA method. Then, the optimal technique found by applying an Entropy MULRIMOORA. Subsequently, the results of the selected PA method are demonstrated in a real-world case study. Therefore, this study is a novel application of the MULTIMOORA approach to a real-world PA problem.

Furthermore, a comparison of the proposed methods has been made using a correlation coefficient of the ranks. Consequently, a comprehensive analysis of an example which has been through the PA process has been demonstrated.

Research methodology

MULTIMOORA approach

The MULTIMOORA method consists of three parts, the ratio system, the reference point and the full multiplicative form which form the multi-objective optimization by ratio analysis (MOORA) method developed by Brauers and Zavadskas (2006). Later on, Brauers and Zavadskas (2010b) extended the method by adding the full multiplicative form to the MOORA procedure to achieve a more robust method. The first step in the MULTIMOORA method is forming the decision matrix X in which x _ij presents the performance index of ith alternative respecting jth attribute i = 1, 2, …m and j = 1, 2, …n , and ${w}_j^s$ denotes the subjective significance coefficients of jth attribute i = 1, 2, …m and j = 1, 2, …n:

$$ \boldsymbol{X}={\left[{x}_{ij}\right]}_{m\times n}, $$

(1)

$$ {\boldsymbol{w}}_{\boldsymbol{j}}^{\boldsymbol{s}}={\left[{w}_j\right]}_n,\kern0.5em \sum {w}_j=1. $$

(2)

In the MULTIMOORA approach, these parameters should be dimensionless in order to make performance indices comparable. Therefore, the decision matrix is a normalization ratio of comparison between each response of an alternative to a criterion as a numerator, and a denominator that is representative for all alternative performances on that attribute, as shown in Eq. (3):

$$ {X}_{ij}^{\ast }=\frac{x_{ij}}{\sqrt[2]{\sum_{i=1}^m{x}_{ij}^2}}, $$

(3)

where, ${X}_{ij}^{\ast }$ denotes the normalized performance index of ith alternative respecting jth attribute i = 1, 2, …m and j = 1, 2, …n and x _ij presents the performance index of ith alternative respecting jth attribute i = 1, 2, …m and j = 1, 2, …n.

The ratio system

The normalization equation i.e., Eq. (3) justifies the foundation of this approach as the ratio system. In a current approach for optimization, the normalized performance indices are added in case of maximization and subtracted in the event of minimization (Brauers and Zavadskas 2011):

$$ {y}_i^{\ast }={\sum}_{j=1}^g{w}_j^s{X}_{ij}^{\ast }-{\sum}_{j=g+1}^n{w}_j^s{X}_{ij}^{\ast }, $$

(4)

in which, g indicates the objectives to be maximized and (n − g) indicates the objectives being minimized, ${y}_i^{\ast }$ denotes the total assessment of alternative j with respect to subjective significance coefficients of all attributes ${w}_j^s$ which can be positive or negative based on the totals of the calculations. The optimal alternative based on the ratio system is an ordinal ranking of the ${y}_i^{\ast }$ which has the highest assessment value:

$$ {A}_{RS}^{\ast }=\left\{{A}_i|{\mathit{\max}}_i\ {y}_i^{\ast}\right\}. $$

(5)

The reference point approach

The second part of the MULTIMOORA approach is based on the foundation of the ratio system shown in Eq. (3). A maximal objective reference point is also concluded in the method obtained by Eq. (6) (Brauers and Zavadskas 2006):

$$ {r}_j=\left\{\begin{array}{c}{\mathit{\max}}_i{X}_{ij}^{\ast}\kern1.5em in case of maximization\\ {}{\mathit{\max}}_i{X}_{ij}^{\ast}\kern1.75em in case of minimization\ \end{array},\right. $$

(6)

where r _j denotes the ith co-ordinate of the maximal objective reference point vector.

Deviation of a performance index from the reference point r _j can be shown as $\left({r}_j-{X}_{ij}^{\ast}\right)$. Subsequently, the maximum value of the deviation for each alternative ${z}_i^{\ast }$ respecting subjective significance coefficients of all criteria ${w}_j^s$ can be calculated as Eq. (7):

$$ {z}_i^{\ast }={\mathit{\max}}_j\left|\left({w}_j^s{r}_j-{w}_j^s{X}_{ij}^{\ast}\right)\right|, $$

(7)

in the reference point approach, calculation of the optimal alternative is obtained by computing the minimum value of Eq. (7) demonstrated in Eq. (8):

$$ {A}_{RP}^{\ast }=\left\{{A}_i|{\mathit{\min}}_i\ {z}_i^{\ast}\right\}. $$

(8)

The full multiplicative form

The third part of the MULTIMOORA method developed by Brauers and Zavadskas is based on an idea from economic mathematics (Brauers and Zavadskas 2010a, 2010b). The formula of the full multiplicative form can be determined as demonstrated in Eq. (10) where g denotes the objectives to be maximized and (n − g) indicates as the objectives to be minimized. The numerator of Eq. (10) indicates the product of performance indices of ith alternative relating to beneficial attributes. The denominator of Eq. (10) represents the product of performance indices of ith alternative relating to non-beneficial attributes respecting subjective significance coefficients of each attribute ${w}_j^s$.

$$ {U}_i^{\prime }=\frac{\prod_{j=1}^g{\left({x}_{ij}\right)}^{w_j^s}}{\prod_{j=g+1}^n{\left({x}_{ij}\right)}^{w_j^s}}, $$

(10)

by using a normalized decision matrix an equivalent equation form of ${U}_i^{\prime }$ can be calculated:

$$ {U}_i^{\ast }=\frac{\prod_{j=1}^g{\left({X}_{ij}^{\ast}\right)}^{w_j^s}}{\prod_{j=g+1}^n{\left({X}_{ij}^{\ast}\right)}^{w_j^s}}, $$

(11)

to maintain harmony among all parts of the calculations in the MULTIMOORA approach. Equation (11) shows the normalized form of the full multiplicative form used. Similar to the ratio system computation of the optimal alternative, it is based on the searching for the maximum among all assessment values of ${U}_i^{\ast }$:

$$ {A}_{MF}^{\ast }=\left\{{A}_i|{\mathit{\max}}_i{U}_i^{\ast}\right\}. $$

(12)

The dominance theory: The final ranking of the MULTIMOORA method

The dominance theory was proposed as a tool for ranking subordinate alternatives with the MULTIMOORA method (Brauers and Zavadskas 2011, 2012). After the calculation of the subordinate ranks, they can be integrated into a final ranking, which is the final MULTIMOORA rank based on the obtained dominance theory. In dominance theory, a summary of the classification of the three MULTIMOORA methods is made based on cardinal and ordinal scales in which rankings rules should be applied (i.e., dominated, transitivity and equability). The theory of dominance can be described as: “(1) the plurality rule assisted with a kind of lexicographic method, (2) the method of correlation of ranks.” demonstrated by Brauers and Zavadskas (2012). For a more detailed explanation of the dominance theory, readers can refer to the study of Brauers and Zavadskas (2012).

MULTIMOORA approach based on integrated Shannon’s entropy

Hafezalkotob and Hafezalkotob (2015b) proposed an extended MULTIMOORA method based on Shannon’s entropy significance coefficient for a material selection problem. As mentioned in Hafezalkotob and Hafezalkotob (2015b), significance coefficients can be obtained in the form of a subjective significance coefficient which comes directly from the completion of a decision matrix by decision makers. Many research studies have integrated the MCDM approaches with subjective weight calculation approaches such as Best-Worst Method (BWM), Shannon’s entropy, etc. Huang et al. (2017) applied a linguistic distribution assessment in order to represent FMEA team members’ risk evaluation information and employed an improved interactive and multiple criteria decision-making approach to determine the risk priority of failure modes. Zhao et al. (2017) suggested an integrated VIKOR approach considering intuitionistic fuzzy data along with both subjective and objective weights of criteria in a supplier selection problem. Liu et al. (2017) developed an integrated risk prioritization method to improve the performance of FMEA by using interval-valued intuitionistic fuzzy sets (IVIFSs) and the multi-attributive border approximation area comparison (MABAC) method. In the current research, two forms of significance coefficient of attributes are demonstrated: the subjective significance coefficient which is already demonstrated i.e., based on expert judgments, and the objective significance coefficient which is obtained through Shannon’s entropy.

Shannon’s entropy significance coefficient

The concept of entropy has been widely employed in numerous fields of research e.g., social sciences, economics, physical sciences, etc. based on a mathematical theory of communication proposed by Claude Shannon (1948). The proposed concept can be effectively employed in the process of decision making because in information theory it can be considered as a criterion for the degree of uncertainty represented by a discrete probability distribution, and it measures existent contrasts between sets of data and clarifies the average intrinsic information transferred to the decision maker (Hafezalkotob and Hafezalkotob 2016a). Normalization of x _ij to determine p _ij which is the total project outcome, obtained by Eq. (13):

$$ {p}_{ij}=\frac{x_{ij}}{\sum_{i=1}^m{x}_{ij}}, $$

(13)

Shannon entropy measure E _j is calculated using the total project outcome p _ij computed by Eq. (14):

$$ {E}_j=-k{\sum \limits}_{i=1}^m\left({p}_{ij}\ \mathit{\ln}\kern0.5em {p}_{ij}\right), in which\kern0.75em k=\frac{1}{\mathit{\ln}(m)}. $$

(14)

calculation of objective significance coefficients achieved by employing E _j as demonstrated in Eq. (15):

$$ {w}_j^o=\frac{d_j}{\sum_{j=1}^n{d}_j}\kern0.62em \mathrm{in}\ \mathrm{which}\ \kern1em {d}_j=1-{E}_j. $$

(15)

Calculation of the integrated Shannon significance coefficients, if the expert assigns subjective significance coefficients ${w}_j^s$ computed by using Eq. (16) which is a combination of subjective and objective significance coefficients:

$$ {w}_j^{\ast }=\frac{w_j^s{w}_j^o}{\sum_{j=1}^n{w}_j^s{w}_j^o}, $$

(16)

when ${w}_j^o,$ i.e., objective significance coefficient is larger, the variation degree of ratings on the attribute is higher, which is a result of a smaller E _j of an attribute. Adversely, larger E _j denotes a lower degree of variation of the ratings, the less information over attribute j, and minor objective significance coefficient ${w}_j^o$ (Hafezalkotob and Hafezalkotob 2016b).

The extended MULTIMOORA method based on Shannon’s entropy

To integrate the MULTIMOORA method with Shannon’s entropy significance coefficients ${w}_j^o$, the subjective significance coefficient ${w}_j^s$ should be replaced by ${w}_j^o$. In current research, the MULTIMOORA method is calculated by using both Shannon significance coefficient ${w}_j^o$ and considering the subjective significance coefficient with Shannon’s entropy ${w}_j^{\ast }$. Considering the subjective significance coefficient with Shannon’s entropy ${w}_j^{\ast }$ the calculations of the extended MULTIMOORA method is obtained in section “The extended ratio system” and section “The extended reference point approach and the extended full multiplicative form”.

The extended ratio system

By replacing the subjective significance coefficient ${w}_j^s$ with the combination of subjective significance coefficient with Shannon’s entropy ${w}_j^{\ast }$ the extended ratio system method is calculated by Eq. (17). Additionally, calculation of the optimal alternative obtained by Eq. (18):

$$ {y}_i^{ew}={\sum}_{j=1}^g{w}_j^{\ast }{X}_{ij}^{\ast }-{\sum}_{j=g+1}^n{w}_j^{\ast }{X}_{ij}^{\ast }, $$

(17)

$$ {A}_{RS}^{ew}=\left\{{A}_i|{\mathit{\max}}_i\ {y}_i^{ew}\right\}. $$

(18)

The extended reference point approach and the extended full multiplicative form

As with the ratio system by replacing the subjective significance coefficient with the combination of subjective significance coefficient with Shannon’s entropy ${w}_j^{\ast }$, the extended reference point approach, the extended full multiplicative form and the optimal alternative rankings of proposed methods are respectively achieved by calculating Eqs. (19), (20), (21) and (22):

$$ {z}_i^{ew}={\mathit{\max}}_j\left|\left({w}_j^{\ast }{r}_j-{w}_j^{\ast }{X}_{ij}^{\ast}\right)\right|, $$

(19)

$$ {A}_{RP}^{ew}=\left\{{A}_i|{\mathit{\min}}_i\ {z}_i^{ew}\right\}, $$

(20)

$$ {U}_i^{ew}=\frac{\prod_{j=1}^g{\left({X}_{ij}^{\ast}\right)}^{w_j^{\ast }}}{\prod_{j=g+1}^n{\left({X}_{ij}^{\ast}\right)}^{w_j^{\ast }}}, $$

(21)

$$ {A}_{MF}^{ew}=\left\{{A}_i|{\mathit{\max}}_i{U}_i^{ew}\right\}. $$

(22)

The complete flow-diagram of the proposed approach i.e., the MULTIMOORA method based on integrated Shannon’s entropy towards selecting the optimal PA method is illustrated in Fig. 1.

Fig. 1

Flowchart of selecting the optimal PA method based on the MULTIMOORA method integrated Shannon’s entropy weight

For the proposed MCDM methodology and based on Fig. 1, the first step is to gather the input data i.e., decision criteria and PA alternatives from experts and literature review to construct the decision matrix. Then the Shannon’s entropy significance coefficient is calculated in order to obtain the weights of the criteria. Ultimately, three steps of the MULTIMOORA approach including the ratio system, the reference point approach and the full multiplicative form are computed. Furthermore, to develop a better understanding of the proposed method a real-world case study is presented in Section “Findings and results”.

Findings and results

The current study is a practical and validation experiment. This type of research is practical because of its purpose. With regards to content and data collection, it is descriptive and quantitative. The type of review is a case study. Given that the success of performance appraisal (PA) methods in an organization is considered from the perspective of the human resources manager and high-level managers, the study population includes the specialists, experts, and officials of the implementation of the PA procedure in the proposed case-study. A set of criteria for selecting the optimal PA approach from the previous research and expert comments on the case study has been collected and classified in Table 1 in which the description of each criterion is available. Table 2 shows a short description of each PA method which has been selected to be included in the ranking procedure. These PA methods have been collected and classified from previous research and expert comments directly dealing with the case study’s PA method implementation.

Table 1 Criteria definition for selection of the optimal PA method

Table 2 Candidate alternatives selection of the optimal PA method (Shaout and Yousif 2014)

In the current study, after identifying the PA methods and measurement criteria by employing expert comments from the case study i.e., experts in human resources and high level management of the suggested cross-industrial company in Iran, and the literature review, the PA methods are assessed by using the multiple criteria decision-making tools of the MULTIMOORA method and the Entropy MULTIMOORA. Additionally, a comparison of rankings has been made and demonstrated based on the proposed method in this study i.e., MULTIMOORA and Entropy MULTIMOORA and the TOPSIS approach and TOPSIS integrated Shannon’s entropy. Subsequently, the optimal PA method is applied to a cross-industrial company in Iran as a real-world case study. The MULTIMOORA and MULTIMOORA approach based on integrated Shannon’s entropy significant coefficients are utilized to select the optimal PA method for an organization. To reach a better understanding of the operative PA selection procedure in a real-world application, a case-study of a multi-national cross-industrial company with the major activities in construction and transportation infrastructure in Iran is presented.

The necessary data for this case-study was collected through interviews. The respondents are high-level managers i.e., three managers, two human resource managers and two manager assistants all directly dealing with the procedure of selecting the optimal PA method. The interview method was question and answer (Q&A) based on the linguistic terms of Table 3. The fundamental purpose of the Q&A was to complete the decision matrix shown in Table 4 based on the linguistic terms and the corresponding numbers of Table 3. This study is based on one case only i.e., that of a cross-industry company in Iran. As explained by Easton (2010), focusing on one case study leads to a better understanding of existing data and a robust exploration and reflection on that data by the researchers. Flyvbjerg (2006) clarified that to employ in-depth research on any topic, “one can study only one case, and the result can be generalized.” Accordingly, the case-study in this research was not chosen randomly. It targeted a specific organization to be able to obtain data that other organizations would not be able to offer. As a result, this practical case has been chosen to gain a thorough knowledge of the selection of the optimal PA method in cross-industry organizations in Iran.

Table 3 Linguistic terms and the corresponding numbers

Table 4 Decision matrix for ranking the optimal PA method

As mentioned before, in the MULTIMOORA procedure, decision matrix numbers which have different dimensions and measurement units transform into dimensionless numbers. This process is so-called normalization which is applied based on Eq. (3) by comparing numbers to each other. The normalized decision matrix is shown in Table 5.

Table 5 Normalized decision matrix for ranking the optimal PA method

Subjective significance coefficient ${w}_j^s$ is imported from expert comments in the same procedure where the decision matrix is completed. Shannon entropy measure E _j is calculated by using Equation (14) and the calculation of objective significance coefficient ${w}_j^o$ is achieved by employing E _j based on the Equation (15). Since both subjective and objective significance coefficients are available in this study, the combined (integrated) significance coefficient ${w}_j^{\ast }$ is calculated by Equation (16) shown in previous sections. Table 6 shows the significant coefficients measurements which are computed from the decision matrix.

Table 6 Significant coefficients measures and weighting factors

The values of the MULTIMOORA is obtained and calculated by employing Equation (4), (7) and (11), respectively. Then, each stage of the MULTIMOORA procedure is calculated by each ranking method. Consequently, the optimal assessment for weighted MULTIMOORA (subjective weights) is calculated by employing dominance theory for the final rank Table 7.

Table 7 Assessment values and rankings of the weighted (subjective) MULTIMOORA for selecting optimal PA method

Based on entropy measurements described in Table 6 and previous calculations for the MULTIMOORA approach respecting subjective and objective significance coefficients combined, the assessment values of Entropy MULTIMOORA is calculated by employing Eqs. (17), (19) and (21) and shown in Table 8. Additionally, the ranking of each procedure of MULTIMOORA is obtained by the assessment values and the final rank of the method respecting dominance theory.

Table 8 Assessment values and rankings of Entropy MULTIMOORA for selecting optimal PA method

Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) is a technique of MCDM which has been proposed and developed by (Hwang and Yoon 1981; Yoon and Hwang 1995). In TOPSIS Euclidean distances after the Minkowski metric would lead to ∞ solutions. Therefore TOPSIS introduces Significance Coefficients, called wrongly weights (Hwang and Yoon 1981). In the current study, a comparison of the proposed method i.e., MULTIMOORA and Entropy MULTIMOORA with TOPSIS and Entropy TOPSIS is demonstrated in Table 9.

Table 9 Comparison between the optimal PA method ranks of the proposed MULTIMOORA approach with TOPSIS method

The Spearman rank correlation coefficient helps with evaluating the similarity of the rankings. A coefficient is a real number in the range of −1 and 1. The Spearman coefficient equal to one denotes identical rankings and −1 indicates opposite rankings. Spearman was a psychologist who in 1904–1910 wrongly used the traditional operations of mathematics for ordinal numbers. It was the statistician Kendall who formulated the rank correlation method: “we shall often operate with these numbers as if they were the cardinals of ordinary arithmetic, adding them, subtracting them and even multiplying them,” but he never gave a proof of this statement (Kendall 1938). Figure 2, illustrates the correlation between ranking lists by utilizing the Spearman rank correlation coefficient obtained by employing Equation (23), the Spearman rank correlation coefficient similarity of the rankings in Table 9.

$$ {r}_s=1-\frac{\sum \limits_{i=1}^n{D}^2}{n\left({n}^2-1\right)}, $$

(23)

where D is differences between the two ranks and n denotes the sample size.

Fig. 2

Correlation between the rankings based on the Spearman coefficient

Figure 2 shows that the proposed methods in this study have high correlation values compared to each other. Based on Table 9, alternative number 9 i.e., 360 Degree has been selected as the optimal PA method based on MULTIMOORA, Entropy MULTIMOORA, and Entropy TOPSIS.

360-degree feedback example: a managerial vision

360-degree feedback which is also known as multi-rater feedback, multi-source feedback or multi-source assessment refers to the process by which PAs are collected from different individual sources i.e., supervisors, peers, subordinates, and customers instead of relying on an appraisal from a single source which would provide less information for the feedback. (Ghorpade 2000; van der Heijden and Nijhof 2004). Today, studies suggest that most organizations use some type of multi-source feedback because of its significant advantages over the traditional PA methods (Espinilla et al. 2013). As mentioned, this study is a case-based research employing the optimal PA method, 360-degree feedback, selected by the proposed MCDM approach i.e., Entropy MULTIMOORA & MULTIMOORA. In order to present a better understanding of the 360-degree approach, the application of the 360-degree approach to a real-world case study is presented.

The necessary research data has been collected using a questionnaire. In this case-study, the questionnaire was created by the organization utilizing the value engineering concepts based on the 360-degree feedback approach. It consists of ten general evaluation factors with 72 questions which were obtained by using the PA approach from the group of experts in the human resource department. Additionally, the rating for the factors in the questionnaire is based on a 0–20 scale. Table 10 shows the statistical population and characteristics of the respondents regarding their occupation in the organization.

Table 10 Characteristics of the questionnaire respondents

Generally, the organizational PA procedure is applied every 6 months in the case-study organization, in this research, the central office of the multi-national cross-industrial organization selected for statistical analysis. The questionnaire has been distributed to 475 employees and 275 complete and accurate questionnaires have been received. The sample size was selected by using the Krejcie and Morgan Table. To control the quality of the questionnaire results, the proper objective features should be examined. Among these features, validity and reliability are more important. The purpose of the validity feature of the questionnaire is to what extent it can accurately measure the variables that are designed for it. In the collected statistical sample from the questionnaires, at least 12 professionals and experts from the human resource department have confirmed the validity of the prepared questionnaire for this research. The purpose of the reliability feature of the questionnaire is to what extent it can measure the same results by using the questionnaire in other different spatial and temporal conditions. Cronbach’s alpha is one of the most common methods for determining the reliability of the questionnaire. When the Cronbach’s alpha value gets closer to 100%, the reliability of the collection tools is increased. The Cronbach’s alpha reliability coefficient of the questionnaire has reached 73% by using SPSS software which indicates the acceptability of the reliability test of the questionnaire. Additionally, to achieve more accurate reliability, the Composite Reliability coefficient (CR) has also been considered, and the rate of 0.70 has been obtained by using Smart PLS software which demonstrates the acceptability of the CR test of the questionnaire. Subsequently, when the PA results for each employee is obtained, the senior management will decide what to do based on the three following stated scenarios:

• I. Employees that obtained less than 70% of the average; for these employees, the PA period will change to 3 months, and if the same results are shown in the next PA, the employees will end up in the penalty process.

• II. Employees that obtained more than 50% of the average; these employees will receive encouragement and financial and non-financial rewards.

• III. Employees that obtained more than 70% of the average; treated the same as II and these employees will be asked to be evaluators for the next PA.

Figure 3 shows an example of an employee performance evaluation from the case-study based on ten factors. The employee Mr. X.Y is on the supervision organizational level, and it is the third time this employee has been involved in PA procedure.

Fig. 3

An example of case-study PA report

Conclusion

In today’s dynamic and competitive environment for organizations, one of the most important issues to discuss is the continuous improvement of the organization itself. One of the main tools to maintain improvement is the periodic evaluation. Therefore, selecting the best PA method is substantial. In large-sized enterprises, selecting the optimal PA method is a challenging task which may require research and special expertise. Selecting the optimal PA method considering the specifications of an organization based on details considering that the possibility of being wrong could impose inappropriate costs on organizations. This is the reason why selecting the optimal approach based on the MCDM method is a good idea, comprehensively described in the current paper. In the present paper, criteria and PA approaches were first identified, and a comprehensive description of each criterion and alternative was provided. Second, the MULTIMOORA integrated Shannon’s entropy was utilized to provide a selection of optimal PA methods applied to a case-study, a multi-national cross-industry company in Iran. Third, the correlations between the rankings of the MULTIMOORA approach and the TOPSIS method were examined by applying correlation coefficients of ranks. Finally, the optimal PA method i.e., 360-degree feedback was selected and employed in the case study and the results of the PA were specified and demonstrated. Ultimately, in this study, a new application of the MULTIMOORA approach has been presented.

Suggestions for future developments of this study may be as follows. First, input data of the MCDM approach can be extended to cases in which the data of the problem has different mathematical forms such as extensions of fuzzy sets, e.g., flou sets, fuzzy multi-sets, bipolar fuzzy sets, and interval data structure. Second, applying the current decision matrix including the same criteria and alternatives to different organizations may have different results which could be compared to the current research. Third, significance coefficients of attributes may be achieved using various techniques. In the present study, subjective significance coefficients were considered, and objective significance coefficients were determined based on Shannon entropy. Subjective significance coefficients may be computed by applying various methods such as ANP, AHP, and BWM.