Skip to content

Advertisement

  • Research
  • Open Access

Ranking and selecting the best performance appraisal method using the MULTIMOORA approach integrated Shannon’s entropy

Frontiers of Business Research in ChinaSelected Publications from Chinese Universities201812:2

https://doi.org/10.1186/s11782-017-0022-6

  • Received: 6 October 2017
  • Accepted: 6 December 2017
  • Published:

Abstract

The selection process of appropriate Performance Appraisal (PA) methods for organizations in today’s dynamic and agile environments along with its funding scales is a complex problem. Performance appraisal in modern organizations has become a part of the strategic approach toward integrating business policies and human resource activities. The existence of multiple criteria in the decision-making procedure makes finding the optimal PA method more challenging. The current study tackles a PA method assessment by applying a multiple criteria decision analysis method i.e., MULTIMOORA integrated Shannon’s entropy significance coefficient. A case study on the optimal PA method selection is analyzed by identifying the criteria and alternatives based on the literature and expert comments of the case-study employing two approaches, that is, MULTIMOORA and Entropy MULTIMOORA. The final rankings of the suggested methods are compared to TOPSIS and TOPSIS integrated Shannon’s entropy methods utilizing correlation coefficients of the final ranks. Eventually, by identifying the optimal PA approach i.e., 360-degree feedback, the selected optimal method employed in the case study and results are demonstrated and described with a comprehensive example.

Keywords

  • Performance appraisal
  • Employee performance evaluation
  • Multiple criteria decision making (MCDM)
  • MULTIMOORA method
  • Shannon’s entropy
  • 360-degrees approach

Introduction

In the 21st century, at the beginning of the post-industrial era, with the development of global trade and rapid rise in economic transactions, complex and competitive environments are formed (Dobbs 2014). By identifying opportunities and threats, organizations can improve their reactions in these competitive environments. One of the structural factors in any organization is Human Resources (HR). Employees are one of the critical assets for organizations to sustain their competitive advantages by utilizing specific knowledge and skills (Ahmed et al. 2013). Performance appraisal (PA) is a formal management procedure which provides an evaluation of the individual’s performance quality in an organization (Macwan and Sajja 2013). In the past few years, PA approaches have attracted considerable attention. Many quantitative and evaluative methods have developed and evolved, and there has been a considerable amount of research work in the PA field over the past few years (Ahmed et al. 2013; Prowse and Prowse 2010; Shaout and Yousif 2014).

Selecting the most appropriate PA approach for organizations is a challenging job. There are advantages and disadvantages to every method which makes them more or less precise. There are different criteria for selecting the best PA method for an organization. Whenever there is a problem with multiple criteria and multiple alternatives, it is a multiple criteria decision making (MCDM) problem. There have been a few studies which have analysed PA approaches with MCDM methods (Carlucci 2010; Jafari et al. 2009; Pereira 2016; Shaout and Yousif 2014) but to the best of the authors’ knowledge, there are no studies regarding the analysis of a case-based PA method selection based on the MULTIMOORA approach. In the current study, the first step was identifying appropriate criteria and PA approaches from the literature review and expert opinion on which to base the research’s case study. Then, the best PA approach was chosen by utilizing a MULTIMOORA approach based on integrated Shannon’s entropy significant coefficient. The MULTIMOORA method is an updated form of multi-objective optimization on the basis of ratio analysis (MOORA), which is an efficient and straightforward multi-attribute decision making (MADM) technique (Brauers and Zavadskas 2010b). Then, a comprehensive employees’ performance evaluation was obtained by applying the optimal PA approach which was ranked first in the decision making process.

This paper is structured as follow; Section “The applications and developments of the MULTIMOORA approach” briefly reviews applications of the MULTIMOORA method; Section “Developments in performance appraisal methods” conducts a short survey of PA approaches; Section “Research gaps and contributions of the current study” reviews research gaps; Section “MULTIMOORA approach” and “MULTIMOORA approach based on integrated Shannon’s entropy” give a short explanation of the MULTIMOORA approach and Shannon’s entropy combined with the MULTIMOORA method respectively; Section “Findings and results” presents the applications of proposed method in a real-world case study for a PA selection problem conducted in a cross-industrial company in Iran; and Section “Conclusion” offers conclusions and recommendations for future researchers.

Literature review

The applications and developments of the MULTIMOORA approach

The multi-objective optimization on the basis of ratio analysis (MOORA) technique extended by Brauers and Zavadskas (2006) to the MULTIMOORA approach, is one of the most efficient and straightforward multiple attribute decision-making methods (MADM). The MULTIMOORA approach is an improved and comprehensive form of the MOORA technique; due to the particular procedure of the MULTIMOORA method which integrates three subordinate ranks, the results can be more robust and accurate than traditional MADM methods and its previous form i.e., the MOORA approach (Brauers and Ginevičius 2010). Brauers and Zavadskas (2010b) amended the MOORA technique into the standard MULTIMOORA form by applying the proposed technique to project management and testing the robustness of the MULTIMOORA approach. Stankevičienė et al. (2014) proposed investigations and calculations of rankings for country risk and sustainability which optimized results by implementing MOORA and MULTIMOORA methods. Baležentis and Baležentis (2014) reviewed the MULTIMOORA method and discussed the extensions of MULTIMOORA with other data structures along with a survey of applications of the MULTIMOORA and MOORA methods. Liu et al. (2014a) proposed an extended version of the MULTIMOORA approach based on interval 2-tuple linguistic variables which is called ITL-MULTIMOORA for evaluating and selecting HCW treatment technologies. Liu et al. (2014b) suggested a novel risk priority model for evaluating the risk of failure modes based on fuzzy set theory and the MULTIMOORA method. Liu et al. (2015) presented a novel hybrid MCDM model by integrating the 2-tuple DEMATEL (Decision-Making Trial and Evaluation Laboratory) technique and fuzzy MULTIMOORA method for the selection of health-care waste (HCW) treatment alternatives. Zavadskas et al. (2015) proposed an IVIF-MULTIMOORA for group decision making in real-world civil engineering problems. Hafezalkotob and Hafezalkotob (2015a) utilized the MULTIMOORA approach with target-based attributes in a materials selection in a biomedical application.

Ceballos et al. (2016) compared rankings obtained by fuzzy MULTIMOORA, fuzzy TOPSIS (The Technique for Order of Preference by Similarity to Ideal Solution), fuzzy VIKOR (VIseKriterijumska Optimizacija I Kompromisno Resenje), and fuzzy WASPAS (Weighted Aggregated Sum-Product Assessment) to answer the question in every MCDM problem, that is “Which method should be used to solve it?”. Although some efforts have been made, the question is still open. Dai et al. (2016) proposed a new multi-attribute group decision-making method based on triangular fuzzy data structure with a MULTIMOORA approach to rank the best investment scenario from four alternatives. Zhao et al. (2016) suggested a novel approach toward Failure mode and effect analysis (FMEA) based on interval-valued intuitionistic fuzzy sets (IVIFSs) and a MULTIMOORA approach to handle the uncertainty and vagueness in a FMEA process and to achieve a more accurate ranking of failure modes identified by FMEA. Hafezalkotob and Hafezalkotob (2015b, 2016a) extended the MULTIMOORA approach based on Shannon’s entropy with crisp and fuzzy data in a material selection problem. Hafezalkotob et al. (2016) integrated the MULTIMOORA approach with interval numbers in an application for a material selection problem. Sahu et al. (2016) modified the MULTIMOORA approach considering generalized interval-valued trapezoidal fuzzy numbers ordered weighted geometric average in order to evaluate CNC machine tools. Tian et al. (2016) proposed several simplified neutrosophic linguistic distance measures by employing a distance-based method to determine criterion weights along with an improved MULTIMOORA approach based on a neutrosophic linguistic normalized weighted Bonferroni mean and simplified neutrosophic linguistic normalized geometric weighted Bonferroni mean operators as well as a simplified neutrosophic linguistic distance measure.

By developing a hesitant fuzzy linguistic term set (HFLTS) into novel concepts named double hierarchy linguistic term set (DHLTS) and double hierarchy hesitant fuzzy linguistic term set (DHHFLTS) and integrating these new concepts with the MULTIMOORA approach which results in the DHHFL-MULTIMOORA method, Gou et al. (2017) applied the proposed method to select the optimal city in China by evaluating the implementation status of haze controlling measures. Stanujkic and Zavadskas (2017) proposed a new extension to the MULTIMOORA approach by using single-valued neutrosophic sets which result in more efficiency in solving complex problems where solving requires assessment and prediction. Awasthi and Baležentis (2017) presented a hybrid approach based on benefits, costs, opportunities and risks (BOCR) and a fuzzy MULTIMOORA approach for the selection of a logistics service provider along with a Monte Carlo simulation based sensitivity analysis to determine the robustness of MULTIMOORA with variation in criterion and decision maker weights. Zavadskas et al. (2017) proposed a hybrid approach by combining the Step-Wise Weight Assessment Ratio Analysis (SWARA) technique with a single-valued neutrosophic set MULTIMOORA approach to create a decision support method for residential house construction materials selection. An approach that integrates fuzzy MULTIMOORA and multi-choice conic goal programming was presented by Deliktas and Ustun (2017) to consider criteria in choosing the best students and define the optimum assignments among predefined programs. Tian et al. (2018) suggested a hybrid QFD-based (Quality Function Deployment) fuzzy MCDM approach based on a fuzzy maximizing deviation method (MDM) and the BWM approach along with the MULTIMOORA approach in Changsha, China in order to increases understand about a smart Bike-Sharing Program (BSP).

Developments in performance appraisal methods

Performance appraisal (PA) is a term refering to “a basic process involving superior annual reports on subordinate’s performance of the organization center or managers” (Fletcher 2001). However, nowadays, there is a vast amount of research and studies analyzing different methods of PA, factors affecting the PA process, and PA methods. PA approaches are one of the evaluation processes for continuous improvement and one of the effective tools used in organizational performance management (DeNisi and Murphy 2017). There have been a few studies on PA approaches including the following. Levy and Williams (2004) conducted a systematic literature review of over 300 articles on PA and found that the focus of recent PA studies is changing from theoretical development and enhancements to practical applications. Caruth and Humphreys (2008) demonstrated the need for a more aligned and integrated framework for PA to enhance effective strategic control. Wei and Bi (2008) applied a performance evaluation based on knowledge management and evaluated the criteria by using the ANP (Analytic Network Process) method. Fan and Tang (2009) proposed a PA method based on fuzzy integrals and analyzed the performance of Industry-University-Research cooperative innovation centers in China. Jafari et al. (2009) proposed a framework based on SAW (Simple Additive Weighting) for the selection of the optimal PA method and compared some PA methods to facilitate the selection process for organizations.

Espinilla et al. (2010) developed a web-based evaluation system by using integral PA based on previous PA and web-based models. Suriyakumari and Kathiravan (2013) proposed a Domain Driven Data Mining (D3M) and opinion mining approach for performance evaluation to evaluate the performance of employees in virtual organizations. Espinilla et al. (2013) proposed a PA modeling based on a heterogeneous framework for a 360-degree feedback approach and implemented it in a multinational clothing company.

Ahmed et al. (2013) developed a PA criteria-based system by using fuzzy logic. There are a few more studies which are similar to the research conducted by Ahmed et al. (2013) (Chen 2015; Monsur and Akkas 2015; Ozkan et al. 2014). Later, Shaout and Yousif (2014) developed the same decision matrix used by Jafari et al. (2009) and proposed AHP (Analytic Hierarchy Process) and FTOPSIS method to select the optimal PA approach. On the other hand, Ishizaka and Pereira (2016) presented a PA method based on PROMTHEE (Preference Ranking Organization Method for Enrichment of Evaluations) and ANP by incorporating the visual techniques GAIA and stacked bar chart. Zhou et al. (2016) examined the roles and methods of PA in hospitals from a strategic management perspective. Ikramullah et al. (2016) developed a conceptual framework for analyzing the processes and procedures involved in PA systems to use a more efficient PA method. Komissarova and Zenin (2016) provided a comparative legal analysis of fundamentals of effective PAs, “concerning matters raised in determining universal core concepts and principles of PA for implementation of PA in various national jurisdictions.” DeNisi and Murphy (2017) examined 100 years of research on PA and performance management and presented a comprehensive overview.

Research gaps and contributions of the current study

To the best of the authors’ knowledge, there is not a single study that uses a MULTIMOORA approach for the selection of an optimal PA method. Therefore, this paper presents a new application for the MULTIMOORA method. Furthermore, the reason that the MULTIMOORA approach might be useful for PA method selection is, as mentioned in the literature review, the MULTIMOORA approach has demonstrated that it is an uncomplicated and fast algorithm that has resulted in optimal rankings in other sectors. In order to show that the proposed approach i.e., MULTIMOORA could also provide an optimal ranking in the selection of PA methods, this study utilizes the MULTIMOORA algorithm which has been compared to the TOPSIS method, in order to present the accuracy of the MULTIMOORA and Entropy MULTIMOORA.

As noted in Section “Developments in performance appraisal methods”, only three studies have investigated the selection of optimal PA methods using MCDM, i.e., selecting the best PA method by employing a novel framework based on weights and multiple linear regression (Jafari et al. 2009), choosing the best performance evaluation method by using Analytic Hierarchy Process (AHP) and fuzzy AHP, and TOPSIS and FTOPSIS methods (Shaout and Yousif 2014), presenting a new PA system based on MCDM methods (i.e., PROMETHEE and ANP) (Ishizaka and Pereira 2016). None of those studies considered a real-world case study for the selection of a PA method. Hence, this study is more realistic, and the assumptions given in this MCDM problem are much closer to what is happening in organizations facing a selection of the appropriate PA method in real-world situations.

The focus of this paper is to identify the best criteria based on the validity of the literature and the practicality of real-world applications for analyzing an optimal PA method. Then, the optimal technique found by applying an Entropy MULRIMOORA. Subsequently, the results of the selected PA method are demonstrated in a real-world case study. Therefore, this study is a novel application of the MULTIMOORA approach to a real-world PA problem.

Furthermore, a comparison of the proposed methods has been made using a correlation coefficient of the ranks. Consequently, a comprehensive analysis of an example which has been through the PA process has been demonstrated.

Research methodology

MULTIMOORA approach

The MULTIMOORA method consists of three parts, the ratio system, the reference point and the full multiplicative form which form the multi-objective optimization by ratio analysis (MOORA) method developed by Brauers and Zavadskas (2006). Later on, Brauers and Zavadskas (2010b) extended the method by adding the full multiplicative form to the MOORA procedure to achieve a more robust method. The first step in the MULTIMOORA method is forming the decision matrix X in which x ij presents the performance index of ith alternative respecting jth attribute i = 1, 2, …m and j = 1, 2, …n , and \( {w}_j^s \) denotes the subjective significance coefficients of jth attribute i = 1, 2, …m and j = 1, 2, …n:
$$ \boldsymbol{X}={\left[{x}_{ij}\right]}_{m\times n}, $$
(1)
$$ {\boldsymbol{w}}_{\boldsymbol{j}}^{\boldsymbol{s}}={\left[{w}_j\right]}_n,\kern0.5em \sum {w}_j=1. $$
(2)
In the MULTIMOORA approach, these parameters should be dimensionless in order to make performance indices comparable. Therefore, the decision matrix is a normalization ratio of comparison between each response of an alternative to a criterion as a numerator, and a denominator that is representative for all alternative performances on that attribute, as shown in Eq. (3):
$$ {X}_{ij}^{\ast }=\frac{x_{ij}}{\sqrt[2]{\sum_{i=1}^m{x}_{ij}^2}}, $$
(3)
where, \( {X}_{ij}^{\ast } \) denotes the normalized performance index of ith alternative respecting jth attribute i = 1, 2, …m and j = 1, 2, …n and x ij presents the performance index of ith alternative respecting jth attribute i = 1, 2, …m and j = 1, 2, …n.

The ratio system

The normalization equation i.e., Eq. (3) justifies the foundation of this approach as the ratio system. In a current approach for optimization, the normalized performance indices are added in case of maximization and subtracted in the event of minimization (Brauers and Zavadskas 2011):
$$ {y}_i^{\ast }={\sum}_{j=1}^g{w}_j^s{X}_{ij}^{\ast }-{\sum}_{j=g+1}^n{w}_j^s{X}_{ij}^{\ast }, $$
(4)
in which, g indicates the objectives to be maximized and (n − g) indicates the objectives being minimized, \( {y}_i^{\ast } \) denotes the total assessment of alternative j with respect to subjective significance coefficients of all attributes \( {w}_j^s \) which can be positive or negative based on the totals of the calculations. The optimal alternative based on the ratio system is an ordinal ranking of the \( {y}_i^{\ast } \) which has the highest assessment value:
$$ {A}_{RS}^{\ast }=\left\{{A}_i|{\mathit{\max}}_i\ {y}_i^{\ast}\right\}. $$
(5)

The reference point approach

The second part of the MULTIMOORA approach is based on the foundation of the ratio system shown in Eq. (3). A maximal objective reference point is also concluded in the method obtained by Eq. (6) (Brauers and Zavadskas 2006):
$$ {r}_j=\left\{\begin{array}{c}{\mathit{\max}}_i{X}_{ij}^{\ast}\kern1.5em in case of maximization\\ {}{\mathit{\max}}_i{X}_{ij}^{\ast}\kern1.75em in case of minimization\ \end{array},\right. $$
(6)
where r j denotes the ith co-ordinate of the maximal objective reference point vector.
Deviation of a performance index from the reference point r j can be shown as \( \left({r}_j-{X}_{ij}^{\ast}\right) \). Subsequently, the maximum value of the deviation for each alternative \( {z}_i^{\ast } \) respecting subjective significance coefficients of all criteria \( {w}_j^s \) can be calculated as Eq. (7):
$$ {z}_i^{\ast }={\mathit{\max}}_j\left|\left({w}_j^s{r}_j-{w}_j^s{X}_{ij}^{\ast}\right)\right|, $$
(7)
in the reference point approach, calculation of the optimal alternative is obtained by computing the minimum value of Eq. (7) demonstrated in Eq. (8):
$$ {A}_{RP}^{\ast }=\left\{{A}_i|{\mathit{\min}}_i\ {z}_i^{\ast}\right\}. $$
(8)

The full multiplicative form

The third part of the MULTIMOORA method developed by Brauers and Zavadskas is based on an idea from economic mathematics (Brauers and Zavadskas 2010a, 2010b). The formula of the full multiplicative form can be determined as demonstrated in Eq. (10) where g denotes the objectives to be maximized and (n − g) indicates as the objectives to be minimized. The numerator of Eq. (10) indicates the product of performance indices of ith alternative relating to beneficial attributes. The denominator of Eq. (10) represents the product of performance indices of ith alternative relating to non-beneficial attributes respecting subjective significance coefficients of each attribute \( {w}_j^s \).
$$ {U}_i^{\prime }=\frac{\prod_{j=1}^g{\left({x}_{ij}\right)}^{w_j^s}}{\prod_{j=g+1}^n{\left({x}_{ij}\right)}^{w_j^s}}, $$
(10)
by using a normalized decision matrix an equivalent equation form of \( {U}_i^{\prime } \) can be calculated:
$$ {U}_i^{\ast }=\frac{\prod_{j=1}^g{\left({X}_{ij}^{\ast}\right)}^{w_j^s}}{\prod_{j=g+1}^n{\left({X}_{ij}^{\ast}\right)}^{w_j^s}}, $$
(11)
to maintain harmony among all parts of the calculations in the MULTIMOORA approach. Equation (11) shows the normalized form of the full multiplicative form used. Similar to the ratio system computation of the optimal alternative, it is based on the searching for the maximum among all assessment values of \( {U}_i^{\ast } \):
$$ {A}_{MF}^{\ast }=\left\{{A}_i|{\mathit{\max}}_i{U}_i^{\ast}\right\}. $$
(12)

The dominance theory: The final ranking of the MULTIMOORA method

The dominance theory was proposed as a tool for ranking subordinate alternatives with the MULTIMOORA method (Brauers and Zavadskas 2011, 2012). After the calculation of the subordinate ranks, they can be integrated into a final ranking, which is the final MULTIMOORA rank based on the obtained dominance theory. In dominance theory, a summary of the classification of the three MULTIMOORA methods is made based on cardinal and ordinal scales in which rankings rules should be applied (i.e., dominated, transitivity and equability). The theory of dominance can be described as: “(1) the plurality rule assisted with a kind of lexicographic method, (2) the method of correlation of ranks.” demonstrated by Brauers and Zavadskas (2012). For a more detailed explanation of the dominance theory, readers can refer to the study of Brauers and Zavadskas (2012).

MULTIMOORA approach based on integrated Shannon’s entropy

Hafezalkotob and Hafezalkotob (2015b) proposed an extended MULTIMOORA method based on Shannon’s entropy significance coefficient for a material selection problem. As mentioned in Hafezalkotob and Hafezalkotob (2015b), significance coefficients can be obtained in the form of a subjective significance coefficient which comes directly from the completion of a decision matrix by decision makers. Many research studies have integrated the MCDM approaches with subjective weight calculation approaches such as Best-Worst Method (BWM), Shannon’s entropy, etc. Huang et al. (2017) applied a linguistic distribution assessment in order to represent FMEA team members’ risk evaluation information and employed an improved interactive and multiple criteria decision-making approach to determine the risk priority of failure modes. Zhao et al. (2017) suggested an integrated VIKOR approach considering intuitionistic fuzzy data along with both subjective and objective weights of criteria in a supplier selection problem. Liu et al. (2017) developed an integrated risk prioritization method to improve the performance of FMEA by using interval-valued intuitionistic fuzzy sets (IVIFSs) and the multi-attributive border approximation area comparison (MABAC) method. In the current research, two forms of significance coefficient of attributes are demonstrated: the subjective significance coefficient which is already demonstrated i.e., based on expert judgments, and the objective significance coefficient which is obtained through Shannon’s entropy.

Shannon’s entropy significance coefficient

The concept of entropy has been widely employed in numerous fields of research e.g., social sciences, economics, physical sciences, etc. based on a mathematical theory of communication proposed by Claude Shannon (1948). The proposed concept can be effectively employed in the process of decision making because in information theory it can be considered as a criterion for the degree of uncertainty represented by a discrete probability distribution, and it measures existent contrasts between sets of data and clarifies the average intrinsic information transferred to the decision maker (Hafezalkotob and Hafezalkotob 2016a). Normalization of x ij to determine p ij which is the total project outcome, obtained by Eq. (13):
$$ {p}_{ij}=\frac{x_{ij}}{\sum_{i=1}^m{x}_{ij}}, $$
(13)
Shannon entropy measure E j is calculated using the total project outcome p ij computed by Eq. (14):
$$ {E}_j=-k{\sum \limits}_{i=1}^m\left({p}_{ij}\ \mathit{\ln}\kern0.5em {p}_{ij}\right), in which\kern0.75em k=\frac{1}{\mathit{\ln}(m)}. $$
(14)
calculation of objective significance coefficients achieved by employing E j as demonstrated in Eq. (15):
$$ {w}_j^o=\frac{d_j}{\sum_{j=1}^n{d}_j}\kern0.62em \mathrm{in}\ \mathrm{which}\ \kern1em {d}_j=1-{E}_j. $$
(15)
Calculation of the integrated Shannon significance coefficients, if the expert assigns subjective significance coefficients \( {w}_j^s \) computed by using Eq. (16) which is a combination of subjective and objective significance coefficients:
$$ {w}_j^{\ast }=\frac{w_j^s{w}_j^o}{\sum_{j=1}^n{w}_j^s{w}_j^o}, $$
(16)
when \( {w}_j^o, \) i.e., objective significance coefficient is larger, the variation degree of ratings on the attribute is higher, which is a result of a smaller E j of an attribute. Adversely, larger E j denotes a lower degree of variation of the ratings, the less information over attribute j, and minor objective significance coefficient \( {w}_j^o \) (Hafezalkotob and Hafezalkotob 2016b).

The extended MULTIMOORA method based on Shannon’s entropy

To integrate the MULTIMOORA method with Shannon’s entropy significance coefficients \( {w}_j^o \), the subjective significance coefficient \( {w}_j^s \) should be replaced by \( {w}_j^o \). In current research, the MULTIMOORA method is calculated by using both Shannon significance coefficient \( {w}_j^o \) and considering the subjective significance coefficient with Shannon’s entropy \( {w}_j^{\ast } \). Considering the subjective significance coefficient with Shannon’s entropy \( {w}_j^{\ast } \) the calculations of the extended MULTIMOORA method is obtained in section “The extended ratio system” and section “The extended reference point approach and the extended full multiplicative form”.

The extended ratio system

By replacing the subjective significance coefficient \( {w}_j^s \) with the combination of subjective significance coefficient with Shannon’s entropy \( {w}_j^{\ast } \) the extended ratio system method is calculated by Eq. (17). Additionally, calculation of the optimal alternative obtained by Eq. (18):
$$ {y}_i^{ew}={\sum}_{j=1}^g{w}_j^{\ast }{X}_{ij}^{\ast }-{\sum}_{j=g+1}^n{w}_j^{\ast }{X}_{ij}^{\ast }, $$
(17)
$$ {A}_{RS}^{ew}=\left\{{A}_i|{\mathit{\max}}_i\ {y}_i^{ew}\right\}. $$
(18)

The extended reference point approach and the extended full multiplicative form

As with the ratio system by replacing the subjective significance coefficient with the combination of subjective significance coefficient with Shannon’s entropy \( {w}_j^{\ast } \), the extended reference point approach, the extended full multiplicative form and the optimal alternative rankings of proposed methods are respectively achieved by calculating Eqs. (19), (20), (21) and (22):
$$ {z}_i^{ew}={\mathit{\max}}_j\left|\left({w}_j^{\ast }{r}_j-{w}_j^{\ast }{X}_{ij}^{\ast}\right)\right|, $$
(19)
$$ {A}_{RP}^{ew}=\left\{{A}_i|{\mathit{\min}}_i\ {z}_i^{ew}\right\}, $$
(20)
$$ {U}_i^{ew}=\frac{\prod_{j=1}^g{\left({X}_{ij}^{\ast}\right)}^{w_j^{\ast }}}{\prod_{j=g+1}^n{\left({X}_{ij}^{\ast}\right)}^{w_j^{\ast }}}, $$
(21)
$$ {A}_{MF}^{ew}=\left\{{A}_i|{\mathit{\max}}_i{U}_i^{ew}\right\}. $$
(22)
The complete flow-diagram of the proposed approach i.e., the MULTIMOORA method based on integrated Shannon’s entropy towards selecting the optimal PA method is illustrated in Fig. 1.
Fig. 1
Fig. 1

Flowchart of selecting the optimal PA method based on the MULTIMOORA method integrated Shannon’s entropy weight

For the proposed MCDM methodology and based on Fig. 1, the first step is to gather the input data i.e., decision criteria and PA alternatives from experts and literature review to construct the decision matrix. Then the Shannon’s entropy significance coefficient is calculated in order to obtain the weights of the criteria. Ultimately, three steps of the MULTIMOORA approach including the ratio system, the reference point approach and the full multiplicative form are computed. Furthermore, to develop a better understanding of the proposed method a real-world case study is presented in Section “Findings and results”.

Findings and results

The current study is a practical and validation experiment. This type of research is practical because of its purpose. With regards to content and data collection, it is descriptive and quantitative. The type of review is a case study. Given that the success of performance appraisal (PA) methods in an organization is considered from the perspective of the human resources manager and high-level managers, the study population includes the specialists, experts, and officials of the implementation of the PA procedure in the proposed case-study. A set of criteria for selecting the optimal PA approach from the previous research and expert comments on the case study has been collected and classified in Table 1 in which the description of each criterion is available. Table 2 shows a short description of each PA method which has been selected to be included in the ranking procedure. These PA methods have been collected and classified from previous research and expert comments directly dealing with the case study’s PA method implementation.
Table 1

Criteria definition for selection of the optimal PA method

ID

Criteria

Functional requirement

Description

C1

Possibility of applying PA method with Organization

Maximum

Based on cultural, economic and technical specifications in every organization it is crucial to measure the opportunity to implement a PA method in the organization. The result of the possibility of method appliance varies in different organizations.

C2

Compatibility of PA method with current Organization state

Maximum

Based on the different specifications of an organization and the current state of being, different PA approaches may have a different compatibility level which is an important criterion for applying the proposed method.

C3

Compatibility with future changes (Reliability)

Maximum

Reliability of the PA method is imperative for organizations. In the modern-day dynamic environment of organizations, it is critical for a method to be upgradable and updatable for future changes in the organizations.

C4

Cost of PA method

Minimum

The cost of the proposed process is one of the most important criteria for applying any improvement procedure in organizations. Implying PA methods in organizations based on the procedure and type of the organizations the cost and price values may vary.

C5

Training needs of method

Minimum

Different PA approaches have different training requirements depending on the type and size of the organizations. Sometimes this training may have some difficulties compared to others.

C6

Proven method application (Validity)

Maximum

To propose and employ an improvement process in any organization, it is crucial for them to apply a valid and proven version of the proposed procedure. This matter is also true for PA methods, to implement a valid and proper PA method of which the validity of process is acceptable scientifically and proven to be positive based on experience.

C7

Employee satisfaction considerations

Maximum

One of the important issues in every organization is employee loyalty. This matter will improve by increasing the employee’s satisfaction factors. Application of any new evaluation and improvement process in organizations affects the human resources. Therefore, it is crucial to consider employee satisfaction factors in applying the assessment process.

C8

Degree of sophistication (Ease of use)

Minimum

Different methods of PA have different levels of sophistication and different levels of application. Availability of technical experts or need of an expert for applying the PA method is a matter to be considered by organizations in hiring their employees.

Table 2

Candidate alternatives selection of the optimal PA method (Shaout and Yousif 2014)

ID

PA method candidate

Description

A1

Ranking Method

A superior ranks his employees based on competencies from best to worst. Although, how the best and why the best competence is selected is based on the superior’s judgments, and it is not described in this method.

A2

Graphic Rating Scales

In this approach, a graphic rating scale is available in which a list of performance indicators and range of graphical grades illustrates the employees’ grade by the score that best defines the level of employees’ performance for each section.

A3

Narrative Essays Management

This method primarily attempts to concentrate on the behavior. Based on explanations that administrators and superiors write about an employee’s (e.g., based on existing capabilities and qualifications, previous performance, and suggestions by others) strengths and weaknesses, points for the evaluation are obtained.

A4

Management by Objectives (MBO)

This method is based on a systematic approach to management by objectives. Employee performance is graded by the achievement of the specified objectives which is described by the administration. MBO includes three main processes; object formulation, execution process, and performance feedback.

A5

Behaviorally Anchored Rating Scale Human (BARS)

This method is based on behavioral statements to explain various stages of performance for each factor in an employee’s performance. This method is a scale of behaviors which is anchored to numerical ratings.

A6

Human Resource Accounting (HRA)

HRA is a method in which performance is specified regarding cost and accounting measurements by which employees’ performances are rated by accounting for the contributions of employees.

A7

Assessment Centers

This method is based on the observation of behaviors conducted by trained observers. In this approach, an evaluation center which includes the observers is a place where managers come together as a workgroup to evaluate employees’ performance.

A8

360 Degree

This method is based on numbers of stakeholders (i.e., immediate supervisors, team members, customers, peers, and self) which provides people with information based on their actions towards others and vice versa.

A9

720 Degree

This method is a 360-degree feedback which has been practiced twice. 720-degree method concentrates on customer and investors’ knowledge of their work. In 720 degree, feedback is taken from external sources (i.e., stakeholders, family, suppliers, and communities).

In the current study, after identifying the PA methods and measurement criteria by employing expert comments from the case study i.e., experts in human resources and high level management of the suggested cross-industrial company in Iran, and the literature review, the PA methods are assessed by using the multiple criteria decision-making tools of the MULTIMOORA method and the Entropy MULTIMOORA. Additionally, a comparison of rankings has been made and demonstrated based on the proposed method in this study i.e., MULTIMOORA and Entropy MULTIMOORA and the TOPSIS approach and TOPSIS integrated Shannon’s entropy. Subsequently, the optimal PA method is applied to a cross-industrial company in Iran as a real-world case study. The MULTIMOORA and MULTIMOORA approach based on integrated Shannon’s entropy significant coefficients are utilized to select the optimal PA method for an organization. To reach a better understanding of the operative PA selection procedure in a real-world application, a case-study of a multi-national cross-industrial company with the major activities in construction and transportation infrastructure in Iran is presented.

The necessary data for this case-study was collected through interviews. The respondents are high-level managers i.e., three managers, two human resource managers and two manager assistants all directly dealing with the procedure of selecting the optimal PA method. The interview method was question and answer (Q&A) based on the linguistic terms of Table 3. The fundamental purpose of the Q&A was to complete the decision matrix shown in Table 4 based on the linguistic terms and the corresponding numbers of Table 3. This study is based on one case only i.e., that of a cross-industry company in Iran. As explained by Easton (2010), focusing on one case study leads to a better understanding of existing data and a robust exploration and reflection on that data by the researchers. Flyvbjerg (2006) clarified that to employ in-depth research on any topic, “one can study only one case, and the result can be generalized.” Accordingly, the case-study in this research was not chosen randomly. It targeted a specific organization to be able to obtain data that other organizations would not be able to offer. As a result, this practical case has been chosen to gain a thorough knowledge of the selection of the optimal PA method in cross-industry organizations in Iran.
Table 3

Linguistic terms and the corresponding numbers

Linguistic term

Alphabetical value of verbal comments

Numerical value of verbal comments

Very Poor

A

1

Poor

B

2

Moderate

C

3

Good

D

4

Very Good

E

5

Table 4

Decision matrix for ranking the optimal PA method

PA methods alternatives

Criteria

C1

(MAX)

C2

(MAX)

C3

(MAX)

C4

(MIN)

C5

(MIN)

C6

(MAX)

C7

(MAX)

C8

(MIN)

A1

2

3

2

2

2

1

1

1

A2

3

1

1

2

4

1

2

1

A3

4

2

2

3

2

2

3

2

A4

3

2

4

2

3

3

4

2

A5

2

3

2

2

2

2

3

3

A6

1

1

4

4

4

5

5

5

A7

2

5

5

5

2

4

4

4

A8

5

4

4

4

2

4

5

3

A9

4

5

4

4

2

5

4

4

As mentioned before, in the MULTIMOORA procedure, decision matrix numbers which have different dimensions and measurement units transform into dimensionless numbers. This process is so-called normalization which is applied based on Eq. (3) by comparing numbers to each other. The normalized decision matrix is shown in Table 5.
Table 5

Normalized decision matrix for ranking the optimal PA method

PA methods alternatives

Criteria

C1

C2

C3

C4

C5

C6

C7

C8

A1

0.2132

0.3094

0.1980

0.2020

0.2480

0.0995

0.0909

0.1084

A2

0.3198

0.1031

0.0990

0.2020

0.4961

0.0995

0.1818

0.1084

A3

0.4264

0.2062

0.1980

0.3030

0.2480

0.1990

0.2727

0.2169

A4

0.3198

0.2062

0.3960

0.2020

0.3721

0.2985

0.3636

0.2169

A5

0.2132

0.3094

0.1980

0.2020

0.2480

0.1990

0.2727

0.3253

A6

0.1066

0.1031

0.3960

0.4040

0.4961

0.4975

0.4545

0.5423

A7

0.2132

0.5157

0.4950

0.5050

0.2480

0.3980

0.3636

0.4338

A8

0.5330

0.4125

0.3960

0.4040

0.2480

0.3980

0.4545

0.3253

A9

0.4264

0.5157

0.3960

0.4040

0.2480

0.4975

0.3636

0.4338

Subjective significance coefficient \( {w}_j^s \) is imported from expert comments in the same procedure where the decision matrix is completed. Shannon entropy measure E j is calculated by using Equation (14) and the calculation of objective significance coefficient \( {w}_j^o \) is achieved by employing E j based on the Equation (15). Since both subjective and objective significance coefficients are available in this study, the combined (integrated) significance coefficient \( {w}_j^{\ast } \) is calculated by Equation (16) shown in previous sections. Table 6 shows the significant coefficients measurements which are computed from the decision matrix.
Table 6

Significant coefficients measures and weighting factors

Significant coefficients

Criteria

C1

C2

C3

C4

C5

C6

C7

C8

\( {w}_j^s \)

0.1923

0.0769

0.1538

0.1153

0.0769

0.1923

0.1153

0.0769

E j

1.310

1.283

1.307

1.326

1.334

1.282

1.317

1.291

\( {w}_j^o \)

0.1263

0.1154

0.1252

0.1330

0.1364

0.1151

0.1294

0.1188

\( {w}_j^{\ast } \)

0.1951

0.0713

0.1547

0.1232

0.0842

0.1778

0.1199

0.0734

The values of the MULTIMOORA is obtained and calculated by employing Equation (4), (7) and (11), respectively. Then, each stage of the MULTIMOORA procedure is calculated by each ranking method. Consequently, the optimal assessment for weighted MULTIMOORA (subjective weights) is calculated by employing dominance theory for the final rank Table 7.
Table 7

Assessment values and rankings of the weighted (subjective) MULTIMOORA for selecting optimal PA method

PA methods alternatives

Assessment values

Rankings

\( {y}_i^{\ast } \)

\( {z}_i^{\ast } \)

\( {U}_i^{\ast } \)

\( {y}_i^{\ast } \)

\( {z}_i^{\ast } \)

\( {U}_i^{\ast } \)

Final rank

A1

0.07415

0.0615

0.4088

8

6

8

8

A2

0.0549

0.0609

0.3749

9

4

9

9

A3

0.1273

0.0456

0.5313

5

2

5

5

A4

0.1690

0.04100

0.6349

3

1

3

3

A5

0.0975

0.06150

0.4873

7

5

6

6

A6

0.1109

0.0820

0.4642

6

9

7

7

A7

0.1646

0.0615

0.6065

4

7

4

4

A8

0.2334

0.0574

0.7391

1

3

1

1

A9

0.2211

0.0765

0.7172

2

8

2

2

Based on entropy measurements described in Table 6 and previous calculations for the MULTIMOORA approach respecting subjective and objective significance coefficients combined, the assessment values of Entropy MULTIMOORA is calculated by employing Eqs. (17), (19) and (21) and shown in Table 8. Additionally, the ranking of each procedure of MULTIMOORA is obtained by the assessment values and the final rank of the method respecting dominance theory.
Table 8

Assessment values and rankings of Entropy MULTIMOORA for selecting optimal PA method

PA methods alternatives

Assessment values

Rankings

\( {y}_i^{ew} \)

\( {z}_i^{ew} \)

\( {U}_i^{ew} \)

\( {y}_i^{ew} \)

\( {z}_i^{ew} \)

\( {U}_i^{ew} \)

Final rank

A1

0.0691

0.0624

0.4247

8

6

8

8

A2

0.0499

0.0612

0.3914

9

3

9

9

A3

0.1224

0.0459

0.5512

5

2

5

5

A4

0.1629

0.0416

0.6556

3

1

3

3

A5

0.0927

0.0624

0.5057

7

5

6

6

A6

0.1010

0.0832

0.4745

6

8

7

7

A7

0.1543

0.0624

0.6188

4

4

4

4

A8

0.2254

0.0530

0.7587

1

3

1

1

A9

0.2108

0.0707

0.7320

2

7

2

2

Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) is a technique of MCDM which has been proposed and developed by (Hwang and Yoon 1981; Yoon and Hwang 1995). In TOPSIS Euclidean distances after the Minkowski metric would lead to ∞ solutions. Therefore TOPSIS introduces Significance Coefficients, called wrongly weights (Hwang and Yoon 1981). In the current study, a comparison of the proposed method i.e., MULTIMOORA and Entropy MULTIMOORA with TOPSIS and Entropy TOPSIS is demonstrated in Table 9.
Table 9

Comparison between the optimal PA method ranks of the proposed MULTIMOORA approach with TOPSIS method

PA methods alternatives

MCDM methods

MULTIMOORA

MULTIMOORA with Shannon’s entropy

TOPSIS

TOPSIS with Shannon’s entropy

A1

8

8

8

9

A2

9

9

9

8

A3

5

5

1

2

A4

3

3

4

4

A5

6

6

6

6

A6

7

7

7

7

A7

4

4

5

5

A8

1

1

2

1

A9

2

2

3

3

The Spearman rank correlation coefficient helps with evaluating the similarity of the rankings. A coefficient is a real number in the range of −1 and 1. The Spearman coefficient equal to one denotes identical rankings and −1 indicates opposite rankings. Spearman was a psychologist who in 1904–1910 wrongly used the traditional operations of mathematics for ordinal numbers. It was the statistician Kendall who formulated the rank correlation method: “we shall often operate with these numbers as if they were the cardinals of ordinary arithmetic, adding them, subtracting them and even multiplying them,” but he never gave a proof of this statement (Kendall 1938). Figure 2, illustrates the correlation between ranking lists by utilizing the Spearman rank correlation coefficient obtained by employing Equation (23), the Spearman rank correlation coefficient similarity of the rankings in Table 9.
$$ {r}_s=1-\frac{\sum \limits_{i=1}^n{D}^2}{n\left({n}^2-1\right)}, $$
(23)
where D is differences between the two ranks and n denotes the sample size.
Fig. 2
Fig. 2

Correlation between the rankings based on the Spearman coefficient

Figure 2 shows that the proposed methods in this study have high correlation values compared to each other. Based on Table 9, alternative number 9 i.e., 360 Degree has been selected as the optimal PA method based on MULTIMOORA, Entropy MULTIMOORA, and Entropy TOPSIS.

360-degree feedback example: a managerial vision

360-degree feedback which is also known as multi-rater feedback, multi-source feedback or multi-source assessment refers to the process by which PAs are collected from different individual sources i.e., supervisors, peers, subordinates, and customers instead of relying on an appraisal from a single source which would provide less information for the feedback. (Ghorpade 2000; van der Heijden and Nijhof 2004). Today, studies suggest that most organizations use some type of multi-source feedback because of its significant advantages over the traditional PA methods (Espinilla et al. 2013). As mentioned, this study is a case-based research employing the optimal PA method, 360-degree feedback, selected by the proposed MCDM approach i.e., Entropy MULTIMOORA & MULTIMOORA. In order to present a better understanding of the 360-degree approach, the application of the 360-degree approach to a real-world case study is presented.

The necessary research data has been collected using a questionnaire. In this case-study, the questionnaire was created by the organization utilizing the value engineering concepts based on the 360-degree feedback approach. It consists of ten general evaluation factors with 72 questions which were obtained by using the PA approach from the group of experts in the human resource department. Additionally, the rating for the factors in the questionnaire is based on a 0–20 scale. Table 10 shows the statistical population and characteristics of the respondents regarding their occupation in the organization.
Table 10

Characteristics of the questionnaire respondents

Number of employees

Organizational level

75

Specialist

50

Engineering Assistant

50

Supervisor

40

Middle-level manager

60

High-level manager

Total = 275

 
Generally, the organizational PA procedure is applied every 6 months in the case-study organization, in this research, the central office of the multi-national cross-industrial organization selected for statistical analysis. The questionnaire has been distributed to 475 employees and 275 complete and accurate questionnaires have been received. The sample size was selected by using the Krejcie and Morgan Table. To control the quality of the questionnaire results, the proper objective features should be examined. Among these features, validity and reliability are more important. The purpose of the validity feature of the questionnaire is to what extent it can accurately measure the variables that are designed for it. In the collected statistical sample from the questionnaires, at least 12 professionals and experts from the human resource department have confirmed the validity of the prepared questionnaire for this research. The purpose of the reliability feature of the questionnaire is to what extent it can measure the same results by using the questionnaire in other different spatial and temporal conditions. Cronbach’s alpha is one of the most common methods for determining the reliability of the questionnaire. When the Cronbach’s alpha value gets closer to 100%, the reliability of the collection tools is increased. The Cronbach’s alpha reliability coefficient of the questionnaire has reached 73% by using SPSS software which indicates the acceptability of the reliability test of the questionnaire. Additionally, to achieve more accurate reliability, the Composite Reliability coefficient (CR) has also been considered, and the rate of 0.70 has been obtained by using Smart PLS software which demonstrates the acceptability of the CR test of the questionnaire. Subsequently, when the PA results for each employee is obtained, the senior management will decide what to do based on the three following stated scenarios:
  1. I.

    Employees that obtained less than 70% of the average; for these employees, the PA period will change to 3 months, and if the same results are shown in the next PA, the employees will end up in the penalty process.

     
  2. II.

    Employees that obtained more than 50% of the average; these employees will receive encouragement and financial and non-financial rewards.

     
  3. III.

    Employees that obtained more than 70% of the average; treated the same as II and these employees will be asked to be evaluators for the next PA.

     
Figure 3 shows an example of an employee performance evaluation from the case-study based on ten factors. The employee Mr. X.Y is on the supervision organizational level, and it is the third time this employee has been involved in PA procedure.
Fig. 3
Fig. 3

An example of case-study PA report

Conclusion

In today’s dynamic and competitive environment for organizations, one of the most important issues to discuss is the continuous improvement of the organization itself. One of the main tools to maintain improvement is the periodic evaluation. Therefore, selecting the best PA method is substantial. In large-sized enterprises, selecting the optimal PA method is a challenging task which may require research and special expertise. Selecting the optimal PA method considering the specifications of an organization based on details considering that the possibility of being wrong could impose inappropriate costs on organizations. This is the reason why selecting the optimal approach based on the MCDM method is a good idea, comprehensively described in the current paper. In the present paper, criteria and PA approaches were first identified, and a comprehensive description of each criterion and alternative was provided. Second, the MULTIMOORA integrated Shannon’s entropy was utilized to provide a selection of optimal PA methods applied to a case-study, a multi-national cross-industry company in Iran. Third, the correlations between the rankings of the MULTIMOORA approach and the TOPSIS method were examined by applying correlation coefficients of ranks. Finally, the optimal PA method i.e., 360-degree feedback was selected and employed in the case study and the results of the PA were specified and demonstrated. Ultimately, in this study, a new application of the MULTIMOORA approach has been presented.

Suggestions for future developments of this study may be as follows. First, input data of the MCDM approach can be extended to cases in which the data of the problem has different mathematical forms such as extensions of fuzzy sets, e.g., flou sets, fuzzy multi-sets, bipolar fuzzy sets, and interval data structure. Second, applying the current decision matrix including the same criteria and alternatives to different organizations may have different results which could be compared to the current research. Third, significance coefficients of attributes may be achieved using various techniques. In the present study, subjective significance coefficients were considered, and objective significance coefficients were determined based on Shannon entropy. Subjective significance coefficients may be computed by applying various methods such as ANP, AHP, and BWM.

Abbreviations

AHP: 

Analytic hierarchy process

CR: 

Composite reliability coefficient

DHHFLTS: 

Double Hierarchy Hesitant Fuzzy Linguistic Term Set

DHLTS: 

Double Hierarchy Linguistic Term Set

FMEA: 

Failure mode and effect analysis

HCW: 

Health-care waste

HFLTS: 

Hesitant Fuzzy Linguistic Term Set

IVIFSs: 

Interval-valued Intuitionistic Fuzzy Sets

MADM: 

Multi-attribute decision making

MCDM: 

Multiple criteria decision making

MOORA: 

The multi-objective optimization on the basis of ratio analysis

PA: 

Performance appraisal

TOPSIS: 

Technique for order preference by similarity to ideal solution

Declarations

Acknowledgements

There is no need for acknowledgement.

Funding

Funding Information is not available and it is not applicable.

Availability of data and materials

Available upon request.

Authors’ contributions

Abteen Ijadi maghsoodi is the corresponding author and the main researcher of the proposed study. In which 85% of the job have been written and calculated by him with collaboration of her assistant Mrs. Gelayol Abouhamzeh. The proposed manuscript is a research based on supervision of Dr. Mohamad Khalilzadeh and Professor Edmundas Kazimieras Zavadskas as consultants. All authors read and approved the final manuscript.

Ethics approval and consent to participate

This article does not contain any studies with human participants performed by any of the authors.

Competing interest

The authors declare that there are no conflicts of interest in this paper.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Department of Industrial Engineering, Science and Research Branch, Islamic Azad University, Daneshgah Blvd, Simon Bulivar Blvd, Tehran, 14515/775, Iran
(2)
Vilnius Gediminas Technical University, Saulėtekio al. 11, 10223 Vilnius, Lithuania

References

  1. Ahmed, I., Sultana, I., Paul, S. K., & Azeem, A. (2013). Employee performance evaluation: A fuzzy approach. International Journal of Productivity and Performance Management, 62(7), 718–734 https://doi.org/10.1108/IJPPM-01-2013-0013.View ArticleGoogle Scholar
  2. Awasthi, A., & Baležentis, T. (2017). A hybrid approach based on BOCR and fuzzy MULTIMOORA for logistics service provider selection. International Journal of Logistics Systems and Management, 27(3), 261–282 https://doi.org/10.1504/IJLSM.2017.10005115.View ArticleGoogle Scholar
  3. Baležentis, T., & Baležentis, A. (2014). A survey on development and applications of the multi-criteria decision making method MULTIMOORA. Journal of Multi-Criteria Decision Analysis, 21(3–4), 209–222 https://doi.org/10.1002/mcda.1501.View ArticleGoogle Scholar
  4. Brauers, W. K. M., & Ginevičius, R. (2010). The economy of the Belgian regions tested with MULTIMOORA. Journal of Business Economics and Management, 11(2), 173–209 https://doi.org/10.3846/jbem.2010.109.View ArticleGoogle Scholar
  5. Brauers, W. K. M., & Zavadskas, E. K. (2006). The MOORA method and its application to privatization in a transition economy by a new method: The MOORA method. Control and Cybernetics, 35(2), 445–469.Google Scholar
  6. Brauers, W. K. M., & Zavadskas, E. K. (2010a). A multi-objective decision support system for project selection with an application for the tunisian textile industry. Ekonomika a Management.Google Scholar
  7. Brauers, W. K. M., & Zavadskas, E. K. (2010b). Project management by multimoora as an instrument for transition economies. Technological and Economic Development of Economy, 16(1), 5–24 https://doi.org/10.3846/tede.2010.01.View ArticleGoogle Scholar
  8. Brauers, W. K. M., & Zavadskas, E. K. (2011). Multimoora optimization used to decide on a Bank loan to buy property. Technological and Economic Development of Economy, 17(1), 174–188 https://doi.org/10.3846/13928619.2011.560632.View ArticleGoogle Scholar
  9. Brauers, W. K. M., & Zavadskas, E. K. (2012). Robustness of MULTIMOORA: A method for multi-objective optimization. Informatica, 23(1), 1–25.Google Scholar
  10. Carlucci, D. (2010). Evaluating and selecting key performance indicators: An ANP-based model. Measuring Business Excellence, 14(2), 66–76 https://doi.org/10.1108/13683041011047876.View ArticleGoogle Scholar
  11. Caruth, D. L., & Humphreys, J. H. (2008). Performance appraisal: Essential characteristics for strategic control. Measuring Business Excellence, 12(3), 24–32 https://doi.org/10.1108/13683040810900377.View ArticleGoogle Scholar
  12. Ceballos, B., Lamata, M. T., & Pelta, D. A. (2016). Fuzzy multicriteria decision-making methods: A comparative analysis. International Journal of Intelligent Systems, 32, 722–738 https://doi.org/10.1002/int.21873.View ArticleGoogle Scholar
  13. Chen, B. R. (2015). Subjective performance feedback, ability attribution, and renegotiation-proof contracts. Journal of Economic Behavior and Organization, 117, 155–174 https://doi.org/10.1016/j.jebo.2015.06.010.View ArticleGoogle Scholar
  14. Dai, W. F., Zhong, Q. Y., & Qi, C. Z. (2016). Multistage Multiattribute Group Decision-Making Method Based on Triangular Fuzzy MULTIMOORA. Mathematical Problems in Engineering, 2016. https://doi.org/10.1155/2016/1687068.
  15. Deliktas, D., & Ustun, O. (2017). Student selection and assignment methodology based on fuzzy MULTIMOORA and multichoice goal programming. International Transactions in Operational Research, 24(5), 1173–1195 https://doi.org/10.1111/itor.12185.View ArticleGoogle Scholar
  16. DeNisi, A. S., & Murphy, K. R. (2017). Performance appraisal and performance management: 100 years of progress? Journal of Applied Psychology, 102(3), 421–433 https://doi.org/10.1037/apl0000085.View ArticleGoogle Scholar
  17. Dobbs, M. E. (2014). Guidelines for applying Porter’s five forces framework: A set of industry analysis templates. Competitiveness Review, 24(1), 32–45 https://doi.org/10.1108/CR-06-2013-0059.View ArticleGoogle Scholar
  18. Easton, G. (2010). One case study is enough. In Academy of marketing annual conference (pp. 1–17).Google Scholar
  19. Espinilla, M., De Andrés, R., Martínez, F. J., & Martínez, L. (2013). A 360-degree performance appraisal model dealing with heterogeneous information and dependent criteria. Information Sciences, 222, 459–471 https://doi.org/10.1016/j.ins.2012.08.015.View ArticleGoogle Scholar
  20. Espinilla, M., Jes, F., & Mart, L. (2010). A Web based Evaluation Support System by Integral Performance Appraisal. In Intelligent Systems and Knowledge Engineering (ISKE), 2010 International Conference on (pp. 646-651). IEEE.Google Scholar
  21. Fan, D. C., & Tang, X. X. (2009). Performance evaluation of industry-university-research cooperative technological innovation based on fuzzy integral. In Management Science and Engineering, 2009. ICMSE 2009. International Conference on (pp. 1789–1795). IEEE.Google Scholar
  22. Fletcher, C. (2001). Performance appraisal and management: The developing research agenda. Journal of Occupational and Organizational Psychology, 74, 473–487 https://doi.org/10.1348/096317901167488.View ArticleGoogle Scholar
  23. Flyvbjerg, B. (2006). Five misunderstandings about case-study research. Qualitative Inquiry, 12(2), 219–245 https://doi.org/10.1177/1077800405284363.View ArticleGoogle Scholar
  24. Ghorpade, J. (2000). Managing five paradoxes of 360-degree feedback. Academy of Management Executive, 14(1), 140–150. https://doi.org/10.5465/AME.2000.2909846
  25. Gou, X., Liao, H., Xu, Z., & Herrera, F. (2017). Double hierarchy hesitant fuzzy linguistic term set and MULTIMOORA method: A case of study to evaluate the implementation status of haze controlling measures. Information Fusion, 38, 22–34 https://doi.org/10.1016/j.inffus.2017.02.008.View ArticleGoogle Scholar
  26. Hafezalkotob, A., & Hafezalkotob, A. (2015a). Comprehensive MULTIMOORA method with target-based attributes and integrated significant coefficients for materials selection in biomedical applications. Materials and Design, 87, 949–959 https://doi.org/10.1016/j.matdes.2015.08.087.View ArticleGoogle Scholar
  27. Hafezalkotob, A., & Hafezalkotob, A. (2015b). Extended MULTIMOORA method based on Shannon entropy weight for materials selection. Journal of Industrial Engineering International, 12(1), 1–13 https://doi.org/10.1007/s40092-015-0123-9.View ArticleGoogle Scholar
  28. Hafezalkotob, A., & Hafezalkotob, A. (2016a). Fuzzy entropy-weighted MULTIMOORA method for materials selection. Journal of Intelligent and Fuzzy Systems, 31(3), 1211–1226 https://doi.org/10.3233/IFS-162186.View ArticleGoogle Scholar
  29. Hafezalkotob, A., & Hafezalkotob, A. (2016b). Risk-based material selection process supported on information theory: A case study on industrial gas turbine. Applied Soft Computing https://doi.org/10.1016/j.asoc.2016.09.018.
  30. Hafezalkotob, A., Hafezalkotob, A., & Sayadi, M. K. (2016). Extension of MULTIMOORA method with interval numbers: An application in materials selection. Applied Mathematical Modelling, 40(2), 1372–1386 https://doi.org/10.1016/j.apm.2015.07.019.View ArticleGoogle Scholar
  31. Huang, J., Li, Z, (Steven) & Liu, H.-C. (2017). New approach for failure mode and effect analysis using linguistic distribution assessments and TODIM method. Reliability Engineering & System Safety, 167(June), 302–309. https://doi.org/10.1016/j.ress.2017.06.014.
  32. Hwang, C. L., & Yoon, K. (1981). Multiple attribute decision making: Methods and applications. New York: Springer-Verlag.View ArticleGoogle Scholar
  33. Ikramullah, M., Van Prooijen, J.-W., Iqbal, M. Z., & Ul-Hassan, F. S. (2016). Effectiveness of performance appraisal. Personnel Review, 45(2), 334–352 https://doi.org/10.1108/PR-07-2014-0164.View ArticleGoogle Scholar
  34. Ishizaka, A., & Pereira, V. E. (2016). Portraying an employee performance management system based on multi-criteria decision analysis and visual techniques. International Journal of Manpower, 37(4). https://doi.org/10.1108/MRR-09-2015-0216.
  35. Jafari, M., Bourouni, A., & Amiri, R. H. (2009). A new framework for selection of the best performance appraisal method. European Journal of Social Sciences, 7(3), 92–100.Google Scholar
  36. Kendall, M. G. (1938). A new measure of rank correlation. Biometrika, 30(1/2), 81 https://doi.org/10.2307/2332226.View ArticleGoogle Scholar
  37. Komissarova, N. V., & Zenin, S. S. (2016). Legal fundamentals of effective performance appraisals. International Journal of Applied Business and Economic Research, 14(9), 5767–5778.Google Scholar
  38. Levy, P. E., & Williams, J. R. (2004). The social context of performance appraisal: A review and framework for the future. Journal of Management, 30(6), 881–905 https://doi.org/10.1016/j.jm.2004.06.005.View ArticleGoogle Scholar
  39. Liu, H. C., Fan, X. J., Li, P., & Chen, Y. Z. (2014). Evaluating the risk of failure modes with extended MULTIMOORA method under fuzzy environment. Engineering Applications of Artificial Intelligence, 34, 168–177 https://doi.org/10.1016/j.engappai.2014.04.011.View ArticleGoogle Scholar
  40. Liu, H.-C., You, J.-X., & Duan, C.-Y. (2017). An integrated approach for failure mode and effect analysis under interval-valued intuitionistic fuzzy environment. International Journal of Production Economics. https://doi.org/10.1016/j.ijpe.2017.03.008.
  41. Liu, H.-C., You, J.-X., Lu, C., & Chen, Y.-Z. (2015). Evaluating health-care waste treatment technologies using a hybrid multi-criteria decision making model. Renewable and Sustainable Energy Reviews, 41, 932–942 https://doi.org/10.1016/j.rser.2014.08.061.View ArticleGoogle Scholar
  42. Liu, H. C., You, J. X., Lu, C., & Shan, M. M. (2014). Application of interval 2-tuple linguistic MULTIMOORA method for health-care waste treatment technology evaluation and selection. Waste Management, 34(11), 2355–2364 https://doi.org/10.1016/j.wasman.2014.07.016.View ArticleGoogle Scholar
  43. Macwan, N., & Sajja, P. S. (2013). Modeling performance appraisal using soft computing techniques: Designing neuro-fuzzy application. In 2013 international conference on intelligent systems and signal processing (ISSP) (pp. 403–407) https://doi.org/10.1109/ISSP.2013.6526943.View ArticleGoogle Scholar
  44. Monsur, M. P., & Akkas, M. A. (2015). Performance appraisal system of the academics of public and private universities of Bangladesh: An Emperical study. International Business Management, 9(5), 712-718.Google Scholar
  45. Ozkan, C., Keskin, G. A., & Omurca, S. I. (2014). A variant perspective to performance appraisal system: Fuzzy-C-means algorithm. International Journal of Industrial Engineering, 21(3), 168–178.Google Scholar
  46. Pereira, A. I. V. E. (2016). Portraying an employee performance management system based on multi-criteria decision analysis and visual techniques. International Journal of Manpower, 37(4), 628–659 https://doi.org/10.1108/02656710210415703.View ArticleGoogle Scholar
  47. Prowse, P., & Prowse, J. (2010). The dilemma of performance appraisal. Business Performance Measurement and Management: New Contexts, Themes and Challenges, 195–206. https://doi.org/10.1007/978-3-642-04800-5_13.
  48. Sahu, A. K., Datta, S., & Mahapatra, S. S. (2016). Evaluation and selection of resilient suppliers in fuzzy environment. Benchmarking: An International Journal, 23(3), 651–673 https://doi.org/10.1108/BIJ-11-2014-0109.View ArticleGoogle Scholar
  49. Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(July 1928), 379–423 https://doi.org/10.1145/584091.584093.View ArticleGoogle Scholar
  50. Shaout, A., & Yousif, M. K. (2014). Performance evaluation – Methods and techniques survey. International Journal of Computer and Information Technology, 3(5), 966–979.Google Scholar
  51. Stankevičienė, J., Sviderskė, T., & Miečinskienė, A. (2014). Dependence of sustainability on country risk indicators in EU Baltic sea region countries. Journal of Business Economics and Management, 15(4), 646–663 https://doi.org/10.3846/16111699.2014.965555.View ArticleGoogle Scholar
  52. Stanujkic, D., & Zavadskas, E. K. (2017). A Neutrosophic extension of the MULTIMOORA method. Informatica, 28(1), 181–192. https://doi.org/10.15388/Informatica.2017.125
  53. Suriyakumari, V., & Kathiravan, A. V. (2013). An ubiquitous domain Driven Data Mining approach for performance monitoring in virtual organizations using 360 Degree data mining & opinion mining. Proceedings of the 2013 International Conference on Pattern Recognition, Informatics and Mobile Engineering, PRIME 2013, 307–311. https://doi.org/10.1109/ICPRIME.2013.6496491.
  54. Tian, Z., Wang, J., Wang, J., & Zhang, H. (2016). An improved MULTIMOORA approach for multi-criteria decision-making based on interdependent inputs of simplified neutrosophic linguistic information. Neural Computing and Applications, 1. https://doi.org/10.1007/s00521-016-2378-5.
  55. Tian, Z., Wang, J., Wang, J., & Zhang, H. (2018). A multi-phase QFD-based hybrid fuzzy MCDM approach for performance evaluation: A case of smart bike-sharing programs in Changsha. Journal of Cleaner Production, 171(Supplement C), 1068–1083 https://doi.org/10.1016/j.jclepro.2017.10.098.View ArticleGoogle Scholar
  56. van der Heijden, B. I. J. M., & Nijhof, A. H. J. (2004). The value of subjectivity: Problems and prospects for 360-degree appraisal systems. The International Journal of Human Resource Management, 15(3), 493–511 https://doi.org/10.1080/0958519042000181223.View ArticleGoogle Scholar
  57. Wei, J. Y., & Bi, R. (2008). Knowledge management performance evaluation based on ANP. Proceedings of the 7th International Conference on Machine Learning and Cybernetics, ICMLC, 1(July), 257–261 https://doi.org/10.1109/ICMLC.2008.4620414.Google Scholar
  58. Yoon, K., & Hwang, C.-L. (1995). Multiple attribute decision making: An introduction. Thousand Oaks CA: Sage Publications.View ArticleGoogle Scholar
  59. Zavadskas, E. K., Antucheviciene, J., Razavi Hajiagha, S. H., & Hashemi, S. S. (2015). The interval-valued intuitionistic fuzzy MULTIMOORA method for group decision making in engineering. Mathematical Problems in Engineering, 2015(2). https://doi.org/10.1155/2015/560690.
  60. Zavadskas, E. K., Bausys, R., Juodagalviene, B., & Garnyte-Sapranaviciene, I. (2017). Model for residential house element and material selection by neutrosophic MULTIMOORA method. Engineering Applications of Artificial Intelligence, 64, 315–324 https://doi.org/10.1016/j.engappai.2017.06.020.View ArticleGoogle Scholar
  61. Zhao, H., You, J. X., & Liu, H. C. (2016). Failure mode and effect analysis using MULTIMOORA method with continuous weighted entropy under interval-valued intuitionistic fuzzy environment. Soft Computing, 1–13. https://doi.org/10.1007/s00500-016-2118-x.
  62. Zhao, J., You, X. Y., Liu, H. C., & Wu, S. M. (2017). An extended VIKOR method using intuitionistic fuzzy sets and combination weights for supplier selection. Symmetry, 9(9). https://doi.org/10.3390/sym9090169.
  63. Zhou, Y., Yuan, H., Li, Y., Zhao, X., & Yi, L. (2016). Roles and methods of performance evaluation of hospital academic leadership. JBUON, 21(2), 261–265.Google Scholar

Copyright

© The Author(s). 2018

Advertisement