Modelling uncertainty in performance appraisal process based on evidence theory and fuzzy theory

Document Type : Industry Article

Authors

1 industrial engineering, payam noor university, tehran, iran

2 industrial engineering, tehran unicersity, tehran, iran

Abstract

Uncertainty involved in human judgments is an important cause of error and loss of credibility in outputs provided by performance appraisal (PA) processes. In few existing studies related to the errors and uncertainties in PA process, solutions mainly based on the use of fuzzy tools have been presented in this regard. These solutions have fundamental deficiencies such as inability to cope with epistemic uncertainty and also problems associated with their implementation. Considering these problems, in this paper, a new model based on evidence theory and fuzzy tools has been proposed to model uncertainty in PA process. The proposed model makes it possible for assessors to provide their ratings commensurate with their level of knowledge and also has the ability to deal with uncertainty caused by randomness and ignorance. In the proposed model which has been designed based on multi-source assessment framework, the uncertainty contained in the data obtained from two common evaluation scales, including Visual Analogue Scale and fuzzy linguistic scale, along with data related to the reliability of evaluator, have been modeled in evidence theory structure. These data then have been aggregated with evidence combination rules. The performance, benefits and improvements resulting from the proposed model, compared with other common aggregation methods in P.A models, have been investigated using simulated data and a numerical example. The results show that the proposed model in addition to improving the ability of dealing with uncertainty in P.A processes and facilitating announcing opinion by raters, provides more accurate results than traditional P.A models.

Keywords

Main Subjects

References

 
[1]          C. E. Lance, B. J. Hoffman, W. A. Gentry, and L. E. Baranik, “Rater source factors represent important subcomponents of the criterion construct space, not rater bias,” Hum. Resour. Manag. Rev., vol. 18, no. 4, pp. 223–232, Dec. 2008.
[2]          S. A. Snell and G. W. Bohlander, Managing Human Resources. Cengage Learning, 2012.
[3]          S. E. Scullen, M. K. Mount, and M. Goff, “Understanding the latent structure of job performance ratings,” J. Appl. Psychol., vol. 85, no. 6, p. 956, 2000.
[4]          A. Nabaum, L. Barry, S. Garr, and A. Liakopoulos, “Performance management is broken: Replace ‘rank and yank’ with coaching and development,” Dupress, pp. 45–52, 2014.
[5]          B. J. Hoffman, C. A. Gorman, C. A. Blair, J. P. Meriac, B. Overstreet, and E. K. Atchley, “Evidence for the Effectiveness of an Alternative Multisource Performance Rating Methodology,” Pers. Psychol., vol. 65, no. 3, pp. 531–563, 2012.
[6]          R. Golman and S. Bhatia, “Performance evaluation inflation and compression,” Accounting, Organ. Soc., vol. 37, no. 8, pp. 534–543, 2012.
[7]          H. Esen, T. Hatipoğlu, and A. İ. Boyacı, “A Fuzzy Approach for Performance Appraisal: The Evaluation of a Purchasing Specialist,” in Studies in Computational Intelligence, vol. 620, Springer, 2016, pp. 235–250.
[8]          P. E. Levy and J. R. Williams, “The social context of performance appraisal: A review and framework for the future,” J. Manage., vol. 30, no. 6, pp. 881–905, 2004.
[9]          T. R. Manoharan, C. Muralidharan, and S. G. Deshmukh, “An integrated fuzzy multi-attribute decision-making model for employees’ performance appraisal,” Int. J. Hum. Resour. Manag., vol. 22, no. 3, pp. 722–745, Jan. 2011.
[10]        G. Shafer, A mathematical theory of evidence, vol. 1. Princeton university press Princeton, 1976.
[11]        Y. Dong and Z. Xiao, “A Group Decision Making Method Based on Dempster-Shafer Fuzzy Soft Sets Under Incomplete Information,” Int. J. Hybrid Inf. Technol., vol. 8, no. 3, pp. 287–296, 2015.
[12]        K. Sentz and S. Ferson, Combination of evidence in Dempster-Shafer theory, vol. 4015. Citeseer, 2002.
[13]        P. A. C. Aguilar, J. Boudy, D. Istrate, B. Dorizzi, J. C. M. Mota, P. Armando, C. Aguilar, J. Boudy, D. Istrate, and B. Dorizzi, “A dynamic evidential network for fall detection,” IEEE J. Biomed. Heal. Informatics, vol. 18, no. 4, pp. 1103–1113, 2014.
[14]        A. Maseleno, N. Tuah, and C. R. Tabbu, “Fuzzy Logic and Dempster-Shafer Theory to Predict the Risk of Highly Pathogenic Avian Influenza H5n1 Spreading Computer Science Program , Universiti Brunei Darussalam , Faculty of Veterinary Medicine , Gadjah Mada University , Indonesia,” vol. 34, no. 8, pp. 995–1003, 2016.
[15]        M. Casanovas and J. M. Merigó, “Fuzzy aggregation operators in decision making with Dempster–Shafer belief structure,” Expert Syst. Appl., vol. 39, no. 8, pp. 7138–7149, 2012.
[16]        A. Moosavian, M. Khazaee, G. Najafi, M. Kettner, and R. Mamat, “Spark plug fault recognition based on sensor fusion and classifier combination using Dempster–Shafer evidence theory,” Appl. Acoust., vol. 93, pp. 120–129, 2015.
[17]        R. U. Kay, “Fundamentals of the Dempster-Shafer theory and its applications to system safety and reliability modelling,” Reliab.  Theory Appl., vol. 2, pp. 173–185, 2007.
[18]        M. A. Lubiano, S. De La Rosa De Sáa, M. Montenegro, B. Sinova, and M. Á. Gil, “Descriptive analysis of responses to items in questionnaires. Why not using a fuzzy rating scale?,” Inf. Sci. (Ny)., vol. 360, pp. 131–148, 2016.
[19]        R. W. Eichinger and M. M. Lombardo, “Patterns of rater accuracy in 360-degree feedback,” Hum. Resour. Plan., vol. 27, no. 4, pp. 23–25, 2004.
[20]        S. Hassan and J. Rohrbaugh, “Incongruity in 360-Degree Feedback Ratings and Competing Managerial Values: Evidence from a Public Agency Setting,” Int. Public Manag. J., vol. 12, no. 4, pp. 421–449, 2009.
[21]        L. R. Aiken, Rating scales and checklists: Evaluating behavior, personality, and attitudes. John Wiley & Sons, 1996.
[22]        J. T. Austin and P. Villanova, “The criterion problem: 1917-1992.,” J. Appl. Psychol., vol. 77, no. 6, pp. 836–874, 1992.
[23]        S. S. Brutus, “Words versus numbers: A theoretical exploration of giving and receiving narrative comments in performance appraisal,” Hum. Resour. Manag. Rev., vol. 20, no. 2, pp. 144–157, 2010.
[24]        B. Kelcey, D. McGinn, and H. Hill, “Approximate measurement invariance in cross-classified rater-mediated assessments,” Front. Psychol., vol. 5, p. 1469, 2014.
[25]        K. M. Nowack and S. Mashihi, “Evidence-based answers to 15 questions about leveraging 360-degree feedback,” Consult. Psychol. J. Pract. Res., vol. 64, no. 3, p. 157, 2012.
[26]        J.-R. Chang, C.-H. Cheng, and L.-S. Chen, “A fuzzy-based military officer performance appraisal system,” Appl. Soft Comput., vol. 7, no. 3, pp. 936–945, 2007.
[27]        N. Arbaiy and Z. Suradi, “Staff performance appraisal using fuzzy evaluation,” Artif. Intell. Innov. 2007 from Theory to Appl., pp. 195–203, 2007.
[28]        S. E. Markham, J. W. Smith, I. S. Markham, K. F. Braekkan, J. Witt, I. S. Markham, and K. F. Braekkan, “A new approach to analyzing the Achilles’ heel of multisource feedback programs: Can we really trust ratings of leaders at the group level of analysis?,” Leadersh. Q., vol. 25, no. 6, pp. 1120–1142, 2014.
[29]        R. de Andrés, J. L. García-Lapresta, and J. González-Pachón, “Performance appraisal based on distance function methods,” Eur. J. Oper. Res., vol. 207, no. 3, pp. 1599–1607, Dec. 2010.
[30]        E. A. J. Hooft, H. Flier, and M. R. Minne, “Construct Validity of Multi‐Source Performance Ratings: An Examination of the Relationship of Self‐, Supervisor‐, and Peer‐Ratings with Cognitive and Personality Measures,” Int. J. Sel. Assess., vol. 14, no. 1, pp. 67–81, 2006.
[31]        T. Gürbüz and Y. E. Albayrak, “An engineering approach to human resources performance evaluation: Hybrid MCDM application with interactions,” Appl. Soft Comput. J., vol. 21, pp. 365–375, 2014.
[32]        W. H. Bommer, J. L. Johnson, G. A. Rich, P. M. Podsakoff, and S. B. MacKenzie, “On the interchangeability of objective and subjective measures of employee performance: A meta-analysis,” Pers. Psychol., vol. 48, no. 587605, pp. 3–27, 1995.
[33]        H. J. Whiting and T. J. B. Kline, “II. Research Tools for the Practicing Manager: Testing a Model of Performance Appraisal Fit on Attitudinal Outcomes,” Psychol. J. (Taylor Fr. Ltd), vol. 10, no. 2, pp. 127–148, 2007.
[34]        D. J. Woehr, M. K. Sheehan, and W. Bennett Jr, “Assessing measurement equivalence across rating sources: a multitrait-multirater approach,” J. Appl. Psychol., vol. 90, no. 3, p. 592, 2005.
[35]        K.-Y. Ng, C. Koh, S. Ang, J. C. Kennedy, and K.-Y. Chan, “Rating leniency and halo in multisource feedback ratings: testing cultural assumptions of power distance and individualism-collectivism,” J. Appl. Psychol., vol. 96, no. 5, p. 1033, 2011.
[36]        B. B. Moore, A consideration of rater effects and rater design via signal detection theory. Columbia University, 2009.
[37]        M. R. Edwards and A. J. Ewen, “360-degree feedback: royal fail or holy grail?,” Career Dev. Int., vol. 1, no. 3, pp. 28–31, 1996.
[38]        T. I. M. Hesketh, “An application of a computerized fuzzy graphic rating scale to the psychological measurement of individual differences,” pp. 21–35, 1988.
[39]        M. Espinilla, R. de Andrés, F. J. Martínez, and L. Martínez, “A 360-degree performance appraisal model dealing with heterogeneous information and dependent criteria,” Inf. Sci. (Ny)., vol. 222, pp. 459–471, 2013.
[40]        C. Moon, J. Lee, and S. Lim, “A performance appraisal and promotion ranking system based on fuzzy logic: An implementation case in military organizations,” Appl. Soft Comput., vol. 10, no. 2, pp. 512–519, 2010.
[41]        X. Min-peng, Z. Xiao-hu, and D. xin, “Modeling of Engineering R&D Staff Performance Appraisal Model Based on Fuzzy Comprehensive Evaluation,” Syst. Eng. Procedia, vol. 4, pp. 236–242, 2012.
[42]        R. Sepehrirad, A. Azar, and A. Sadeghi, “Developing a Hybrid Mathematical Model for 360-Degree Performance Appraisal: A Case Study,” Procedia - Soc. Behav. Sci., vol. 62, no. June 2016, pp. 844–848, 2012.
[43]        M. Anisseh, J. Dodangeh, M. A. Dashti, and F. Piri, “360 Degree personnel performance appraisal using the MADM models and presenting a model for overall ranking,” in IEEE International Conference on Industrial Engineering and Engineering Management, 2007.
[44]        A. Neogi, A. C. A. A. C. A. Mondal, and S. S. K. Mandal, “A cascaded fuzzy inference system for university non-teaching staff performance appraisal,” J. Inf. Process. Syst., vol. 7, no. 4, pp. 595–612, 2011.
[45]        I. Ahmed, I. Sultana, S. K. Paul, and A. Azeem, “Employee performance evaluation: a fuzzy approach,” Int. J. Product. Perform. Manag., vol. 62, no. 7, pp. 718–734, 2013.
[46]        S. S. Jamsandekar and R. R. Mudholkar, “Performance Evaluation by Fuzzy Inference Technique,” no. 2, pp. 158–164, 2013.
[47]        O. W. Samuel, M. O. Omisore, and E. J. Atajeromavwo, “Online fuzzy based decision support system for human resource performance appraisal,” Measurement, vol. 55, pp. 452–461, 2014.
[48]        A. A. A. Lasserre, M. V. L. Solabac, R. Hernandez-torres, R. Posada-gomez, U. Juárez-Martínez, G. Fernández Lambert, A. Alfonso, A. A. A. Lasserre, M. Violeta, L. Solabac, R. Hernandez-torres, R. Posada-gomez, U. Juárez-Martínez, and G. F. Lambert, “Expert system for competences evaluation 360° feedback using fuzzy logic,” Math. Probl. Eng., vol. 2014, 2014.
[49]        Y. Deng, R. Sadiq, W. Jiang, and S. Tesfamariam, “Risk analysis in a linguistic environment: A fuzzy evidential reasoning-based approach,” Expert Syst. Appl., vol. 38, no. 12, pp. 15438–15446, 2011.
[50]       ف. بنکداری، ش. جعفری, “طراحی و پیاده ‏سازی رفع‏ کننده‏ ی برخورد در یک سیستم تصمیم‏ همیار هوشمند مبتنی بر تنوع نقطه نظرات,” مدل سازی در مهندسی, vol. 8, no. 22, pp. 1–12, Oct. 2010.
[51]        G. Meenakshi, “Multi source feedback based performance appraisal system using Fuzzy logic decision support system,” Int. J. Soft Comput., vol. 3, no. 1, pp. 91–106, 2012.
[52]        A. Shaout, M. K. Yousif, and M. Khalid Yousif, “Employee Performance Appraisal System Using Fuzzy Logic,” Int. J. Comput. Sci. Inf. Technol., vol. 6, no. 4, pp. 1–19, 2014.
[53]        N. Macwan and D. P. S. Sajja, “Performance Appraisal using Fuzzy Evaluation Methodology,” Int. J. Eng. Innov. Technol. Vol., vol. 3, 2013.
[54]        M. A. Lubiano, M. Montenegro, B. Sinova, S. de la Rosa de Sáa, and M. Á. Gil, “Hypothesis testing for means in connection with fuzzy rating scale-based data: algorithms and applications,” Eur. J. Oper. Res., vol. 251, no. 3, pp. 918–929, 2016.
[55]        S. De La Rosa De Sáa, M. Á. Gil, M. T. L. García, and M. A. Lubiano, “Fuzzy rating vs. fuzzy conversion scales: An empirical comparison through the MSE,” Adv. Intell. Syst. Comput., vol. 190 AISC, pp. 135–143, 2013.
[56]        R. de Andrés, J. L. García-Lapresta, and L. Martínez, “A multi-granular linguistic model for management decision-making in performance appraisal,” Soft Comput., vol. 14, no. 1, pp. 21–34, 2010.
[57]        M. Ha-Duong, “Hierarchical fusion of expert opinions in the Transferable Belief Model, application to climate sensitivity,” Int. J. Approx. Reason., vol. 49, no. 3, pp. 555–574, 2008.
[58]        C. Fu, J.-B. Yang, and S.-L. Yang, “A group evidential reasoning approach based on expert reliability,” Eur. J. Oper. Res., vol. 246, no. 3, pp. 886–893, 2015.
[59]        D. Tang, T. C. Wong, K. S. Chin, and C. K. Kwong, “Evaluation of user satisfaction using evidential reasoning-based methodology,” Neurocomputing, vol. 142, pp. 86–94, 2014.
[60]        T. Reineking, “Belief functions: theory and algorithms,” Staats-und Universit{ä}tsbibliothek Bremen, 2014.
[61]        P. Smets, “Decision making in the TBM: The necessity of the pignistic transformation,” Int. J. Approx. Reason., vol. 38, no. 2, pp. 133–147, 2005.
[62]        D. Dubois, H. Prade, and P. Smets, “A definition of subjective possibility,” Int. J. Approx. Reason., vol. 48, no. 2, pp. 352–364, 2008.
[63]        R. R. Yager, “On the Dempster-Shafer framework and new combination rules,” Inf. Sci. (Ny)., vol. 41, no. 2, pp. 93–137, 1987.
[64]        D. Dubois and H. Prade, “Representation and combination of uncertainty with belief functions and possibility measures,” Comput. Intell., vol. 4, no. 3, pp. 244–264, 1988.
[65]        W. Liu, “Analyzing the degree of conflict among belief functions,” Artif. Intell., vol. 170, no. 11, pp. 909–924, 2006.
[66]        E. Lefevre, O. Colot, and P. Vannoorenberghe, “Belief functions combination and conflict management,” Inf. Fusion J., vol. 3, no. 2, pp. 149–162, 2002.
[67]        L. A. Zadeh, “Fuzzy sets,” Inf. Control, vol. 8, no. 3, pp. 338–353, 1965.
[68]        C.-H. Yeh and Y.-H. Chang, “Modeling subjective evaluation for fuzzy group multicriteria decision making,” Eur. J. Oper. Res., vol. 194, no. 2, pp. 464–473, 2009.
[69]       ح. کوهساری, ا. نجفی, ح. علی الهی, م. آدم پیرا, “بررسی عوامل مؤثر بر عملیات تراکم دینامیکی در خاک های دانه ای مبتنی بر روش فازی,” مدل سازی در مهندسی, vol. 13, no. 43, pp. 143–158, Dec. 2015.
[70]       م. اژدری مقدم, ن. ا. امانیان,ا. جعفری ندوشن, “بهینه یابی هندسه سرریز کنگره‌ای مثلثی با استفاده از مدل فازی- عصبی و الگوریتم ژنتیک (مطالعه موردی سدHyrum درایالت یوتای آمریکا),” مدل سازی در مهندسی, vol. 5, no. 19, pp. 57–68, Dec. 2009.
[71]        E. E. Karsak, “A fuzzy multiple objective programming approach for personnel selection,” in Systems, Man, and Cybernetics, 2000 IEEE International Conference on, 2000, vol. 3, pp. 2007–2012.
[72]        A. Cannavacciuolo, G. Capaldo, A. Ventre, and G. Zollo, “Linking the fuzzy set theory to organizational routines: a study in personnel evaluation in a large company,” in Fuzzy Systems, 1993., Second IEEE International Conference on, 1993, pp. 667–672.
[73]        A. Cannavacciuolo, G. Capaldo, A. Ventre, A. Volpe, and G. Zollo, “An approach to the evaluation of human resources by using fuzzy set theory,” in Fuzzy Systems, 1994. IEEE World Congress on Computational Intelligence., Proceedings of the Third IEEE Conference on, 1994, pp. 1165–1170.
[74]        D.-I. J. Stahl, “A method for personnel selection in concurrent engineering using fuzzy sets,” in Fuzzy sets in engineering design and configuration, Springer, 1996, pp. 265–276.
[75]       ش. ‏الدین ناظمی, ش. ‏الدین ناظمی, م. کاظمی,ا. اخروی, “ارائه مدل تلفیق شکاف عملکردی با AHP گروهی- فازی برای تعیین اولویت‏ های بهبود,” مدل سازی در مهندسی, vol. 9, no. 27, pp. 1–12, Feb. 2012.
[76]        L. Podofillini, V. Dang, E. Zio, P. Baraldi, and M. Librizzi, “Using expert models in human reliability analysis—a dependence assessment method based on fuzzy logic,” Risk Anal., vol. 30, no. 8, pp. 1277–1297, 2010.
[77]        O. Taylan, A. O. Bafail, R. M. S. Abdulaal, and M. R. Kabli, “Construction projects selection and risk assessment by fuzzy AHP and fuzzy TOPSIS methodologies,” Appl. Soft Comput., vol. 17, pp. 105–116, 2014.
[78]        D. V Petrović, M. Tanasijević, V. Milić, N. Lilić, S. Stojadinović, and I. Svrkota, “Risk assessment model of mining equipment failure based on fuzzy logic,” Expert Syst. Appl., vol. 41, no. 18, pp. 8157–8164, 2014.
[79]        W. Pedrycz, “Why triangular membership functions?,” Fuzzy Sets Syst., vol. 64, no. 1, pp. 21–30, 1994.
[80]        M. Mizumoto and K. Tanaka, “Fuzzy sets and their operations,” Inf. Control, vol. 48, no. 1, pp. 30–48, 1981.
[81]        R.-J. Li, “Fuzzy method in group decision making,” Comput. Math. with Appl., vol. 38, no. 1, pp. 91–101, 1999.
[82]        T. Kuhlmann, M. Dantlgraber, and U.-D. Reips, “Investigating measurement equivalence of visual analogue scales and Likert-type scales in Internet-based personality questionnaires,” Behav. Res. Methods, pp. 1–9, 2017.
[83]        X. Deng, Y. Li, and Y. Deng, “A decision making method based on Dempster-Shafer theory of evidence under the constraint of uncertain subjective information,” J. Inf. &COMPUTATIONAL Sci., vol. 9, no. 8, pp. 2049–2056, 2012.
[84]        D.-L. Xu, “An introduction and survey of the evidential reasoning approach for multiple criteria decision analysis,” Ann Oper Res, vol. 195, no. 1, pp. 163–187, 2012.
[85]        J.-B. Yang, Y.-M. Wang, D.-L. Xu, and K.-S. Chin, “The evidential reasoning approach for MADA under both probabilistic and fuzzy uncertainties,” Eur. J. Oper. Res., vol. 171, no. 1, pp. 309–343, 2006.
[86]        C. Fu and S. Yang, “An evidential reasoning based consensus model for multiple attribute group decision analysis problems with interval-valued group consensus requirements,” Eur. J. Oper. Res., vol. 223, no. 1, pp. 167–176, 2012.
[87]        D.-L. Xu, J.-B. Yang, and Y.-M. Wang, “The evidential reasoning approach for multi-attribute decision analysis under interval uncertainty,” Eur. J. Oper. Res., vol. 174, no. 3, pp. 1914–1943, 2006.
Journal of Modeling in Engineering
Volume 15, Issue 51
October 2017
Pages 411-432

Files

History

  • Receive Date: 15 May 2017
  • Revise Date: 14 July 2017
  • Accept Date: 22 August 2017

Share

How to cite

Statistics