Developing an integrated model for evaluating R&D organizations’ performance: combination of DEA-ANP
Abstract
Assessing the performance of the Research and development (R&D) organizations to achieve higher productivity, growth, and development is always a critical necessity. Therefore, developing a more accurate model to evaluate the performance is always required. For this purpose, this study is aimed at developing a decision-making model for evaluating R&D performance. The model comes up with determining the most proper evaluative criteria for assessing R&D organizations. Then, it integrates Data Envelopment Analysis (DEA) with Analytical Network Process (ANP) to assess R&D performance. This paper is aimed to develop an integrated model for evaluating R&D performance. The findings of the study show that the DEA-ANP model is an accurate and acceptable model for evaluating R&D organizations’ performance.
Keyword : R&D organizations, efficiency, Data Envelopment Analysis (DEA), Analytical Network Process (ANP), evaluation, decision-making
How to Cite
Hoseini, S. A., Fallahpour, A., Wong, K. Y., & Antuchevičienė, J. (2021). Developing an integrated model for evaluating R&D organizations’ performance: combination of DEA-ANP . Technological and Economic Development of Economy, 27(4), 970-991. https://doi.org/10.3846/tede.2021.15144
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
Abedi Gheshlaghi, H., Feizizadeh, B., &Blaschke, T. (2020). GIS-based forest fire risk mapping using the analytical network process and fuzzy logic. Journal of Environmental Planning and Management, 63(3), 481–499. https://doi.org/10.1080/09640568.2019.1594726
Aparicio, J., & Kapelko, M. (2019). Enhancing the measurement of composite indicators of corporate social performance. Social Indicators Research, 144(2), 807–826. https://doi.org/10.1007/s11205-018-02052-1
Asmara, I. J., Achelia, E., Simamora, N. G., & Sartono, B. (2019). Measuring R&D performance using Data Envelopment Analysis (DEA): Case Indonesia. International Journal of Social Science and Humanity, 9(4), 91–96. https://doi.org/10.18178/ijssh.2019.V9.997
Belgin, O. (2019). Analysing R&D efficiency of Turkish regions using data envelopment analysis. Technology Analysis & Strategic Management, 31(11), 1341–1352. https://doi.org/10.1080/09537325.2019.1613521
Calik, A., Pehlivan, N. Y., &Kahraman, C. (2018). An integrated fuzzy AHP/DEA approach for performance evaluation of territorial units in Turkey. Technological and Economic Development of Economy, 24(4), 1280–1302. https://doi.org/10.3846/20294913.2016.1230563
Cao, Y., You, J., Shi, Y., & Hu, W. (2019). Evaluating R&D and transformation functional platforms’ operational performance using a data envelopment analysis model: A comparative study. Sustainability, 11(18), 5023. https://doi.org/10.3390/su11185023
Carrillo, M. (2019). Measuring and ranking R&D performance at the country level. Economics and Sociology, 12(1), 100–114. https://doi.org/10.14254/2071-789X.2019/12-1/5
Chen, K., Kou, M., & Fu, X. (2018). Evaluation of multi-period regional R&D efficiency: An application of dynamic DEA to China’s regional R&D systems. Omega, 74, 103–114. https://doi.org/10.1016/j.omega.2017.01.010
Dobrzanski, P., & Bobowski, S. (2020). The Efficiency of R&D Expenditures in ASEAN Countries. Sustainability, 12(7), 2686. https://doi.org/10.3390/su12072686
Ersoyak, E. B., & Ozcan, S. (2019). A performance measurement system for R&D activities in the software sector. International Journal of Technology Intelligence and Planning, 12(3), 242–272. https://doi.org/10.1504/IJTIP.2019.099214
Fallahpour, A., Olugu, E. U., Musa, S. N., Khezrimotlagh, D., & Wong, K. Y. (2016). An integrated model for green supplier selection under fuzzy environment: application of data envelopment analysis and genetic programming approach. Neural Computing and Applications, 27(3), 707–725. https://doi.org/10.1007/s00521-015-1890-3
Gangopadhyay, D., Roy, S., & Mitra, J. (2018). Public sector R&D and relative efficiency measurement of global comparators working on similar research streams. Benchmarking: An International Journal, 25(3), 1059–1084. https://doi.org/10.1108/BIJ-07-2017-0197
Ge, H., & Yang, S. Y. (2017). Study on the R&D performance of high-tech industry in China-based on data envelopment analysis. Journal of Interdisciplinary Mathematics, 20(3), 909–920. https://doi.org/10.1080/09720502.2017.1358890
Geng, Z., Bai, J., Zhu, Q., Xu, Y., & Han, Y. (2018, May). Energy saving and management of the industrial process based on an improved DEA cross-model. In 2018 IEEE 7th Data Driven Control and Learning Systems Conference (DDCLS) (pp. 154–159). IEEE. https://doi.org/10.1109/DDCLS.2018.8515973
Gibson, E., Daim, T. U., & Dabic, M. (2019). Evaluating university industry collaborative research centers. Technological Forecasting and Social Change, 146, 181–202. https://doi.org/10.1016/j.techfore.2019.05.014
Halásková, M., & Bazsová, B. (2016). Evaluation of the efficiency of research and development in EU countries. Acta academica karviniensia, 16(4), 32–45. https://doi.org/10.25142/aak.2016.030
Hou, Q., Wang, M., & Zhou, X. (2018). Improved DEA cross efficiency evaluation method based on ideal and anti-ideal points. Discrete Dynamics in Nature and Society, 2018, 1604298. https://doi.org/10.25142/aak.2016.030
Javaid, B., Arshad, M. A., Ahmad, S., & Kazmi, S. A. A. (2019, January). Comparison of different multi criteria decision analysis techniques for performance evaluation of loop configured micro grid. In 2019 2nd International Conference on Computing, Mathematics and Engineering Technologies (iCoMET) (pp. 1–7). IEEE. https://doi.org/10.1109/ICOMET.2019.8673536
Karadayi, M. A., & Ekinci, Y. (2019). Evaluating R&D performance of EU countries using categorical DEA. Technology Analysis & Strategic Management, 31(2), 227–238. https://doi.org/10.1080/09537325.2018.1493191
Karsak, E. E., & Goker, N. (2020). Improved common weight DEA-based decision approach for economic and financial performance assessment. Technological and Economic Development of Economy, 26(2), 430–448. https://doi.org/10.3846/tede.2020.11870
Khoshnevis, P., & Teirlinck, P. (2018). Performance evaluation of R&D active firms. Socio-Economic Planning Sciences, 61, 16–28. https://doi.org/10.1016/j.seps.2017.01.005
Kim, K., & Cho, N. (2018). A case study on qualitative efficiency of national r&d projects: focused on agricultural research area. Journal of the Korea Society of Digital Industry and Information Management, 14(3), 115–125.
Kou, M., Zhang, Y., Zhang, Y., Chen, K., Guan, J., & Xia, S. (2020). Does gender structure influence R&D efficiency? A regional perspective. Scientometrics, 122(1), 477–501. https://doi.org/10.1007/s11192-019-03282-x
Lim, Y., & Jeon, J. (2019). Analyzing the performance of defense R&D projects based on DEA. Journal of the Korea Institute of Military Science and Technology, 22(1), 106–123.
Nourani, M., Devadason, E. S., & Chandran, V. G. R. (2018). Measuring technical efficiency of insurance companies using dynamic network DEA: an intermediation approach. Technological and Economic Development of Economy, 24(5), 1909–1940. https://doi.org/10.3846/20294913.2017.1303649
Park, J. H., & Shin, K. (2018). Efficiency of government-sponsored R&D projects: A metafrontier DEA approach. Sustainability, 10(7), 2316. https://doi.org/10.3390/su10072316
Qin, X., Du, D., & Kwan, M. P. (2019). Spatial spillovers and value chain spillovers: evaluating regional R&D efficiency and its spillover effects in China. Scientometrics, 119(2), 721–747. https://doi.org/10.1007/s11192-019-03054-7
Salimi, N., & Rezaei, J. (2018). Evaluating firms’ R&D performance using best worst method. Evaluation and Program Planning, 66, 147–155. https://doi.org/10.1016/j.evalprogplan.2017.10.002
Sun, X., Lin, Z., Zhang, L., Lian, Y., & Ding, L. (2018, July). The R&D efficiency evaluation of the university teachers based on a two-stage DEA Model. Journal of Physics: Conference Series, 1060(1), 012–029. https://doi.org/10.1088/1742-6596/1060/1/012029
Tian, C., & Peng, J. (2020). An integrated picture fuzzy ANP-TODIM multi-criteria decision-making approach for tourism attraction recommendation. Technological and Economic Development of Economy, 26(2), 331–354. https://doi.org/10.3846/tede.2019.11412
Wu, H. Y., Chen, I. S., Chen, J. K., & Chien, C. F. (2019). The R&D efficiency of the Taiwanese semiconductor industry. Measurement, 137, 203–213. https://doi.org/10.1016/j.measurement.2019.01.053
Wu, Y. C., Kweh, Q. L., Lu, W. M., Hung, S. W., & Chang, C. F. (2016). Capital stock and performance of R&D organizations: A dynamic DEA-ANP hybrid approach. In Handbook of operations analytics using data envelopment analysis (pp. 167–186). Springer. https://doi.org/10.1007/978-1-4899-7705-2_7
Xie, L., Zhou, J., Zong, Q., & Lu, Q. (2020). Gender diversity in R&D teams and innovation efficiency: Role of the innovation context. Research Policy, 49(1), 103885. https://doi.org/10.1016/j.respol.2019.103885
Xiong, X., Yang, G. L., & Guan, Z. C. (2018). Assessing R&D efficiency using a two-stage dynamic DEA model: A case study of research institutes in the Chinese Academy of Sciences. Journal of Informetrics, 12(3), 784–805. https://doi.org/10.1016/j.joi.2018.07.003
Yeh, L. T., & Chang, D. S. (2020). Using categorical DEA to assess the effect of subsidy policies and technological learning on R&D efficiency of IT industry. Technological and Economic Development of Economy, 26(2), 311–330. https://doi.org/10.3846/tede.2019.11411
You, T., & Jung, W. S. (2019). A system dynamics analysis of national R&D performance measurement system in Korea. Industrial Engineering & Management Systems, 17(4), 833–839. https://doi.org/10.7232/iems.2018.17.4.833
Yu, D., Lim, D., & Seol, H. (2018). A framework for deriving investment priority in national defense R&D-using DEA based on TRA. Journal of the Korea Institute of Military Science and Technology, 21(2), 217–224.
Zuo, K., & Guan, J. (2017). Measuring the R&D efficiency of regions by a parallel DEA game model. Scientometrics, 112(1), 175–194. https://doi.org/10.1007/s11192-017-2380-4
Aparicio, J., & Kapelko, M. (2019). Enhancing the measurement of composite indicators of corporate social performance. Social Indicators Research, 144(2), 807–826. https://doi.org/10.1007/s11205-018-02052-1
Asmara, I. J., Achelia, E., Simamora, N. G., & Sartono, B. (2019). Measuring R&D performance using Data Envelopment Analysis (DEA): Case Indonesia. International Journal of Social Science and Humanity, 9(4), 91–96. https://doi.org/10.18178/ijssh.2019.V9.997
Belgin, O. (2019). Analysing R&D efficiency of Turkish regions using data envelopment analysis. Technology Analysis & Strategic Management, 31(11), 1341–1352. https://doi.org/10.1080/09537325.2019.1613521
Calik, A., Pehlivan, N. Y., &Kahraman, C. (2018). An integrated fuzzy AHP/DEA approach for performance evaluation of territorial units in Turkey. Technological and Economic Development of Economy, 24(4), 1280–1302. https://doi.org/10.3846/20294913.2016.1230563
Cao, Y., You, J., Shi, Y., & Hu, W. (2019). Evaluating R&D and transformation functional platforms’ operational performance using a data envelopment analysis model: A comparative study. Sustainability, 11(18), 5023. https://doi.org/10.3390/su11185023
Carrillo, M. (2019). Measuring and ranking R&D performance at the country level. Economics and Sociology, 12(1), 100–114. https://doi.org/10.14254/2071-789X.2019/12-1/5
Chen, K., Kou, M., & Fu, X. (2018). Evaluation of multi-period regional R&D efficiency: An application of dynamic DEA to China’s regional R&D systems. Omega, 74, 103–114. https://doi.org/10.1016/j.omega.2017.01.010
Dobrzanski, P., & Bobowski, S. (2020). The Efficiency of R&D Expenditures in ASEAN Countries. Sustainability, 12(7), 2686. https://doi.org/10.3390/su12072686
Ersoyak, E. B., & Ozcan, S. (2019). A performance measurement system for R&D activities in the software sector. International Journal of Technology Intelligence and Planning, 12(3), 242–272. https://doi.org/10.1504/IJTIP.2019.099214
Fallahpour, A., Olugu, E. U., Musa, S. N., Khezrimotlagh, D., & Wong, K. Y. (2016). An integrated model for green supplier selection under fuzzy environment: application of data envelopment analysis and genetic programming approach. Neural Computing and Applications, 27(3), 707–725. https://doi.org/10.1007/s00521-015-1890-3
Gangopadhyay, D., Roy, S., & Mitra, J. (2018). Public sector R&D and relative efficiency measurement of global comparators working on similar research streams. Benchmarking: An International Journal, 25(3), 1059–1084. https://doi.org/10.1108/BIJ-07-2017-0197
Ge, H., & Yang, S. Y. (2017). Study on the R&D performance of high-tech industry in China-based on data envelopment analysis. Journal of Interdisciplinary Mathematics, 20(3), 909–920. https://doi.org/10.1080/09720502.2017.1358890
Geng, Z., Bai, J., Zhu, Q., Xu, Y., & Han, Y. (2018, May). Energy saving and management of the industrial process based on an improved DEA cross-model. In 2018 IEEE 7th Data Driven Control and Learning Systems Conference (DDCLS) (pp. 154–159). IEEE. https://doi.org/10.1109/DDCLS.2018.8515973
Gibson, E., Daim, T. U., & Dabic, M. (2019). Evaluating university industry collaborative research centers. Technological Forecasting and Social Change, 146, 181–202. https://doi.org/10.1016/j.techfore.2019.05.014
Halásková, M., & Bazsová, B. (2016). Evaluation of the efficiency of research and development in EU countries. Acta academica karviniensia, 16(4), 32–45. https://doi.org/10.25142/aak.2016.030
Hou, Q., Wang, M., & Zhou, X. (2018). Improved DEA cross efficiency evaluation method based on ideal and anti-ideal points. Discrete Dynamics in Nature and Society, 2018, 1604298. https://doi.org/10.25142/aak.2016.030
Javaid, B., Arshad, M. A., Ahmad, S., & Kazmi, S. A. A. (2019, January). Comparison of different multi criteria decision analysis techniques for performance evaluation of loop configured micro grid. In 2019 2nd International Conference on Computing, Mathematics and Engineering Technologies (iCoMET) (pp. 1–7). IEEE. https://doi.org/10.1109/ICOMET.2019.8673536
Karadayi, M. A., & Ekinci, Y. (2019). Evaluating R&D performance of EU countries using categorical DEA. Technology Analysis & Strategic Management, 31(2), 227–238. https://doi.org/10.1080/09537325.2018.1493191
Karsak, E. E., & Goker, N. (2020). Improved common weight DEA-based decision approach for economic and financial performance assessment. Technological and Economic Development of Economy, 26(2), 430–448. https://doi.org/10.3846/tede.2020.11870
Khoshnevis, P., & Teirlinck, P. (2018). Performance evaluation of R&D active firms. Socio-Economic Planning Sciences, 61, 16–28. https://doi.org/10.1016/j.seps.2017.01.005
Kim, K., & Cho, N. (2018). A case study on qualitative efficiency of national r&d projects: focused on agricultural research area. Journal of the Korea Society of Digital Industry and Information Management, 14(3), 115–125.
Kou, M., Zhang, Y., Zhang, Y., Chen, K., Guan, J., & Xia, S. (2020). Does gender structure influence R&D efficiency? A regional perspective. Scientometrics, 122(1), 477–501. https://doi.org/10.1007/s11192-019-03282-x
Lim, Y., & Jeon, J. (2019). Analyzing the performance of defense R&D projects based on DEA. Journal of the Korea Institute of Military Science and Technology, 22(1), 106–123.
Nourani, M., Devadason, E. S., & Chandran, V. G. R. (2018). Measuring technical efficiency of insurance companies using dynamic network DEA: an intermediation approach. Technological and Economic Development of Economy, 24(5), 1909–1940. https://doi.org/10.3846/20294913.2017.1303649
Park, J. H., & Shin, K. (2018). Efficiency of government-sponsored R&D projects: A metafrontier DEA approach. Sustainability, 10(7), 2316. https://doi.org/10.3390/su10072316
Qin, X., Du, D., & Kwan, M. P. (2019). Spatial spillovers and value chain spillovers: evaluating regional R&D efficiency and its spillover effects in China. Scientometrics, 119(2), 721–747. https://doi.org/10.1007/s11192-019-03054-7
Salimi, N., & Rezaei, J. (2018). Evaluating firms’ R&D performance using best worst method. Evaluation and Program Planning, 66, 147–155. https://doi.org/10.1016/j.evalprogplan.2017.10.002
Sun, X., Lin, Z., Zhang, L., Lian, Y., & Ding, L. (2018, July). The R&D efficiency evaluation of the university teachers based on a two-stage DEA Model. Journal of Physics: Conference Series, 1060(1), 012–029. https://doi.org/10.1088/1742-6596/1060/1/012029
Tian, C., & Peng, J. (2020). An integrated picture fuzzy ANP-TODIM multi-criteria decision-making approach for tourism attraction recommendation. Technological and Economic Development of Economy, 26(2), 331–354. https://doi.org/10.3846/tede.2019.11412
Wu, H. Y., Chen, I. S., Chen, J. K., & Chien, C. F. (2019). The R&D efficiency of the Taiwanese semiconductor industry. Measurement, 137, 203–213. https://doi.org/10.1016/j.measurement.2019.01.053
Wu, Y. C., Kweh, Q. L., Lu, W. M., Hung, S. W., & Chang, C. F. (2016). Capital stock and performance of R&D organizations: A dynamic DEA-ANP hybrid approach. In Handbook of operations analytics using data envelopment analysis (pp. 167–186). Springer. https://doi.org/10.1007/978-1-4899-7705-2_7
Xie, L., Zhou, J., Zong, Q., & Lu, Q. (2020). Gender diversity in R&D teams and innovation efficiency: Role of the innovation context. Research Policy, 49(1), 103885. https://doi.org/10.1016/j.respol.2019.103885
Xiong, X., Yang, G. L., & Guan, Z. C. (2018). Assessing R&D efficiency using a two-stage dynamic DEA model: A case study of research institutes in the Chinese Academy of Sciences. Journal of Informetrics, 12(3), 784–805. https://doi.org/10.1016/j.joi.2018.07.003
Yeh, L. T., & Chang, D. S. (2020). Using categorical DEA to assess the effect of subsidy policies and technological learning on R&D efficiency of IT industry. Technological and Economic Development of Economy, 26(2), 311–330. https://doi.org/10.3846/tede.2019.11411
You, T., & Jung, W. S. (2019). A system dynamics analysis of national R&D performance measurement system in Korea. Industrial Engineering & Management Systems, 17(4), 833–839. https://doi.org/10.7232/iems.2018.17.4.833
Yu, D., Lim, D., & Seol, H. (2018). A framework for deriving investment priority in national defense R&D-using DEA based on TRA. Journal of the Korea Institute of Military Science and Technology, 21(2), 217–224.
Zuo, K., & Guan, J. (2017). Measuring the R&D efficiency of regions by a parallel DEA game model. Scientometrics, 112(1), 175–194. https://doi.org/10.1007/s11192-017-2380-4