Exploring a Better Search–based Implementation on Second–Order Mutant Generation

Mohamad Syafri Tuloli, Benhard Sitohang, Bayu Hendradjaya

Abstract


Pengujian perangkat lunak adalah bagian dari proses pengembangan perangkat lunak, dengan tujuan utama untuk mengurangi/menghilangkan kesalahan pada perangkat lunak, hal ini umumnya dilakukan dengan menjalankan kasus-uji. Salah satu teknik untuk mengukur dan meningkatkan kualitas dari kasus uji adalah pengujian mutasi, tetapi walaupun sudah terbukti keefektifannya, teknik ini masih memiliki suatu kendala besar, yaitu tidak praktis untuk digunakan karena melibatkan pembangkitan dan eksekusi dari jumlah mutan yang besar. Belakangan ini penggunaan optimisasi berbasis-pencarian pada permasalahan pengujian perangkat lunak sedang popular. Pada penelitian ini, dilakukan eksplorasi penggunaan optimasi berbasis-pencarian pada pembangkitan mutan (variasi dari program), dengan tujuan untuk menghasilkan mutan yang tidak dapat dideteksi oleh kasus-uji, karena mutan jenis ini memiliki dapat kekurangan dari kasus-uji. Metode usulan dibandingkan dengan algoritma pembangkitan second-order mutant yang umum digunakan, dan juga dibandingkan dengan pendekatan berbasis pencarian lainnya. Hasil menunjukkan bahwa metode usulan dapat membangkitkan lebih banyak mutan tidak-terdeteksi (undetected-mutant) daripada dengan metode pembangkitan mutan yang umum. Metode usulan memiliki performansi yang lebih rendah daripada metode pembangkitan berbasis-pencarian benchmark, tetapi performansinya dapat ditingkatkan dengan melakukan perubahan pada representasi solusi, dan dengan adopsi parameter optimasi yang digunakan oleh metode pembanding.

Software testing is a part of a software development process with a major concern is to reduce/eliminate fault in the software, and mainly done by executing a test case. One of the techniques for measuring and improving test case quality is mutation testing, but despite it is good effectiveness, this technique has a major problem that is impractical because it involves generation and execution of huge amount of mutant. This trend also happens in software testing, with the main focus on optimizing the test case generation. In this research, we explore the used of search-based optimization to the mutant (program variant) generation, with a goal to generate mutants that can escape test case detection, because these mutants have a probability to show test case deficiency. In this research, the proposed method is compared with a general second-order mutant generation algorithm and with other search-based mutant generation. The result shows that the proposed method can generate more undetected-mutant than a general second-order mutant generation. The proposed method performs worse than the benchmark search-based mutant generation, but this performance improved by altering it is solution representation and by the adoption of an optimization parameter.

Keywords


analisis mutasi; pengujian mutasi; pengujian perangkat lunak berbasis pencarian; rekayasa perangkat lunak berbasis pencarian

Full Text:

PDF (English)

References


Delgado-Perez, P., Medina -Bulo, I., Segura, S., Garcia-Dominguez, A., & Dominguez -Jimenez, J. J. (2017). GiGAn : Evolutionary Mutation Testing for C ++ Object-Oriented Systems. 32nd ACM SIGAPP Symposium On Applied Computing.

Deng, L., Offutt, J., & Li, N. (2013). Empirical evaluation of the statement deletion mutation operator. Proceedings - IEEE 6th International Conference on Software Testing, Verification and Validation, ICST 2013, 84–93. https://doi.org/10.1109/ICST.2013.20

Harman, M., Mansouri, S. A., & Zhang, Y. (2009). Search Based Software Engineering : A Comprehensive Analysis and Review of Trends Techniques and Applications, 1–78.

Harman, M., McMinn, P., Souza, J. De, & Yoo, S. (2012). Search-based software engineering: Techniques, taxonomy, tutorial. Empirical Software Engineering and Verification, 1–59. Retrieved from http://link.springer.com/chapter/10.1007/978-3-642-25231-0_1

Jia, Y., & Harman, M. (2008). Constructing Subtle Faults Using Higher Order Mutation Testing. Retrieved from http://discovery.ucl.ac.uk/1302155/

Jia, Y., & Harman, M. (2011). An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 37(5), 649–678. https://doi.org/10.1109/TSE.2010.62

Just, R., Jalali, D., Inozemtseva, L., & Ernst, M. (2014). Are mutants a valid substitute for real faults in software testing?, 654–665. https://doi.org/10.1145/2635868.2635929

Mcminn, P. (2004). Search-based Software Test Data Generation : A Survey. Software Testing, Verification & Reliability, 14(2), 105–156.

Nguyen, Q. V., & Madeyski, L. (2014). Problems of Mutation Testing and Higher Order Mutation Testing. Advanced Computational Methods for Knowledge Engineering, 157– 172. https://doi.org/10.1007/978-3-319-06569-4_12

Offutt, a. J., Lee, A., Rothermel, G., Untch, R. H., & Zapf, C. (1996). An experimental determination of sufficient mutant operators. ACM Transactions on Software Engineering and Methodology, 5(2), 99–118. https://doi.org/10.1145/227607.227610

Orso, A., & Rothermel, G. (2014). Software testing: a research travelogue (2000–2014). Proceedings of the on Future of Software Engineering - FOSE 2014, 117–132. https://doi.org/10.1145/2593882.2593885

Polo, M., Piattini, M., & Garc, I. (2009). Decreasing the cost of mutation testing with second-order mutants. Software: Testing, Verification, and Reliability, (June 2008), 111–131.

Tuloli, M. S., Sitohang, B., & Hendradjaya, B. (2016). Regex Based Mutation Testing Operator Implementation. In International Conference on Data and Software Engineering.

Tuloli, M. S., Sitohang, B., & Hendradjaya, B. (2017). On the Implementation of Search-Based Approach to Mutation Testing. In International Conference on Data and Software Engineering (ICoDSE). in press.




DOI: https://doi.org/10.37905/jji.v1i1.2329

Refbacks

  • There are currently no refbacks.


JJI has been indexed by:
Sinta Crossref Scholar Garuda
Base Dimension ROAD SIS
ASCI







Editorial Office
Department of Informatics Engineering, Universitas Negeri Gorontalo
Engineering Faculty Building, 1st Floor
Jl. Prof. Dr. Ing. B. J. Habibie, Bone Bolango, Gorontalo, 96119, Indonesia.Whatsapp: +6281314270499Email: jji.ft@ung.ac.id
 
Creative Commons Licence
Jambura Journal of Informatics (JJi), is licensed under a Lisensi Creative Commons Atribusi 4.0 Internasional.