Reduzindo esforços manuais na análise de equivalência dos testes de mutação

Página do item simplificado
dc.contributor.advisor1Oliveira, Leonardo Fernandes Mendonça de
dc.contributor.advisor1Latteshttp://lattes.cnpq.br/4226060363182113
dc.contributor.advisor2Oliveira
dc.contributor.referee1Oliveira, Augusto César Melo de
dc.contributor.referee1Latteshttp://lattes.cnpq.br/0015527839272634
dc.contributor.referee2Kamei, Fernando Kenji
dc.contributor.referee2Latteshttp://lattes.cnpq.br/5033020411757389
dc.creatorAmorim, Samuel Veloso de
dc.creator.Latteshttp://lattes.cnpq.br/3793521720549401
dc.date.accessioned2025-03-05T16:53:26Z
dc.date.available2025-03-05T16:53:26Z
dc.date.issued2024-10-30
dc.description.abstractMutation testing has attracted significant interest due to its reputation as a powerful criterion for test suite adequacy and its ability to guide test case generation. However, the presence of equivalent mutants hinders its use in industry. The Equivalent Mutant Problem has been proven undecidable, but manually detecting equivalent mutants is an error-prone and time- consuming task. Therefore, even partial solutions can help reduce this cost. To minimize this problem, we introduce an approach to suggest equivalent mutants. Our approach is based on automated behavioral tests, which consist of test cases based on the original program's behavior. We perform static analysis to automatically generate tests for the entities impacted by the mutation. For each analyzed mutant, our approach can suggest whether the mutant is equivalent or non-equivalent. In the case of non-equivalent mutants, our approach provides a test case capable of killing the mutant. For the suggested equivalent mutants, we also provide a ranking of mutants with a higher or lower likelihood of being truly equivalent. We tested our approach on a set of 1,542 mutants manually classified in previous works as either equivalent or non-equivalent. We found that the approach effectively suggests equivalent mutants, achieving over 93% accuracy in five of the eight studied subjects. Compared to the manual analysis of surviving mutants, our approach takes one-third of the time to suggest equivalent mutants and is 25 times faster in identifying non-equivalent mutants. We also conducted a study to discern the specific characteristics of mutants that our approach mistakenly classified as equivalent, generating false positives. Furthermore, our investigation delves into a comprehensive analysis of mutation operators, providing essential insights for professionals seeking to improve equivalent mutant detection accuracy and effectively mitigate the associated costs.
dc.description.resumoOs testes de mutação têm atraído muito interesse devido à sua reputação como um poderoso critério de adequação para suítes de teste e por sua capacidade de guiar a geração de casos de teste. No entanto, a presença de mutantes equivalentes dificulta seu uso na indústria. O Problema do Mutante Equivalente já foi provado indecidível, mas detectar manualmente mutantes equivalentes é uma tarefa sujeita a erros e que consome muito tempo. Assim, soluções, mesmo que parciais, podem ajudar a reduzir esse custo. Para minimizar esse problema, introduzimos uma abordagem para sugerir mutantes equivalentes que consistem em casos de teste baseados no comportamento do programa original. Realizamos uma análise estática para gerar automaticamente testes para as entidades impactadas pela mutação. Para cada mutante analisado, nossa abordagem pode classificar o mutante como equivalente ou não equivalente. No caso de mutantes não equivalentes, nossa abordagem fornece um caso de teste capaz de matá-lo. Para os mutantes equivalentes sugeridos, também fornecemos um ranking de mutantes com maior ou menor chance de serem de fato equivalentes. Testamos nossa abordagem em um conjunto de 1.542 mutantes classificados manualmente em trabalhos anteriores como equivalentes ou não equivalentes. Notamos que a abordagem sugere efetivamente mutantes equivalentes, atingindo mais de 93% de precisão em cinco dos oito projetos estudados. Comparado à análise manual dos mutantes sobreviventes, nossa abordagem leva um terço do tempo para sugerir equivalentes e é 25 vezes mais rápida para indicar os não equivalentes. Fizemos também um estudo para discernir as características específicas dos mutantes que nossa abordagem classificou erroneamente como equivalentes, gerando falsos positivos. Além disso, nossa investigação se aprofunda em uma análise abrangente dos operadores de mutação, fornecendo informações essenciais para profissionais que buscam melhorar a precisão da detecção de mutantes equivalentes e mitigar efetivamente os custos associados.
dc.identifier.urihttps://repositorio.ifal.edu.br/handle/123456789/890
dc.language.isopt
dc.publisher.countryBrasil
dc.publisher.departmentCampus Maceió
dc.relation.referencesACREE, J. A. T. On Mutation. Tese (Doutorado) — Georgia Institute of Technology, 1980. ANDREWS, J.; BRIAND, L.; LABICHE, Y. Is mutation an appropriate tool for testing experiments? In: ICSE. [S.l.: s.n.], 2005. p. 402–411. ARCURI, A.; FRASER, G.; JUST, R. Private api access and functional mocking in automated unit test generation. In: ICST. [S.l.: s.n.], 2017. p. 126–137. BINDER, R. Design for testability in object-oriented systems. Communications of the ACM, v. 37, p. 87–101, 1994. BRAIONE, P. et al. Combining symbolic execution and search-based testing for programs with complex heap inputs. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. [S.l.: s.n.], 2017. p. 90–101. BRITO, C. et al. A preliminary investigation into using machine learning algorithms to identify minimal and equivalent mutants. In: ICSTW. [S.l.: s.n.], 2020. p. 304–313. BUDD, T.; ANGLUIN, D. Two notions of correctness and their relation to testing. Acta Informatica, v. 18, n. 1, p. 31–45, 1982. DEMILLO, R.; LIPTON, R.; SAYWARD, F. Hints on test data selection: Help for the practicing programmer. Computer, v. 11, n. 4, p. 34–41, 1978. FERNANDES. Nimrod - Experimental Pack Repository. [S.l.]: GitHub, 2023. Disponível em: https://github.com/leofernandesmo/nimrod/tree/main. Acesso em: 18 jan. 2024. FERNANDES, L. et al. Avoiding useless mutants. In: GPCE. [S.l.: s.n.], 2017. p. 187–198. FRASER, G.; ARCURI, A. Evosuite: automatic test suite generation for object-oriented software. In: ESEC/FSE. [S.l.: s.n.], 2011. p. 416–419. FRASER, G. et al. Does automated unit test generation really help software testers? a controlled empirical study. ACM Transactions on Software Engineering and Methodology, ACM, v. 24, n. 4, p. 23:1–23:49, 2015. GOSLING, J. et al. The Java Language Specification. 2022. Disponível em: https://docs.oracle.com/javase/specs. Acesso em: 12 jul. 2022. JIA, Y.; HARMAN, M. An analysis and survey of the development of mutation testing. TSE, v. 37, n. 5, p. 649–678, 2011. JUST, R. et al. Are mutants a valid substitute for real faults in software testing? In: ESEC/FSE. [S.l.: s.n.], 2014. p. 654–665. KINTIS, M.; MALEVRIS, N. Medic: A static analysis framework for equivalent mutant identification. IST, v. 68, p. 1 – 17, 2015. KINTIS, M. et al. Detecting trivial mutant equivalences via compiler optimisations. TSE, v. 44, n. 4, p. 308–333, 2018. LAKHOTIA, K.; MCMINN, P.; HARMAN, M. Automated test data generation for coverage: Haven’t we solved this problem yet? In: Testing: Academic and Industrial Conference Practice and Research Techniques. [S.l.: s.n.], 2009. p. 95–104. LI, S. et al. Measuring code behavioral similarity for programming and software engineering education. In: Proceedings of the 38th International Conference on Software Engineering Companion. [S.l.: s.n.], 2016. p. 501–510. ISBN 978-1-4503-4205-6. LUO, Q. et al. An empirical analysis of flaky tests. In: ESEC/FSE. [S.l.: s.n.], 2014. p. 643–653. MADEYSKI, L. et al. Overcoming the equivalent mutant problem: A systematic literature review and a comparative experiment of second order mutation. TSE, v. 40, n. 1, p. 23–42, 2014. MONGIOVI, M. et al. Making refactoring safer through impact analysis. SCP, Elsevier, v. 93, p. 39–64, 2014. MONGIOVI, M. et al. Detecting Overly Strong Preconditions in Refactoring Engines. TSE,v. 44, n. 5, p. 429–452, 2018. NAEEM, M. R. et al. A machine learning approach for classification of equivalent mutants. Journal of Software: Evolution and Process, v. 32, n. 5, 2020. OFFUTT, J.; CRAFT, M. Using compiler optimization techniques to detect equivalent mutants. STVR, Wiley Online Library, v. 4, n. 3, p. 131–154, 1994. OFFUTT, J.; MA, Y.-S.; KWON, Y.-R. The class-level mutants of mujava. In: AST. [S.l.:s.n.], 2006. p. 78–84. PACHECO, C. et al. Feedback-directed random test generation. In: ICSE. [S.l.: s.n.], 2007. p. 75–84. PAPADAKIS, M. et al. Mutation testing advances: an analysis and survey. In: Advances in Computers. [S.l.]: Elsevier, 2019. v. 112, p. 275–378. PAPADAKIS, M. et al. Are mutation scores correlated with real fault detection?: A large scale empirical study on the relationship between mutants and real faults. In: ICSE. [S.l.: s.n.], 2018. 537–548. PEACOCK, S. et al. Automatic equivalent mutants classification using abstract syntax tree neural networks. In: ICSTW. [S.l.: s.n.], 2021. p. 13–18. PIZZOLETO, A. V. et al. A systematic literature review of techniques and metrics to reduce the cost of mutation testing. JSS, v. 157, 2019. SCHULER, D.; DALLMEIER, V.; ZELLER, A. Efficient mutation testing by checking invariant violations. In: ISSTA. [S.l.: s.n.], 2009. p. 69–80. SCHULER, D.; ZELLER, A. Covering and uncovering equivalent mutants. STVR, v. 23, n. 5, p. 353–374, 2013. SHAMSHIRI, S. et al. Do automatically generated unit tests find real faults? an empirical study of effectiveness and challenges. In: 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). [S.l.: s.n.], 2015. p. 201–211. SOARES, G.; GHEYI, R.; MASSONI, T. Automated behavioral testing of refactoring engines. TSE, v. 39, n. 2, p. 147–162, 2013. SOARES, G. et al. Comparing approaches to analyze refactoring activity on software repositories. JSS, v. 86, n. 4, p. 1006–1022, 2013. SOARES, G. et al. Making program refactoring safer. IEEE software, v. 27, n. 4, p. 52–57, 2010. STEIMANN, F.; THIES, A. From behaviour preservation to behaviour modification: Constraint- based mutant generation. In: ICSE. [S.l.: s.n.], 2010. p. 425–434. VOAS, J.; MCGRAW, G. Software fault injection: inoculating programs against errors. [S.l.]: John Wiley & Sons, Inc., 1997. YAO, X.; HARMAN, M.; JIA, Y. A study of equivalent and stubborn mutation operators using human analysis of equivalence. In: Proceedings of the 36th International Conference on Software Engineering. [S.l.: s.n.], 2014. p. 919–930.
dc.subjectSistema de Informação
dc.subjectMutantes equivalente
dc.subjectTestes de mutação
dc.subjectMutation testing
dc.subjectEquivalente mutant
dc.subjectAutomated testing
dc.subject.cnpqCIENCIAS EXATAS E DA TERRA::CIENCIA DA COMPUTACAO::METODOLOGIA E TECNICAS DA COMPUTACAO::SISTEMAS DE INFORMACAO
dc.titleReduzindo esforços manuais na análise de equivalência dos testes de mutação
dc.typeTrabalho de Conclusão de Curso

Arquivos

Pacote Original
Agora exibindo 1 - 1 de 1
Imagem de Miniatura
Nome:
TCC_Graduação_Samuel Veloso de Amorim.pdf
Tamanho:
1.51 MB
Formato:
Adobe Portable Document Format
Baixar
Licença do Pacote
Agora exibindo 1 - 1 de 1
Nenhuma Miniatura disponível
Nome:
license.txt
Tamanho:
1.66 KB
Formato:
Item-specific license agreed upon to submission
Descrição:
Baixar

Coleções

TCC Sistemas de Informação