AmPyfier (Test Amplification for Python)

Test Amplification

Software test amplification is the act of strengthening manually written test-cases to exercise the boundary conditions of the system under test. It has been demonstrated extensively by the research community to work for the programming language Java, relying on the type system to safely transform the code under test.

The AnSyMo research group, has created the AmPyfier tool for the Python language (a programming languages without a type system ).

Example

Consider the following unit test which excercises a class under test SmallBank, representing a bank with operations deposit and withdraw. This test is just a happy day scenario where line 02 deposits 10 eur (and then asserts that the balance is indeed 10 in line 3) and then depossits twice 100 eur (lines 04 and 05) to arrive at a final balance of 210 (line 6)

01     def testDeposit(self):
02         self.b.deposit(10)
03         self.assertEqual(self.b.get_balance(), 10)
04         self.b.deposit(100)
05         self.b.deposit(100)
06         self.assertEqual(self.b.get_balance(), 210)
 

AmPyfier takes this test and amplifies it by adding extra assertions (lines 09 - 12) that verifies all observable output on the object under test. It modifies the deposit call in line 04 with a very large negative integer which is expected to raise an exception (lines 13-14). The rest of the test proceeds as before, however now with extra assertions (lines 16-17)

07     def testDeposit_amplified(self):
08         self.b.deposit(10)
09         self.assertEqual(self.b.get_transactions(), [10])
10         self.assertFalse(self.b.is_empty())
11         self.assertEqual(self.b.owner, 'Iwena Kroka')
12         self.assertEqual(self.b.get_balance(), 10)
13         with self.assertRaises(Exception):
14             self.b.deposit(-56313)
15         self.b.deposit(100)
16         self.assertEqual(self.b.get_transactions(), [10, 100])
17         self.assertFalse(self.b.is_empty())
18         self.assertEqual(self.b.owner, 'Iwena Kroka')
19         self.assertEqual(self.b.get_balance(), 110)


Validation

We evaluated Ampyfier on 3 open-source projects in the Python ecosystem containing 5 test classes and approximately 75 test methods. We show an average improvement of 497.05% mutation score in 4 out of 5 test classes.

This illustrates that in AmPyfier really is capable of strengtening existing test suites.

Language

Python 3: AmPyfier is completely written in Python (PSF - https://docs.python.org/3/license.html)

Dependencies

AmPyfier depends on the following modules not included in the Python Standard Library

 

License

The tool is available under exclusive license. Interested individuals should contact Prof. Serge Demeyer for further details.