Software Testing

Who ?

• Master Computer Science; Period: 2nd semester 2022-2023

Where ? When ?

• Theory
• Time: ...
• Location: ...
• Exercises (Lab Sessions)
• Time: ...
• Location: ...
• Industrial Guest Lectures
• Time: ...
• Location: ...
• Map

Prerequisites

Before starting this course a student must meet the following prerequisites.

• You have profound experience with programming in an object-oriented language (e.g. C++, Java). Advanced programming constructs (exceptions, threads) contain no secrets for you.
• You have hands-on experience with the testing of software (e.g. you wrote unit tests using XUnit).
• You are capable of reading and interpreting a design written down in the most commonly used design notations (UML, statecharts, ...)
• You can demonstrate deep knowledge concerning the formal foundations of computer science (logics, algorithms, complexity theory, finite automata)
• You can specify the pre- and post-conditions and the invariants for a given software module (cfr. design by contract).
• You are well aware of the various phases in a software project (requirements, analysis, design, implementation, testing, maintenance).
• You have knowledge about techniques for managing quality during a project (cost estimation, code reviews, metrics)

The easiest way to satisfy these prerequisites is to have a passing grade for the course Software Engineering (BA 3 Computer Science).

Course content

De student will acquire experience with thorough testing and verification of a software system, to guarantee with a certain degree of confidence that a given software system meets its specification.

The course has a practical ring to it with a minimal theoretical content (taught as testing best practices and involving self-training), several lab-sessions (trying out several test techniques and strategies on an existing software system) and industrial guest speakers illustrating how testing is done in real projects.

Objectives (expected learning outcomes)

After this course, a student will be able to

1. apply white-box and black-box test techniques to build a test-suite;
2. assess and improve the coverage of a test suite;
3. distinguish between various test automation strategies;
4. select the most appropriate test techniques for a given test strategy.

Practical Organisation (incl. Time Schedule)

• Theory
• The course starts with three introductory lectures by prof. Demeyer.
• Afterwards we switch to a regime of flipped class room, where mini-lectures or recorded on video and students should read provided background material and participate in an on-line quizz.
• Next, the course features a few invited speakers from a testing department / QA department in the software industry. They will share their experiences on testing practices in an industrial context.
• Exercises
• During the exercise session you will experiment with numerous test techniques and strategies on a small yet representative system, namely a simplified version of the famous pac-man game.

Below is the detailed time-schedule for the academic year 2021 - 2022. This will be adapted at the start of the semester.

 Theory Lectures

• Lecture 1 - Introduction
• Covers: Terminology / Risk / V-Model / RIPR / Triangle problem
• Lecture 2 - Test Design I
• Covers: Domain-Based Testing (Equivalence partitioning & Boundary value analysis) / Associations
• Lecture 3 - Test Design II
• Covers: Decision table testing / State Transition Testing / Scenario-Based Testing (Use Cases)

Lab Sessions Assignments

The lab sessions is the part of the course which requires the most work. We organized it such that most of the work can be done in the lab and that you only need to spend some time at work finishing the report. Block the necessary time in your agendas every week; there is a penalty (= extra work) for late submissions.

For the exercise session in the computer lab here is the course material:

• Assignment 1: Assignment - Resources
• Assignment 2: Assignment
• Assignment 3: Assignment
• Assignment 4: Assignment
• Assignment 5: Assignment
• Assignment 6: Assignment
• Assignment 7: Assignment
• Assignment 8: Assignment - Resources

Groups (These groups are created arbitrarily for Exercises 7 and 8 of Assignment 7.)

• Group 1
• Presentation Date: 6 May 2023
• Presenters: Dogukan Altay, Mert Can, Evelien De Groot, Luna Geens
• Opponents: Thimoty Smet, Cristian Tironi, Matteo Vedovati, Joshua Moelans
• Group 2
• Presentation Date: 13 May 2023
• Presenters: Maxim Gonnissen, Katsiaryna Makayed, Tim Mees, Joshua Moelans
• Opponents: Annila Munsaf, Anthony Peeters, Bjorn Rexhepi, Christoph Ronken
• Group 3
• Presentation Date: 20 May 2023
• Presenters: Annila Munsaf, Anthony Peeters, Bjorn Rexhepi, Christoph Ronken
• Opponents: Luna Geens, Maxim Gonnissen, Katsiaryna Makayed, Tim Mees
• Group 4
• Presentation Date: 27 May 2023
• Presenters: Thimoty Smet, Cristian Tironi, Matteo Vedovati
• Opponents: Dogukan Altay, Mert Can, Evelien De Groot

Course Material

The book Software Testing: A Craftsman’s Approach, Fourth Edition 4th Edition by Paul C. Jorgensen (Author) [https://anet.be/record/opacuantwerpen/c:lvd:15089379/N]  will be used heavily during the first lectures. Electronic copies are available via the library (VPN access required). Buying the book for the course is not necessary. Web-site for the 5th edition of this book(newer than the one we adopt): https://www.routledge.com/Software-Testing-A-Craftsmans-Approach-Fifth-Edition/Jorgensen-DeVries/p/book/9780367767624

The book Practical Test Design: Selection of traditional and automated test design techniques by István Forgács, Attila Kovács [https://anet.be/record/opacuantwerpen/c:lvd:15089355/N]  will be used for the later student lectures on Test Automation. Electronic copies are available via the library (VPN access required). Buying the book for the course is not necessary. Web-site for the book: https://www.oreilly.com/library/view/practical-test-design/9781780174723/

For copyright purposes, digital material used during the lectures will be distributed via blackboard

• Lecture 1 - Lecture 2 - Lecture 3
• Flipped class room (videos and background reading)

• Industrial Guest Lectures

Exam

The final grade will be based on four components; (a) and (b) will be assessed during the semester while (c) and (d) will be assessed during the exam.

• (a) Flipped class room:
• Each student will need to study the flipped class room material and participate in a small on-line quizz.
• You must hand in the answers to the quizz.
• (b) Lab-sessions:
• You must finish the lab sessions and hand in the solutions on time via blackboard.
• To be eligible for the final exam, you need to get at least 50/100 for each assignment.
• (c) Theory chapters:
• You will be questioned on one of the lectures (theory + flipped class room). During this questioning you may consult the books (a copy will be present during the exam), and your own notes (which you should bring yourself). A digital copy of all material will be availble.
• Pay special attention to elements like coverage; exit-entry criteria, strategy.
• (d) Guest lecture:
• You will be questioned on one of invited guest lectures (random selection). In particular, you should be able to summarize the main points of a given lecture, to relate the guest lecture to some of the chapters in the book(s), to relate the guest lecture to some of the test process principles and finally to relate the guest lecture to some of the test automation strategies.

 Copied Code - Code created by Generative AI

Reports written with support from generative AI

Exam Schedule

The exam is scheduled on Friday, June 9th. We will discuss for approximately 20 minutes about (c) a randomly selected chapter of one the theory lectures; (d) a randomly selected invited lecture.

You can use annotated copies of all material used during the lectures during the exam. For (c) the random selection is one of

1. Introduction
2. Test Design 1
3. Test Design 2
4. Test Automation: 41 Reasons
5. Test Automation: Setting Up
6. Exploiting Automated Tests

For (d) the random selection is one of

1. Andreas Faes (Delaware) - https://www.delaware.pro/
2. Nele Van Asch - Kevin Boutsen (CTG) - https://www.ctg.com
3. Bernard Sputh (Siemens) - https://www.siemens.be/
4. Geert-Jan Vandenbogaerde (Garvis) - https://www.garvis.ai/
5. Patrick Ruyskensveld (Colruytgroup) - https://www.colruytgroup.com
6. Andy Kellens (Corporify) - https://www.corporify.com/
7. Koen De Hondt (yesplan) - https://yesplan.be/
8. Tars Joris (Inventive Designers - OMP) - https://www.unifiedpost.com/ - https://omp.com/
9. Simon Temmerman — Brecht Arnou (dscribedata) - https://dscribedata.com
10. Jef Neefs (Barco - clickshare) - https://www.barco.com/

For (d) you should be able to summarize the main points of a given lecture and to relate the guest lecture to the relevant topics in (b) and the testing assignments you performed over the year.

Friday, June 9th 2023
Room CMI G106b (office Prof. Serge Demeyer) 			
09:00	09:20	Altay,Dogukan
09:20	09:40	Can,Mert
09:40	10:00	Geens,Luna
10:00	10:20	Gonnissen,Maxim
10:20	10:30	 -- break
10:30	10:50	Mees,Tim
10:50	11:10	Moelans,Joshua
11:10	11:30	Peeters,Anthony
11:30	11:50	Ronken,Christoph
11:50	12:00	 -- break
12:00	12:20	Smet,Thimoty
12:20	12:40	Tironi,Cristian
12:40	13:00	Vedovati,Matteo

Postponed to next exam session
	...