Software Testing

Professor: 
Serge Demeyer
Assistants: Mutlu Beyazıt, Onur Kilincceker

Who ?

  • Master Computer Science; Period: 2nd semester 2022-2023

Where ? When ?

  • Theory
    • Time: Monday 08:30 - 10:30
    • Location: CMI room G 017
  • Exercises (Lab Sessions)
    • Time: Monday 10:45 - 12:45
    • Location: CMI room G 026 (PC class)
  • Industrial Guest Lectures
    • Time: Monday 09:45 - 12:45
    • Location: CMI room G 015
  • Map

Annotated Map Campus Middelheim

Prerequisites

Before starting this course a student should have:

  • Practical programming experience with an object-oriented programming language (e.g., Java, C++), preferably in a team context.
  • Basic awareness about the testing of software (unit tests, regression tests).
  • Knowledge about the various techniques used in the different phases a software project (requirements, analysis, design, implementation, testing, maintenance).
  • Knowledge about techniques for managing quality during a project (cost estimation, code reviews, metrics).

The easiest way to satisfy these prerequisites is to have a passing grade for the course Software Engineering (BA 3 Computer Science).

Course content

De student will acquire experience with thorough testing and verification of a software system, to guarantee with a certain degree of confidence that a given software system meets its specification.

The course has a practical ring to it with a minimal theoretical content (taught as testing best practices and involving self-training), several lab-sessions (trying out several test techniques and strategies on an existing software system) and industrial guest speakers illustrating how testing is done in real projects.

From the academic year 2009-2012 onward, this courses is offered every 2 years. It will not be offered in 2009-2010; …; 2019-2020; 2021-2022; ... but will be offered in 2018-2019; 2020-2021; 2022-2023. The motivation for this 2 yearly schedule follows from the explicit invitation of industrial guest speakers; a feature well-appreciated by the fellow students. Offering the course every two years must guarantee that these industrial lectures attract sufficient participants.

 

Objectives (expected learning outcomes)

After this course, a student will be able to

  1. apply white-box and black-box test techniques to build a test-suite;
  2. assess and improve the coverage of a test suite.
  3. distinguish between various test automation strategies;
  4. select the most appropriate test techniques for a given test strategy.

Practical Organisation (incl. Time Schedule)

  • Theory
    • The course starts with three introductory lectures by prof. Demeyer.
    • Afterwards we switch to a regime of flipped class room, where mini-lectures or recorded on video and students should read provided background material and participate in an on-line quizz.
    • Next, the course features a few invited speakers from a testing department / QA department in the software industry. They will share their experiences on testing practices in an industrial context.
  • Exercises
    • During the exercise session you will experiment with numerous test techniques and strategies on a small yet representative system, namely a simplified version of the famous pac-man game.

Below is the detailed time-schedule, which is subject to change. Changes will be notified over e-mail. [Last modified on: Friday, February, 10th 2022.]

1 13-Feb [T] Introduction 13-Feb [P] Java testing tools
2 20-Feb [T] Test Design 1
(**Lecture starts at 9:00 instead of 8:30)
20-Feb [P] Category Partitioning and Boundary Values
3 27-Feb [T] Test Design 2
(**Lecture starts at 9:00 instead of 8:30)
27-Feb [P] Enforcing proper associations with asserts
4 06-Mar [F] Test Automation: 41 Reasons
(**No physical presence required)
06-Mar [P] Decision Structures
5 13-Mar [F] Test Automation: Setting Up
(**No physical presence required)
13-Mar [P] State Machines 
6 20-Mar [F] Test Automation: Test Architecture
(**No physical presence required)
20-Mar [P] Mutation testing
7 27-Mar [F] Exploiting Automated Tests
(**No physical presence required)
27-Mar [P] GUI Testing - Selenium
  03-Apr   -- easter holiday 03-Apr  -- easter holiday
  10-Apr   -- easter holiday 10-Apr  -- easter holiday
8 17-Apr  -- 17-Apr [P] Fuzzing
9 24-Apr [G] Industrial guest lecture 24-Apr -- no lab
10 01-May  -- holiday -- labour day 01-May  -- holiday -- labour day
11 08-May [G] Industrial guest lecture 08-May -- no lab
12 15-May [G] Industrial guest lecture 15-May -- no lab
13 22-May [G] Industrial guest lecture 22-May -- no lab

 Theory Lectures

  • Lecture 1 - Introduction
    • Covers: Terminology / Risk / V-Model / RIPR / Triangle problem
  • Lecture 2 - Test Design I
    • Covers: Domain-Based Testing (Equivalence partitioning & Boundary value analysis) / Associations
  • Lecture 3 - Test Design II
    • Covers: Decision table testing / State Transition Testing / Scenario-Based Testing (Use Cases)
 

Flipped class room Lectures

  • Lecture 4 - Test Automation: 41 Reasons
  • Lecture 5 - Test Automation: Setting Up
  • Lecture 6 - Test Automation: Test Architecture
  • Lecture 7 - Exploiting Automated Tests
 

Industrial Guest Lectures

Assignments


For the exercise session in the computer lab here is the course material:

 

Groups (These groups are created arbitrarily for Exercises 7 and 8 of Assignment 7.)

  • Group 1
    • Presentation Date: 6 May 2023
    • Presenters: Dogukan Altay, Mert Can, Evelien De Groot, Luna Geens
    • Opponents: Thimoty Smet, Cristian Tironi, Matteo Vedovati, Joshua Moelans
  • Group 2
    • Presentation Date: 13 May 2023
    • Presenters: Maxim Gonnissen, Katsiaryna Makayed, Tim Mees, Joshua Moelans
    • Opponents: Annila Munsaf, Anthony Peeters, Bjorn Rexhepi, Christoph Ronken
  • Group 3
    • Presentation Date: 20 May 2023
    • Presenters: Annila Munsaf, Anthony Peeters, Bjorn Rexhepi, Christoph Ronken
    • Opponents: Luna Geens, Maxim Gonnissen, Katsiaryna Makayed, Tim Mees
  • Group 4
    • Presentation Date: 27 May 2023
    • Presenters: Thimoty Smet, Cristian Tironi, Matteo Vedovati
    • Opponents: Dogukan Altay, Mert Can, Evelien De Groot

 

Course Material

The book Software Testing: A Craftsman’s Approach, Fourth Edition 4th Edition by Paul C. Jorgensen (Author) [https://anet.be/record/opacuantwerpen/c:lvd:15089379/N will be used heavily during the first lectures. Electronic copies are available via the library (VPN access required). Buying the book for the course is not necessary.

The book Practical Test Design: Selection of traditional and automated test design techniques by István Forgács, Attila Kovács [https://anet.be/record/opacuantwerpen/c:lvd:15089355/N]  will be used for the later student lectures on Test Automation. Electronic copies are available via the library (VPN access required). Buying the book for the course is not necessary.

Practical Test Design: Selection of traditional and automated test design techniques

For copyright purposes, digital material used during the lectures will be distributed via blackboard

  • Lecture 1 - Lecture 2 - Lecture 3
  • Flipped class room (videos and background reading)
  • Industrial Guest Lectures

 

 

Exam

The exam is organised as an oral round where you will be questioned for approximately 1/2 and hour on (a) the application of white-box and black-box test techniques to set-up an automated test suite; (b) the selection of the most appropriate test automation strategy.

The final grade will be based on four components; (a) and (b) will be assessed during the semester while (c) and (d) will be assessed during the exam.

  • (a) Flipped class room:
    • Each student will need to study the flipped class room material and participate in a small on-line quizz.
    • You must hand in the answers to the quizz.
  • (b) Lab-sessions:
    • You must finish the lab sessions and hand in the solutions on time via blackboard.
    • To be eligible for the final exam, you need to get at least 50/100 for each assignment.
  • (c) Theory chapters:
    • You will be questioned on one of the lectures (theory + flipped class room). During this questioning you may consult the books (a copy will be present during the exam), and your own notes (which you should bring yourself). A digital copy of all material will be availble.
    • Pay special attention to elements like coverage; exit-entry criteria, strategy.
  • (d) Guest lecture:
    • You will be questioned on one of invited guest lectures (random selection). In particular, you should be able to summarize the main points of a given lecture, to relate the guest lecture to some of the chapters in the book(s), to relate the guest lecture to some of the test process principles and finally to relate the guest lecture to some of the test automation strategies.
 

Exam Schedule

The exam is scheduled on Friday, June 9th. We will discuss for approximately 20 minutes about (c) a randomly selected chapter of one the theory lectures; (d) a randomly selected invited lecture.

You can use annotated copies of all material used during the lectures during the exam. For (c) the random selection is one of

  1. Introduction
  2. Test Design 1
  3. Test Design 2
  4. Test Automation: 41 Reasons
  5. Test Automation: Setting Up
  6. Exploiting Automated Tests

For (d) the random selection is one of

  1. Andreas Faes (Delaware) - https://www.delaware.pro/
  2. Nele Van Asch - Kevin Boutsen (CTG) - https://www.ctg.com
  3. Bernard Sputh (Siemens) - https://www.siemens.be/
  4. Geert-Jan Vandenbogaerde (Garvis) - https://www.garvis.ai/
  5. Patrick Ruyskensveld (Colruytgroup) - https://www.colruytgroup.com
  6. Andy Kellens (Corporify) - https://www.corporify.com/
  7. Koen De Hondt (yesplan) - https://yesplan.be/
  8. Tars Joris (Inventive Designers - OMP) - https://www.unifiedpost.com/ - https://omp.com/
  9. Simon Temmerman — Brecht Arnou (dscribedata) - https://dscribedata.com
  10. Jef Neefs (Barco - clickshare) - https://www.barco.com/

For (d) you should be able to summarize the main points of a given lecture and to relate the guest lecture to the relevant topics in (b) and the testing assignments you performed over the year.

 

The fine-grained schedule for the oral exam (20 minutes per student + 10 minutes breaks in between).

Friday, June 9th 2023
Room CMI G106b (office Prof. Serge Demeyer) 			
09:00	09:20	Altay,Dogukan
09:20	09:40	Can,Mert
09:40	10:00	Geens,Luna
10:00	10:20	Gonnissen,Maxim
10:20	10:30	 -- break
10:30	10:50	Mees,Tim
10:50	11:10	Moelans,Joshua
11:10	11:30	Peeters,Anthony
11:30	11:50	Ronken,Christoph
11:50	12:00	 -- break
12:00	12:20	Smet,Thimoty
12:20	12:40	Tironi,Cristian
12:40	13:00	Vedovati,Matteo

Postponed to next exam session
	...
If names are missing from the above list, or if you spot any other mistakes, please contact prof. Serge Demeyer immediately.