Software Testing

Professor: 
Serge Demeyer
Assistants: Sten Vercammen, Mehrdad Abdi

Who ?

  • Master Computer Science; Period: 2nd semester 2020-2021

Where ? When ?

  • Theory
    • Time: Monday 10:45 - 12:45
    • Location: Virtual (Blackboard Collaborate)
  • Exercises (Lab Sessions)
    • Time: Thursday 08:30 - 10:30
    • Location: Virtual (Blackboard Collaborate)

Prerequisites

Before starting this course a student should have:

  • Practical programming experience with an object-oriented programming language(e.g., Java, C++), preferably in a team context.
  • Basic awareness about the testing of software (unit tests, regression tests).
  • Knowledge about the various techniques used in the different phases a software project (requirements, analysis, design, implementation, testing, maintenance).
  • Knowledge about techniques for managing quality during a project (cost estimation, code reviews, metrics).

The easiest way to satisfy these prerequisites is to have a passing grade for the course Software Engineering (BA 3 Computer Science).

Course content

De student will acquire experience with thorough testing and verification of a software system, to guarantee with a certain degree of confidence that a given software system meets its specification.

The course has a practical ring to it with a minimal theoretical content (taught as testing best practices; students have to deliver some of the lectures themselves), several lab-sessions (trying out several test techniques and strategies on an existing software system) and industrial guest speakers illustrating how testing is done in real projects.

From the academic year 2009-2012 onward, this courses is offered every 2 years. It will not be offered in 2009-2010; …; 2019-2020; 2021-2022; ... but will be offered in 2018-2019; 2020-2021; 2022-2021. The motivation for this 2 yearly schedule follows from the explicit invitation of industrial guest speakers; a feature well-appreciated by the fellow students. Offering the course every two years must guarantee that these industrial lectures attract sufficient participants.

 

Objectives (expected learning outcomes)

After this course, a student will be able to

  1. distinguish between various test strategies;
  2. select the most appropriate test techniques for a given test strategy;
  3. assess and improve the coverage of a test suite.

Practical Organisation (incl. Time Schedule)

  • Theory
    • The course starts with a few introductory lectures by prof. Demeyer.
    • Afterwards each student will prepare part of a lecture based on a chapter in one of the following books:
    • Next, the course features a few invited speakers from a testing department / QA department in the software industry. They will share their experiences on testing practices in an industrial context.
  • Exercises
    • During the exercise session you will experiment with numerous test techniques and strategies on a small yet representative system, namely a simplified version of the famous pac-man game.

Below is the detailed time-schedule, which is subject to change. Changes will be notified over e-mail. [Last modified on: Saturday, February, 9th 2019.]

1 08-Feb [T] Introduction (Serge Demeyer) 11-Feb [P] Java testing tools
2 15-Feb [T] Test Design 1 (Serge Demeyer) 18-Feb [P] Category Partitioning and Boundary Values
3 22-Feb [T] Test Design 2 (Serge Demeyer) 25-Feb [P] Enforcing proper associations with asserts
4 01-Mar  --  04-Mar [P] Decision Structures
5 08-Mar [S] Student Lecture
Integration Testing / System Testing 
11-Mar [P] State Machines 
6 15-Mar

[S] Student Lecture
Object-Oriented Testing

18-Mar [P] Mutation testing
7 22-Mar [S] Student Lecture
Model Based testing 
25-Mar [P] GUI Testing - Selenium
8 29-Mar [S] Student Lecture
Manual and Automated Testing 
01-Apr [P] Fuzzing
  05-Apr   -- easter holiday 08-Apr  -- easter holiday
  12-Apr   -- easter holiday 15-Apr  -- easter holiday
9 19-Apr  -- easter; no lectures 22-Apr  -- easter; no lectures
10 26-Apr [G] Industrial guest lecture 29-Apr -- no lab
11 03-May [G] Industrial guest lecture 06-May -- no lab
12 10-May [G] Industrial guest lecture 13-May -- no lab
13 17-May [G] Industrial guest lecture 30-May -- no lab

 Theory Lectures

  • Lecture 1 - Introduction
    • Covers: Terminology / Risk / V-Model / RIPR / Triangle problem
  • Lecture 2 - Test Design I
    • Covers: Domain-Based Testing (Equivalence partitioning & Boundary value analysis) / Associations
  • Lecture 3 - Test Design II
    • Covers: State Transition Testing / Decision table testing / Scenario-Based Testing (Use Cases)

 

Student lectures

Lectures will be given by (Groups to be confirmed)

  • Group 1: De Sagher Freek; EliasSylvain
  • Group 2: Kisaakye Joanna;Latonas,Haroldas
  • Group 3: Marynen Wannes; Ndzenyuy Diboma Henry; Schittekat Igor
  • Group 4: Schoofs Ebert; Van Roy Lars
 
  • Mon 8-Mar Mon 1-Mar — [S] Lecture 4 (Student) - Integration Testing / System Testing
      • (Software Testing: A Craftman's approach - Chapters 13 - 14 [pp. 229 - 292 = 63 pages])
      • Covers: Decomposition-Based Integration / Call Graph–Based Integration / Path-Based Integration (Integration Testing) Threads / Use Cases / Operational Profiles / Stress testing (System Testing)
      • Teachers: Group 1
      • Opponents: Group 2
  • Mon 15-Mar Mon 8-Mar — [S] Lecture 5 (Student) - Object-Oriented Testing
      • Software Testing: A Craftman's approach - Chapter 15 [pp. 295 - 330 = 35 pages pages]
      • Covers: Issues / Unit Testing / Integration Testing / System Testing
      • Teachers: Group 2
      • Opponents: Group 3
  • Mon 22-Mar — [S] Lecture 6 (Student) - Model Based testing
      • Practical Test Design - Chapters 11 - 12 [pp. 217 - 255 = 38 pages]
      • Covers: Modelling languages / Test generation
      • Software Testing: A Craftman's approach - Chapter 17 [pp. 349 - 367 = 18 pages]
      • Covers: Systems of Systems based on SysML notation
      • Teachers: Group 3
      • Opponents: Group 4
    • Mon 29-Mar — [S] Lecture 7 (Student) - Manual and Automated Testing
      • Software Testing: A Craftman's approach - Chapters 18 - 19 - 21 [pp. 369 - 390 / 407 - 415 = 29 pages]
      • Covers: Exploratory Testing / Test-Driven Development / Mutation Testing / Test Cases
      • Teachers: Group 4
      • Opponents: Group 1
 

Industrial Guest Lectures

  • 26-Apr [G] Industrial guest lecture
  • 03-May [G] Industrial guest lecture
    • Sigrid Eldh - Ericsson, Sweden
    • Kristian Wiklund Ericsson, Sweden
    • Andy Kellens - Corporify en Yesplan (test automation in start ups)
  • 10-May [G] Industrial guest lecture
  • 17-May [G] Industrial guest lecture

Assignments


For the exercise session in the computer lab here is the course material:

 

Course Material

The book Software Testing: A Craftsman’s Approach, Fourth Edition 4th Edition by Paul C. Jorgensen (Author) [https://anet.be/record/opacuantwerpen/c:lvd:15089379/N will be used heavily during the first lectures. Electronic copies are available via the library (VPN access required). Buying the book for the course is not necessary.

The book Practical Test Design: Selection of traditional and automated test design techniques by István Forgács, Attila Kovács [https://anet.be/record/opacuantwerpen/c:lvd:15089355/N]  will be used for the later student lectures on Test Automation. Electronic copies are available via the library (VPN access required). Buying the book for the course is not necessary.

Practical Test Design: Selection of traditional and automated test design techniques

For copyright purposes, the copies of the slides used during the lectures will be distributed via blackboard

  • Lecture 1 - Lecture 2 - Lecture 3
  • Student lectures
  • Industrial Guest Lectures

 

 

Exam

The exam is organised as an oral round where you will be questioned for approximately 1/2 and hour on (a) the application of white-box and black-box test techniques to set-up a test suite; (b) the selection of the most appropriate test strategy.

The final grade will be based on four components; (a) and (b) will be assessed during the semester while (c) and (d) will be assessed during the exam.

  • (a) Student lecture:
    • Each student will teach one part of a lecture, and your colleagues will peer review your contribution based on a given criteria check-list.
    • The criteria check list can be found at [CriteriaSwTestingMAS.rtf]
  • (b) Lab-sessions:
    • You must finish the lab sessions and hand in the solutions on time via blackboard.
    • To be eligible for the final exam, you need to get at least 50/100 for each assignment.
  • (c) Theory chapters:
    • You will be questioned on one of the lectures (randomly selected, but not one which you taught yourself). During this questioning you may consult the books (a copy will be present during the exam), and your own notes (which you should bring yourself). A digital copy of all material will be availble.
    • Pay special attention to elements like coverage; exit-entry criteria
  • (d) Guest lecture:
    • You will be questioned on one of invited guest lectures (random selection). In particular, you should be able to summarize the main points of a given lecture, to relate the guest lecture to some of the chapters in the book(s), to relate the guest lecture to some of the test process principles and finally to relate the guest lecture to some of the tooling principles in the book
 

Exam Schedule

The exam is scheduled on XXdate to be confirmedxX in Prof. Demeyer's office (Room CMI.G.106b). We will discuss for approximately 20 minutes about (a) a randomly selected chapter of the software testing book (not one you presented yourself); (b) a randomly selected invited lecture.

You can bring annotated copies of all material used during the lectures. The material will be available in digital format as well.

 

The fine-grained schedule for the oral exam (20 minutes per student).

xxx, June xxx 2019 - room CMI.G106b				
	09:30	09:50	xxx
	09:50	10:10	xxx
	10:10	10:30	xxx
	10:30	10:50	xxx
	10:50	11:10	xxx
	11:10	11:30	xxx	
	11:30	11:50	xxx	
	11:50	12:10	xxx
	12:10	12:30	xxx
	12:30	12:50	xxx
If names are missing from the above list, or if you spot any other mistakes, please contact prof. Serge Demeyer immediately.