Software Testing

Professor: 
Serge Demeyer
Assistant: Ali Parsai

Who ?

  • Master Computer Science; Period: 2nd semester 2018-2019

Where ? When ?

  • Theory
    • Time: Monday 10:45 - 12:45
    • Location: CMI.G.016 (Campus Middelheim)
  • Exercises
    • Time: Monday 13:45 - 15:45
    • Location: CMI.G.025 (PC-lab; Campus Middelheim)

Prerequisites

Before starting this course a student should have:

  • Practical programming experience with an object-oriented programming language(e.g., Java, C++), preferably in a team context.
  • Basic awareness about the testing of software (unit tests, regression tests).
  • Knowledge about the various techniques used in the different phases a software project (requirements, analysis, design, implementation, testing, maintenance).
  • Knowledge about techniques for managing quality during a project (cost estimation, code reviews, metrics).

The easiest way to satisfy these prerequisites is to have a passing grade for the course Software Engineering (BA 3 Computer Science).

Course content

De student will acquire experience with thorough testing and verification of a software system, to guarantee with a certain degree of confidence that a given software system meets its specification.

The course has a practical ring to it with a minimal theoretical content (taught as testing patterns; students have to deliver some of the lectures themselves), several lab-sessions (trying out several test techniques and strategies on an existing software system) and industrial guest speakers illustrating how testing is done in real projects.

From the academic year 2009-2012 onward, this courses is offered every 2 years. It will not be offered in 2009-2010; 2011-2012; ... but will be offered in 2014-2015; 2016-2017; 2018-2019. The motivation for this 2 yearly schedule follows from the explicit invitation of industrial guest speakers; a feature well-appreciated by the fellow students. Offering the course every two years must guarantee that these industrial lectures attract sufficient participants.

 

Objectives (expected learning outcomes)

After this course, a student will be able to

  1. distinguish between various test strategies;
  2. select the most appropriate test techniques for a given test strategy;
  3. assess and improve the coverage of a test suite.

Practical Organisation (incl. Time Schedule)

Theory
The course starts with a few introductory lectures by prof. Demeyer.
Next, the course features a few invited speakers from a testing department / QA department in the software industry; they will share their experiences on testing practices in an industrial context.
Afterwards each student will prepare part of a lecture based on a chapter in one of the following books: Testing Object-Oriented Systems. (Robert V. Binder); Addison-Wesley, 2000 or Agile Testing: A Practical Guide for Testers and Agile Teams (Lisa Crispin, Janet Gregory); Pearson-Education 2008). We will use peer review to rate the quality of the lectures using the following criteria [CriteriaSwTestingMAS.rtf].
Exercises
During the exercise session you will experiment with numerous test techniques and strategies on a small yet representative system, namely a simplified version of the famous pac-man game.

Below is the detailed time-schedule, which is subject to change. Changes will be notified over e-mail. [Last modified on: Monday, February, 25th 2019.]

1 11-Feb [T] Introduction (Ch. 3 en 4) 11-Feb [P] Java testing tools
2 18-Feb [T] Test Models (Ch. 5 — 7) 18-Feb [P] Category Partitioning and Boundary Values
3 25-Feb [G] Industrial guest lecture (**Cancelled) 25-Feb [P] Enforcing proper associations with asserts
4 04-Mar [G] Industrial guest lecture 04-Mar [P] Decision Structures
5 11-Mar [G] Industrial guest lecture 11-Mar [P] State Machines 
6 18-Mar [G] Industrial guest lecture 18-Mar [P] Mutation testing
7 25-Mar [G] Industrial guest lecture 25-Mar [P] GUI Testing (Selenium)
8 01-Apr [G] Industrial guest lecture 01-Apr  -- no lab
  08-Apr   -- easter holiday 08-Apr  -- easter holiday
  15-Apr   -- easter holiday 15-Apr  -- easter holiday
9 22-Apr  -- easter; no lectures 22-Apr  -- easter; no lectures
10 29-Apr [S] Ch.10-11 (Classes - Reusable Components) 29-Apr -- no lab
11 06-May [S] Ch.12-13 (Subsystems - Integration) 06-May -- no lab
12 13-May [S] Ch.14-15 (Application Systems - Regression Testing) 13-May -- no lab
13 20-May

[S] Continuous Integration / Test Automation / Agile Testing

[G] Industrial Guest Lecture with Hands On Session (Lego Mindstorms)

20-May

**Room CMI G.006**

[G] Industrial Guest Lecture with Hands On Session (Lego Mindstorms)

 

Industrial Guest Speakers

All lectures on monday 10:30-13:00

25-Feb	[G] Industrial guest lecture
 - cancelled
04-Mar	[G] Industrial guest lecture
 - Jacques Wouters (Jamo Solutions)
 - Bert Jagers (RealDolmen)
11-Mar	[G] Industrial guest lecture
 - Jef Neefs (Barco)
 - ...
18-Mar	[G] Industrial guest lecture
 - Tars Joris (Inventive Designers)
 - Geert Vanhove (Sogeti)

25-Mar	[G] Industrial guest lecture
 - Andy Kellens (Yesplan)
 - Emanuel Dejonghe (Delaware)
1-Apr	[G] Industrial guest lecture
 - Sigrid Eldh + Kristian Wiklund (Ericsson, Sweden)

20-May [G]  Industrial guest lecture
 - Wouter Emael en Michael Pilaeten (CTG)

Student lectures

Lectures will be given by (Groups to be confirmed)

  • Group 1: Claessens,Robbe; Geeraert,Lander
  • Group 2: Ismailaj,Kleidi; Lu,Zhong-Xi
  • Group 3: Paredis,Randy; Parezys,Jordan; Van Muylder,Ben
  • Group 4: xxx
 
  • Mon 29-Apr — [S] Testing Patterns — Ch.10-11 (Classes - Reusable Components)
    • Chapter 10 (Classes) - p. 347 - 522 [175 pages]
      • Includes long introduction covering important concepts (Small pop; Alpha - Omega cycle; Domain Analysis; ...)
    • Chapter 11 (Reusable Components) - p. 525 - 562 [37 pages]
      • Teachers: Group 1
      • Opponents: Group 2
  • Mon 06-May — [S] Testing Patterns — Ch.12-13 (Subsystems - Integration)
    • Chapter 12 (Subsystems) - p. 563 - 624 [61 pages]
    • Chapter 13 (Integration) - p. 627 - 715 [88 pages]
      • Teachers: Group 2
      • Opponents: Group 3
  • Mon 13-May — [S] Testing Patterns — Ch.14-15 (Application Systems - Regression Testing) 
    • Chapter 14 (Application Systems) - p. 715 - 753 [38 pages]
      • Also covers important material in non-pattern form (Configuration and Compatibility + Performance + Concurrency Testing + Stress testing + Human Computer Interaction + Alpha/Beta testing + Batch Systems + Interactive Systems + Real Time Systems)
    • Chapter 15 (Regression Testing) - p. 755 - 797 [42 pages]
      • Includes long intro + 5 patterns
      • Teachers: Group 3
      • Opponents: Group 1
  • Fri 20-May — [S] Continuous Integration / Test Automation / Agile Testing
      • … 
 

Assignments


For the exercise session in the computer lab here is the course material:

 

Course Material

The book Testing Object-Oriented Systems. will be used heavily during the sessions. Several copies are available in the library (see http://anet.ua.ac.be/record/opacua/c:lvd:3072797 and http://anet.be/record/opacuantwerpen/c:lvd:6506056/N). Buying the book for the course is not necessary.

Testing book cover

(To be confirmed) The book Agile Testing. A practical Guide for Testers and Agile Teams will be used for the last student lecture on Agile Testing and Test Automation.

  • Linda Crispin and Janet Gregory. Agile Testing. A practical Guide for Testers and Agile Teams. Addison-Wesley, 2009.
  • Web-site for the book: http://agiletester.ca/agile-testing/
Agile Testing Book Cover

Besides the book, there are the copies of the slides used during the lectures (of which the originals are available in keynote):

The presentation material from the industrial guest speakers

  • Invited lectures: Distributed via blackboard

 

 

Exam

The exam is organised as an oral round where you will be questioned for approximately 1/2 and hour on (a) the application of white-box and black-box test techniques to set-up a test suite; (b) the selection of the most appropriate test strategy.

The final grade will be based on four components; (a) and (b) will be assessed during the semester while (c) and (d) will be assessed during the exam.

  • (a) Student lecture:
    • Each student will teach one part of a lecture, and your colleagues will peer review your contribution based on a given criteria check-list.
    • The criteria check list can be found at [CriteriaSwTestingMAS.rtf]
  • (b) Lab-sessions:
    • You must finish the lab sessions and hand in the solutions on time via blackboard.
    • To be eligible for the final exam, you need to get at least 50/100 for each assignment.
  • (c) Testing patterns:
    • You will be questioned on one of the chapters in the book (randomly selected, but not one which you taught yourself). During this questioning you may consult the book (a copy will be present during the exam), and your own notes (which you should bring yourself). A digital copy of all material will be availble.
    • Pay special attention to elements like coverage; exit-entry criteria
  • (d) Guest lecture:
    • You will be questioned on one of invited guest lectures (random selection). In particular, you should be able to summarize the main points of a given lecture, to relate the guest lecture to some of the patterns in the book, to relate the guest lecture to some of the test process principles and finally to relatethe guest lecture to some of the tooling principles in the book
 

Exam Schedule

The exam is scheduled on Friday, June 21st 2019 in Prof. Demeyer's office (Room CMI.G.106b). We will discuss for approximately 20 minutes about (a) a randomly selected chapter of the software testing book (not one you presented yourself); (b) a randomly selected invited lecture.

You can bring annotated copies of all material used during the lectures. The material will be available in digital format as well.

The fine-grained schedule for the oral exam (20 minutes per student).

Friday, June 21st 2019 - room CMI.G106b				
	10:00	10:20	Claessens,Robbe
	10:20	10:40	Geeraert,Lander
	10:40	11:00	Ismailaj,Kleidi
	11:00	11:20	Lu,Zhong-Xi
	11:20	11:40	Paredis,Randy
	11:40	12:00	Parezys,Jordan	
	12:00	12:20	Van Muylder,Ben
If names are missing from the above list, or if you spot any other mistakes, please contact prof. Serge Demeyer immediately.