Software Testing

Professor: 
Serge Demeyer
Assistant: Ali Parsai

Who ?

  • Master Computer Science; Period: 2nd semester 2016-2017

Where ? When ?

  • Theory
    • Time: Monday 10:45 - 12:45
    • Location: CMI.G.016 (Campus Middelheim)
  • Exercises
    • Time: Friday 10:45 - 12:45
    • Location: CMI.G.026 (PC-lab; Campus Middelheim)

Prerequisites

Before starting this course a student should have:

  • Practical programming experience with an object-oriented programming language(e.g., Java, C++), preferably in a team context.
  • Basic awareness about the testing of software (unit tests, regression tests).
  • Knowledge about the various techniques used in the different phases a software project (requirements, analysis, design, implementation, testing, maintenance).
  • Knowledge about techniques for managing quality during a project (cost estimation, code reviews, metrics).

The easiest way to satisfy these prerequisites is to have a passing grade for the course Software Engineering (BA 3 Computer Science).

Course content

De student will acquire experience with thorough testing and verification of a software system, to guarantee with a certain degree of confidence that a given software system meets its specification.

The course has a practical ring to it with a minimal theoretical content (taught as testing patterns; students have to deliver some of the lectures themselves), several lab-sessions (trying out several test techniques and strategies on an existing software system) and industrial guest speakers illustrating how testing is done in real projects.

From the academic year 2009-2012 onward, this courses is offered every 2 years. It will not be offered in 2009-2010; 2011-2012; ... but will be offered in 2014-2015; 2016-2017; 2018-2019. The motivation for this organisation is the fact that this course explicitly invites industrial guest speakers; a feature well-appreciated by the fellow students. Offering the course every two years must guarantee that these industrial lectures attract sufficient participants.

 

Objectives (expected learning outcomes)

After this course, a student will be able to

  1. distinguish between various test strategies;
  2. select the most appropriate test techniques for a given test strategy;
  3. assess and improve the coverage of a test suite.

Practical Organisation (incl. Time Schedule)

Theory
The course starts with a few introductory lectures by prof. Demeyer. Afterwards each student will prepare part of a lecture based on a chapter in one of the following books: Testing Object-Oriented Systems. (Robert V. Binder); Addison-Wesley, 2000 or Agile Testing: A Practical Guide for Testers and Agile Teams (Lisa Crispin, Janet Gregory); Pearson-Education 2008). We will use peer review to rate the quality of the lectures using the following criteria [CriteriaSwTestingMAS.rtf].
At the end of the semester the course features a few invited speakers from a testing department / QA department in the software industry; they will share their experiences on testing practices in an industrial context.
Exercises
During the exercise session you will experiment with numerous test techniques and strategies on a small yet representative system, namely a simplified version of the famous pac-man game.

Below is the detailed time-schedule, which is subject to change. Changes will be notified over e-mail. [Last modified on: Monday, March, 27th 2017.]

1 13-Feb [T] Introduction (Ch. 3 en 4) 17-Feb   -- no lab
2 20-Feb [T] Test Models (Ch. 5 — 7) 24-Feb [P] Java testing tools
3

27-Feb

[S] Ch.10-11 (Classes - Reusable Components) 03-Mar [P] Category Partitioning and Boundary Values
4 06-Mar [G] Industrial guest lecture 10-Mar [P] Enforcing proper associations with asserts
5 13-Mar [G] Industrial guest lecture 17-Mar [P] Decision Structures
6 20-Mar [G] Industrial guest lecture 24-Mar [P] State Machines 
7 27-Mar [G] Industrial guest lecture 31-Mar  -- no lab
  03-Apr   -- easter holiday 07-Apr  -- easter holiday
  10-Apr   -- easter holiday 14-Apr  -- easter holiday
8 17-Apr   -- easter; no lectures 21-Apr [P] Mutation testing
9 24-Apr [G] Industrial guest lecture
(Room CMI G.016)
28-Apr [G] Industrial guest lecture
(Room CMI G.006)
10 01-May   -- no lecture holidays 05-May [S] Ch.12-13 (Subsystems - Integration)
(Room CMI G.015)
11 08-May [S] Ch.14-15 (Application Systems - Regression Testing)
(Room CMI G.016)
12-May [S] Test Automation / Agile Testing
(Room CMI G.015)
12 15-May   -- no lecture 19-May   -- no lecture
13 22-May  -- no lecture  26-May   -- no lecture

 

Industrial Guest Speakers

All lectures on monday 10:30-13:00

Monday 6 march 2017 - 10:30 to 12:45
	What is wrong with my Metrics ? Olivier Denoo (PS Testware)
	Context-Driven testing. Bert Jagers (RealDolmen)
Monday 13 march 2017 - 10:30 to 12:45
 	Agile & testing in the real world - Stefan Patry (Test Leaders)
 	A history of UI testing - Filip Van Rysselberghe & Soizic Launay (OMPartners)
Monday 20 march 2017 - 10:30 to 12:45
  	UI based testing (Past – Present - Future) - Jacques Wouters (JAMO Solutions)
  	Software Testing @ Inventive Designers - Tars Joris (Inventive Designers)
Monday 27 march 2017 - 10:30 to 12:45
  	Test Process Improvement - Geert Vanhove (Sogeti)
  	Virtual Banks at Credit Suisse - Daniel Künzle (Credit-Suisse)
  —-  easter holidays
Monday 24 april 2017 - 10:30 to 12:45
 	** 2 hour session 
 	Software Testing Hands on Session - Michaël Pilaeten (CTG)
Friday 28 april 2017  - 10:30 to 12:45
 	Test Execution Environments for Embedded Software Testing - Tom Philips (Nokia - BE)   

Student lectures

Lectures will be given by

Group 1: Mathias Beke, Kristof De Middelaer
Group 2: Jonas Vanden Branden, Jony Van Puymbroeck
Group 3: Sten Vercammen, Björn de Nijs, Gitte Bluekens
Group 4: Dominique de Heer, Lucas de Heer, Andrés Carrasco
 
  • Mon 27-Feb — [S] Testing Patterns — Ch.10-11 (Classes - Reusable Components)
    • Chapter 10 (Classes) - p. 347 - 522 [175 pages]
      • Includes long introduction covering important concepts (Small pop; Alpha - Omega cycle; Domain Analysis; ...)
    • Chapter 11 (Reusable Components) - p. 525 - 562 [37 pages]
      • Teachers: Group 1
      • Opponents: Group 3
  • Fri 05-May — [S] Testing Patterns — Ch.12-13 (Subsystems - Integration)
    • Chapter 12 (Subsystems) - p. 563 - 624 [61 pages]
    • Chapter 13 (Integration) - p. 627 - 715 [88 pages]
      • Teachers: Group 2
      • Opponents: Group 1
  • Mon 08-May — [S] Testing Patterns — Ch.14-15 (Application Systems - Regression Testing) 
    • Chapter 14 (Application Systems) - p. 715 - 753 [38 pages]
      • Also covers important material in non-pattern form (Configuration and Compatibility + Performance + Concurrency Testing + Stress testing + Human Computer Interaction + Alpha/Beta testing + Batch Systems + Interactive Systems + Real Time Systems)
    • Chapter 15 (Regression Testing) - p. 755 - 797 [42 pages]
      • Includes long intro + 5 patterns
      • Teachers: Group 4
      • Opponents: Group 2
  • Fri 12-May — [S] Agile Testing — Ch.6 + Ch.13 + Ch.14
    • Chapter 6 (The Purpose of Testing; The Agile Testing Quadrants) - p. 97 - 108 [11 pages]
    • Ch 13 (Why we want to Automate) - p. 257 - 272 [15 pages]
    • Ch 14 (Agile Test Automation Strategy) - p. 273 - 326 [53 pages]
      • Teachers: Group 3
      • Opponents: Group 4 
 

Assignments


For the exercise session in the computer lab here is the course material:

 

Course Material

The book Testing Object-Oriented Systems. will be used heavily during the sessions. Several copies are available in the library (see http://anet.ua.ac.be/record/opacua/c:lvd:3072797 and http://anet.be/record/opacuantwerpen/c:lvd:6506056/N). Buying the book for the course is not necessary.

Testing book cover

The book Agile Testing. A practical Guide for Testers and Agile Teams will be used for the last student lecture on Agile Testing and Test Automation.

  • Linda Crispin and Janet Gregory. Agile Testing. A practical Guide for Testers and Agile Teams. Addison-Wesley, 2009.
  • Web-site for the book: http://agiletester.ca/agile-testing/
Agile Testing Book Cover

Besides the book, there are the copies of the slides used during the lectures (of which the originals are available in keynote):

The presentation material from the industrial guest speakers

  • Invited lectures: Distributed via blackboard

 

 

Exam

The exam is organised as an oral round where you will be questioned for approximately 1/2 and hour on (a) the application of white-box and black-box test techniques to set-up a test suite; (b) the selection of the most appropriate test strategy.

The final grade will be based on four components; (a) and (b) will be assessed during the semester while (c) and (d) will be assessed during the exam.

  • (a) Student lecture:
    • Each student will teach one part of a lecture, and your colleagues will peer review your contribution based on a given criteria check-list.
    • The criteria check list can be found at [CriteriaSwTestingMAS.rtf]
  • (b) Lab-sessions:
    • You must finish the lab sessions and hand in the solutions on time via blackboard.
  • (c) Testing patterns:
    • You will be questioned on one of the chapters in the book (randomly selected, but not one which you taught yourself). During this questioning you may consult the book (a copy will be present during the exam), and your own notes (which you should bring yourself). A digital copy of all material will be availble.
    • Pay special attention to elements like coverage; exit-entry criteria
  • (d) Guest lecture:
    • You will be questioned on one of invited guest lectures (random selection). In particular, you should be able to summarize the main points of a given lecture, to relate the guest lecture to some of the patterns in the book, to relate the guest lecture to some of the test process principles and finally to relatethe guest lecture to some of the tooling principles in the book
 

Exam Schedule

The exam is scheduled for ??? in Prof. Demeyer's office (Room CMI.G.106b). We will discuss for approximately 1/2 an hour about (a) a randomly selected chapter of the software testing book (not one you presented yourself); (b) a randomly selected invited lecture.

You can bring annotated copies of all material used during the lectures. The material will be available in digital format as well.

 

The fine-grained schedule for the oral exam (20 minutes per student).

Thursday, June 4th - room CMI.G106b				
	9:30	10:00	...
	10:00	10:30	...
	10:30	11:00	...
	11:00	11:30	...
	11:30	12:00	...
	12:00	12:30	 -- break	
	12:30	13:00	 -- break	
	13:00	13:30	...
	13:30	14:00	...
	14:00	14:30	...
	14:30	15:00	...
	15:00	15:30	...
	15:30	16:00	 -- break	
	16:00	16:30	...
	16:30	17:00	...
	17:00	17:30	...
	17:30	18:00	...
	18:00	18:30	...
	18:30	19:00	...	
If names are missing from the above list, or if you spot any other mistakes, please contact prof. Serge Demeyer immediately.