Performance and Test, Spring 2007

Lecture [test 3]: Friday, 9 Mar, 10:00-12:00, room 2A14

Subject : Guided inspection, mock objects, interaction testing, coverage, a test plan example. Slides available here (after the lecture)

Text : [MS] Chapter 5 and 6. [Expected reading time 8 hours.] Some typos:
[MS, p. 116] line -2, "..the test coverage is 67%" (not 75% since 12/18 ~ 0.67)
[MS, p. 177] in the table on the middle of the page, third column, third row: should be "OUT.speed=100" (not 1000).

Comment : In the lecture this time we will talk about guided inspection (from Chapter 4) and move on to interaction testing described in Chapter 6. Chapter 5 describes how to semi-systematically construct test-cases from pre- and post-conditions as we talked about in the last lecture. We will talk about mock objects as a means for extending automatic unit testing towards automatic interaction testing. After this lecture we have now completed describing test methods for all phases in an iterative, incremental software development process.
We will show an example of a full test plan for a Brickles project using the methods.

Lab Exercises, Friday, March 9

The following warm-up exercises will not be discussed at the exercise session, but we recommend studying them before the session [expected time 2h]:

MS exercise 1.1 p. 14
MS exercise 2.3 p. 63
Exercise T.0

The following exercises will be discussed at the exercise session on March 9, 1-4pm in room 2A52:

(Optional) Make test suites using JUnit for your solutions to the programming contest
Exercises T.1-T.2 below
(Optional) exercise T.5
Exercises T.6-T.7
Exercise T.9

Exercise T.0

Consider the FifteenPuzzle as described in the lecture (link to slides available above). Write down a test suite of at least 5 test cases you would like to run on any implementation of FifteenPuzzle. The test suite should test for at least functionality and robustness.

Exercise T.1

Study this implementation of FifteenPuzzle. It implements the FifteenPuzzle as described in the slides from the lecture (link available above). Implement you test cases from T.0 as methods test1(),...,test5() in the class FifteenPuzzle. It is ok to check the outcome of the tests by manual inspection and not automatically by having code do it.

Run your tests on the implementation. Any surprises? Fix any bugs found. Rerun. How well would you say the code is now tested?

Exercise T.2

Write a unit test using the JUnit framework to implement the test suite from exercise T.1. Consider whether you have to make changes to the class implementation in order to make it possible to perform the tests you want. Run your tests from within Eclipse. Change the implementation of FifteenPuzzle so that you can make the tests fail one at a time. Run the tests and see them fail.

Exercise T.3

Extend you test suite to a test suite with at least 10 test cases covering all the major aspects of functionality and robustness. Run your tests from within Eclipse.

Exercise T.4

Void.

Exercise T.5

Set up your environment including classpath and installation of JUnit so that you can run unit tests from a command prompt. Do that on the tests from T.2 or T.3.

Exercise T.6

In the note Systematic software test, Peter Sestoft, 1998 examples of erroneous programs and tests for them are given. In this exercise we use example 1 in section 2.1, p.3.

Consider to what extent this program is ready for making automatic tests using JUnit. Would you have to make some changes? Which?
Write a test suite with JUnit executing the proposed test sequences. Run it. Detect the bug.
Fix the bug and rerun the tests to see that the program now passes all tests.

Exercise T.7

Proceed as in T.6 but with example 2, section 3.3, p. 5 from Sestoft's note.

Exercise T.8

Solve exercise T.6 using a mock object based on jMock to imitate the output stream System.out. Rewrite and rerun the tests. Hint: it is okay to first rewrite the code to use an interface class which is the easiest to mock using jMock.

Exercise T.9

In this exercise we will implement and test Quicksort using JUnit. It is described in RS Chapter 7.

Implement a class Quicksort with the methods quicksort (Program 7.1, p. 317) and partition (Program 7.2, p. 319). Use "int" for the type "ITEM". You will need to write the auxiliary function "exch(a, i, j)" which exchanges the contents of array a at the positions i and j.
Write test cases for testing "partition". Test for functionality and robustness. Make sure all branches in "partition" are exploited. Run them.
Write test cases for testing the method quicksort. Again, test for functionality and robustness.
(Performance) Write test cases to evaluate the performance of quicksort on large instances. First consider how you are going to generate large relevant instances. Then consider how you are going to measure performance (time?, space?). Run the tests. What is the asymptotic time behaviour? Does it fit expectations? How can you see this?

Performance and Test, Spring 2007 : Episode 6.2