Fork me on GitHub

Getting started

To complete this workshop, you should have installed locally:

  • JDK 8

  • GIT

  • Docker 17-CE

  • Linux environment recommended

    • Using Docker for Windows and Docker for Mac should work as well, but we might struggle in helping with problems on these systems ;)

Generals experience in using the following technologies is expected. You can tag along if you don’t know all the details, but some topics might get too complex if you are missing the basics.

  • Java

  • JUnit4

  • Docker

Also, although not mandatory, it’s highly recommended to use IntelliJ IDEA, arguably the best IDE for Java and JVM language development. You can just use the Community Edition if you want to (which will miss Groovy support).

Documentation

During the course of this workshop, it’s highly likely you want to check out some details in the documentation of any of the used technologies. While the excercises mention relevant docs where necessary, here is a list of links:

Doing the Workshop

This workshop is structured into multiple excercises. Each excercise introduces a new concept and provides you with a task to solve by coding.

Although not mandatory, we do recommend you to try and work in pairs. This will allow you to get feedback for your ideas quickly, and also help you spot errors right away. Just keep in mind to switch positions at the keyboard every few minutes!

If you have any questions or get stuck, don’t hesitate to ask!

Excercises

Excercise 0: Setting up

You have just been assigned to an existing project. You and your team are responsible for creating an inventory tool for bookstores. It’s a Gradle project. You are told that most of the functionality has already been done. This sounds a bit suspicious: It’s never that easy, is it?

Goals

  • Checkout the project from GitHub.

  • Take a look at the codebase and notice the technologies used:

    • It’s a Gradle project

    • It has a JUnit integration test against a H2 in memory database

  • Run a quick Gradle build in order to check if everything works.

git clone git@github.com:kiview/testcontainers-groovy-integration-tests-workshop-workspace.git
cd testcontainers-groovy-integration-tests-workshop-workspace
./gradlew build

Excercise 1: Enter Testcontainers

You just saw that the existing tests passed. Just then, you are notified about a customer complaint: Apparently, they just integrated your application, but somewhere along the line, the program crashes. As you ask your team about it, you’re met with a classic response: "It works on my machine!". As you tell them the customer is using PostgreSQL, their expressions turn grim: The tests currently only run against a local H2 Database.

This situation is a perfect fit for Testcontainers. You can find an extensive documentation about using Testcontainers here, so we will just do a quick overview in this excercise.

Testcontainers is a Java 8 library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.

In JUnit4 tests, Testcontainers can be used by initializing Testcontainers objects as fields of the test class and by either annotating these fields with @Rule or @ClassRule JUnit4 annotation. @Rule means, the container will be restarted between each test method, @ClassRule will result in reusing the same container for all test methods of the test class. Testcontainers utilizes a fluent builder API for creating containers. Here are two examples:

// Set up a redis container
// @Rule restarts containers between test methods
@Rule
public GenericContainer redis =
    new GenericContainer("redis:3.0.2")
               .withExposedPorts(6379);


// Set up a plain OS container and customize environment,
// command and exposed ports. This just listens on port 80
// and always returns '42'
// @ClassRule starts container once before all test methods
@ClassRule
public static GenericContainer alpine =
    new GenericContainer("alpine:3.2")
            .withExposedPorts(80)
               .withEnv("MAGIC_NUMBER", "42")
               .withCommand("/bin/sh", "-c",
               "while true; do echo \"$MAGIC_NUMBER\" | nc -l -p 80; done");

A classic use case for integration testing is the persistence layer. An often found default approach is to use an embedded H2 in-memory database and run each test inside its own database transaction, which is rolled back after each test method. This might work for a lot use case, but sometimes it’s beneficial to use a real database for the integration tests, which will provide you with the extra level of confidence regarding vendor specific SQL features or more complex queries.

TestContainers provides out of the box support for the following databases:

  • MySQL

  • PostgreSQL

  • Oracle XE

  • Virtuoso

Specialized containers can simply be instantiated like this:

@Rule
public MySQLContainer mysql = new MySQLContainer();

We want our tests to be as portable as possible and so one shouldn’t make assumptions regarding the environment they are running in (like i.e. free ports). Luckily Testcontainers will already do all the heavy lifting for you and start the database on a free port (by leveraging the underlying container technology). By using methods like postgreSQLContainer.getJdbcUrl() it’s possible to get the concrete values a runtime. Of course, production code needs to be written in such a way that you can inject those values into the SUT at test runtime, i.e. specify such values in the constructor.

Goals

  • Take a good look at the examples and the online documentation for Testcontainers, especially the PostgreSQLContainer class.

  • Replace or extend the existing test. Make it use a real postgres database utilizing a PostgreSQLContainer.

    • Notice: This test has to fail - which is good! You successfully reproduced the customers' bug!

Excercise 1.1: A small fix

Your new teammates are amazed by how quickly you found that bug - and by writing a portable test, too! Your team’s database expert takes a quick look and provides you with a fix for the count method of the BookRepository. Apparently, some debug code was left over there in line 44 - you only need to remove it:

c.createStatement().execute("SET @foobar = 4");

Goals

  • Apply the fix.

  • Run the tests again.

  • Once the tests are green, reflect a moment on how quick you were in doing so.

Excercise 2: Use Spock

You just came back from a really good Groovy conference where you learned about Spock. You immediately want to convert your entire test suite.

Goals

  • Convert your test suite to Spock.

  • Run your Spock tests and make sure they are green!

Excercise 2.1: Use the Spock-Extension for Testcontainers

You probably feel already much more at ease using Spock, but somehow, the Testcontainer stuff does not seem to fit the newfound elegance now. Luckily, there is the Testcontainers Spock extension to make things even easier.

With that, we don’t need to add the @Rule or @ClassRule annotations to our containers anymore. The testcontainers-spock extension does a great job taking this off our hands, just make sure your Specification is annotated @Testcontainers. For recreating the behaviour of @ClassRule annotated contains, the static modifier is dropped and the annotation replaced with @Shared. This again makes our container persistent across all test methods:

@Testcontainers
class RedisTest extends Specification {

    @Shared
    // @Shared starts container once before all test methods
    public GenericContainer redis =
        new GenericContainer("redis:3.0.2")
                .withExposedPorts(6379);

    // ...
}

Goals

  • Enhance your testing suite further by using the features from the testcontainers-spock extension.

  • Run your tests and make sure they are green!

Excercise 3: Spawn the Database using the JDBC-URL

Someone in your team really misses the short and concise way they could connect to the H2 test database. Now is you chance to brighten their day!

As long as you have Testcontainers and the appropriate JDBC driver on your classpath, you can simply modify regular JDBC connection URLs to get a fresh containerized instance of the database each time your application starts up (meaning on initialization of the JDBC connection pool). This can be used as an alternative to the way we’ve seen in the last exercise.

jdbc:tc:postgresql://hostname/databasename

Goals

  • Try out this implicit way of spawning containers.

  • Run your new tests and make sure they are green!

Excercise 4: Interact with an external HTTP-Server

Now we want to think about testing the integration with a real external application. This could be anything which we’d be able to run inside a container, but in order to keep things simple, we have a very basic example: Downloading a file from an HTTP-Server.

Let’s start with a service class skeleton, which looks like this:

class HttpDownloaderService {

    private String serverIp

    private String serverPort

    HttpDownloaderService(String serverIp, String serverPort) {
        // ...
    }

    String downloadFile(String path) {
        // ...
    }
}

Generic Container

For this integration test, we want to use an Apache web server. Fortunately, there is a ready to use Docker image: httpd:alpine

Testcontainers provides a generic API for Docker images called GenericContainer. We also need to tell Testcontainers which port we want the container to expose and as before, Testcontainers will find use a free port on our host system and setup up the appropriate mapping.

We might also want to have some specific files on the server we can use for our tests and Testcontainers will allow us to mount files on the classpath into the container:

GenericContainer httpContainer = new GenericContainer("httpd:alpine")
            .withExposedPorts(80)
            .withClasspathResourceMapping("foo.txt", "/usr/local/apache2/htdocs/foo.txt", BindMode.READ_ONLY);

The GenericContainer interface also provides the methods to retrieve the actual container IP and port at runtime:

httpContainer.getContainerIpAddress();
httpContainer.getMappedPort(80);

Goals

Write an integration test as well as the corresponding production code to make the test green. You might want to use the wonderful new HttpBuilder-NG for the implementation code:

compile 'io.github.http-builder-ng:http-builder-ng-core:0.16.1'

(As an alternative solution, you can also check out the Groovy enhanced URL class.)

Excercise 5: Demo Functional Testing using Geb and Selenium

We’ve prepared an example we might want to look into:

git clone https://github.com/kiview/example-voting-app.git

BONUS Excercise: Testing your Docker containers with Testcontainers and Groovy scripts

Imagine we have a setup with a Nginx proxy redirecting to our service container (for the sake of simplicity we will simply use an Apache container for this example). The nginx config should go into a default.conf file and could look like this:

server {
    listen       80;
    server_name  localhost;
    rewrite ^/$ http://apache redirect;
}

Next create a groovy script file, i.e. test.groovy and insert the following boilerplate code:

#!/usr/bin/env groovy

@GrabResolver(name='jitpack', root='https://jitpack.io', m2Compatible='true')
@Grab(group='org.testcontainers', module='testcontainers', version='1.6.0')
@Grab('com.github.testcontainers:testcontainers-groovy-script:1.4.2')
@Grab(group='io.rest-assured', module='rest-assured', version='3.0.7', scope='test')
@GrabExclude('org.codehaus.groovy:groovy-xml')
@groovy.transform.BaseScript(TestcontainersScript)
import io.restassured.*
import org.testcontainers.containers.*
import org.junit.*
import static io.restassured.RestAssured.*
import static org.hamcrest.Matchers.*

You can think of a Groovy script as a Java main-class without the additional class syntax overhead. This means you can start writing your test code directly below the import statements.

You should look into withClasspathResourceMapping for your container setup (the nginx-config file needs to be mounted to /etc/nginx/conf.d/default.conf). Also you will need to wire your apache and nginx together into one docker network. Create a new Network instance and pass it to withNetwork(network) for this. If you want to leverage Docker’s name based DNS feature, you can define a network alias for your container using withNetworkAlias(alias).

Network testNetwork = newNetwork()

GenericContainer nginx = new GenericContainer("nginx:1.9.4")
        .withNetwork(testNetwork)

You can start containers manually by calling the start() method on the container object. You’ll also need an additional apache container (you can use the httpd:2.4 image for this exercise).

To get started with using RestAssured, refer to their Website. Set it up to refer to the nginx:

def myBaseUrl = // obtain the correct base url from your containers
RestAssured.baseURI = "http://myBaseUrl"

A simple RestAssured test will look like this:

when()
            .get("/somePath")
    .then()
            .statusCode(200)
            .body(containsString("foobarBaz"))

Of course setting up containers and the base url can be encapsulated in i.e. a @BeforeClass method, but for sake of simplicity, we can simply perform this stuff directly in a test method (annotated with @Test).

Acknowledgements