Connect SDK Integration Test Suite

Home
Essential Knowledge
By Type
Building Blocks
APIs
Releases

Feature

Changed on:

17 Oct 2023

Overview

There are two ways to perform integration tests. The first involves creating the tests within the same project as your custom connector code. Alternatively, if your custom connector logic is in a separate module from your main class, you can create integration tests for that module instead.

When your integration tests are in module that doesn't have a main class, then you need to have a class like below in your src/test/java folder to allow Springboot tests to start the application for the integration tests.

Detailed Technical Description

1@EnableScheduling
2@SpringBootApplication
3public class ConnectorApplication {
4    /**
5     * Main method to start the connector
6     * @param args no arguments expected
7     */
8    public static void main(final String[] args) {
9        SpringApplication.run(ConnectorApplication.class, args);
10    }
11}

Language: java

Name: Example

Description:

[Warning: empty required content area]

It is also required to create the necessary configuration files on your src/test/resources folder.

application.yml
application-connector.yml
bootstrap.yml

If you want logging, you will have to add a

`logback-spring.xml`

file for the spring profile

`it`

Integration Tests Modules

Connect SDK includes test modules to assist with integration tests:

connect-sdk-test-core: Connect SDK base test utilities
connect-sdk-test-core-aws: Extension of the SDK test utilities for projects using AWS
connect-sdk-test-core-kafka: Extension of the SDK test utilities for projects using Kafka

Creating Integration Tests

The first step is to create the base integration test class where all dependencies are wired and set up for all of your integration tests. It also has some helper methods that can be used in your initialization of tests to set up the correct queues and secrets as well as wiremock stubs.

The base test class may or may not have a specific spring profile scope. It entirely depends on the integration test and coverage required.

Without Spring Profiles

In this approach, all tests implemented using this base class are not bound to run with AWS for example. To control what

`MessageSender`

the base class is using in the example below, it requires one to specify the spring profiles at the time of the execution of the tests. For modules that support multiple implementations, it may be required to run integration tests as many times as required to cover all the different ways it can be used. For example, the Connect SDK can work with both Kafka and SQS, and it requires running integration tests to run against both profiles.

1# To run the integration tests using SQS
2mvn clean integration-test -Dspring.profiles.active=connectors,it,aws,localstack
3
4AND/OR
5
6# To run the integration tests using Kafka 
7mvn clean integration-test -Dspring.profiles.active=connectors,it,kafka

Language: plain_text

Name: Example

Description:

[Warning: empty required content area]

Spring profile will ensure the correct bean of

`MessageSender`

is available to the test class.

1@Slf4j
2@Feature("Commerce Tools Integration Tests")
3@Testcontainers
4public abstract class CTBaseIntegrationTest extends BaseIntegrationTest {
5
6    @Autowired
7    protected MessageSender messageSender;
8
9    @Autowired
10    private SecretManagerContainerSetupService secretManagerContainerSetupService;
11    ...

Language: java

Name: Example

Description:

[Warning: empty required content area]

With Spring Profiles

Whenever the profile is specified, the spring profile is preset and there is no need to specify them at the time of the execution of the tests.

1mvn clean integration-test

Language: plain_text

Name: commands

Description:

[Warning: empty required content area]

1@Slf4j 
2@Feature("AWS specific Integration Tests")
3@Testcontainers
4@SpringBootTest(properties = "spring.profiles.active=connector,it,aws,localstack")
5public class AwsBaseIntegrationTest extends BaseIntegrationTest {
6
7    @Autowired
8    protected MessageSender messageSender;
9
10    @Autowired
11    private SecretManagerContainerSetupService secretManagerContainerSetupService;
12    ...

Language: java

Name: Example

Description:

[Warning: empty required content area]

Commonly Used Spring Profiles

connector: connector configurations (routes and other fluent connect settings)
it: any configuration override for the integration tests
aws: any aws specific configuration as well as enabling the correct libraries to be used for AWS tests
localstack: by adding this profile the SDK will auto-create queues in SQS
kafka: any aws specific configuration as well as enabling the correct libraries to be used for Kafka tests

Integration Test Example

Here is a full example of a base integration test class:

1@Slf4j 
2@Feature("AWS Integration Tests")
3@Testcontainers
4@SpringBootTest(properties = "spring.profiles.active=connector,it,aws,localstack")
5public class AwsBaseIntegrationTest extends BaseIntegrationTest {
6    public static final String EVENTS = "events";
7    public static final String BATCH = "batch";
8    private static final String SQS_BATCH = "SQS_BATCH";
9    private static final String SQS_EVENTS = "SQS_EVENTS";
10    @Autowired
11    protected MessageSender messageSender;
12    @Autowired
13    private SecretManagerContainerSetupService secretManagerContainerSetupService;
14
15    @BeforeAll
16    static void setup() {
17        //sets up the minimal required queues for the SDK, more can be added if necessary 
18        System.setProperty(SQS_BATCH, BATCH);
19        System.setProperty(SQS_EVENTS, EVENTS);
20        startWiremockServer();
21    }
22
23    @AfterAll
24    static void tearDown() {
25        fluentWireMockServer.stop();
26    }
27
28    @BeforeEach
29    void testSetup() {
30        baseSetup(fluentWireMockServer);
31    }
32
33    @AfterEach
34    void afterTest() {
35        fluentWireMockServer.findAll(RequestPatternBuilder.allRequests()).forEach(request -> log.info(request.getAbsoluteUrl() + " " + request.getBodyAsString()));
36        fluentWireMockServer.resetAll();
37    }
38
39    @PostConstruct
40    private void setupSecretManager() throws IOException, InterruptedException {
41        //Sets up all required secrets used by your tests
42        secretManagerContainerSetupService.createSecret(
43                "fc/connect/testconnector/api/fluent/activeAccounts",
44                "{\"accounts\":[{\"name\":\"test\", \"retailers\":[1]}]}"
45        );
46        secretManagerContainerSetupService.createSecret(
47                "fc/connect/testconnector/TEST/api/fluent-account/1",
48                "{\"retailer\":\"1\", \"userName\":\"username\", \"password\":\"password\"}"
49        );
50    }
51}

Language: java

Name: Example

Description:

[Warning: empty required content area]

Once you have done the above, you should be able to start writing tests. Any integration test class should extend the base test class created above.

1public class EventQueueIntegrationTest extends AwsBaseIntegrationTest {
2    private static final String EVENTS_DLQ = EVENTS + "-dlq";
3
4    @Autowired
5    private AmazonSQSAsync amazonSQSAsync;
6
7    @Test
8    void shouldPublishInvalidMessageToDlq() {
9        // when: publishing an invalid message to the events queue
10        messageSender.send(EVENTS, "hello");
11
12        // then: the message should be published to the dlq as thera are no handlers to process it
13        given()
14                .await()
15                .atMost(Duration.ofSeconds(2))
16                .untilAsserted(() -> Assertions.assertEquals(1, receiveMessagesFromQueue(EVENTS_DLQ, 5).size()));
17    }
18
19    private List<Message> receiveMessagesFromQueue(final String queue, final int messageNumber) {
20        final ReceiveMessageRequest request = new ReceiveMessageRequest(queue);
21        request.setVisibilityTimeout(2);
22        request.setWaitTimeSeconds(1);
23        request.setMaxNumberOfMessages(messageNumber);
24        return amazonSQSAsync.receiveMessage(request).getMessages();
25    }
26}

Language: java

Name: Example

Description:

[Warning: empty required content area]

If you want to test any handler logic you may need to setup stubs and assert that wiremock is called. Some basic utility functions to assist with that are provided. Below is an example of how you might do this.

1import static com.fluentcommerce.connect.common.utils.IntegrationTestUtils.FLUENT_GRAPHQL_URL;
2import static com.fluentcommerce.connect.common.utils.IntegrationTestUtils.assertURLCalledAndBodyMatchesJson;
3import static com.fluentcommerce.connect.common.utils.IntegrationTestUtils.getResourceAsString;
4import static com.fluentcommerce.connect.common.utils.IntegrationTestUtils.setupGraphQL;
5import static com.fluentcommerce.connect.common.utils.IntegrationTestUtils.setupURL;

Language: java

Name: Example

Description:

[Warning: empty required content area]

1    @Test
2    @DisplayName("Should create a customer and send to Fluent")
3    void shouldCreateCustomerInFluent() {
4        // given: a customer does not exist in Fluent
5        setupGraphQL(
6                fluentWireMockServer,
7                ".*GetCustomer.*",
8                "it/customer/fc/graphql/get_customer_empty_response.json"
9        );
10        setupGraphQL(fluentWireMockServer, ".*CreateCustomer.*", "it/common/fc/empty_response.json");
11        setupURL(ctWireMockServer, CT_GRAPHQL_URL, CT_CUSTOMER_QUERY_REGEX,
12                "it/customer/ct/graphql/get_customer_response.json"
13        );
14
15        // when: sending a customer message to the commercetools_events queue
16        messageSender.send(
17                COMMERCETOOLS_EVENTS,
18                getResourceAsString("it/message/customer_created.json")
19        );
20
21        // then: we should send a request to create a customer in Fluent
22        assertURLCalledAndBodyMatchesJson(
23                fluentWireMockServer,
24                FLUENT_GRAPHQL_URL,
25                ".*createCustomer.*",
26                getResourceAsString("it/customer/fc/graphql/create_customer_request_body.json")
27        );
28    }

Language: java

Name: Example

Description:

[Warning: empty required content area]

Running the tests

Depending on how the integration tests were written (with or without spring profiles fixed), you may or may not need to specify the spring profiles when executing the integration tests.

Note that

`mvn test`

does not run integration tests, only JUnits.

1# without specifying profiles  
2mvn clean integration-test
3
4# with spring profiles
5mvn clean integration-test -Dspring.profiles.active=connectors,it,aws,localstack
6mvn clean integration-test -Dspring.profiles.active=connectors,it,kafka

Language: java

Name: Example

Description:

[Warning: empty required content area]

You will also need to configure some properties depending which services you are using. To enable

`SecretsManager`

`S3`

and

`SQS`

test containers, add the following to your application.yml in your test resources folder.

1integration-test:
2  aws:
3    core-services:
4      enabled: true
5    queue:
6      enabled: true

Language: java

Name: Example

Description:

[Warning: empty required content area]

If you want to use Kafka instead of AWS SQS, then just add the following to your configuration:

1integration-test:
2  kafka:
3    queue:
4      enabled: true
5  aws:
6    core-services:
7      enabled: true

Language: java

Name: Example

Description:

[Warning: empty required content area]

Connect SDK Integration Test Suite

Overview

Detailed Technical Description

Integration Tests Modules

Creating Integration Tests

Commonly Used Spring Profiles

Integration Test Example

Running the tests

Building Blocks

By Type

Helpful Resources

Quick Links