Connect SDK Integration Test Suite
Changed on:
17 Oct 2023
Overview
There are two ways to perform integration tests. The first involves creating the tests within the same project as your custom code. Alternatively, if your custom logic is in a separate module from your main class, you can create integration tests for that module instead.
When your integration tests are in module that doesn't have a main class, then you need to have a class like below in your src/test/java folder to allow Springboot tests to start the application for the integration tests.
Detailed Technical Description
1@EnableScheduling
2@SpringBootApplication
3public class ConnectorApplication {
4 /**
5 * Main method to start the connector
6 * @param args no arguments expected
7 */
8 public static void main(final String[] args) {
9 SpringApplication.run(ConnectorApplication.class, args);
10 }
11}It is also required to create the necessary configuration files on your src/test/resources folder.
- application.yml
- application-connector.yml
- bootstrap.yml
If you want logging, you will have to add a `logback-spring.xml` file for the spring profile `it`.
Integration Tests Modules
Connect SDK includes test modules to assist with integration tests:
- connect-sdk-test-core: Connect SDK base test utilities
- connect-sdk-test-core-aws: Extension of the SDK test utilities for projects using AWS
- connect-sdk-test-core-kafka: Extension of the SDK test utilities for projects using Kafka
Creating Integration Tests
The first step is to create the base integration test class where all dependencies are wired and set up for all of your integration tests. It also has some helper methods that can be used in your initialization of tests to set up the correct queues and secrets as well as wiremock stubs.
The base test class may or may not have a specific spring profile scope. It entirely depends on the integration test and coverage required.
- Without Spring Profiles
In this approach, all tests implemented using this base class are not bound to run with AWS for example. To control what `MessageSender` the base class is using in the example below, it requires one to specify the spring profiles at the time of the execution of the tests. For modules that support multiple implementations, it may be required to run integration tests as many times as required to cover all the different ways it can be used. For example, the Connect SDK can work with both Kafka and SQS, and it requires running integration tests to run against both profiles.
1# To run the integration tests using SQS
2mvn clean integration-test -Dspring.profiles.active=connectors,it,aws,localstack
3
4AND/OR
5
6# To run the integration tests using Kafka
7mvn clean integration-test -Dspring.profiles.active=connectors,it,kafkaSpring profile will ensure the correct bean of `MessageSender` is available to the test class.
1@Slf4j
2@Feature("Commerce Tools Integration Tests")
3@Testcontainers
4public abstract class CTBaseIntegrationTest extends BaseIntegrationTest {
5
6 @Autowired
7 protected MessageSender messageSender;
8
9 @Autowired
10 private SecretManagerContainerSetupService secretManagerContainerSetupService;
11 ... - With Spring Profiles
Whenever the profile is specified, the spring profile is preset and there is no need to specify them at the time of the execution of the tests.
1mvn clean integration-test1@Slf4j
2@Feature("AWS specific Integration Tests")
3@Testcontainers
4@SpringBootTest(properties = "spring.profiles.active=connector,it,aws,localstack")
5public class AwsBaseIntegrationTest extends BaseIntegrationTest {
6
7 @Autowired
8 protected MessageSender messageSender;
9
10 @Autowired
11 private SecretManagerContainerSetupService secretManagerContainerSetupService;
12 ...Commonly Used Spring Profiles
- connector: connector configurations (routes and other fluent connect settings)
- it: any configuration override for the integration tests
- aws: any aws specific configuration as well as enabling the correct libraries to be used for AWS tests
- localstack: by adding this profile the SDK will auto-create queues in SQS
- kafka: any aws specific configuration as well as enabling the correct libraries to be used for Kafka tests
Integration Test Example
Here is a full example of a base integration test class:
1@Slf4j
2@Feature("AWS Integration Tests")
3@Testcontainers
4@SpringBootTest(properties = "spring.profiles.active=connector,it,aws,localstack")
5public class AwsBaseIntegrationTest extends BaseIntegrationTest {
6 public static final String EVENTS = "events";
7 public static final String BATCH = "batch";
8 private static final String SQS_BATCH = "SQS_BATCH";
9 private static final String SQS_EVENTS = "SQS_EVENTS";
10 @Autowired
11 protected MessageSender messageSender;
12 @Autowired
13 private SecretManagerContainerSetupService secretManagerContainerSetupService;
14
15 @BeforeAll
16 static void setup() {
17 //sets up the minimal required queues for the SDK, more can be added if necessary
18 System.setProperty(SQS_BATCH, BATCH);
19 System.setProperty(SQS_EVENTS, EVENTS);
20 startWiremockServer();
21 }
22
23 @AfterAll
24 static void tearDown() {
25 fluentWireMockServer.stop();
26 }
27
28 @BeforeEach
29 void testSetup() {
30 baseSetup(fluentWireMockServer);
31 }
32
33 @AfterEach
34 void afterTest() {
35 fluentWireMockServer.findAll(RequestPatternBuilder.allRequests()).forEach(request -> log.info(request.getAbsoluteUrl() + " " + request.getBodyAsString()));
36 fluentWireMockServer.resetAll();
37 }
38
39 @PostConstruct
40 private void setupSecretManager() throws IOException, InterruptedException {
41 //Sets up all required secrets used by your tests
42 secretManagerContainerSetupService.createSecret(
43 "fc/connect/testconnector/api/fluent/activeAccounts",
44 "{\"accounts\":[{\"name\":\"test\", \"retailers\":[1]}]}"
45 );
46 secretManagerContainerSetupService.createSecret(
47 "fc/connect/testconnector/TEST/api/fluent-account/1",
48 "{\"retailer\":\"1\", \"userName\":\"username\", \"password\":\"password\"}"
49 );
50 }
51}Once you have done the above, you should be able to start writing tests. Any integration test class should extend the base test class created above.
1public class EventQueueIntegrationTest extends AwsBaseIntegrationTest {
2 private static final String EVENTS_DLQ = EVENTS + "-dlq";
3
4 @Autowired
5 private AmazonSQSAsync amazonSQSAsync;
6
7 @Test
8 void shouldPublishInvalidMessageToDlq() {
9 // when: publishing an invalid message to the events queue
10 messageSender.send(EVENTS, "hello");
11
12 // then: the message should be published to the dlq as thera are no handlers to process it
13 given()
14 .await()
15 .atMost(Duration.ofSeconds(2))
16 .untilAsserted(() -> Assertions.assertEquals(1, receiveMessagesFromQueue(EVENTS_DLQ, 5).size()));
17 }
18
19 private List<Message> receiveMessagesFromQueue(final String queue, final int messageNumber) {
20 final ReceiveMessageRequest request = new ReceiveMessageRequest(queue);
21 request.setVisibilityTimeout(2);
22 request.setWaitTimeSeconds(1);
23 request.setMaxNumberOfMessages(messageNumber);
24 return amazonSQSAsync.receiveMessage(request).getMessages();
25 }
26}If you want to test any handler logic you may need to setup stubs and assert that wiremock is called. Some basic utility functions to assist with that are provided. Below is an example of how you might do this.
1import static com.fluentcommerce.connect.common.utils.IntegrationTestUtils.FLUENT_GRAPHQL_URL;
2import static com.fluentcommerce.connect.common.utils.IntegrationTestUtils.assertURLCalledAndBodyMatchesJson;
3import static com.fluentcommerce.connect.common.utils.IntegrationTestUtils.getResourceAsString;
4import static com.fluentcommerce.connect.common.utils.IntegrationTestUtils.setupGraphQL;
5import static com.fluentcommerce.connect.common.utils.IntegrationTestUtils.setupURL;1 @Test
2 @DisplayName("Should create a customer and send to Fluent")
3 void shouldCreateCustomerInFluent() {
4 // given: a customer does not exist in Fluent
5 setupGraphQL(
6 fluentWireMockServer,
7 ".*GetCustomer.*",
8 "it/customer/fc/graphql/get_customer_empty_response.json"
9 );
10 setupGraphQL(fluentWireMockServer, ".*CreateCustomer.*", "it/common/fc/empty_response.json");
11 setupURL(ctWireMockServer, CT_GRAPHQL_URL, CT_CUSTOMER_QUERY_REGEX,
12 "it/customer/ct/graphql/get_customer_response.json"
13 );
14
15 // when: sending a customer message to the commercetools_events queue
16 messageSender.send(
17 COMMERCETOOLS_EVENTS,
18 getResourceAsString("it/message/customer_created.json")
19 );
20
21 // then: we should send a request to create a customer in Fluent
22 assertURLCalledAndBodyMatchesJson(
23 fluentWireMockServer,
24 FLUENT_GRAPHQL_URL,
25 ".*createCustomer.*",
26 getResourceAsString("it/customer/fc/graphql/create_customer_request_body.json")
27 );
28 }Running the tests
Depending on how the integration tests were written (with or without spring profiles fixed), you may or may not need to specify the spring profiles when executing the integration tests.
Note that `mvn test` does not run integration tests, only JUnits.
1# without specifying profiles
2mvn clean integration-test
3
4# with spring profiles
5mvn clean integration-test -Dspring.profiles.active=connectors,it,aws,localstack
6mvn clean integration-test -Dspring.profiles.active=connectors,it,kafkaYou will also need to configure some properties depending which services you are using. To enable `SecretsManager`, `S3` and `SQS` test containers, add the following to your application.yml in your test resources folder.
1integration-test:
2 aws:
3 core-services:
4 enabled: true
5 queue:
6 enabled: trueIf you want to use Kafka instead of AWS SQS, then just add the following to your configuration:
1integration-test:
2 kafka:
3 queue:
4 enabled: true
5 aws:
6 core-services:
7 enabled: true