Open Banking Quality Assurance: Comprehensive Testing

 Current Status
BuildBuild Status
ReleaseGitHub release (latest by date)

Secure API Gateway functional tests (Open Banking UK)

Software testing to validate and coverage the implemented open banking functionalities and ensure that these satisfies the functional requirements.


  • Gradle 7.4.2
  • Kotlin 1.4.20
  • Java 14
  • OB certificates

It's important to have the VM argument -Djunit.jupiter.extensions.autodetection.enabled=true when running the tests. Fortunately this is already in the tasks defined in the gradle configuration in the build.gradle.kts. This VM argument tells JUnit to automatically load the ExecutionCondition modules defined in org.junit.jupiter.api.extension.Extension.

For more information

source Folders structure

src/test/kotlin/com/forgerock/sapi/gateway: The root folder source structure.

{Root}/framework: Folder source container that contains all related with the functional tests transversal framework for generic purposes.

{Root}/ob/uk: Folder source container that contains all related with a specific functional test implementation for Open banking UK spec.

  • /framework: configuration, constants etc for uk spec tests
  • /support: instrumentalization support classes for uk spec tests
  • /tests: Where functional tests implementation live.

Set up the certificates for test purposes

The certificates are protected, and you can't find them in the repository, for that reason to run the functional tests in local environments is necessary set the OB certificates:

  • Create the folder certificates in the root project folder

ForgeRock code owners

  • Get the certificates from sapig-ci-secrets and decrypt them:
    • OBWac.key
    • OBWac.pem
    • OBSeal.key
    • OBSeal.pem


Obtain your TPP OB certificates for test purposes from OB directory:

  • OBWac.key
  • OBWac.pem
  • OBSeal.key
  • OBSeal.pem

Copy the certificates to certificates folder created in the above step.

Adding a new ExecutionCondition

  1. Copy from examples
  2. Update the execution condition logic
  3. Add FQN of ExecutionCondition class to org.junit.jupiter.api.extension.Extension
  4. Add your annotations to classes


Functional test project has been implemented with a gradle strategy to be able to run the tests against different environments using different profiles files, let's go how works.

The profiles file lives in gradle/profiles folder, and they need to be named following the pattern profile-${profile-name}.gradle.kts as example we provide the default profile profile-dev.gradle.kts, with all mandatory properties, where dev is the ${profile-name}.

For default the functional tests are executed with the dev profile if another profile is not provided.

The profile-${profile-name}.gradle.kts file contains all properties/values that will be necessary to run the tests against an environment.

How run tests with a custom profile?

First you will need to create a new profile file in gradle/profile following the pattern defined for the profile file name. Example: gradle/profile/profile-MY_CUSTOM_PROFILE_NAME.gradle.kts

To use your new profile you can pass the profile name by command line.


Check your values

gradle properties -q -Pprofile=dev
gradle properties -q -Pprofile=dev

How overwrite a property supplied by a profile file?

To overwrite a property defined in a profile file we can use system properties passed by command line.

Example command line to overwrite a project property defined by profile

gradle -DpropertyName=someValue -DpropertyName2=someValue -DpropertyName3=someValue ...
gradle -Pprofile=MY_CUSTOM_PROFILE_NAME -DpropertyName=newValue -DpropertyName2=newValue -DpropertyName3=newValue ...

The logic implemented in build gradle will overwrite the project property propertyName with the new value passed by command line. The properyName need match with a property defined in the profile file to be overwrites


The tasks are grouped by types Task

Code GenerationDescription
gen-pain00100108-sourceGenerate sources for the schema schema-pain.001.001.08
xjcGenerationRun all XJC tasks
generateTestJarGenerate a non-executable jar library tests
accounts_v3_1_8Runs the accounts tests for version v3.1.8
domestic_payments_v3_1_8Runs the domestic payments tests for version v3.1.8
tests_v3_1_8Runs the accounts and domestic payments tests for version v3.1.8
tests_v3_1_9Runs the accounts and domestic payments tests for version v3.1.9
tests_v3_1_10Runs the accounts and domestic payments tests for version v3.1.10
N/AAll test tasks that have been deprecated for that strategic version

Run single Test on Intellij using JUnit platform

  1. Go to IntelliJ IDEA > preferences > build, execution, deployment > build tools > Gradle
  2. Set Build and run using to Gradle
  3. Set Run tests using to Gradle
  4. Set use gradle from to '' file
  5. Set Gradle JVM to java version 14 gradle-config
  6. Go to Run/Debug configuration and Edit configuration edit-config
  7. Go to properties for Gradle template and add a new run configuration edit-junit-template
  8. Add in Run the task name and the profile -Pprofile=jorge (the profile is optional, the default profile will be applied if -Pprofile isn't set)

Run configuration examples

accounts_v3_1_8devruns the task accounts_v3_1_8 with profile profile-dev.gradle.kts
accounts_v3_1_8 -Pprofile=devdevruns the task with the profile profile-dev.gradle.kts
:singleTest --tests "" -Pprofile=devdevRuns the single test method shouldGetParty_v3_1_8 defined on GetPartyTest with profile-dev.gradle.kts profile
:singleTest --tests "" -Pprofile=devdevRuns the single test method shouldGetAccounts_v3_1_8 defined on GetAccountsTest with profile-dev.gradle.kts profile

Run gradle tests manually

When a task has been run, and you want to run again the test task you need use the flag --rerun-tasks to avoid the UP-TO-DATE gradle check

gradle all [--rerun-tasks] [-Pprofile]

All test

gradle all [-Pprofile]


gradle all
# The default 'DOMAIN' value has been set in the variable 'domain' defined on the 'build.gradle.kts' file
gradle all -Pprofile=my-profile

A Specific task, set of tests

gradle <TASK> [-Pprofile]


gradle payment
gradle payment -Pprofile=my-profile

Single test

gradle test --tests "x.x.y.y.TestClass.testMethod" [-Pprofile]


gradle test --tests "" -Pprofile=my-profile


Codefresh pipeline variables table

SPEC_IMAGEYESDefine the value of the image used for functional tests by spec implementation (ex. uk-functional-tests, -functional-tests)
TAGYESDefine the image tag
TEST_TASKYESDefine the test task to be executed by gradle (ex. tests_v3_1_8, accounts_v3_1_8)
PROFILENO (default profile: default)Define which properties profile file will be load by gradle in the execution to inject the properties of a specific environment. (ex. default, dev, nightly)

Run Codefresh functional tests pipeline manually

  • Go to codefresh and sing in up.
  • Go to the pipeline ForgeCloud/sbat-infra/functional-tests
  • When Run you need to ensure that the variables values are the correct values that you want.

Run Codefresh functional tests pipeline from github actions

  • To run the codefresh pipeline from a github action add the below step to your action descriptor.

Run the functional tests pipeline from another codefresh pipeline

  • To run the functional tests pipeline from another codefresh pipeline add the below step to your pipeline.
  title: Initialise cluster
  stage: configure-kubernetes
  type: codefresh-run
    PIPELINE_ID: ForgeCloud/sbat-infra/functional-tests
      - SPEC_IMAGE=uk-functional-tests
      - TAG=latest
      - TEST_TASK=tests_v3_1_8
      # PROFILE isn't mandatory, no profile provided to use the default
  # add conditions directive if necessary
- name: 'run functional tests'
  uses: codefresh-io/codefresh-pipeline-runner@master
    args: '-v SPEC_IMAGE=uk-functional-tests -v TAG=latest -v TEST_TASK=tests_v3_1_8' # PROFILE isn't mandatory, no profile provided to use the default
    PIPELINE_NAME: 'ForgeCloud/sbat-infra/functional-tests'
    CF_API_KEY: ${{ secrets.CF_API_KEY }}
  id: run-pipeline

Download Details:

Author: SecureApiGateway

Official Github: 

License: Apache-2.0 license

#Software Engineering  #software #engine #engineering 

Open Banking Quality Assurance: Comprehensive Testing
1.30 GEEK