--- title: Running SoapUI tests on Jenkins description: Explaining all the steps needed to enable fully automated SoapUI test on Jenkins server. image: https://www.cegal.com/hubfs/Thoughtful%20young%20programmer%20coding%20on%20computer%20in%20the%20evening%20at%20home.jpeg --- # Running SoapUI tests on Jenkins Cegal Tech Advisors Fighting fuzz by striving to make your coding life more straight forward. 11/28/2018 | ## It is fine to run small and ad hoc API and functional tests from the [SoapUI](https://www.soapui.org/) GUI environment but it is not suitable for running automated regression tests that are running on a test automation server by using some form of scheduling mechanism. Fortunately, there is a possibility to run [SoapUI](https://www.soapui.org/) tests in a fully automated fashion on e.g. [Jenkins](https://jenkins.io/) server. In this article, we are going to explain all the steps needed to enable a fully automated [SoapUI](https://www.soapui.org/) test on the [Jenkins](https://jenkins.io/) server. ## Automating SoapUI to run on Jenkins The easiest way to automate [SoapUI](https://www.soapui.org/) tests to run them on the [Jenkins](https://jenkins.io/) server is to follow these steps: 1. Split big [SoapUI](https://www.soapui.org/) project 2. Create a managing IDE project 3. Prepare the [Maven](https://maven.apache.org/) build 4. Enable versioning 5. Running tests on Jenkins ## 1. Split big SoapUI project Big SoapUI can create many problems for management and versioning as a project file, essentially an XML file, can become too big and thus cumbersome to: maintain, version, and fix, if necessary. In the screenshot below, we can see such a big project that is composed of many different [Test Suites](https://www.soapui.org/docs/functional-testing/reference/testsuite-reference.html). Each [Test Suite](https://www.soapui.org/docs/functional-testing/structuring-and-running-tests.html#1-Test-Structure) is then composed of many different [Test Cases,](https://www.soapui.org/docs/functional-testing/structuring-and-running-tests.html#1-Test-Structure) and in turn, each [Test Case](https://www.soapui.org/docs/functional-testing/structuring-and-running-tests.html#1-Test-Structure) is composed of various [Test Steps](https://www.soapui.org/docs/functional-testing/working-with-teststeps.html): ![Project with many test suites, test cases and test steps initialization](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/big_project.png) Fig. 1. Project with many [Test Suites](https://www.soapui.org/docs/functional-testing/reference/testsuite-reference.html), [Test Cases,](https://www.soapui.org/docs/functional-testing/structuring-and-running-tests.html#1-Test-Structure) and [Test Steps](https://www.soapui.org/docs/functional-testing/working-with-teststeps.html) A more manageable way is to split this big project into many smaller projects, each containing essentially one [Test Suite](https://www.soapui.org/docs/functional-testing/reference/testsuite-reference.html): ![Big project split in many smaller](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/smaller_projects.png) Fig. 2. Big project split into several smaller projects NOTE: Split projects have to be saved in specific folders (see below) ## 2. Create a managing IDE project Because we have to deal with versioning, [Maven](https://maven.apache.org/) and [Jenkins](https://jenkins.io/) files, it is easier if we create a project with a specific folder structure in development with an environment of choice. Here we have used [IntelliJ](https://www.jetbrains.com/idea/) to create such a project: ![IntelliJ project for SoapUI projects](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/intellij_project.png) Fig. 3. [IntelliJ](https://www.jetbrains.com/idea/) project for [SoapUI](https://www.soapui.org/) projects ## Project structure To run successfully on [Jenkins](https://jenkins.io/), the [IntelliJ](https://www.jetbrains.com/idea/) project structure has to be composed as follows: ``` ├── Jenkinsfile ├── README.md ├── elemif.iml ├── ext │   └── ojdbc8-12.2.0.1.jar ├── global-groovy.log ├── pom.xml ├── soapui-errors.log ├── soapui-settings.xml ├── soapui.log ├── src │   └── test │       ├── resources │       │   ├── UAT.jks │       │   ├── database.json │       │   └── qa.jks │       └── soapui │           ├── EMIF-archive.xml │           ├── EMIF-compression.xml │           ├── EMIF-content-type.xml │           ├── EMIF-duplicate-messages.xml │           ├── EMIF-header-issues.xml │           ├── EMIF-library.xml │           ├── EMIF-security.xml │           └── EMIF-wrong-message.xml └── target     ├── surefire-reports     │   ├── Archive-Initialization-InitDB-0-OK.txt     │   ├── Archive-Initialization-InitLib-0-OK.txt     │   └── TEST-Archive.xml     └── test-classes         ├── UAT.jks         ├── database.json         └── qa.jks ``` ```   ``` - Jenkinsfile - contains [Jenkins](https://jenkins.io/) pipeline build instructions - README.md - optional README file with the setup instructions - elemif.iml - file generated by [IntelliJ](https://www.jetbrains.com/idea/) - ext - folder containing additional JAR libraries used during test execution (database driver in this case) - *.log - various log Files - pom.xml - [Maven](https://maven.apache.org/) build file - soapui-settings.xml - File that contains [SoapUI Preferences](https://www.soapui.org/docs/preferences-and-settings.html) - src/test/resources - folder containing [Java Keystore](https://en.wikipedia.org/wiki/Keystore) databases and database credentials - src/test/soapui - folder containing projects that we have split in the previous step and saved to this folder. Normally [SoapUI](https://www.soapui.org/) stores preferences files (soapui-settings.xml) in a user's home folder. We can copy this file to our IDE project location and update it every time we make some important preference changes through [SoapUI](https://www.soapui.org/) GUI, or we can discard the file from the original location and make a symbolic link to the file in our IDE project folder. ## Setting absolute and relative paths When we are running our tests from within [SoapUI](https://www.soapui.org/) GUI environment we have to supply absolute paths but when we are running our tests from the command line or from e.g. [Jenkins](https://jenkins.io/) server than we have to use relative paths. The same goes for specifying the location of [Java Keystore](https://en.wikipedia.org/wiki/Keystore) used to sign and/or encrypt our messages. So in a project file (e.g. `EMIF-archive.xml`), we have to search for the location of [Java Keystore](https://en.wikipedia.org/wiki/Keystore) database (e.g. `qa.jks`) and then assure that the absolute path is used: ![Absolute Java Keystore path in project file](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/Absolute-path-xml.png) Fig. 4. Absolute [Java Keystore](https://en.wikipedia.org/wiki/Keystore) path in a project file When we then open our project in [SoapUI,](https://www.soapui.org/) and we open Project->[Show Project View](https://www.soapui.org/soapui-projects/reference/project-window.html) menu, we can verify that [SoapUI](https://www.soapui.org/) can recognize and verify our Keystore ![Absolute Java Keystore path in Show Project View dialog box](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/Absolute-path.png)Fig. 5. Absolute [Java Keystore](https://en.wikipedia.org/wiki/Keystore) in [Show Project View](https://www.soapui.org/soapui-projects/reference/project-window.html) dialog However, to run the same project on [Jenkins](https://jenkins.io/) server, we have to supply relative paths. Thus we have to modify the project file to use the relative path: ![Relative Java Keystore path in project file](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/relative-path-xml.png) Fig. 6. Relative [Java Keystore](https://en.wikipedia.org/wiki/Keystore) path in project file When we than open the same project in [SoapUI](https://www.soapui.org/) we can see that [SoapUI](https://www.soapui.org/) is not able to recognise location of [Java Keystore](https://en.wikipedia.org/wiki/Keystore) and thus not able to use it: ![Relative Java Keystore path in Show Project View dialog box](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/relative-path.png) Fig. 7. Relative [Java Keystore](https://en.wikipedia.org/wiki/Keystore) in [Show Project View](https://www.soapui.org/soapui-projects/reference/project-window.html) dialog The same principle has to be applied to all other files. Unfortunately, [SoapUI](https://www.soapui.org/) is not capable of working with relative paths, so in order to test our test cases in [SoapUI](https://www.soapui.org/) we have to use absolute paths but to run it on the [Jenkins](https://jenkins.io/) server, we have to use relative paths. ## 3. Prepare the Maven build [SoapUI](https://www.soapui.org/) [Maven](https://maven.apache.org/) artifacts are not stored on the [Maven Central](https://mvnrepository.com/) repository. Thus, we have to add an additional repository location for the [Maven](https://maven.apache.org/) settings file, which is located in the users' home folder (usually: `$HOME/.m2/settings.xml`), assuming that we have downloaded and installed [Maven](https://maven.apache.org/) support previously (Installation instructions can be found [here](https://maven.apache.org/install.html)). In this case, the easiest way to modify `settings.xml` the file is by adding additional `` XML element under the `` section and then reference it in `` section, as in this code excerpt from `settings.xml`: ```   .   .   .           maven.central               true                               central           Maven2 Central                       true             always             warn                                 true             always             warn                     https://repo.maven.apache.org/maven2           default                               com.smartbear                 false                                   smartbear-sweden-plugin-repository             http://www.soapui.org/repository/maven2/                                           smartbear-sweden-plugin-repository             http://www.soapui.org/repository/maven2/                                      com.smartbear   ```   After we have enabled additional repository locations for [SoapUI](https://www.soapui.org/) specific artefacts, we are able to specify content of `pom.xml` which is a [Maven](https://maven.apache.org/) build file and is located at the root of our IDE project location. Content of a `pom.xml` file can look something like this: ```     4.0.0       emif     1.0-SNAPSHOT                                               com.smartbear.soapui                 soapui-maven-plugin                 5.4.0                                                            EMIF-archive                                                   soapui-settings.xml                            src/test/soapui/EMIF-archive.xml                              target/surefire-reports                            Archive                                                                                                             true                            true                            true                            true                                                                          test                                               test                                                                  EMIF-compression                                                     soapui-settings.xml                             src/test/soapui/EMIF-compression.xml                               target/surefire-reports                             Compression                                                         true                             true                             true                             true                                                                             test                                                 test                                                                 ``` ```   ``` The [Maven](https://maven.apache.org/) plugin responsible for executing the build process is: `soapui-maven-plugin` and this plugin allows for one or more `` blocks inside `` element. Each `` block is responsible for running one [SoapUI](https://www.soapui.org/) project. Which project file to run, which test suite to run, or even specific test case to run is defined with: ``, `` and `` elements. Now we can test our build and execute all tests from the command line by executing the following maven command: ``` mvn clean test ``` If everything is successful, [Maven](https://maven.apache.org/) will execute all tests specified in `pom.xml` file, print results on the console, and store them in text files in `target/surefire-reports` folder ## 4. Enable versioning To have full Continuous Integration (CI) cycle, we have to initialize our IDE project folder as a version controlled folder. The GIT version control system is implemented with the commands: ``` git init git add . git commit -m "first commit" ``` ```   ``` Then we have to create a repository on the version control server that will contain our project. For [GitHub,](https://github.com/) it is done by clicking on the “New” button under Repositories section. After we have given the name to the repository, we can upload our project to the version control server. For [GitHub](https://github.com/): ``` git remote add origin https://github.com//emif.git git push -u origin master ``` ``` ``` ``` git commit -a -m "description of change" git push ``` ```   ``` each time we have modified our projects, and we would like to push changes to [Jenkins](https://jenkins.io/) ## 5. Running tests on Jenkins ### Prepare Jenkins file We want to use new [Jenkins](https://jenkins.io/) pipeline build facilities that treat the build process as a code. Therefore we have to create a [Jenkins file](https://jenkins.io/doc/book/pipeline/jenkinsfile/) that contains build instructions in a declarative form or in the form of [Groovy](http://groovy-lang.org/) code. Our [Jenkins file](https://jenkins.io/doc/book/pipeline/jenkinsfile/) looks like this: ``` pipeline {     options {         disableConcurrentBuilds()         buildDiscarder(logRotator(numToKeepStr: '50', artifactNumToKeepStr: '50'))     }       agent any       tools {         maven 'maven3'         jdk 'jdk8'         allure 'allure2'     }       triggers {         pollSCM 'H/5 * * * *'           cron '''TZ=Europe/Oslo         H H(13-14) * * *'''     }       stages {         stage('Test') {             steps {                 sh '''                     mvn clean test                 '''             }             post {                 always {                     archiveArtifacts 'target/surefire-reports/**/*.xml'                     junit 'target/surefire-reports/**/*.xml'                 }             }          }     } } ``` ``` ``` It essentially says that we do not allow parallel builds. We use Maven to build and the build will start whenever we post changes to the source version control server and then on a daily schedule. It also says that the test outputs are going to be available in the `target/surefire-reports/` folder on the server. ## Prepare Jenkins server After we have prepared the [Jenkins file](https://jenkins.io/doc/book/pipeline/jenkinsfile/) we have to create a new [Jenkins](https://jenkins.io/) project by clicking on the New Item button in Jenkins classic user interface web console: ![ Jenkins classic user interface web console](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/jenkins1.png)Fig. 8. Jenkins classic user interface web console From the new page that opens, we can choose to create “Pipeline” project or “Multibranch pipeline” if we plan to have multiple branches in our version control system (e.g. “master”, “stage”) that we build separately with different options: ![ Miltibranch Pipeline creation option](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/jenkins2.png)Fig. 9. Multibranch Pipeline creation option When we have chosen the appropriate option, we have to configure the project. First, we establish a connection with the version control system: ![ Connection to version control system](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/jenkins3.png)Fig. 10. Connection to version control system Finally, we define how our build process is going to be executed. As we have opted to use the [Jenkins file](https://jenkins.io/doc/book/pipeline/jenkinsfile/) to define our build process, we are going to specify that option in the “Build Configuration” tab: ![ Build configuration tab](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/jenkins4.png)Fig. 11. Build configuration tab ## Checking test executions on Jenkins After we have saved the project, we are ready to run our first test build. Build can be run automatically by pushing changes to the version control server, waiting on a schedule, or running it manually. To run it manually, we have to go to the [Jenkins](https://jenkins.io/) web console and then click the project name we gave to the Jenkins project in the previous step. Then we can choose the version control branch we want to execute: ![Jenkins project page](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/jenkins5.png)Fig. 12. Jenkins project page By clicking on a branch name, the page that contains executions statuses and links to reports are opened. We can then click on the “Build Now” button to start our build manually. To check build results, we click on build execution number: ![Branch execution status page](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/jenkins6.png)Fig. 13. Branch execution status page Clicking on the build execution number opens the page, giving us access to various execution status information. Like, access to text console output, pipeline steps, etc., and most important, test results: ![Execution status information](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/jenkins7.png)Fig. 14. Execution status information The test result page displays all successfully executed tests and all failed tests: ![Test results page](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/jenkins8.png)Fig. 15. Test results page By clicking on the failed test name link, we can see a detailed explanation of why the tests failed that allows for easier investigation and fix: ![Failure details page](https://blog.sysco.no/images/2018-11-28-soapui-tests-on-jenkins/jenkins9.png)Fig. 16. Failure details page ## Conclusion Setting up and running [SoapUI](https://www.soapui.org/) tests on [Jenkins](https://jenkins.io/) is not a trivial process but is worth the time and effort invested because this allows us to run fully automated regression tests and automatic system checking regularly. This process also generates reports for every failure discovered in the process, allowing us to apply fixes and patches immediately or roll back our deployments. - [Automation ](https://www.cegal.com/en/tech-blog/tag/automation) - [TechBlog ](https://www.cegal.com/en/tech-blog/tag/techblog) - [soap ui ](https://www.cegal.com/en/tech-blog/tag/soap-ui) - [Jenkins ](https://www.cegal.com/en/tech-blog/tag/jenkins) ## Related articles How to be a better developer Cegal Tech Advisors Fighting fuzz by striving to make your... ![arrow](https://www.cegal.com/hubfs/cegal-website/vectors/resource-card-arrow.svg) Writing Groovy Script Libraries in SoapUI Cegal Tech Advisors Fighting fuzz by striving to make your... ![arrow](https://www.cegal.com/hubfs/cegal-website/vectors/resource-card-arrow.svg) Setting Up and Initializing Database Connection from Groovy Test... Cegal Tech Advisors Fighting fuzz by striving to make your... ![arrow](https://www.cegal.com/hubfs/cegal-website/vectors/resource-card-arrow.svg) ```json { "@context" : "https://schema.org", "@type" : "BlogPosting", "author" : { "@type" : "Person", "name" : "Cegal Tech Advisors", "url" : "https://www.cegal.com/en/tech-blog/author/cegal-tech-advisors" }, "dateModified" : "2023-11-28T18:41:17.764Z", "datePublished" : "2018-11-28T14:00:00.000Z", "headline" : "Running SoapUI tests on Jenkins", "image" : [ "https://www.cegal.com/hubfs/Thoughtful%20young%20programmer%20coding%20on%20computer%20in%20the%20evening%20at%20home.jpeg" ], "mainEntityOfPage" : { "@id" : "https://www.cegal.com/en/tech-blog/running-soapui-tests-on-jenkins", "@type" : "WebPage" }, "publisher" : { "@type" : "Organization", "logo" : { "@type" : "ImageObject", "url" : "https://www.cegal.com/hubfs/Cegal%20white.png" }, "name" : "Cegal " } } ``` ```json { "@context" : "https://schema.org", "@type" : "Article", "author" : [ { "@type" : "Person", "name" : "Cegal Tech Advisors", "url" : "https://www.cegal.com/en/tech-blog/author/cegal-tech-advisors" } ], "dateModified" : "2023-11-28T18:41:17", "datePublished" : "2018-11-28T14:00:00", "headline" : "Running SoapUI tests on Jenkins", "image" : [ "https://7270560.fs1.hubspotusercontent-na1.net/hubfs/7270560/Thoughtful%20young%20programmer%20coding%20on%20computer%20in%20the%20evening%20at%20home.jpeg" ] } ```