Phi Nguyen is a QA engineer with 11+ years of experience . His areas of experience include manual testing, automation testing, acceptance testing, regression testing, database testing, API testing and web/mobile apps testing. He currently works full time as a freelancer.

Become a JMeter and Continuous Testing Pro

Start Learning

Test Your Website Performance NOW!

arrowPlease enter a URL with http(s)
Assert:is found in response
Sep 25 2018

How to Check Swagger Docs in Jenkins with the Validator-badge Tool

A Swagger/OpenAPI definition is one of the most popular frameworks for defining and documenting API projects. The Swagger/OpenAPI Docs document APIs to help developers or end users interact with API resources. A Swagger Spec/OpenAPI file is actually an API resources file that is formatted in JSON/YAML.


Suppose you have a Swagger spec file and you want to check if it follows "Swagger guidelines" like: how to describe parameters, requests/responses formats, syntax accuracy, etc. To do that, you can use the open source validator-badge tool.


The Validator-badge tool is an open-source tool for validating if a Swagger spec file follows "Swagger rules". If the spec abides by them, the response will be a blank string. Otherwise, you will get a detailed failed string.


But instead of manually running the Validator-badge, you can add it to Jenkins with your OpenAPI doc. This will enable you to automate the test, so you can run it whenever you need: every night, every time you update your Swagger file, or whenever you choose.


Let’s get started. I will show a solution for Linux.


1. Go to Jenkins and click on “New Item”.


run swagger in jenkins


2. Enter a project name (ex: Project3), click on “Freestyle project” and then “OK”.


validate swagger in jenkins


3. Scroll down to the “Build Environment” section and choose “Add build step->Execute shell”.


use the validate badge tool in jenkins for swagger


4. Let’s suppose we are checking the Swagger spec url


Enter a shell script like the one below in the “Execute shell” section and click on the “Save” button.


curl -o swagger.json
CHECK=$(curl -X "POST" "" --silent -d @swagger.json)
if [[ $CHECK != "{}" ]]; then 
  echo -e "\nSorry this is an invalid Swagger:\n$CHECK\n"
  exit 1


Shell script explanation:

  • The first line tells Linux that the file is to be executed by /bin/bash. This is the standard location of the Bourne shell on every Linux system.
  • The second line downloads the Swagger spec file and locates it in the current project directory (/var/lib/jenkins/workspace/<current project directory> .
  • The third line uses the curl command to send the JSON file to the validator-badge tool with the POST method. The result is assigned to the CHECK variable, either a blank value (passed) or a string value (failed).
  • The last line is an if statement condition. If the CHECK variable is not blank then fail the result. In this case it will print out the sentence “Sorry this is an invalid Swagger:”, followed by the content of CHECK variable.


swagger and open api testing in jenkins


5. Click on the “Build Now” button.


check your swagger docs in jenkins


Jenkins is executing the shell script.


check your openapi doc in jenkins


6. After each execution, you will see either a blue (passed) or red (failed) dot next to the build number. In this case we have a blue dot next to build number #6. This proves that your Swagger Docs passed.


test open api specs with jenkins


7. For a more detailed output log, click on the build number and choose “Console Output”. You will be able to see more information such as: %Total bytes, %Received bytes, Average Download, Status...


test swagger automatically in jenkins


8. Now let’s look at what a failed case would look like. We will rewrite shell script like below. In this case we will check for another swagger spec url


curl -o swagger.json
CHECK=$(curl -X "POST" "" --silent -d @swagger.json)
if [[ $CHECK != "{}" ]]; then 
  echo -e "\nSorry this is invalid Swagger:\n$CHECK\n"
  exit 1


9. Click on the “Configure” button to edit the project.


swagger, jenkins, openapi


10. Update the new shell script and perform the steps one more time. Here is the result. As you can see, there is a red dot next to the build number:


updating jenkins for checking swagger specs


11. The Console Output will look like this:


swagger, openapi, shell script


From here you can see detailed information such as: %Total bytes, %Received bytes, Average Download, Status, Detailed Errors, etc.


After checking that your Swagger file is properly written, upload it to BlazeMeter and run functional API tests. Learn more from this blog post.


Start testing now!

arrowPlease enter a URL with http(s)

Interested in writing for our Blog?Send us a pitch!