In this part of our series, we will take our existing platform composed of Postman, Newman and Oracle Intelligent Advisor and plug it into Jenkins, to allow us to run lights-out tests according to a schedule, and to enable the testing team to receive emails when the tests have completed.

For reference, the other parts of this series are here : Part 1, Part 2, Part 3.  For the record, let’s be absolutely clear about one thing – this is not the only way to automate this process, and there are other tools and platforms that can do the same job. Heck, if you are lucky you will not even have to worry about this because some other team or department already has this thought through and all you have to do is deliver projects. But you might want to understand the mechanics of this process anyway, or if you are unlucky (or lucky, depending on your point of view)  then when the musical chairs are happening and you are the one left standing so you inherit responsibility for this, at least you will have a head start.

So, off to work. Many of you will know Jenkins, the stable, powerful open source automation platform that allows us to pretty much build, deploy or test anything. The biggest bonuses of Jenkins are that it is free to use, easy to install and that it comes with an incredibly large set of plugins, which extend the functionality to include lots of other things.

If you have installed Jenkins, which is about as easy as it can be, you should find yourself looking at the basic setup screen of Jenkins:

If you are pressed for time, just accept the suggested plugins first, and then return to the list of plugins later. You will need to add several, if they are not already present. The most important one is the Node.JS plugin, since we want to use Jenkins to communicate via Newman, and you will remember from the last post in this series that Newman runs on Node.js.

1. Manage Jenkins, Plugin Manager
2. Click Available to find more plugins
3. Filter for nodejs
4. Select the plugin
5. Install without a restart or with a restart as you wish depending on whether you have more to install.

Other important plugins include potentially the Email Extension Plugin which you might find useful, and potentially lots of others. You will also need to make sure that you have installed any plugins you want to use with Newman, recall that in the last post in the series you added an HTML report generation to Newman. If you didn’t do that yet, you should do it now. There are a few other settings in Jenkins that you need to setup, notably the System Admin Email Address but most of it should be fairly straightforward and you will find out that you need it when Jenkins complains.

Now you need to tell Jenkins a little more about Newman. In the Global Tool Configuration of Jenkins, via the Manage Jenkins option, scroll down to the Node.js seection and ensure you have added the details about your Node.js installation, and that you want to use Newman with it.

1. Add NodeJS
2. Give it a name whatever you would like to see in the logs
3. Install Automatically
4. Choose the version that you installed (the installer would have told you that)
5. Add this to ensure that Newman and the HTML Reporter you used with Newman are available

Save all of that and you are just about ready to use Jenkins for real.

Create a new Freestyle Project in Jenkins and give it whatever name you wish. Then in the Build Environment section, ensure that you signal the need for Node.js. In the Build Step, choose Execute Windows batch command (unless you are not running windows, in which case there is a non-Windows equivalent). Add, for example, the following command line to run Newman with Node.js and use the HTML reporter, with your data file from the previous post in this series.

newman run c:\temp2\mycollection_2.json -d c:\temp2\mydata.csv -e c:\temp2\myenv.json -r html --reporter-html-export myfile.html

The file name ensures that this file is overwritten each time you run this project. If you don’t you will end up with lots of files, each using a timestamp as the name, which can get confusing.

Having set up our Jenkins Project, we are going to add a Post Build Action to ensure that your team (or just you) gets an email with the report from Newman. The screenshot below assumes you have added the Extended Email Plugin mentioned above. Note the items below the image.

1. Without going into too much detail, the extended email plugin provides all sorts of options for creating email groups and deciding which email (there can be many templates) goes to which person.
   
2. This is where you add the Newman HTML report. Note the syntax of the  attachment path is not maybe what you are used to. It uses Ant FileSet syntax. The easiest thing for a test is to not include any path either in the build command or in the attachment. Then the file will be generated in the workspace folder of the project.
   
3. The Jenkins build log can also be included with the mail, and can be useful if you have added many build steps and want to see what they did.

At a minimum you might want to set up an email that is sent when there are failures:

You can click the advanced options and begin that process to set up who receives emails in different trigger circumstances. Or you can just send a mail always.

Save your project. Now, as a final point, you may which to schedule these tests to run when you are not in the office. So you can add Build Triggers from the top of the Project Configuration and use CRON-style syntax to decide when this will run.

Now all you have to do is save your Jenkins Project. Of course, if you don’t want to wait until midnight you can always Build Now. You will recall from the third part of this series that my data file had an iteration in it that was set up to cause the Postman test to fail, because the journey time was too long. So, sure enough, I get an email in my Inbox:

The HTML report is attached to my email, and I can view the details of the failure, which is indeed the iteration 3 from my data file:

So we are now in a position to test, in an automated fashion, our Web Service from Oracle Intelligent Advisor. Although the example given was using the REST assessor, Postman will happily let you run SOAP XML calls as well, so this would be applicable too.

In the next part of the series we will look at using Jenkins to test our HTML-based Interview for the same project.

See you soon!