Considerations for testing TeamMentor with Jenkins CI

The Jenkins Continuous Integration server is a Java based application that manages builds, tests and notifications. The idea is that it monitors the version control system for new code, when it detects a new code drop, it can:

  1. Check it out
  2. Build it
  3. Deploy it
  4. Test it
  5. Notify stakeholders of any build/test failures
  6. Display the results

BDD-Security can hook into the testing and reporting phases of a Jenkins build, but there are some constraints when using it:

Firstly, it must test a live instance of the web application/web service, so the “deploy” step of the build is necessary.

Secondly, since BDD-Security relies on Burp the server hosting Jenkins needs to be able to run Burp…or needs to be able to connect to another server running Burp.

So a few questions about how to integrate TeamMentor into a Jenkins setup:

  1. By design, Jenkins can build software as well as test it. So if we host Jenkins on a Windows server then Jenkins could be used as a build server for TeamMentor if TeamMentor can be built using MSBuild. The process is described here. So the question is, do we want Jenkins to build TeamMentor as well as test it – or just to test it?
  2. If we decide to build with Jenkins, then Jenkins can build, deploy and test on the same server. But if we decide to only test using Jenkins, then BDD-Security would need a live instance of the latest running TeamMentor code to test. So when it detects a TeamMentor commit to git, which server should it test against?
  3. Should Jenkins also run Arvind’s Python unit tests against TeamMentor as part of the testing phase using the Tox tool?
  4. Where should notifications be sent to for: Build failures, Python test failures and BDD-Security test failures?

Personally, I think it would be great to take full advantage of all of the Jenkins functionality by using it to build, deploy and test on the same server- but that will involve a bit more work, firstly in getting TeamMentor to build and deploy using MSBuild and secondly in getting both Arshan’s and my tests integrated with it.

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

7 Responses to Considerations for testing TeamMentor with Jenkins CI

  1. Dinis Cruz says:

    I like it, lets use Jenkins to build, test and deploy TM (ideally triggered from a GitHub push (see http://www.foraker.com/hudson-github-hooks/ for an idea on how to do it))

    Let’s use an Amazon EC2 windows image (any special requirements, or default windows 2008 will do?)

    • stephendv says:

      Default windows 2008 will do nicely. The only other requirements are that you’ll need a commercial license for Burp if you want to run the automated scanning tests.
      Does TeamMentor build using the MSBuild tool?

      • Dinis Cruz says:

        ok, and yes you can build TeamMentor using MSBuild (although if you do a git pull you will get the latest build (i.e. dlls included in the distribution))

  2. Dinis Cruz says:

    Is there a cloud service that we can use for the testing part? (even if using an externally hosted TM site)

    for example http://www.cloudbees.com/dev-features.cb?

    • stephendv says:

      Can’t use them, because we can’t run a Burp instance. BDD-Sec uses Burp both for inspecting HTTP requests and also for scanning. So even if you choose not to scan, you still need a Burp instance to read HTTP requests.

      • Dinis Cruz says:

        Humm, an cloud based Burp solution , that would be an interesting service 🙂

        Isn’t there a command line version of Burp?

        Also what about ZAP proxy? can we use ZAP in addition to Burp?

  3. stephendv says:

    Am discussing licensing questions with Portswigger at the moment to clarify. You can run the normal burp as a headless server, but that’s not the issue- it’s that we have to have another service running and listening on a port which cloudbees doesn’t allow.

    I’m strongly considering ZAP now, given that most open source projects will be reluctant to pay for a Burp license.

    Luckily ZAP already has an API, so should be possible to make it a configuration option to choose Burp or ZAP or Both, ..but it will require some time to refactor BDD-Sec!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s