So if you read my last post what I pushed to Git last time was a bunch of unit tests for each user. So at 102 tests per user that’s a total of 408 unit tests for an admin,editor,reader and an anonymous user. Quite a bit. For me anyway 🙂
Now I wanted to find a way to generate a report of these 408 tests coz otherwise its quite useless. Luckily py.test has this awesome little feature where I can export the results of all my tests to an XML file. Here’s the command that does it: py.test –junitxml=/tmp/Admin_consume.xml –tb=no
If you look at the sample XML file I pushed to Git you can clearly see which tests passed and failed for all the ‘consume’ methods for Admin. We can now repeat this for every method and every user and perform our analysis of the authorization logic; comparing against this main spreadsheet.
The only thing though is that you’ll notice that certain methods fail despite everything being perfectly correct, programatically, and the user being allowed to invoke that method. For example: 1 method that failed is GetActiveSession. Why? Because the value used isn’t valid any more. If you look at the code I used, the value I used was a=client.service.GetActiveSession(‘376639fe-90f6-4e2c-a08f-06c5f1c28f77’) .. and that isn’t valid any more. Get it? Now that means that the unit test should be changed in such a way that it always picks up a “valid” value; in this case, a valid session ID.
Thanks to py.test and XML though…I can quickly get a complete list of all the methods that failed and fix all of them so they never fail again…if the user is a valid user. Once this is done… all the Unit tests…will be truly truly independent :). Cya