[Buildbot-devel] WebpartsRecursive test added (Was: Re: WebStatus ponderings)

[Marcus Lindblom]

On 2010-01-07 21:12, Marcus Lindblom wrote:
> On 2010-01-06 17:25, Dustin J. Mitchell wrote:
>> On Wed, Jan 6, 2010 at 4:14 AM, Marcus Lindblom<macke at yar.nu>   wrote:
>>> 1. We need to write a testcase that does this. ;)
>>
>> Anyone want to have some fun with htmlunit?
>
> I was thinking 'drunk monkey style' testing. Follow every link and make
> sure nothing is broken.
>
> Checking every page for some known content OTOH, is a much grander goal,
> but yes, I suppose some form of web-based testing framework would help,
> esp with testimg forms and all.
>
> Otherwise, we just need test data and some more test_webparts.py hacking.

I've added a new test called WebpartsRecursive that does the above, i.e. 
fetches root, then checks for links, follow all those and makes sure 
every page is returned with 200 OK _and_ that each html-page is 
*actually a valid xhtml document*.

That last part took a while to fix, but I believe I've gotten most parts 
right (at least the DTD validates everywhere now). The console took the 
brunt of these changes, so I'm hoping to see some more checking done 
there, for those that have VC-system that work with it.

The next step is to make sure the CSS is valid as well.

A few notes:

The test currently it uses lxml (http://codespeak.net/lxml) and is 
skipped if that is not installed.

It prints some lines context on the validation errors, as well as 
pointing out the column, so it's gives pretty good feedback.

I had some problems with the redirect from '/rebuild', so I just skipped 
that link, and also the twisted.web.client.getPage() doesn't give me the 
content-type at the moment, so I've manually hacked that pages ending in 
'/text' aren't validated.

I've tried to add as much data as possible to the mockup (builders, 
builds, changes, logs, etc) to make all pages visible. We need more 
categoeries, branches, failed builds, exceptions, etc to exercise all 
code paths. Hopefully that'll be easier with the scheduler-db.

Anyway, if you hack the webparts, please install lxml and run this test, 
as it covers a lot of code pretty well. You're welcome to add to it too 
(i.e. if you see that there are parts in the templates that are wrong 
but we don't have the mockup-data for it in the test.)

Happy Hacking

Cheers
/Marcus