Ask the question on timetriallingforum.co.uk
There are a number of TT'ers there who are diehard on fixed gear (usu BIG fixed gear), reckon it's quicker than a cassette.
Obree was one when TT'ing domestically in the UK : set a record in the Tour of the Trossachs which was considered 'hilly' (for Scotland !)
I guess it will force you to push hard uphill to maintain cadence and hence speed, when having multiple gears could tempt you to drop down a gear or two and hence slow down. This uphill gain will more than make up for the speed you'd lose spinning downhill rather than being able to change up and push a bigger gear coming back the other way.
In hillclimbs, then assuming you have the gear right, the lower weight and no temptation to faff-about changing gears must make it quicker too.
For myself, I'm not a strong enough fixed rider to do it, I feel I need my gears although generally I don't use that many of them even on rolling, not dragstrip DC, courses.
I do run a 12-21 9sp cassette, with 1-tooth gaps apart from 19-21, and 55/42 chainrings : the 42 is there for getting over steeper hills on training rides, where I might struggle on 55-21