Yeah, you'd have to be a power meter geek, or a physics professor, to get to the end of this article before zoning out.

Are you aware that you can check the torque calibration (not just the zeroing) of the PT by doing a static torque check? Google "stomp test" and "powertap" together and you should find all the info you should need.

That will simply and quickly tell you if the PT is reading accurately or not. It doesn't sound like that's possible with the P2Max or the Look/Polar setups.

Well, it may be geeky but it does give you an idea of how difficult it is to do these kinds of comparisons. It's easy to slap a piece of equipment on your bike, go for a ride, and say, "it's firm but supple and responds instantaneously as if reading my mind." In particular, when power meters are so expensive you should probably be thankful that Jeff is willing to do careful work.

I'm no physics professor* but I like it. It's clear these devices aren't as consistent as you'd like them to be. No regular rider would ever use a set-up like this, but for a magazine it's a great asset IF the readings were consistent. You could use it to test the efficiency of different frames etc in a way that is a bit more quantitative, but it seems that's still a long way off. Keep us posted on the rest of the test!

* OK, I admit, I do have a phd in the subject

@Jmayhem: condensed version "I'm not satisfied with either of these power meters yet" - the blog was more to illustrate the testing process, as I wrote in the opening two pars.

@Tom A. - Yep, although I've not done one on this PowerTap. I will.

@RChung thank you :-)

@ak-47 - they're not and it's important to be able to test this, as measuring power accurately is their primary function. Otherwise you may as well use a cheap speedo.

I'll update this again next week after I do some more testing and calibrations, as well as comments from Power2Max and Look

Couple of things to point out from a PowerTap perspective-

1- The initial release of Joule 2.0 did not fully manage the calibration of crank based power meters but the latest version of released firmware (version 14.072) now supports this function.

2- The PowerTap also "auto-zeros" during coasting.

3- There are significant differences in how ANT+ displays are configured out of the box that could be responsible for some of the differences you are seeing in your results. For example, the Garmin Edge 705 uses a recording method that optimizes data for storage but compromises some fidelity from the power sensor. Also, some displays average power with zeros included and some do not.

To complicate things further, if you're numbers posted above are from a software program there may be differences there as well. Speaking from experience, it seems best to keep all displays and software identical and only vary the power meter as much as possible. That being said, it is possible to set up Joule and a Garmin identically from a data collection perspective and most software programs can handle the data the same as well.

Jesse Bartholomew

CycleOps Product Manager

Thanks Jesse - fwiw I own three PowerTaps and they all 'agree' with each other as far as I can tell. They are all set to auto-zero and my experience with them is that they don't drift much in changing temperatures.

The Garmin has Smart recording turned off and it does include zeros in power averaging (as does the Joule).

I have the 14.072 firmware but the calibration value it gave me for the Power2Max didn't look right. I'll recheck, although I think the head unit pairing I have is better because the Joule will auto-record off the PowerTap (speed) whereas I don't have a speed or an ANT+ heart rate sensor for the Power2Max, so it won't record. Given that I eyeballed the same trends with the units swapped, I don't think the head units are the problem.

I used WKO+ 3.0 with Device Agent to download the PowerTap and Power2Max data then exported as csv files for VE analysis (I've used Power Agent too and never noticed a difference in data). I used the Protrainer 5 software to download the Polar data then exported that. Again I'm pretty sure that's not causing problems as the data I'm seeing on screen seems to match up with what's downloaded.

My PowerTap, which Saris looked at, tests significantly different than a "control" wheel (my GF's) which agrees closely with theory:

http://djconnel.blogspot.com/2010/05/powertap-torque-test-post-service.html

So I don't trust PowerTaps without clear data. That said, if foil strain gauges stretch, loosen, or fatigue, I'd expect them to underreport (as mine appears to) rather than over-report.

Sounds like you've got the pieces in place for a good comparison. Happy testing!

Doesn't the Powertap have options to be set as Auto Zeroing and Manual Zeroing? Because the Power2Max and the Look Pedals are autozeroed every time you stop pedaling, I assume you should've done the same to the Powertap. If you manually zeroed the Powertap, IMO the virtual elevation modelling should have differences, as I can't see exactly where in the ride you zeroed the Powertap.

In the virtual elevation graph, I can see that the measured power is relativley same in the climbs and the flats, but drift drastically in the descents. This could be because of the changing temperatures in the windy, fast and cold descents. Power2max should've zeroed itself, but assuming you manually zeroed Powertap, it would've read different values, thus reason for power drift.

KeithF: Actually, both the P2Max and the Look read lower than the PT across all terrain but the difference is greatest not in the descents but on the climbs. There are two ways to make these comparisons: first, if one created wattage "mean-difference" plots (for example, plotting the mean of the PT and P2Max watts on the x-axis and their difference on the y-axis; and repeating this for PT vs. Look/Polar and for P2Max vs. Look/Polar) one could see that the P2Max and Look/Polar read lower than the PT at all power levels, but the discrepancy grows for the P2M compared to both the PT and the Look/Polar at the highest power.

Second, we can plot power against the "virtual slope." As is common, power increases during climbs and in general the steepest slopes demand the highest (sustained) power. As before, the P2Max and Look/Polar are "shifted down" compared to the PT but in addition there's a "twist" for the P2Max so it reads much lower than the PT at steeper slopes, where the highest power occurs.

One small thing to note is that on the second descent, the P2M started reading really really low for some reason and it wouldn't correct through autozero. It lasted a few minutes until I went to the calibration screen and did a manual one on the fly (no pressure on the pedals, but still clipped in)

That's why the green line dips below the blue at the 30km mark

Really interesting comparison, well done for being so methodical.

Some thoughts. Make sure the data recording is the same for each unit....eg 1 second. Careful about their averaging functions and read whilst stopped functions.

I would suggest to allow each unit to acclimitise to the ambient temp before starting (I notice a a 10w variation in powertap without this).

Repeat the exact same ride several times (as this will at least offer repeat test comparability esp for altitude and distance).

Consider riding a controlled route at sealevel, altitide 1, altitude 2 etc.

Await your final report with interest!!

also check out this very interesting comparison for PT and SRM (for their methods)

GARDNER, A. S., S. STEPHENS, D. T. MARTIN, E. LAWTON, H. LEE, and D. JENKINS. Accuracy of SRM and Power Tap Power Monitoring Systems for Bicycling. Med. Sci. Sports Exerc., Vol. 36, No. 7, pp. 1252-1258, 2004.

Purpose: Although manufacturers of bicycle power monitoring devices SRM and Power Tap (PT) claim accuracy to within 2.5%, there are limited scientific data available in support. The purpose of this investigation was to assess the accuracy of SRM and PT under different conditions.

Methods: First, 19 SRM were calibrated, raced for 11 months, and retested using a dynamic CALRIG (50-1000 W at 100 rpm). Second, using the same procedure, five PT were repeat tested on alternate days. Third, the most accurate SRM and PT were tested for the influence of cadence (60, 80, 100, 120 rpm), temperature (8 and 21°C) and time (1 h at ~300 W) on accuracy. Finally, the same SRM and PT were downloaded and compared after random cadence and gear surges using the CALRIG and on a training ride.

Results: The mean error scores for SRM and PT factory calibration over a range of 50-1000 W were 2.3 ± 4.9% and -2.5 ± 0.5%, respectively. A second set of trials provided stable results for 15 calibrated SRM after 11 months (-0.8 ± 1.7%), and follow-up testing of all PT units confirmed these findings (-2.7 ± 0.1%). Accuracy for SRM and PT was not largely influenced by time and cadence; however, power output readings were noticeably influenced by temperature (5.2% for SRM and 8.4% for PT). During field trials, SRM average and max power were 4.8% and 7.3% lower, respectively, compared with PT.

Conclusions: When operated according to manufacturers instructions, both SRM and PT offer the coach, athlete, and sport scientist the ability to accurately monitor power output in the lab and the field. Calibration procedures matching performance tests (duration, power, cadence, and temperature) are, however, advised as the error associated with each unit may vary.

also here is a protocol for checking calibration with weights, including a calculator

http://www.cyclepowermeters.com/powertap-garmin-calibration-check-76-c.asp

Thanks Jeff and Robert, this should be very interesting. Do you have any info or experience regarding the Garmin Vector?

It seems like there are two issues. One is the accuracy and repeatability of P2M when zeroed and the other its drift.

Regarding drift, current knowledge says that after an initial 10 to 20 minute settling period the unit stabilizes and drift becomes mainly temperature related.

My observations over the last 4 months only partially confirm this. For example, my unit varies up to 8 units, typically 5, (nominal is around -1060) , just when performing multiple Calibrates (Garmin speak for Zeroing) within 1 or 2 minutes, with some gentle pedaling in between, outdoors in the shade. These are performed while stopped and unclipped.

I have also performed multiple Calibrates while riding, no pedaling and feet clipped, just as P2M supposedly does. The values bounce around the same as when stopped and unclipped but are usually a few units more negative than the corresponding stopped and unclipped values.

Also, my rides start in the early morning. If the theory about the temperature drift was correct I would expect to see a steady trend of increasing or decreasing zero values along the ride as the ambient temperature increased. This is not the case.

My take on the P2M is that its variation is due to mechanical (chain rings to crank to P2M?) and Electromechanical settling as well as due to temperature changes and that the first two are not restricted to an initial settling period of 10 to 20 minutes. Instead, after the settling period there is still some important influence from the mechanical and electromechanical factors.

It would be interesting to ask someone who rides with a significant temperature chznge to plot autozero values vs temperature over the course of a ride in an attempt to obtain a liner relationship.

FWIW, this is my 2nd unit as P2M requested I return the 1st one when I described a 36 unit drift ( -768 to -732) after the settling period and without significant temperature changes (less than 10 C). I also described "jumps" in power when compared to the PT SL+ I was simultaneously using at the time. In fairness to P2M they were very prompt in returning another unit. I was later told it was the result of faulty electronics.

Noc

Did some further testing regarding temperature induced drift.

After the bike had been stored indoors at 19º C for 2 hours, the Edge 800 repeatedly reported -1070. When moved outside, in the shade, where the temperature was 13º C it stabilized at -1056. Approximately 2 units per degree C. About 1 unit per degree F.

Once the bike was brought back inside, the values eventually returned back to -1070.

No pedaling, just enough motion of the crank to wake the PM up before each reading.

Update to temperature induced changes to zero offset values:

Bike indoors, 2 hours at 19ºC (as mentioned above) = -1070

Bike outdoors, 2 hours in shade at 13ºC (as mentioned above) = -1057

Bike outdoors, 2 hours in direct sunlight = - 1150

These tests were done with the bike stopped, on 2 consecutive days with similar results. I live at 38º N so that even though the bike was in direct sunlight (between 11 AM and 1PM), the angle of incidence at this time of year is moderate when compared to other latitudes nearer the equator and/or nearer to summer.

I don’t have a thermocouple able to read the metallic surface temperature of the PM but will repeat this test again next week when I do have one

The immediate conclusion is that when using this PM utmost attention must be given to variables that can affect the core temperature of the device.

Further temperature vs zero offset information:

A P2M Rotor crank assembly (minus left crank) was placed in a climatic chamber. Zero offset values were read for each temperature once the crank assembly had reached surrounding temperature (typically 30mins). The stable readings of the zero offset value further confirmed this.

The door of the chamber would be opened briefly to wake up the PM then closed again. The Edge was able to read the zero offset values with the door closed. These tests were repeated 3 times with similar (+-3) zero offset values.

The temperatures (in degrees Celsius) and zero offset values were the following:

-10 -1040

6 -1042

10 -1047

15 -1068

20 -1088

30 -1158

40 -1205

At the lower temperatures the temperature appears to have less effect on the zero offset but as the temperature increases the impact becomes much larger. In other words, the relationship is not linear.

Granted the crank was not in use (on a bike with force being applied) so I really do not whether if this had been the case if the values would have been different.

FWIW, one time when the crank was removed from the -10C trial I noticed that the zero offset values briefly changed to -1011. I have no idea what caused this to happen.

Using the above data, with the temperature between 10 and 30C, the calibration curve info provided by P2M and assuming a 90RPM cadence the following would be true:

+1 Degree C = -5.55 Zero offset units

+1 Degree C = -8.0475 Watts @90 RPM

Regards,

Interesting article - this is the first thoughtful piece I've seen on these two power meters. I've been using the keo power pedals since August last year after scoring an early set. I've found them pretty good save a couple of gripes which I'll get to in a moment. I chose the Keo power system as they are easily moved from my road bike to TT bike as required. Also as I was recovering from a serious injury I appreciated the left right balance feature which is unique to the keo power system (at least until the Garmin Vector pedals are release in a few months). Now for the gripes.

The attachment of the pedals to the crank is not as well resolved as it should be. As the pedal axles need to rotated in a specific geometry relative to the crank they must be tightened by lock rings between the crank surface and pedal body. With my DA 7900 cranks I cannot tighten the pedals all the way on as the axle just protrudes beyond the inner surface of the crank which means the transmitters cannot be fitted flush to the inner surface of the crank as they need to be. As a result I need to back the axle out a full turn and tighten with the lock ring. As a consequence the pedals have a wider Q factor than my regular Keo Carbons, requiring a repositioning of the cleat on my shoes. Also it is imperative that the lock rings are tightened to the required torque otherwise the pedals can intermittently loosen mid ride which is quite disconcerting. This freaked me the first time it occurred as the pedal body became unstable during a ride, but by the time I got home to check they had re-tightened again. It took a while to work out what was happening, but now I know to tighten then really firmly (they won't tend to tighten as you ride like normal pedals will). It seems to me that the Garmin Vector pedals are better resolved in this regard as they don't use a lock ring.

Second gripe - every ride there might be an abnormal power spike or two (maybe only two or three power values out of thousands). It is easily fixed when downloaded but I don't know why they should occur.

Regarding the review I'm intrigued by the 'low' power readings reported in this article for the Keo's (I can't imagine Look would be happy with the power readings being 5-10% out (if you believe their testing the pedals are only a few watts different from a SRM lab quality ergo / power meter see http://www.youtube.com/watch?v=QhxOvIi-SMw pause @1min 23 sec for the hard data). One of the articles referenced above also notes that SRM powermeters (which are considered the gold standard) returned power readings 5-7% below a power tap. So I'm not sure you should be using the Power Tap as the reference point for this review. I guess your forthcoming update with the ergo will clear it up.

Finally I'm further amazed at the temperature drift issues. I understand SRM power meters have temperature compensation. I cannot believe Look and others have not incorporated compensation into their systems. My Polar CS600x records temperature at the same intervals as all other data so I can't see why compensation could not be done in the head unit on the fly or software once downloaded when the recorded temp varies during a ride. This is pretty basic stuff for a consumer product. Thanks for the article, can't wait for the update!

Thanks for this review. I race TTs and use a powertap with disc cover. Pedal based power will allow me to invest in race-day wheelsets.

cheers

Roadrunner: thanks for sharing your experiences. Had you used any other power meter prior to the Look/Polar system?

Robert, the Look/Polar System is the first I have owned but I have used a PT before. To follow up on my previous post, I may be giving the Look system a bit of a bum rap on the very occasional power spike. I just noticed this pop-up in golden cheetah when correcting power spikes:

"Occasionally power meters will erroneously report high values for power. For crank based power meters such as SRM and Quarq this is caused by an erroneous cadence reading as a result of triggering a reed switch whilst pushing off."

Sure enough when I look at my files the power spikes are accompanied by an erroneous cadence reading of 199rpm which immediately follows a 0 cadence reading.

Also regarding my issue on the fitting of the KeO pedals. I should qualify this and say they are in no way flimsy. The pedals themselves are very robustly engineered and are of very high quality (which you would hope given their price), its just that the lock ring system for attachment could be better designed. Properly tightened to spec (read the instructions carefully people...) they are as solid as a rock.

Also something I forgot to mention, personally I would like to see some more sophisticated during and post ride analysis options for the Keo power system which allows analysis of pedal technique. At a minimum this could occur when downloaded but ideally in real time (in much the same way that the WattBike does). Apparently this might be in the works as Polar is reported to be updating its software and head units in the coming year. At the very least the head units should allow real time display of 5,10,30 sec average power (as selected by the user), as well as easy to access interval summaries showing average power as the CycleOps Joule 2.0 does (although to be fair the CS600x does display average power at the end of each rep of a programmed interval for a few seconds on the fly).

+1 on the usefulness of this work. I am surprised by the size of the differences you uncovered, since all the manufacturers state 1.5% etc. I hope you can repeat a similar comparison with other meters when they come out.

I'd like to see the error plots mentioned by RChung, along with plots showing error per gear. This is great work because bottom line, if you're unable to get reliable outputs from power meters, despite time, expert support and manufacturer feedback, then they are not worth buying.

It's not so much the $ cost of the meter, but more the hours of potentially wasted training time which you can't get back, recalibration malarkey and need to send it back for servicing....