This paper compares the relative performance of quality adjusted life years (QALYs) based on quality weights elicited by rating scale (RS), time trade-off (TTO) and standard gamble (SG). The standard against which relative performance is assessed is individual preference elicited by direct ranking. The correlation between predicted and direct ranking is significantly higher for TTO-QALYs than for RS-QALYs and SG-QALYs. This holds both based on mean Spearman rank correlation coefficients calculated per individual and based on two social choice rules: the method of majority voting and the Borda rule. Undiscounted TTO-QALYs are more consistent with direct ranking than discounted TTO-QALYs.