One of the greatest challenges in the vQTL detection is how to choose the scale of measurement on which to draw conclusions about estimated effects. When we detect a SNP whose alleles increase both the mean and the variance, should we interpret it as a vQTL with a significant marginal effect, or a mean-controlling QTL for a trait that was analyzed on the wrong scale? Suppose, for instance, that we had a cylindrical organism (such as a snake or a worm) whose body width increases with its body length, and we have a gene with a strong additive effect on body length, which is itself normally distributed. A GWAS on the volume of this cylindrical organism is likely to detect a QTL controlling both mean and the variance, despite the fact that a simpler explanation is available.

To circumvent such ambiguities, some researchers have used the coefficient of variation (CV; the standard deviation divided by the mean) to detect genotypic effects on the variance [12]. This approach is highly applicable for a trait that is positive-valued, and for which (under a null model) the standard deviation would be expected to increase with the mean. A more flexible remedy is to find a transformation of the trait for which the residuals under a mean-effects model are approximately Gaussian (and thereby symmetric), and estimate genotypic effects on this transformed phenotype. This helps guard against vQTL interpretations that could be otherwise explained by a combination of mean effects and transformation. We [1] and Yang et al. [35] promote the use of the Box-Cox procedure to help guide such transformations, with Yang et al. making this procedure an intrinsic and simultaneously estimated component of their Bayesian model. The QTL and vQTL effects detected on a transformed scale can be back-transformed to the original scale to study their effects on the scale of interest.

In some cases, the trait is known to be well approximated by a distribution that has a known mean-variance relationship, such as the Poisson or gamma distribution. In such cases, it will often be preferable to model that distribution explicitly and define vQTL parameters as those that alter the variance in a way not already anticipated by concomitant effects on the mean. DGLMs provide a natural basis for such models when the known sampling distribution is member of the exponential family [1].

To illustrate the effect of a misspecified distribution on vQTL detection, we consider the following simulation study in which vQTL-detection methods are applied to a non-normal phenotype. Let the phenotype _{
Y
i
} of individual *i* be distributed as _{
Y
i
}∼Poisson(_{
μ
i
}), where the individual’s genotype _{
g
i
} affects the phenotype through the relation
, with *μ*=−1,_{
β
g
}=0.05. This set up is similar to that used in Struchalin et al. [7] with the difference that whereas they use a normally distributed phenotype subject to interactions (ie, a true vQTL signal), we use a Poisson distribution with no interaction effect (ie, no vQTL signal). We performed 1000 simulation trials. In each, we generated 10,000 phenotypes and tested the significance of a vQTL effect using three methods: SVLM, DGLM with a Gaussian response (DGLM-normal), and DGLM with a Poisson response (DGLM-Poisson). At a 95% significant level, Levene’s test, SVLM and DGLM-normal had strongly inflated FPR, whereas DGLM-Poisson had a more reasonable FPR of 5.5% (Table 1). Hence, SVLM or a misspecified DGLM are likely to produce a large proportion of false positives for a trait whose variance is a function of its mean. Further research is required to distinguish trait values generated by a distribution where the variance explicitly depends on the mean from a process where the variance is controlled by a vQTL.