Use a parallel GRT if you have an intervention that operates at a group level, manipulates the social or physical environment, or simply cannot be delivered to individuals without serious risk of contamination. If you can deliver your intervention to individuals without risk of contamination and can avoid interaction among participants post-randomization, it is more efficient and easier to use a traditional RCT.

A pragmatic trial is one that helps users choose between options for care. These trials are usually done in the real world, under less well-controlled conditions than more traditional clinical trials. Pragmatic trials can use a traditional RCT design, or they can use a parallel GRT design. Stepped wedge group-randomized trials (SW-GRTs) are also used in pragmatic trials. Of 21 pragmatic trials supported by the Health Care Systems Research Collaboratory at the NIH, two are RCTs, 10 are GRTs, four are IRGTs, and five are SWGRTs.

There are five published textbooks on the design and analysis of group- or cluster-randomized trials (Campbell and Walters, 2014; Donner and Klar, 2000; Eldridge and Kerry, 2012; Hayes and Moulton, 2017; Murray, 1998). A recent textbook is devoted to power and sample size calculation for multilevel designs, including parallel GRTs, IRGTs, and stepped wedge group-randomized trials (Moerbeek and Teerenstra, 2016).

The most accurate result will be available with t-scores. For studies in which the number of units randomized to conditions is 50 or more, z-scores will work well. As the number of randomization units decreases, the df available for the test of the intervention effect also decrease, and the difference between z-scores and t-scores increases.

Yes. Sometimes investigators randomize months or weeks within clinics to study conditions. As an example, consider a study in which over the course of a year, six months are spent delivering the intervention condition and six months are spent delivering the control condition, with the order randomized within each clinic. The unit of assignment in this case is the time block within the clinic, rather than the clinic itself. Patients receive the intervention or control condition appropriate to the time block when they come to the clinic. While these groups are not structural groups like whole clinics, they are still groups, and this is still a parallel group- or cluster-randomized trial with the time block as the group. The key number in this case for power or sample size calculations is the number of time blocks, not the number of clinics. In this example, the clinic is crossed with study conditions as there are both interventions and control participants in each clinic; the clinic can be included in the analysis as a fixed effect stratification factor and that may improve power. 

It is important to distinguish between changing study conditions or study arms and changing groups or clusters. In a parallel GRT or IRGT, it is important to ensure that each participant remains in the study condition to which they were randomized. Those assigned to the intervention condition should not move to the control condition, and vice versa. Sometimes that is unavoidable, but it should be uncommon. If it does happen, standard practice is to analyze as randomized, under the intention-to-treat principle.

The other possibility is that a participant in a GRT or IRGT would change groups or clusters even as they stay in the same study condition or study arm. In a school-based trial, a participant from one intervention school might move to another intervention school. Or in an IRGT, a participant who usually went to the Tuesday night class might sometimes go to the Saturday morning class. Recent studies have shown that failure to account for changing group membership can result in an inflated type I error rate (Andridge et al., 2014). Several authors provide methods for analyzing data to account for such changes (Candlish et al., 2018; Luo et al., 2015; Roberts and Walwyn, 2013; Sterba, 2017).

Standard sources assume that each group or cluster has the same number of observations, but that is almost never true in practice. So long as the coefficient of variation (CV) of group size is less than 0.23, such variation can be ignored (Eldridge et al., 2006). But as the variation grows more marked, analysts risk an inflated type I error rate if they ignore it (Johnson et al., 2015).

In addition, power falls as the variation in group or cluster size increases, so that it needs to be addressed in the sample size calculations. There are a number of publications on this issue for GRTs (Candel and van Breukelen, 2010; Candel and van Breukelen, 2016; Hemming et al., 2020; Lauer et al., 2015; Liu et al., 2021; Moerbeek and Teerenstra, 2016; van Breukelen et al., 2007; Wang et al., 2020; Xu et al., 2019; You et al., 2011). There are also a few publications on this issue for IRGTs (Candel and van Breukelen, 2009; Moerbeek and Teerenstra, 2016).

We have known for some time that the magnitude of the ICC is inversely related to the level of aggregation (Donner, 1982). The smaller the level of aggregation, the larger the ICC. Spouse pairs and family units are small clusters, so their ICCs are often large. Moving to larger aggregates, like worksites or schools, the ICCs are usually smaller. Moving to even larger aggregates, like communities, the ICCs are usually even smaller. However, the ICC is not the only factor that determines sample size in a GRT. The variance inflation factor is defined as (1+(m-1)ICC) where m is the average number of observations in the groups randomized in the study. In a spouse pair, m=2, so that the formula is reduced to 1+ICC, and the VIF will be less than 2. If a school study, the ICC may be much smaller, e.g., 0.05, but the number of observations may be much larger, e.g., 400, and the VIF=1+(400-1)0.05=20.95, which will have a much more deleterious effect on the power of the study. It is important to account for the ICC, but also the average number of observations expected in each group randomized to the study conditions, as well as the number of groups randomized, as that dictates the df available for the test of the intervention effect. These issues are discussed in Part 4 of the Pragmatic and Group-Randomized Trials in Public Health and Medicine Course.

It is true that this approach will improve the fidelity of implementation. But the problem is that this approach completely confounds the instructor/facilitator with the study condition: everyone who gets the intervention gets exposed to that instructor/facilitator. In that situation, it is impossible to separate the effect of the intervention from the effect of the instructor/facilitator. It is possible that a charismatic instructor/facilitator could generate beneficial effects on the outcome of interest, even if the intervention itself is completely ineffective, and the investigator would not be able to distinguish those two effects. It is better to use an IRGT design in which multiple instructors/facilitators are used in the intervention condition so that variability due to instructors/facilitators can be separated from variability due to the intervention. 

In a parallel GRT, the groups are the units of assignment and are nested within study conditions, with different groups in each condition. In an IRGT, the groups are created in the intervention condition to facilitate delivery of the intervention; those groups may be defined by their instructor or facilitator, surgeon, therapist, or other interventionist, or they may be virtual groups. So long as the groups are nested within study conditions, they must be included in the analysis as levels of a random effect; ignoring them, or including them as levels of a fixed effect, will result in an inflated type 1 error rate. That is true for GRTs (Campbell and Walters, 2014; Donner and Klar, 2000; Eldridge and Kerry, 2012; Hayes and Moulton, 2017; Murray, 1998) and for IRGTs (Baldwin et al., 2011; Bauer et al., 2008; Candlish et al., 2018; Kahan and Morris, 2013; Lee and Thompson, 2005a; Lee and Thompson, 2005b; Pals et al., 2008; Pals et al., 2011; Roberts and Roberts, 2005; Roberts and Walwyn, 2013). This is because nested factors must be modeled as random effects (

). 

This explanation also offers a potential solution – if the investigator can avoid nesting groups within study conditions, the requirement to model those groups as levels of a random effect disappears. The alternative to nesting is crossing, so if it is possible to cross the levels of the grouping factor with study conditions, then the grouping factor becomes a stratification factor and the investigator is free to model the grouping factor as a random effect, as a fixed effect, or to ignore the grouping factor in the analysis. –

For example, if schools are randomized to study conditions, the study is a GRT. But if students within schools are randomized to study conditions, the schools will be crossed with study conditions and we have a stratified RCT; the investigator can model the schools as a random effect, as a fixed effect, or ignore it in the analysis. As another example, if the therapists used to deliver the intervention in an IRGT also deliver an alternative intervention in the control condition, the therapists will be crossed with study condition and the investigator can model therapist as a random effect, as a fixed effect, or ignore therapist in the analysis. In either example, the choice between modeling the grouping factor as random, as fixed, or ignoring it will depend on factors like power and generalizability.

The best estimate for the ICC will reflect the circumstances for the trial being planned. That estimate will be from the same target population, so that it reflects the appropriate groups or clusters (e.g., schools vs. clinics vs. worksites vs. communities); age groups (e.g., youth vs. young adults vs. seniors); ethnic, racial, and gender diversity; and other characteristics of the target population. That estimate will derive from data collected for the same outcome using the same measurement methods to be used for the primary outcome in the trial being planned. For example, if planning a trial to improve servings of fruits and vegetables in inner-city third graders, it would be important to get an ICC estimate for servings of fruits and vegetables, measured in the same way as servings would be measured in the trial being planned, from third-graders in inner-city schools like the schools that would be recruited for the trial being planned.

Regression adjustment for covariates often improves power in a GRT or IRGT by reducing the residual error variance or the ICC (Murray and Blitstein, 2003). At the same time, it is important to remember that regression adjustment for covariates can reduce power in a GRT or IRGT by increasing the ICC (Murray, 1998). As such, it is important to choose covariates carefully. The best covariates will be related to the outcome and unevenly distributed between the study conditions or among the groups or clusters randomized to the study conditions.

A priori matching can improve power in a GRT, but it can also reduce power, so investigators need to be thoughtful about a priori matching in their design and analysis. A priori matching reduces the df for the test of the intervention effect by half, and if the correlation between the matching factor and the outcome is not large enough to overcome the loss of df, power will be reduced in the matched analysis compared to the unmatched analysis.

A priori matching is often used to balance potential confounders, and it is then up to the investigator to decide whether to reflect that a priori matching in the analysis. It is not required, because the type 1 error rate is unaffected when the matching or stratification factor is ignored in the analysis of intervention effects (Diehr et al., 1995; Proschan, 1996). However, Donner et al., 2007 have warned against ignoring matching in analyses that do not involve intervention effects, e.g., in an analysis to examine the association between a risk factor and an outcome. Ignoring matching in the analysis in this situation can lead to an inflated type 1 error rate when the correlation between the matching factor and either the outcome or the risk factor is at least modest (>0.2) and the number of members per group is not large (<100). Stratification with strata of size four avoids this problem and improves efficiency almost as much as matching. For this reason, stratification with strata of size four is a prudent strategy for balancing potential confounders across study conditions or study arms.

A priori stratification can also improve power in a GRT, but the situation is more complicated, because it depends on how the stratification is reflected in the analysis. As with a priori matching, a priori stratification can be used to balance potential confounders, and it is then up to the investigator to decide whether and how to reflect that a priori stratification in the analysis.

If the primary interest is to balance on potential confounders, the stratification factor could be included in the analysis as a covariate, but without creating interactions with study condition or other factors. To the extent that the stratification factor is related to the outcome, there is likely to be benefit to power, because the gain from the regression adjustment is likely to outweigh any reduction due to lost df.

If the primary interest is differential intervention effects, the stratification factor is included in the analysis as a main effect, but additional interaction terms are required, both for fixed and random effects. The number and nature of the additional fixed and random effects will depend on the design and analytic plan (Murray, 1998; Murray, 2001). Inclusion of the correct fixed and random effects is essential to a valid analysis, so investigators are strongly encouraged to work with a methodologist familiar with stratified designs to ensure that the analysis is structured correctly. Regarding power, detection of differential intervention effects will always require a larger study than detection of uniform intervention effects.

Constrained randomization is an alternative to a priori matching or stratification (Li et al., 2016; Li et al., 2017b). It can be used to balance across a larger number of covariates than is typically possible with matching or stratification, usually improves power, and can be used either with model-based or permutation-based tests.

Some have suggested that 4 groups or clusters per study condition should be considered as an absolute minimum (Hayes and Moulton, 2017). Investigators should be cautious about such rules of thumb because it is quite possible that 4 groups or clusters per study condition would result in a badly underpowered trial. ICCs in public health and medicine often fall in the range of 0.01–0.05, and if the ICC does fall in that range, 8–12 groups or clusters will often be needed in each study condition. The best advice is to estimate sample size requirements for the trial under consideration, using the best parameter estimates available.

There is no general answer to this question. Instead, investigators should estimate sample size requirements for the trial under consideration, using the best parameter estimates available. At the same time, it is fair to say that increasing the number of groups or clusters per condition will more effectively increase power than will increasing the number of members per group or cluster.

No. When a priori matching or stratification is used for balance, the matching or stratification factor may be included in the analysis of intervention effects, but that is not required, and it may be inefficient to do so. It is not required because the type 1 error rate is unaffected when the matching or stratification factor is ignored in the analysis of intervention effects (Diehr et al., 1995; Proschan, 1996). Both procedures reduce the df available for the test of the intervention effect, and if the number of df is limited, the unmatched or unstratified analysis may be more powerful than the matched or stratified analysis. In that circumstance, it is to the investigator’s advantage to match or stratify in the design to achieve balance on potential confounders, but to ignore the matching or stratification in the analysis to improve power or reduce sample size (Diehr et al., 1995). The choice of whether to include the matching or stratification factor in the analysis should be made a priori based on sample size calculations comparing the matched or stratified analysis to the unmatched or unstratified analysis.

The choice between a priori matching and a priori stratification for balance should be guided by whether the investigator anticipates doing analyses that do not involve intervention effects. Donner et al., 2007 have warned against ignoring matching in analyses that do not involve intervention effects, e.g., in an analysis to examine the association between a risk factor and an outcome. Ignoring matching in the analysis in this situation can lead to an inflated type 1 error rate when the correlation between the matching factor and either the outcome or the risk factor is at least modest (>0.2) and the number of members per group is not large (<100). Stratification with strata of size four avoids this problem and improves efficiency almost as much as matching. For this reason, stratification with strata of size four is a prudent strategy for balancing potential confounders across study conditions because it is almost as efficient as matching, and it does not limit the range of analyses that can be applied to the data.

No. In public health and medicine, ICCs in group- or cluster-randomized trials are often small, usually ranging from 0.01–0.05 (Moerbeek and Teerenstra, 2016). While it is tempting to ignore such small correlations, doing so risks an inflated type I error rate, and the risk is substantial both in parallel GRTs (Campbell and Walters, 2014; Donner and Klar, 2000; Eldridge and Kerry, 2012; Hayes and Moulton, 2017; Murray, 1998) and in IRGTs (Baldwin et al., 2011; Bauer et al., 2008; Candlish et al., 2018; Kahan and Morris, 2013; Lee and Thompson, 2005b; Lee and Thompson, 2005a; Pals et al., 2011; Pals et al., 2008; Roberts and Roberts, 2005; Roberts and Walwyn, 2013). The prudent course is to reflect all nested factors as random effects and to plan the study to have sufficient power given a proper analysis.

No. That is another tempting strategy that can risk an inflated type I error rate. The standard error for the variance component is not well estimated when the value is close to zero, and if the df are limited, the power will be limited. As such, it is likely that the result will suggest that the ICC or variance component is negligible, when ignoring it will inflate the type I error rate. The prudent course is to reflect all nested factors as random effects and to plan the study to have sufficient power given a proper analysis.

There are three common analytic models used for pretest-posttest parallel GRTs. (Murray, 1998).

First, one could analyze the posttest data, ignoring the pretest data altogether.

Second, one could analyze the posttest data with regression adjustment for covariates measured at baseline, including adjustment for the baseline measure of the outcome, as is common in a cohort design.

Because the second approach includes regression adjustment for covariates, it is often more powerful than the first. Both the first and second approaches focus on the simple difference between the two study conditions or study arms at a single point in time, regardless of the number of measurement occasions included in the design. They can be applied to cohort or cross-sectional designs, to designs that collect only posttest data, or to designs that include two or more observations on the same members or groups but focus on a single point of time in the analysis. They are most often applied to a pretest-posttest design, where the difference between the two conditions is evaluated at posttest. The analysis of a simple difference provides results that are typically displayed in a bar graph, where the two bars represent the two conditions. The intervention effect is interpreted as the unadjusted or adjusted difference between the two conditions at a particular point in time.

Third, one could analyze the pretest and posttest data in a repeated measures analysis, equivalent to an analysis of a net difference, with or without regression adjustment for covariates. The analysis of a net difference provides results that are typically displayed in a line graph, where the two lines represent the trends over time in the two conditions, and the error bars placed on one line represent the standard error of the difference between the two conditions. The intervention effect is interpreted as the unadjusted or adjusted net difference between the two conditions over time. Because the analysis of a net difference is based on a comparison of four means, proportions, slopes, or other statistics, it is usually less powerful than the analysis of a simple difference, which is based on a comparison of two means, proportions, slopes, or other statistics. These models can be fit using the general linear mixed model for normally distributed outcomes and using the generalized linear mixed model for outcomes that have one of many non-normal distributions.

The most common design in a parallel GRT is a pretest-posttest design (Murray, 1998). However, some trials include additional baseline measurements and/or follow-up measurements. If the investigator wants to include no more than two time points in the analysis (e.g., pretest and posttest, or pretest and one year follow-up), a mixed-model repeated measures ANOVA/ANCOVA can be used and is expected to carry the nominal type 1 error rate (Murray, 1998). However, if the investigator wants to include three or more time points in the analysis (e.g., baseline, posttest, one year follow-up), the mixed-model repeated measures ANOVA/ANCOVA should not be used (Murray, 1998). The mixed-model repeated measures ANOVA/ANCOVA assumes that the group-specific time trends within a study arm are homogeneous and if that assumption does not hold, the mixed-model repeated measures ANOVA/ANCOVA will have an inflated type I error rate. Because there is no test for this assumption within the mixed-model repeated measures ANOVA/ANCOVA, the prudent course is to avoid this analytic model. Instead, a random coefficients or growth-curve model can be used and is expected to have the nominal type 1 error rate even in the presence of heterogeneity for the group-specific slopes within a study arm (Murray, 1998). Some have suggested that the mixed-model repeated measures ANOVA can be used with more than two time points in the analysis if it includes an unstructured covariance matrix (Bell and Rabe, 2020), but more recent work has shown that is not always the case, again recommending the random coefficients model when the analysis will include three or more time points (Moyer and Murray, 2021).

There are a variety of methods that can provide an appropriate analysis of data from a parallel GRT, including mixed models, two-stage methods, randomization tests, methods based on generalized estimating equations (GEE), and non-parametric or semi-parametric methods (Campbell and Walters, 2014; Donner and Klar, 2000; Eldridge and Kerry, 2012; Hayes and Moulton, 2017; Murray, 1998; Murray et al., 2008; Turner, Prague, et al., 2017b). Used properly, these methods will give similar results when applied to data from a GRT with many groups or clusters (>20 per condition). Mixed-model regression methods are the most common methods used to analyze data from GRTs (Murray et al., 2018); in addition, mixed models, two-stage methods, and randomization tests will give similar results even for smaller studies. As such, most of the material on this website assumes that the analysis will employ mixed-model regression methods. Randomization tests will be preferred for very skewed or heavy-tailed distributions as they preserve the type 1 error rate while model-based methods may be conservative (Fu, 2006; Murray et al., 2006). Standard GEE will have an inflated type 1 error rate as the degrees of freedom for the test of the intervention effect fall below 40, with the inflation growing worse as the degrees of freedom decline (Bellamy et al., 2000; Huang et al., 2016; Kauermann and Carroll, 2001; Lu et al., 2007; Mancl and DeRouen, 2001; Murray et al., 2004); small sample corrections are available but users should take care to select a correction that will work as intended in the circumstances at hand (Bie et al., 2021; Fay and Graubard, 2001; Jackson et al., 2021; Kahan et al., 2016; Leyrat et al., 2018; Li and Redden, 2015; Liu et al., 2021; Lu et al., 2007; Mancl and DeRouen, 2001; McCaffrey and Bell, 2006; McNeish and Stapleton, 2016; Morel et al., 2003; Pan and Wall, 2002; Preisser et al., 2008).