Individually Randomized Group-Treatment (IRGT) Trials

In an individually randomized group-treatment (IRGT) trial, also called a partially clustered design, individuals are randomized to study conditions but receive at least some of their intervention with other participants or through an interventionist or facilitator shared with other participants. (Baldwin et al., 2011 ; Bauer et al., 2008 ; Kahan and Morris, 2013 ; Lee and Thompson, 2005b ; Lee and Thompson, 2005a ; Pals et al., 2008b ; Pals et al., 2011 ; Pals et al., 2008a ; Roberts and Roberts, 2005 ; Roberts and Walwyn, 2013).

Special methods are required for analysis and sample size estimation for these studies, and investigators will need to show that their methods are appropriate.

Features and Uses

Groups or Common Interventionist or Facilitator

An IRGT trial is a randomized trial in which participants in one or more study conditions receive at least some of their treatment in groups or through a common interventionist or facilitator. This design is common in surgical trials, where each surgeon operates on multiple patients (Cook et al., 2012 ; Oltean and Gagnier, 2015).

It is common in psychotherapy trials, where a therapist may treat multiple patients, either in groups or as individuals (Baldwin et al., 2005 ; Carlbring et al., 2006 ; Sterba, 2017). It is common in a variety of intervention trials addressing health behaviors such as weight loss, smoking cessation, and physical activity, which may include group activities as well as individual activities (Jeffery et al., 2006).

Nested or Hierarchical Design

IRGTs always have a hierarchical structure in the intervention condition. Participants may receive some of their treatment in groups, or they may receive their intervention individually, but through a common interventionist or facilitator, whether in person or through a video or other virtual connection (Lee and Thompson, 2005b). These issues are especially challenging in an IRGT because the design may not have the same hierarchical structure in all conditions. In that case, the analytic model must accommodate a heterogeneous variance-covariance structure, allowing for intraclass correlation (ICC) in one condition, but not in the other. For this reason, it is even more important for investigators to rely on an experienced methodologist in developing design and analytic plans for an IRGT.

Appropriate Uses

IRGTs can be employed in a wide variety of settings and populations to address a wide variety of research questions. They are an appropriate design when the investigator wants to evaluate an intervention that:

  • involves at least one component that is delivered in a group format,
  • it is necessary to use a limited number of intervention delivery staff, or interventionists or facilitators, so that each one interacts with multiple participants, or
  • it is necessary to have participants interact with one another in a virtual environment.

Potential for Confounding

IRGTs randomize individuals to study conditions. If the number is large, confounding is not likely to be a threat to the internal validity of the design. If the number is small, confounding could be a threat, and a priori matching or a priori stratification would be useful strategies to protect against confounding.

Intraclass Correlation (ICC)

The more challenging feature of IRGTs is that participants in at least the intervention condition will interact post-randomization. Even if their groups or virtual networks are constructed using random assignment, those participants will interact with one another directly in their groups or indirectly through their common interventionist or facilitator. This interaction creates the expectation that some level of ICC will develop. The magnitude of the ICC in an IRGT will depend on the type, duration, and intensity of these interactions. That ICC may be negligible at baseline, but it can develop over the course of the trial. With a limited number of groups or interventionists or facilitators, the degrees of freedom (df) available to estimate the ICC, or the component of variance associated with the group or interventionist or facilitator, will be limited. As for group-randomized trials (GRTs), any analysis that ignores the extra variation (or positive ICC) or the limited df will have a type 1 error rate that is inflated, often badly. (Baldwin et al., 2011 ; Bauer et al., 2008 ; Kahan and Morris, 2013 ; Lee and Thompson, 2005b ; Lee and Thompson, 2005a ; Pals et al., 2008b ; Pals et al., 2011 ; Pals et al., 2008a ; Roberts and Roberts, 2005 ; Roberts and Walwyn, 2013).

Solutions

The recommended solution to these challenges is similar to the solution proposed for GRTs. It is important to employ a priori matching or a priori stratification to balance potential confounders if the number of assignment units is limited, to reflect the hierarchical or partially hierarchical structure of the design in the analytic plan, and to estimate the sample size for the IRGT based on realistic and data-based estimates of the ICC and the other parameters indicated by the analytic plan. Extra variation and limited df always reduce power, so it is essential to consider these factors while the study is being planned, and particularly as part of the sample size estimation.

The sections below provide additional resources for investigators considering an individually randomized group-treatment trial, or partially clustered design.

 

Use an IRGT if you can randomize individual participants but need to deliver the intervention in groups or through a common interventionist or facilitator either for pedagogic or practical reasons. If you can randomize individual participants and can deliver the intervention to participants one at a time without going through a common interventionist or facilitator, it is more efficient and easier to use a traditional RCT.

It is important to distinguish between changing study conditions or study arms and changing groups or clusters. In a parallel group RCT, GRT, or IRGT, it is important to ensure that each participant remains in the study condition to which they were randomized. Those assigned to the intervention condition should not move to the control condition, and vice versa. Sometimes that is unavoidable, but it should be uncommon. If it does happen, standard practice is to analyze as randomized, under the intention-to-treat principle.

The other possibility is that a participant in a GRT or IRGT would change groups or clusters even as they stay in the same study condition or study arm. In a school-based trial, a participant from one intervention school might move to another intervention school. Or in an IRGT, a participant who usually went to the Tuesday night class might sometimes go to the Saturday morning class. Recent studies have shown that failure to account for changing group membership can result in an inflated type I error rate (Andridge et al., 2014). Roberts and Walwyn, 2013 provide a good method for analyzing data to account for such changes.

Standard sources assume that each group or cluster has the same number of observations, but that is almost never true in practice. So long as the ratio of the largest to the smallest group is no worse than about 2:1, such variation can be ignored. But as the variation grows more marked, analysts risk an inflated type I error rate if they ignore it (Johnson et al., 2015).

In addition, power falls as the variation in group or cluster size increases, so that it needs to be addressed in the sample size calculations. There are a number of publications on this issue for GRTs (Candel and van Breukelen, 2010; Candel and van Breukelen, 2016; Moerbeek and Teerenstra, 2016; van Breukelen et al., 2007; You et al., 2011). There are also a number of publications on this issue for IRGTs (Candel and van Breukelen, 2009; Moerbeek and Teerenstra, 2016).

This will depend on several factors, including the expected ICC that reflects the average correlation among observations taken on members of the same small group, and the number of participants in each group. There is no one answer that will be correct for all studies. The best approach is to perform a power analysis, or calculate sample size, using methods appropriate to IRGTs. There are a number of papers that address this question (Moerbeek and Teerenstra, 2016; Pals et al., 2008b; Pals et al., 2008a).

It is true that this approach will improve the fidelity of implementation. But the problem is that this approach completely confounds the instructor/facilitator with the study condition: everyone who gets the intervention gets exposed to that instructor/facilitator. In that situation, it is impossible to separate the effect of the intervention from the effect of the instructor/facilitator. It is possible that a charismatic instructor/facilitator could generate beneficial effects on the outcome of interest, even if the intervention itself is completely ineffective, and the investigator would not be able to distinguish those two effects.

The best estimate for the ICC will reflect the circumstances for the trial being planned. That estimate will be from the same target population, so that it reflects the appropriate groups or clusters (e.g., schools vs. clinics vs. worksites vs. communities); age groups (e.g., youth vs. young adults vs. seniors); ethnic, racial, and gender diversity; and other characteristics of the target population. That estimate will derive from data collected for the same outcome using the same measurement methods to be used for the primary outcome in the trial being planned. For example, if planning a trial to improve servings of fruits and vegetables in inner-city third-graders, it would be important to get an ICC estimate for servings of fruits and vegetables, measured in the same way as servings would be measured in the trial being planned, from third-graders in inner-city schools similar to the schools that would be recruited for the trial being planned.

Regression adjustment for covariates often improves power in a GRT or IRGT by reducing the residual error variance or the ICC (Murray and Blitstein, 2003). At the same time, it is important to remember that regression adjustment for covariates can reduce power in a GRT or IRGT by increasing the ICC (Murray, 1998). As such, it is important to choose covariates carefully. The best covariates will be related to the outcome and unevenly distributed between the study conditions or among the groups or clusters randomized to the study conditions.

No. When a priori matching or stratification is used for balance, the matching or stratification factor may be included in the analysis of intervention effects, but that is not required, and it may be inefficient to do so. It is not required because the type 1 error rate is unaffected when the matching or stratification factor is ignored in the analysis of intervention effects (Diehr et al., 1995; Proschan, 1996). Both procedures reduce the df available for the test of the intervention effect, and if the number of df is limited, the unmatched or unstratified analysis may be more powerful than the matched or stratified analysis. In that circumstance, it is to the investigator’s advantage to match or stratify in the design to achieve balance on potential confounders, but to ignore the matching or stratification in the analysis to improve power or reduce sample size (Diehr et al., 1995). The choice of whether to include the matching or stratification factor in the analysis should be made a priori based on sample size calculations comparing the matched or stratified analysis to the unmatched or unstratified analysis.

The choice between a priori matching and a priori stratification for balance should be guided by whether the investigator anticipates doing analyses that do not involve intervention effects. Donner et al., 2007 have warned against ignoring matching in analyses that do not involve intervention effects, e.g., in an analysis to examine the association between a risk factor and an outcome. Ignoring matching in the analysis in this situation can lead to an inflated type 1 error rate when the correlation between the matching factor and either the outcome or the risk factor is at least modest (>0.2) and the number of members per group is not large (<100). Stratification with strata of size four avoids this problem, and improves efficiency almost as much as matching. For this reason, stratification with strata of size four is a prudent strategy for balancing potential confounders across study conditions because it is almost as efficient as matching and it does not limit the range of analyses that can be applied to the data.

These studies are individually randomized group treatment trials (IRGTs), sometimes called partially clustered designs. While these trials are common, most investigators do not recognize the implications of this design. IRGTs always have a hierarchical structure in the intervention condition. Participants may receive some of their treatment in groups, or they may receive their intervention individually, but through a common interventionist or facilitator, whether in person or through a video or other virtual connection (Lee and Thompson, 2005b). There may or may not be a similar structure in the control condition, depending on the nature of the control condition. Whether it exists in one or both study conditions, the hierarchical structure requires that the positive ICC expected in the data be accounted for in the sample size estimation and in the data analysis. Any analysis that ignores the positive ICC or what may be limited df will have a type 1 error rate that is inflated, often badly (Baldwin et al., 2011; Bauer et al., 2008; Kahan and Morris, 2013Lee and Thompson, 2005b; Lee and Thompson, 2005a; Pals et al., 2008b; Pals et al., 2011; Pals et al., 2008a; Roberts and Roberts, 2005; Roberts and Walwyn, 2013).

The recommended solution to these challenges is similar to the solution recommended for GRTs. It is important to employ a priori matching or stratification to balance potential confounders if the number of assignment units is limited, to reflect the hierarchical or partially hierarchical structure of the design in the analytic plan, and to estimate the sample size for the IRGT based on realistic and data-based estimates of the ICC and the other parameters indicated by the analytic plan. Extra variation and limited df always reduce power, so it is essential to consider these factors while the study is being planned, and particularly as part of the sample size estimation.

No. In public health and medicine, ICCs in group- or cluster-randomized trials are often small, usually ranging from 0.01–0.05 (Moerbeek and Teerenstra, 2016). While it is tempting to ignore such small correlations, doing so risks an inflated type I error rate, and the risk is substantial both in GRTs (Campbell and Walters, 2014; Donner and Klar, 2000; Eldridge and Kerry, 2012; Hayes and Moulton, 2017; Murray, 1998) and in IRGTs (Baldwin et al., 2011; Bauer et al., 2008; Kahan and Morris, 2013; Lee and Thompson, 2005b; Lee and Thompson, 2005a; Pals et al., 2008b; Pals et al., 2011; Pals et al., 2008a; Roberts and Roberts, 2005; Roberts and Walwyn, 2013). The prudent course is to reflect all nested factors as random effects and to plan the study to have sufficient power given a proper analysis.

No. That is another tempting strategy that can risk an inflated type I error rate. The standard error for the variance component is not well estimated when the value is close to zero, and if the df are limited, the power will be limited. As such, it is likely that the result will suggest that the ICC or variance component is negligible, when ignoring it will inflate the type I error rate. The prudent course is to reflect all nested factors as random effects and to plan the study to have sufficient power given a proper analysis.

CONSORT Statement
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: Explanation and elaboration. Ann Intern Med. 2008;148(4);295-309.
PMID: 18283207.
Key References

IRGTs

Turner EL, Li F, Gallis JA, Prague M, Murray DM. Review of recent methodological developments in group-randomized trials: Part 1-Design. Am J Public Health. 2017;107(6);907-915. Epub 2017/04/20.
PMID: 28426295.
Turner EL, Prague M, Gallis JA, Li F, Murray DM. Review of recent methodological developments in group-randomized trials: Part 2-Analysis. Am J Public Health. 2017;107(7);1078-1086. Epub 2017/05/18.
PMID: 28520480.
Lai MH, Kwok OM. Estimating standardized effect sizes for two- and three-level partially nested data. Multivariate Behav Res. 2016;51(6);740-756. Epub 2016/11/01.
PMID: 27802077.
Hedges LV, Citkowicz M. Estimating effect size when there is clustering in one treatment group. Behav Res Methods. 2015;47(4);1295-1308.
PMID: 25425393.
Andridge RR, Shoben AB, Muller KE, Murray DM. Analytic methods for individually randomized group treatment trials and group-randomized trials when subjects belong to multiple groups. Statistics in Medicine. 2014;33(13);2178-90. Epub 2014/01/08.
PMID: 24399701.
Kahan BC, Morris TP. Assessing potential sources of clustering in individually randomised trials. BMC Med Res Methodol. 2013;13;58.
PMID: 23590245.
Baldwin SA, Bauer DJ, Stice E, Rohde P. Evaluating models for partially clustered designs. Psychological Methods. 2011;16(2);149-65.
PMID: 21517179.
Pals SL, Murray DM, Alfano CM, Shadish WR, Hannan PJ, Baker WL. Individually randomized group treatment trials: A critical appraisal of frequently used design and analytic approaches: Erratum. American Journal of Public Health. 2008b;98(12);2120.
PMID: 18556603.
Pals SL, Murray DM, Alfano CM, Shadish WR, Hannan PJ, Baker WL. Individually randomized group treatment trials: A critical appraisal of frequently used design and analytic approaches. American Journal of Public Health. 2008a;98(8);1418-1424. Epub 2008/06/12.
PMID: 18556603.
Roberts C, Roberts SA. Design and analysis of clinical trials with clustering effects due to treatment. Clinical Trials. 2005;2(2);152-162.
PMID: 16279137.
Hoover DR. Clinical trials of behavioral interventions with heterogeneous teaching subgroup effects. Statistics in Medicine. 2002;21(10);1351-1364.
PMID: 12185889.

 

State of the Practice Reviews for IRGTs

Oltean H, Gagnier JJ. Use of clustering analysis in randomized controlled trials in orthopaedic surgery. BMC Med Res Methodol. 2015;15;17.
PMID: 25887529.
Pals SL, Wiegand RE, Murray DM. Ignoring the group in group-level HIV/AIDS intervention trials: A review of reported design and analytic methods. AIDS. 2011;25(7);989-96.
PMID: 21487252.
Pals SL, Murray DM, Alfano CM, Shadish WR, Hannan PJ, Baker WL. Individually randomized group treatment trials: A critical appraisal of frequently used design and analytic approaches. American Journal of Public Health. 2008a;98(8);1418-1424. Epub 2008/06/12.
PMID: 18556603.
Lee KJ, Thompson SG. Clustering by health professional in individually randomised trials. BMJ. 2005a;330(7483);142-4.
PMID: 15649931.

 

Sample Size Estimation for IRGTs

Heo M, Litwin AH, Blackstock O, Kim N, Arnsten JH. Sample size determinations for group-based randomized clinical trials with different levels of data hierarchy between experimental and control arms. Stat Methods Med Res. 2017;26(1);399-413. Epub 2016/07/11.
PMID: 25125453.
Moerbeek M, Teerenstra S. Power analysis of trials with multilevel data. 2016. Boca Raton: CRC Press.
Roberts C, Walwyn R. Design and analysis of non-pharmacological treatment trials with multiple therapists per patient. Statistics in Medicine. 2013;32(1);81-98. Epub 2012/08/02.
PMID: 22865729.
Candel MJ, van Breukelen GJ. Varying cluster sizes in trials with clusters in one treatment arm: Sample size adjustments when testing treatment effects with linear mixed models. Statistics in Medicine. 2009;28(18);2307-24.
PMID: 19472169.
Moerbeek M, Wong WK. Sample size formulae for trials comparing group and individual treatments in a multilevel model. Statistics in Medicine. 2008;27(15);2850-64.
PMID: 17960589.
Pals SL, Murray DM, Alfano CM, Shadish WR, Hannan PJ, Baker WL. Individually randomized group treatment trials: A critical appraisal of frequently used design and analytic approaches: Erratum. American Journal of Public Health. 2008b;98(12);2120.
PMID: 18556603.
Pals SL, Murray DM, Alfano CM, Shadish WR, Hannan PJ, Baker WL. Individually randomized group treatment trials: A critical appraisal of frequently used design and analytic approaches. American Journal of Public Health. 2008a;98(8);1418-1424. Epub 2008/06/12.
PMID: 18556603.
Roberts C, Roberts SA. Design and analysis of clinical trials with clustering effects due to treatment. Clinical Trials. 2005;2(2);152-162.
PMID: 16279137.
Body
More Questions?

You may find answers in the Reference Materials.

Last updated on April 8, 2021