Random-effects summaries can establish an average association while leaving clinicians uncertain about the effect expected in another service or population. Online psychotherapy for coronavirus disease 2019 (COVID-19)-related distress is a pertinent example because trials differed markedly in delivery channel, therapist involvement, treatment dose, population, and measured outcome. To determine whether pooled improvements in depression, anxiety, stress, and sleep remain persuasive when between-study dispersion is translated into probabilities for a comparable new trial, and to identify which delivery contrasts retain decision value under propagated uncertainty. The analytic collection comprised 13 randomized trials (1,897 participants), four outcome-level random-effects summaries, and prespecified delivery contrasts. Standard errors were recovered from published 95% confidence intervals. For each outcome, a \(t\)-based 95% prediction interval and predictive probabilities of any benefit, a benefit of at least 0.20 standardized mean difference, a benefit of at least 0.50, and harm were calculated from the reported mean effect and between-study variance. A 200,000-draw Monte Carlo check evaluated numerical agreement. Normalized entropy described design concentration, and uncertainty in two delivery contrasts was propagated from subgroup confidence intervals. Mean effects favored online psychotherapy for depression (\(smd=-0.45\)), anxiety (\(-0.67\)), stress (\(-0.73\)), and sleep (\(-0.53\)), but all 95% prediction intervals crossed no effect: \(-1.22\) to \(0.32\), \(-1.72\) to \(0.38\), \(-2.06\) to \(0.60\), and \(-3.74\) to \(2.68\), respectively. Probabilities that a comparable new trial would achieve at least a 0.20-SMD benefit were 0.779, 0.856, 0.826, and 0.671. The corresponding harm probabilities were 0.083, 0.065, 0.098, and 0.239. Therapist guidance had a 0.991 probability of outperforming self-help for anxiety, whereas short daily treatment had a 0.997 probability of outperforming seven- to eight-week weekly treatment for depression; the latter contrast depended on a single short-daily comparison. The evidence supports probable benefit for emotional distress, yet does not justify assuming benefit in every comparable implementation. Anxiety has the strongest combination of predictive benefit and low harm probability. Sleep remains too uncertain for a directional recommendation. Guidance for anxiety is more transportable than the apparent depression advantage of an intensive short dose, which requires direct replication.
Random-effects summaries can establish an average association while leaving clinicians uncertain about the effect expected in another service or population. Online psychotherapy for coronavirus disease 2019 (COVID-19)-related distress is a pertinent example because trials differed markedly in delivery channel, therapist involvement, treatment dose, population, and measured outcome. To determine whether pooled improvements in depression, anxiety, stress, and sleep remain persuasive when between-study dispersion is translated into probabilities for a comparable new trial, and to identify which delivery contrasts retain decision value under propagated uncertainty. The analytic collection comprised 13 randomized trials (1,897 participants), four outcome-level random-effects summaries, and prespecified delivery contrasts. Standard errors were recovered from published 95% confidence intervals. For each outcome, a \(t\)-based 95% prediction interval and predictive probabilities of any benefit, a benefit of at least 0.20 standardized mean difference, a benefit of at least 0.50, and harm were calculated from the reported mean effect and between-study variance. A 200,000-draw Monte Carlo check evaluated numerical agreement. Normalized entropy described design concentration, and uncertainty in two delivery contrasts was propagated from subgroup confidence intervals. Mean effects favored online psychotherapy for depression (\(smd=-0.45\)), anxiety (\(-0.67\)), stress (\(-0.73\)), and sleep (\(-0.53\)), but all 95% prediction intervals crossed no effect: \(-1.22\) to \(0.32\), \(-1.72\) to \(0.38\), \(-2.06\) to \(0.60\), and \(-3.74\) to \(2.68\), respectively. Probabilities that a comparable new trial would achieve at least a 0.20-SMD benefit were 0.779, 0.856, 0.826, and 0.671. The corresponding harm probabilities were 0.083, 0.065, 0.098, and 0.239. Therapist guidance had a 0.991 probability of outperforming self-help for anxiety, whereas short daily treatment had a 0.997 probability of outperforming seven- to eight-week weekly treatment for depression; the latter contrast depended on a single short-daily comparison. The evidence supports probable benefit for emotional distress, yet does not justify assuming benefit in every comparable implementation. Anxiety has the strongest combination of predictive benefit and low harm probability. Sleep remains too uncertain for a directional recommendation. Guidance for anxiety is more transportable than the apparent depression advantage of an intensive short dose, which requires direct replication.
