Our main goal was to create a web-based application that allows users to perform a complete meta-analysis of clinical trials with minimal effort. For this, we established a user-friendly portal that is accessible in any major web browser. The web application is running on an Ubuntu server (Canonical Ltd) powered by Apache. The application can be used to perform meta-analyses using binary (using total and event numbers), continuous (using mean and SD data), and time-to-event data (using hazard ratio and CI data). The results of each study can be pasted from commonly used spreadsheet applications like Microsoft Excel.

The calculations and plots used by the web-based system are done using meta, RTSA, and metafor packages in R programming environments (version 4.2.2; R Foundation for Statistical Computing). The Shiny user interface was created by using the shinyjs, shinydashboard, and rhandsontable R packages. The portal generates a forest plot to summarize the results of the meta-analysis. A 2-tailed P value below .05 indicates significance and is displayed in the analysis results. MetaAnalysisOnline.com uses the DerSimonian-Laird estimator to estimate tau-square [7].

In addition to the forest plot, a funnel plot is displayed to detect potential bias visually, and a z score plot is used for assessing the power achieved by the cumulative sample number. While there is commercial software available for performing a meta-analysis [8], the portal we developed can be accessed completely free, without the need for registration or logging in.

Experimental data are typically classified into 2 main categories: clinical data, which detail the effects of specific treatments on individual participants; and epidemiological data, which unveil patterns of disease or mortality in groups of participants exposed to either single agents or various substances. The most common classification in such studies uses categorical variables. A categorical variable is nonnumerical, relying on qualitative properties such as treatment arm, race, or gender, among others. Unlike numerical variables, categorical variables lack a specific ordering and assume values from a finite set of possibilities.

In MetaAnalysisOnline.com, the 2 study arms can be compared by either the fixed-effect model or the random-effect model. The fixed effect model operates on the assumption that a single true effect size underlies all studies in the meta-analysis, hence the term “fixed effect” or “common-effect” or sometimes “equal-effects.” Any variations in observed effects are attributed to sampling error. The random effects model posits that the true effect may vary across studies due to inherent differences or heterogeneity between them. As studies encompass diverse participant compositions and different interventions, individual studies may yield varying effect sizes. If an infinite number of studies were conducted, the effect estimates from all studies would conform to a normal distribution and the pooled estimate would then represent the mean or average effect. The assumption is that the effect sizes observed in conducted studies represent a random sample from the universe of all possible effect sizes, hence the term “random effects” [9]. The random effects approach for aggregating evidence from a series of experiments that compare 2 treatments was originally proposed by DerSimonian and Laird [7]. They integrated the heterogeneity of effects into the analysis, providing a comprehensive assessment of the overall treatment efficacy. The heterogeneity can be assessed using the I² and 𝜏² statistics, where the 𝜏² represents the between-trial variance of the effect size. The square root of the 𝜏² provides an estimate of the SD of the effects in the analyzed studies. High values of 𝜏² indicate substantial heterogeneity among the effect sizes, suggesting that the true effects are not consistent across studies and may be influenced by other factors [10]. The I² method, developed by Higgins and Thompson [11], offers a percentage value representing the variability observed in the effect size, independent of sampling error. One of its advantages lies in the ease of interpreting its results: I² percentages of 25% correspond to low, 50% to medium, and 75% to high levels of heterogeneity, respectively [12].

There are 4 commonly used methods of meta-analysis for dichotomous outcomes: 4 fixed-effect methods (Mantel-Haenszel [13], Peto, Bakbergenuly [14], and inverse variance) and 1 random-effect method (DerSimonian and Laird inverse variance [7]). The Peto method is limited to combining odds ratios (ORs), while the other 3 methods can combine ORs, risk ratios, or risk differences. The Mantel-Haenszel method is a fixed effect meta-analysis method that uses a different weighting scheme depending on the type of effect measure being used (eg, risk ratio or OR) [13]. A relatively recent development is the Bakbergenuly sample-size-weighted estimator approach, which offers an alternative by using a weighted average where each study’s weight is determined by its effective sample size [14]. In the fixed effect model, the weights are calculated as w_i=1/v₁ where v₁ represents the variance of the study. In the calculation, also known as the inverse variance method, studies with lower variances are weighted higher. In the random effect model, the calculation of weights is different, and the formula is 𝑤_i=1/ (𝜏²+v_i), where 𝜏² represents the square of the variance of the distribution of true effect sizes. The inverse-variance method’s key advantage is its wide applicability, as it can be used for all effect measures and not only for OR, risk ratio, or risk difference. For its execution, we need to have the point estimate and the variance of the treatment effect in each study [7].

Quantitative variables can be either discrete or continuous. Discrete variables are constrained to a finite number of values (such as whole numbers), whereas continuous variables can assume any value, including all possible values within a given range (extending to an infinite number of decimal places). Standard methods for meta-analysis of continuous data rely on the assumption that the outcomes follow a normal distribution within each intervention arm in each study. The 3 commonly used summary statistics for a meta-analysis of continuous data include the mean difference (MD), the standardized mean difference (SMD), and the ratio of means. The choice of summary statistics for continuous data primarily depends on whether studies report the outcome using the same scale (in which case MD is used) or different scales (in which case the SMD is typically used). In the MD approach, the SDs and sample sizes are used to calculate the weight assigned to each study and studies with smaller SDs are given relatively larger weights. In the SMD approach, the SDs are used to standardize the MDs to a single scale and are also used in the computation of study weights. When comparing with MD, interpreting the results can be challenging due to the need to categorize the SMD. Typically, we use the categories outlined by Cohen, which indicate that an SMD of approximately 0.2 corresponds to a small effect, around 0.5 signifies a moderate effect, and approximately 0.8 indicates a large effect [15]. In addition to MD and SMD, MetaAnalysisOnline.com also includes the ratio of means, a more recent solution where instead of using the difference between groups, the log-transformed ratio of the means of the groups is used in the analysis. The advantage of this approach is that the output of the result is easily interpretable similar to the odds or risk ratio [16].

In recent years, the addition of immune checkpoint blockade using anti–PD-1 drugs, such as pembrolizumab, nivolumab, and cemiplimab, into cancer therapy has improved clinical outcomes. However, the safety of the immune checkpoint blockade needs further evaluation. As a demonstration of the functionalities of MetaAnalysisOnline.com, a set of previous studies was analyzed to assess how incorporating immune checkpoint blockade into perioperative cancer therapy affects treatment-related adverse events in patients treated with anti–PD-1 therapy.

A comprehensive forest plot displays a set of the following information.

The features discussed in this section are marked in the plot generated by demo data in Figure 1. In addition, a screenshot of the analysis page with a generated forest plot is shown in Figure 2.

The second analysis available in MetaAnalysisOnline.com is the funnel plot designed to detect publication bias. Whether the primary studies for pooling are observational studies or randomized controlled trials, heterogeneity between studies is probable. We show the components of a funnel plot used to assess study heterogeneity in Figure 3. Each dot on the plot represents an individual study. The x-axis represents the hazard ratio, or in other cases the log of OR, indicating the effect size of the treatment in each study. The y-axis represents the standard error, reflecting study precision, with larger studies having better precision (within the green area) and smaller studies having worse precision (within the orange area). In the plot, the vertical solid black or red line serves as the reference line corresponding to no effect or a hazard rate of 1 (or log OR=0). The overall effect and 95% CIs are depicted by the dashed black lines.

If the funnel plot suggests the presence of publication bias, the trim-and-fill method can be used to estimate the number of missing studies and impute their effect sizes. This allows the meta-analysis to be “trimmed” of asymmetric studies, and “filled” with the estimated missing studies to create a more symmetric funnel plot. The web-based platform automatically displays a button to generate trim-and-fill plots in case of a significant Egger test.

MetaAnalysisOnline.com performs a trial sequential analysis by drawing the z score plot to illustrate the reliability of the sample size. The y-axis of a z score plot depicts the cumulative z score and the x-axis the cumulative sample size. The cumulative z curve is built by sequentially adding studies based on chronological order. The plot depicts the relationship between the collective sample size and the achieved significance. When the ideal sample size is attained, a decisive conclusion regarding the examined condition can be made. The components of a z score plot are visualized in Figure 4.

At the end of the z curve lies the most recently added study, falling into one of the following zones: “favoring outcome A,” “favoring outcome B,” “no correlation,” or “statistically not significant.” The first 2 zones indicate statistically significant results, which may vary until the optimal sample size is reached, but after this, a reliable final conclusion can be determined. Placement in the “no correlation” signifies strong evidence suggesting that subsequent studies are unlikely to alter the no-effect outcome significantly. Placement within the “not statistically significant” zone indicates the definitive requirement for further studies as neither correlation nor missing correlation could be determined.

To verify the output of the digital portal, a set of 10 samples was drawn from the dataset of a simulated randomized trial developed for the metafor package and a meta-analysis was performed. Analysis was repeated using IBM SPSS (version 29). The following outputs were compared: OR, the calculated weight, overall effect, as well as measures of heterogeneity (I² and τ²). For all parameters, the corresponding 95% CIs were also checked. The output results of the 2 methods were identical, indicating sufficient reliability of the implemented procedures.

To show the entire analysis pipeline, the association between immune checkpoint blockade using anti–PD-1 agents and adverse events was analyzed. Altogether 10 studies were included with a total of 4379 participants in the anti–PD-1–treated cohort and 3720 participants in the control cohort [17-26]. Based on the analysis performed using the random effects model with the Mantel-Haenszel method to compare the risk ratio, there is a statistical difference between the 2 cohorts, the overall risk ratio is 2.15 with a 95% CI of 1.39-3.32. The test for overall effect reached significance as depicted in Figure 5A.

Notably, substantial heterogeneity was detected, suggesting inconsistent effects in magnitude and/or direction. The I² value indicates that 95% of the variability among studies stems from heterogeneity rather than random chance.

The funnel plot indicates a potential publication bias. The Egger test supports the presence of funnel plot asymmetry (intercept: 3.85, 95% CI 1.39-6.3; t₈=3.072; P=.02; visualized in Figure 5B).

Based on the z score plot, the total number of samples did not reach the optimal number, and more research is needed to establish a conclusion (Figure 5C).