Table 2 Risk of bias assessment, JBI assessments and study biases
From: Respiratory health effects of e-cigarette substitution for tobacco cigarettes: a systematic review
Study/rating | JBI | JBI blinding items | Other JBI items | Potential sources of research bias | Reporting bias |
|---|---|---|---|---|---|
Barna [24] HIGH RISK OF BIAS | 6/9 | NA | Dissimilar participants Limited measurements Differences in outcome measurement | Small sample size All participants male Excessive cigarette exposure No wash-out period | Test measurements NR Spin: pro-ENDS conclusion |
Chaumont [23] HIGH RISK OF BIAS | 8/13 | Allocation not concealed No participant blinding No treater blinding | Subgroup for pulmonary tests randomly selected from full participant group Testing only non-nicotine use | Excessive ENDS exposure Very small sample size Participants occasional smokers, short smoking history | Focus on surrogate markers Substantial and multiple discrepancies between data and discussion Spin: over-generalization |
Cravo [25] HIGH RISK OF BIAS | 9/13 | Allocation not concealed No participant blinding No treater blinding Assessor blinding unclear | No additional items | Lung function test conducted only on cohort 1 | Spin: NS findings described as causal |
D’Ruiz [26] HIGH RISK OF BIAS | 6/13 | Allocation unclear No participant blinding No treater blinding Assessor blinding unclear | Randomization unclear Treatment groups dissimilar Mixed model statistical basis unclear | Ad libitum use, consumption levels not recorded Potential volunteer bias: high compensation | None noted |
Flouris [27] SOME CONCERNS OF BIAS | 8/8 | NA | No items | Indirectness of ENDS (device design no longer on the market) Participants: wide variation in smoking history | Spin: pro-ENDS conclusion |
Hickling [28] HIGH RISK OF BIAS | 7/9 | NA | No control groups. Follow-up incomplete | Small sample size. Participants > 70% male. No recruitment information | Test measurements NR Spin: emphasis on pro-ENDS results |
Kerr [29] HIGH RISK OF BIAS | 6/10 | No allocation concealment No participant blinding No treater blinding Assessors blinding unclear | No items | Small sample size. Participants all male | Discrepancy between text and Fig. 3a |
Kotoulas [30] HIGH RISK OF BIAS | 9/12 | Allocation not concealed No participant blinding | No randomization | No recruitment information Very high nicotine strength | Spin: secondary tests presented, not health outcomes Spin: over-generalization |
Lappas [22] SOME CONCERNS OF BIAS | 8/8 | NA | No items | Current smoker definition overly broad (≥ 1 cigarette past 30 days) | Spin: focus on secondary endpoints |
Palmidas [31] HIGH RISK OF BIAS | 6/8 | NA | Participants dissimilar Limited measurement, puffs not counted | Indirectness of ENDS (device design no longer on the market) | No items noted |
Polosa [33] SOME CONCERNS OF BIAS | 9/10 | NA | Participants dissimilar between exclusive ENDS users and dual users | ENDS type and quantity used not documented | No items noted |
Polosa [32] SOME CONCERNS OF BIAS | 9/10 | NA | Participants dissimilar | No items noted | Data discrepancies in Table 2 |
Pulvers [38] Arnold [34] SOME CONCERNS OF BIAS | 10/13 | No participant blinding No treater blinding No assessor blinding | No other issues | Volunteer bias: high compensation with low-income participants | Spin: conclusion not supported by findings |
Vardavas [35] HIGH RISK OF BIAS | 8/10 | NA | Dissimilar participants in comparisons Limited measurement, puffs not counted | Indirectness of ENDS (device type no longer on the market) No reporting of wash-out period | Spin: conclusion focus on results of one sub-group Spin: statistical significance interpreted as clinical relevance |
Veldheer [36] SOME CONCERNS | 12/13 | No issues | Not all participants completed testing | One arm high nicotine ENDS Drop-outs not examined | Spin: previous study quoted as evidence for conclusion Power insufficient to support conclusions |
Walele [37] HIGH RISK OF BIAS | 6/9 | NA | No control group Participants included persons with and without previous ENDS use Between group analyses mixed for completion and treatment compliance participants | Volunteer bias: recruitment from prior study, potential spill-over effect | Spin: Emphasis on secondary outcome Spin: effect modification on findings not supported |