The TRILUMINATE study, a randomized trial presented at the American College of Cardiology Scientific Sessions 2023 and simultaneously published in the New England Journal of Medicine, ¹ evaluated the usefulness of a new device for correcting severe tricuspid regurgitation. The primary end point was a composite that included death, readmission for heart failure and quality of life measured by a score. The statistical analysis was performed using the win ratio, a method proposed by Stuart Pocock in 2012, ² which is not yet well-known by professionals. The study is presented as successful, with a win ratio (we will explain its meaning later in this article) of 1.48 (95% CI 1.06 to 2.13), indicating that, overall, the intervention was 50% better than the control. Critical reading will show us the limitations of this interpretation in the trial and a somewhat more prudent view of its results.

Let us start with the medical issue: tricuspid regurgitation is a valvular heart disease that is usually associated with other structural heart conditions. One of the big questions when faced with a patient with this dysfunction is how patients' outcome and quality of life would improve if we could correct it. Thus, if the intervention was effective in producing a significant reduction in the magnitude of tricuspid regurgitation, we would have key information on the role of the intervention in patients' clinical condition. This study provides a clear answer to the question, although, in our opinion, it does not coincide with the authors' interpretation.

The second issue is the validity of the end point chosen to determine the win ratio. Here we should describe the concept and technical aspects of this method.

In studies with a short duration, as in the acute phase of myocardial infarction, we obtain an event rate in each group and the relative risk is reliable. Thus, if mortality is 8% in the intervention group and 10% in the control group, the relative risk is 8/10 = 0.8, or in other words, a 20% reduction in the occurrence of the event. In studies with long follow-up, events may occur at different times; death within the first month is not the same as death at five years, although in the crude analysis at five years both events are expressed with a deceased patient, which requires an actuarial correction. One issue in the analysis of composite end points is that the usual methods use the first event without prioritizing whether it is death or other. The comparison is made on an actuarial basis, adjusting the times at which the event occurs, and the Cox proportional hazard method is used to estimate the relative risk, expressed as hazard ratio, which has also limitations when the effect is not constant over time.

In 2012, Pocock proposed a different approach to overcome many of these limitations, the win ratio, which considers a hierarchical order of events, prioritizing death event over any other event. The example published in the original paper was a study with a composite end point of death and readmission for heart failure.

The win ratio is a method that compares matched pairs of patients between the intervention group and the control group. In this case, two alternatives are possible, which we illustrate using a study of 100 patients per group.

Alternative A: patients are easily matched by risk criteria.

In that case we will make 100 comparisons in a hierarchical order. The first event will be the most serious, in this case mortality. Who is the winner? There are many easy possibilities.

1) None of the patients of the pair died: tie.

2) One patient of the control group died: the patient in the intervention group is the winner (and vice versa).

3) Both patients died: the one who died later is the winner.

Let us assume that with this analysis we obtained winners in 30 cases, 20 in the intervention group and 10 in the control group.

Now we will compare the event hospitalization in the remaining 70 cases, using the same criteria as above. Let us assume that with this analysis we obtained winners in 30 cases, 18 in the intervention group and 12 in the control group.

The win ratio is calculated by comparing the winners in each group.

In the example, the intervention group has 20 winners in the event death and 18 winners in the event hospitalizations, 38 winners in total, and the control group has 10 and 12 winners respectively, 22 winners in total. Win ratio = 38/22 = 1.73, that is, 73% more winners in the intervention group. The confidence interval is calculated by statistical software programs and provide the statistical significance: win ratio 1.73 (95% CI 1.05-3.1), p = 0.04. This calculation can be estimated in the form attached (Appendix).

https://gedic.files.wordpress.com/2023/03/calculador-win-ratio.xlsx.

Alternative B: there is no formation of matched pairs and win ratio can be obtained by comparing all possible unmatched pairs. This is the method used in this trial.

In most studies it is not easy to choose matched pairs which could result subjective or wrong, and instead each patient in one group is compared with all the patients in the other group. With 100 patients in each group, we multiply 100 x 100 comparisons = 10 000 total comparisons. The estimation of win ratio is similar, but the statistical analysis is more complex. The win ratio has been recognized by the FDA as a valid method to evaluate and approve patents of drugs and devices and has been applied in prospective studies with robust results. ^3,4

The use of win ratio in the TRILUMINATE study The authors prospectively used the unmatched pair approach even for calculating the sample size. Each of the 175 patients in each group was compared with each patient of the other group. Thus, 30 625 comparisons were made (175 x 175).

The primary end point was a hierarchical composite that included death, hospitalization for heart failure, and an increase of at least 15 points in the quality-of-life score.

First conceptual critique: as the win ratio provides one point per winner, it is obvious that winning in terms of not dying or avoiding hospitalization is much more relevant than winning by points in a quality-of- life score in an open study, that is, where patients know whether they have been intervened or not. Furthermore, if the events do not go in the same direction, as was the case in this trial.

Let us see the results reported (table). The final result was: winners in the intervention group 11 348, winners in the control group 7643, tie 11 638. By summing up 1348+7643+11638 = 30625 is the total number of possible comparisons.

Although hospitalization-free survival is not shown in the main study, it appears in a figure in the supplementary material, clearly illustrating the temporal advantage of the control group compared with the intervention group. (Figure 1)

Fig. 1 Actuarial curve of freedom from heart failure hospitalization. Patients in the control group had better outcome. (Reference 1 - Supplementary material) 

Second conceptual argument: the primary endpoint favored the intervention group because of a significant improvement in the quality-of-life score, which was 3 times higher, although the trend in the first two pooled events of death and readmissions favored the control group. As quality-of-life assessment involves some bias that could be enhanced by the open-label nature of a study without sham procedure, one cannot strongly conclude that the procedure is beneficial.

We will try to make questions that we believe the study answers, although not exactly as the authors suggest.

1) Was device placement successful for correcting tricuspid regurgitation?

Tricuspid regurgitation significantly improved with the procedure; 87% remained with moderate regurgitation or lower. Severity of tricuspid regurgitation did not change significantly in the control group, only 4.8% presented moderate regurgitation or lower.

2) Did tricuspid valve repair play an important role in patients' outcome?

The answer is a matter of debate, since the events that we could consider major, as mortality and readmissions, tended to favor the control group, while quality of life scores and the six-minute walk test favored the intervention group. The difference in the six-minute walk was similar to that usually seen in other studies with drugs, for example for pulmonary hypertension, which have no impact on major events as hospitalizations or mortality. This observation provides an important message about the clinical role of tricuspid regurgitation: the study demonstrates that the device greatly reduces tricuspid regurgitation without a significant clinical impact on outcome. Undoubtedly, larger trials will be required to justify this intervention with its possible costs.

3) Which aspects of the presentation of the results can be considered unusual or subject to criticism for this type of trial?

a) An initial aspect is that the study was not carried out by an independent research group, but rather each of the steps involved the participation of the laboratory manufacturing the device, with the obvious bias of corporate interests.

b) There are no tables in the study comparing mortality and hospitalization rates, and no figures on survival and event-free survival. They are only shown in the supplementary material.

c) The abstract does not provide information about mortality and hospitalization with their relative risks; it just mentions that they did not differ between the two groups. This omission is obviously due to the negative trend associated with both events in the intervention group.

d) The lack of partial analysis of win ratio, which demonstrates better outcomes in the rate of major events for the control group and only in quality of life, is not clearly expressed. Winners and losers are even presented in a rather confusing way in the table, implying that the analysis of the table requires considerable time.

Given that professionals have limited time for reading, these omissions contribute to increase confusion and lead to believe the opinion of the authors expressed in their conclusions. It is interesting to note, as a final comment, how the conclusions are worded: tricuspid valve repair with the device was safe for patients, reduced the severity of tricuspid regurgitation, and was associated with an improvement in quality of life.

The greatest strength of randomized studies is that they are validated methods for demonstrating causality. If the baseline characteristics of the intervention group and control group are identical, the only difference between them is the intervention tested; the better outcome in one group indicates that the intervention is the cause of the difference in the outcome. In this case, the authors were cautious in not attributing causality for the improvement in quality of life due to the treatment performed, using the phrase "was associated with an improvement" as an observational phenomenon. Possibly this was the suggestion of the editorial board to accept the publication of this paper with its limitations.