Entitled “Where do the outputs from GetReal fit in medicines development and decision-making?” the plenary discussion included the following panelists:

• Wim Goettsch of EUnetHTA and ZIN, The Netherlands
• Bettina Ryll, Melanoma Patient Network Europe
• Robert Hemmings, MHRA, United Kingdom
• Anton (Tony) Hoos, Amgen
• Moderator: Duane Schulthess, Vital Transformation

Wim Goettsch of EUnetHTA opened the plenary discussion with a look at what needs to be done for the results of the GetReal programme to be applied in the “real world” following its conclusion. Providing the HTA perspective, he spoke on EUnetHTA’s Joint Action 3 (JA3) initiative as related to GetReal, pinpointing key areas of alignment between the two. The essence of JA3, which began on 1 June 2016, is to speed up the joint production of national assessments and to build on joint assessments—specifically using joint assessments in national projects.

Over the next four years, the JA3—which includes 77 partner organisations from 27 Member States—will aim to build a sustainable structural network of HTA bodies. Compared to previous, more academic endeavors, this is a more practical endeavor that places HTAs as an active part of the entire dialogue and assessment of technologies from early on. Noting that the tools developed by GetReal can help in EUnetHTA’s initiative, Goettsch emphasised four key JA3 work packages linked to GetReal’s work in the areas of:

• Production of joint and collaborative HTA reports;
• Evidence generation;
• Quality management;
• National implementation and impact.

There is a lot of willingness on EUnetHTA’s part to use the tools being provided by GetReal and to incorporate these into the real world, Goettsch noted—this interaction is necessary. As an HTA organisation, he admitted to being initially skeptical that GetReal could reach the level of diverse stakeholder collaboration that it has now attained. Figuring out how to continue GetReal’s activities, and retain this environment, after the programme’s formal end, he concluded, is a must.

Bettina Ryll of the Melanoma Patient Network Europe took over the conversation to present the patient’s perspective on GetReal’s work. From her perspective, the day’s conversations about “acceptability of data sets” neglected a crucial point: the acceptability of evidence generation for patients, who are the ultimate beneficiaries of healthcare. It’s not only about whether the dataset itself cuts it, she said, but more importantly if the process of obtaining that data is acceptable for patients.

Ryll pointed to melanoma as an extreme example of how rapid scientific progress has had enormous impact on the clinical side for a disease, which unfortunately was not matched by equal progress on the regulatory science side (or HTA assessment). The ones who paid a price for this were patients. Looking at phase three trials, she noted shocking differences between the control and treatment arms—differences so stark that they should have been picked up in phase 2. Whether it’s a result of a failure to notice or an equipoise violation, either option is unacceptable from a patient perspective. Ethics committees, she said, should pick up on this.

When it comes to GetReal, Ryll sees huge value in the programme’s work to further the use of RWE. The hope is that real world data will be able to compensate for some of the biases seen in RCTs. When it comes to melanoma, trials have huge effect sizes, making it easy to optimise trial populations to allow for healthy stage 4 patients. Brain metastases, for instance, were chronically excluded—although some 80% of melanoma patients have brain metasteses present at death. Similarly, there was a systematic exclusion of patients with auto-immune diseases. Ultimately, these measures do not reflect real world clinical practice, Ryll concluded.

For these reasons, the melanoma community got involved in alternative medicines development designs, such as Medicines Adaptive Pathways to Patients (MAPPs). This interest is also motivated by a desire to encourage more appropriate benefit-risk assessments that reflect patient’s specific situations. As a patient, you have to make a decision at some point in time, Ryll noted—untreated melanoma is deadly and you cannot stand around and wait. For these patients, hearing from a doctor that a specific treatment can’t be provided because of possible side effects—doesn’t ring true. A melanoma stage 4 patient has a very different benefit-risk assessment than a healthy person does. RCTs are nothing but a scientific model, Ryll concluded: The problem is that that model has become a religion.

Robert Hemmings of MHRA continued the panel discussion with insights into the regulator’s point of view on GetReal’s work. Acknowledging that GetReal has accomplished a great deal already, he focused on how GetReal can continue to move forward, specifically taking into consideration scientific advice and benefit-risk assessment.

In terms of efficacy-effectiveness gap, it’s important to establish whether we are talking about looking at effectiveness on a population level – or at specific effect modifiers. There should absolutely be an expectation that effectiveness will differ from efficacy, however, better understanding is needed here. This, Hemmings noted, is something that traditional clinical trials cannot provide. That said, regulators of course understand that RCTs are conducted under controlled conditions (in terms of end points, target populations, etc.) and that trials cannot answer all questions. However, it’s still important to remember that inferences based from RCTs can still be made beyond the target population, particularly if you have a good understanding of the pharmacology.

While acknowledging the utility of the tools that are being rolled out by GetReal, Hemmings asked an important question: Is there a risk of rolling out tools so that they are used by people who don’t know the risks involved in using them? Having worked in the industry for many years, Hemmings—a statistician by background – noted that he would not trust himself to attempt a network meta analysis with observational data – and that he doesn’t know many people who he would trust with this task.

Hemmings also called for a greater focus on assumptions and validations. While methodologies and best practices can be defined, these still need to be validated. In particular when network meta-analyses rely on a combination of assumptions that can and cannot be tested—how will the discussion on untestable assumptions be governed? These assumptions can be used but more work needs to be done not just in presenting methodologies but also in validating them – an issue he presented to the GetReal leaders for further consideration.

Anton (Tony) Hoos of Amgen concluded the presentations from panelists with the industry point of view, asking: Where do the outputs from GetReal fit in medicines development and decision making? He came back to the original objectives outlined by GetReal, as outlined on the programme’s website:

• GetReal aims to show how robust new methods of RWE collection and synthesis could be developed and considered for adoption earlier in pharmaceutical R&D and the healthcare decision making process. This will require companies, healthcare decision makers and other stakeholders to work together to generate a consensus on best practice in the use of RWE in regulatory and reimbursement decision-making. Alternative evidence generating strategies will deliver more focused research in pharmaceutical R&D, and allow healthcare decision makers to be more certain when providing patients with access to new treatments.

Calling on those present to conduct a “feasibility check” he posed the question of whether these objectives have been fully reached—and concluded that this is not the case. While applauding the progress made, especially in aligning stakeholders, there is still work to be done in achieving a more efficient and focused process for offering access to medicines. Ultimately, the solution lies in questions around benefit-risk and uncertainty, and how patients are granted access to new medicines. This requires bringing patients more closely into the process, Hoos noted, asking: How can they inform us much more?

Hoos also pointed to the ongoing difficulties posed by isolation among stakeholders—not only in terms of diverse stakeholders but also in terms of geography. While different stakeholders have worked hard to optimise data sets, this has been done largely in isolation, resulting in a lack of alignment.

Some stakeholders are keen on RCTs and others on RWE. Meanwhile—citing his personal experience at a recent conference—stakeholders also express differences of opinion when it comes to joint advice for studies. In the particular case described by Hoos, stakeholder opinions on joint advice for studies resulted in a percentage alignment of 77% when it came to population; 44% when it came to comparators; and 60% when it came to endpoints. Additionally, different stakeholders have different levels of participation.

These numbers are without a doubt, sobering—and show that the goal of achieving the most efficient and streamlined process is not yet met. If we continue to work in isolation, Hoos noted, costs will continue to explode—and fewer patients will get the medicines they need. Reimbursement and access decisions are national. How can we use initiatives like GetReal to come up with one view? What opportunities can GetReal and similar initiatives provide, if we do them well?

Picking up on this thread, Elaine Irving, GSK and GetReal, turned the discussion towards the future, asking the panelists for their input on two questions:

1. What should GetReal aim to achieve in the last six months of the programme; if it was your money, what would you spend it on for the next six months?
2. Is there a need for a platform like this after the next six months? How can we address the challenges that have been mentioned regarding i.e. some of the methodological processes and tools?

Inviting participation not only from panelists but also from those in the audience, Irving opened the floor for comments. In the discussion that ensued, a number of objectives and suggestions were made regarding the points above, including:

• Apply activities to real cases, with more detail: While GetReal has done a great deal of work on particular case studies, it would be extremely valuable if some of the programme’s current activities could be linked to real cases—including actual assessments and products. From the regulator standpoint, this means that more information will be needed. For a regulator to commit to a proposed methodology (i.e. “While I still have to assess your data, I do not object to your methodology”) much more information is required. How is the quality of the data source? How is the comparison being made? What assumptions are going to be tested and how will they be validated?

• Expand the scope of participants: There is a problem in that GetReal still deals with a very limited group of people. These stakeholders are at the forefront of GetReal’s activities and more needs to be done to get others in the door and in the discussion. Clinicians, for example, could be better represented going forward.

• Consider “small ticket” decision-makers: Instead of targeting only “big ticket” decision-makers, the “small ticket” decision-makers or “decision influencers” could be more comprehensively included. These might include healthcare provider organisations, health insurers, clinical professional associations, and patient associations. They may not make decisions on behalf of a country like national regulators, but they are relevant actors whose opinion matters.

• Bring patients to the forefront: The significance of the patient in GetReal’s work is implicit, but could be more specifically delineated—i.e. in the programme’s objectives. To outside parties not familiar with the specifics of the work packages, it appears that the objectives focus on the interests of regulators and other stakeholders.

• Continue working on stakeholder alignment in a non-politicized atmosphere: While progress has been made, more needs to be done towards stakeholder alignment. GetReal is both multi-stakeholder and not very politicized, and in this regard it is very useful and really capable of accomplishing something. Greater alignment is still needed, however.

Click here to download a copy of the report