Expected Value of Sample Information - Example

Example

A regulatory agency is to decide whether to approve a new treatment. Before making the final approve/reject decision, they ask what the value would be of conducting a further trial study on subjects. This question is answered by the EVSI.

The diagram shows an influence diagram depiction of an Analytica model for computing the EVSI in this example. For the reader who wishes to study this computation in greater detail, the model can be viewed and evaluated from Analytica Web Player.

The model classifies the outcome for any given subject into one of five categories:

{"Cure", "Improvement", "Ineffective", "Mild side-effect", "Serious side-effect"}

And for each of these outcomes, assigns a utility equal to an estimated patient-equivalent monetary value of the outcome.

A decision state, in this example is a vector of five numbers between 0 and 1 that sum to 1, giving the proportion of future patients that will experience each of the five possible outcomes. For example, a state denotes the case where 5% of patients are cured, 60% improve, 20% find the treatment ineffective, 10% experience mild side-effects and 5% experience dangerous side-effects.

The prior, is encoded using a Dirichlet distribution, requiring five numbers (that don't sum to 1) whose relative values capture the expected relative proportion of each outcome, and whose sum encodes the strength of this prior belief. In the diagram, the parameters of the Dirichlet distribution are contained in the variable dirichlet alpha prior, while the prior distribution itself is in the chance variable Prior. The probability density graph of the marginals is shown here:

In the chance variable Trial data, trial data is simulated as a Monte Carlo sample from a Multinomial distribution. For example, when Trial_size=100, each Monte Carlo sample of Trial_data contains a vector that sums to 100 showing the number of subjects in the simulated study that experienced each of the five possible outcomes. The following result table depicts the first 8 simulated trial outcomes:

Combining this trial data with a Dirichlet prior requires only adding the outcome frequencies to the Dirichlet prior alpha values, resulting in a Diriclet posterior distribution for each simulated trial. For each of these, the decision to approve is made based on whether the mean utility is positive, and using a utility of zero when the treatment is not approved, the Pre-posterior utility is obtained. Repeating the computation for a range of possible trial sizes, an EVSI is obtained at each possible candidate trial size as depicted in this graph: