AdhereR: Interactive plotting (and more) with Shiny

Dan Dediu ddediu@gmail.com

2020-05-12

Introduction

AdhereR is an R package that implements, in an open and standardized manner, various methods linked to the estimation of adherence to treatment from a variety of data sources and formats (please see the other vignettes in the package, by involving, for example browseVignettes(package="AdhereR") or by visiting the package’s site on CRAN). One of the main aims of the package is to allow users to produce high quality, publication-ready and highly customizable graphical representations of both the patterns in the raw data and of the various estimates of adherence. This can be normally achieved from an R session or script using the plot() function applied to an estimated CMA object (the raw patterns are plotted by creating a basic CMA0 object), as detailed in the AdhereR: Adherence to Medications vignette. However, while allowing for a very fine-grained control over the resulting plots, this requires a certain level of familiarity with R (loading the source data, creating the appropriate CMA object, invoking the plot() function with the desired parameters, and the export of the resulting plot in the desired format at the quality and with the other desired characteristics), on the one hand, and the process is rather cumbersome when the user wants to explore and understand the data, or to try various types of plotting in search for the optimal visualization, on the other.

These reasons prompted us to develop a fully interactive user interface that should hide the “gory details” of data loading, CMA computation and plot() invocation under an intuitive and easy to use point-and-click interface, while allowing fast exploration and customization of the plots. However, because this interactive user interface covers a rather particular set of use cases, tends to be rather “heavy” in terms of dependencies, and may not install or run properly in some environments (e.g., headless servers or older systems), we decided to implement it in a separate package, AdhereRViz that extends AdhereR (i.e., AdhereRViz requires AdhereR, but AdhereR cn happily run without AdhereRViz).

Overview

We use Shiny, which allows us to build a self-contained app that can be run locally or remotely inside a standard web browser (such as Firefox, Google Chrome, Safari, Internet Explorer, Edge or Opera) on multiple Operating Systems (such as Microsoft Windows, Apple’s macOS and iOS, Google’s Android, and several flavors of Linux – e.g., Debian, Ubuntu, Fedora, RedHat, CentOS, Arch… – and BSD – e.g., FreeBSD or TrueOS) and devices ranging from desktop and laptop computers to mobile phones and tablets. The app’s interface uses standard controls and paradigms, ensuring a similar user experience across browsers, platforms and devices.

Launching the app

Locally, the app can be launched from a normal R session (including from within RStudio) or script with a single command; of course, the latest version of AdhereR and AdhereRViz must be installed on the system (using, for example, install.packages("AdhereRViz", dep=TRUE) or RStudio’s ToolsInstall Packages… menu; or, in case it is already installed, updated using update.packages() or RStudio’s ToolsCheck for Package Updates… menu), and loaded in the current session (using, for example, library(AdhereRViz) or require(AdhereRViz)). With these prerequisites in order, the app can be launched without any parameters with

plot_interactive_cma()

or, if so desired, by specifying a data source and the important column names and optionally the desired CMA, as in the following example, where we use CMA0 (i.e., the raw data) from the sample dataset med.events (see its structure in the Table below) included with the AdhereR package:

plot_interactive_cma(data=med.events, # included sample dataset
                     cma.class="simple", # simple cma, defaults to CMA0
                     # The important column names:
                     ID.colname="PATIENT_ID",
                     event.date.colname="DATE",
                     event.duration.colname="DURATION",
                     event.daily.dose.colname="PERDAY",
                     medication.class.colname="CATEGORY",
                     # The format of dates in the "DATE" column:
                     date.format="%m/%d/%Y");
The structure of the sample dataset med.events included in the AdhereR package. We shows the rows 23 to 33 (out of a total of 1080 rows), each row representing one event which is characterized at the minimum by: the patient it refers to (identified by the patient’s unique ID in column PATIENT_ID), the date it happened (in column DATE, recorded in a uniform format, here MM/DD/YYYY), its duration in days (in column DURATION); optionally we can also have info concerning the prescribed daily quantity or dose (column PERDAY) and the class or type of treatment (column CATEGORY). We will use this dataset throughout this vignette.
PATIENT_ID DATE PERDAY CATEGORY DURATION
1 03/22/2035 2 medB 30
1 03/31/2035 2 medB 30
2 01/20/2036 4 medA 50
2 03/10/2036 4 medA 50
2 08/01/2036 4 medA 50
2 08/01/2036 4 medB 60
2 09/21/2036 4 medB 60
2 01/24/2037 4 medB 60
2 04/16/2037 4 medB 60
2 05/08/2037 4 medB 60
3 04/13/2042 4 medA 50

The app can also be launched in the “standard way” using runApp() or RStudio’s ▶︎ Run app button.

Alternatively, the app may be made available on a remote server, such as on https://www.shinyapps.io/, in which case it can be accessed simply by pointing the web browser to the app’s internet address.

Please note that launching the App with no parameters, opens with a different screen (see the Selecting/changing the data source section for details).

Also note that there we provide a “stub” function plot_interactive_cma() in the package AdhereR, but this simply checks if AdhereRViz is installed and functional, and then tries to invoke plot_interactive_cma() from AdhereRViz.

The App’s User Interface (UI)

The App’s UI has several main elements which can be seen below. Most UI elements have tooltips that show up on hovering the mouse over the element and that offer specific information (but please note that these tooltips might need some time before showing up).

Overview of the User Interface. Screenshot (App is running in Firefox on macOS 10.13) of some of the main UI elements (dotted black ellipses or rectangles identified with black numbers). 1 is a button that opens a box giving information about the App. 2 is a button that exist the App cleanly (i.e., stops and disconnects the session). 3 is the main plotting area which displays the current plot. 4 shows some of the UI elements controlling the plot (element 4) size. 5 is the area where various parameters can be modified. 6 opens UI elements specific for saving the current plot to file. 7 shows the R code that can be used to generate the current plot. 8 gives access to UI controls that allow the computation of the current CMA for many more patients and saving the results to file. 9 displays messages, warnings or errors that might have been geenrated while constructing the current plot.

Overview of the User Interface. Screenshot (App is running in Firefox on macOS 10.13) of some of the main UI elements (dotted black ellipses or rectangles identified with black numbers). 1 is a button that opens a box giving information about the App. 2 is a button that exist the App cleanly (i.e., stops and disconnects the session). 3 is the main plotting area which displays the current plot. 4 shows some of the UI elements controlling the plot (element 4) size. 5 is the area where various parameters can be modified. 6 opens UI elements specific for saving the current plot to file. 7 shows the R code that can be used to generate the current plot. 8 gives access to UI controls that allow the computation of the current CMA for many more patients and saving the results to file. 9 displays messages, warnings or errors that might have been geenrated while constructing the current plot.

About the App

Clicking on the About button (element 1 in the overview figure) opens a box with info about the App, such as the version of the AdhereR package, and overview of the package and links to where more help (such as vignettes) can be found.

Cleanly exiting the App

It is recommended to cleanly exit the App by clicking the Exit… button (element 2 in the overview figure), as simply closing the browser will not normally also stop the R process running in the background. Please note that currently, exiting the App will not also close the browser window or tab in which the App was running…

The plotting area (and the messages)

The current plot is displayed in the UI element 3 (in the overview figure), a canvas that can be re-sized using the UI elements 4 in the overview figure (see also below) and which, when too big, can be scrolled horizontally and vertically at will. This canvas is currently passive in the sense that it simply displays a plot which which interaction is possible only using the other elements of the UI, but almost all aspects of this plot can be tweaked using controls from the left-hand side vertical panel (element 5 in the overview figure; see details below).

While the interpretation of these plots should be relatively intuitive, it is nevertheless detailed in the AdhereR: Adherence to Medications vignette.

Element 9 in the overview figure displays most of the information messages, warnings and errors generated during the plotting process (please note that currently some messages, warnings and errors might not be captured and only shown in the R console). For example, here, the informational message Plotting patient ID ‘1’ with CMA ‘CMA9’ Plotting patient ID ‘2’ with CMA ‘CMA9’ means that the computation and plotting of CMA9 for patients with IDs 1 and 2 was successful.

Elements 4 in the overview figure allow the control of the horizontal and vertical dimensions of the plot 3 either coupled (i.e., keeping the current width–to-height ratio), when the Keep ratio switch is ON, or independently of each other, when the switch is OFF (in which case a new slider controlling the plot height appears). The interaction with the slider(s) can be done either with mouse or with the arrow keys.

Please note that there is a minimum size requirement for a plot to be displayed, otherwise an error of the type

Plotting area is too small (it must be at least 10 x 0.5 characters per event, but now it is only 31.1 x 0.5)!

is thrown, in which case either the plotting area needs to be increased using the Plot width (and, if visible, Plot height) slider(s), or the number of patients or the duration to be shown need to be reduced. Alternatively, the Advanced section (see Section Setting parameters for details) can be used to decrease these minimum requirements (but this not recommended in most cases).

Saving the current plot to file

The current plot can be exported to a variety of formats by turning the Save plot! switch ON:

Saving the current plot to file. A zoom-in of the new controls (element 10) revealed by turning the Save plot! switch (UI element 6) ON. Also visible is an explanatory tooltip .

Saving the current plot to file. A zoom-in of the new controls (element 10) revealed by turning the Save plot! switch (UI element 6) ON. Also visible is an explanatory tooltip .

These new UI elements (10) allow:

By pressing the Save plot button, the user can select the location and file name (relative to the local machine) under which the plot will be exported.

Viewing, copying and using the R code that would produce the current plot

While the main use scenarios for this App are built around interactivity, the user may want to generate the same (or similar) plots as the one currently displayed (in element 3 in the overview figure). To allow this, we provide the Show R code… button (element 7 in the overview figure), which opens a box with the clearly commented R code:

Viewing the R code that can generates the current plot.

Viewing the R code that can generates the current plot.

Clicking the Copy to clipboard button copies the R code to the clipboard, from where it can be pasted into an editor of choice (such as RStudio). In particular, for the plot shown in the overview figure, the R code displayed is:

# The R code corresponding to the currently displayed Shiny plot:
# 
# Extract the data for the selected 2 patient(s) with ID(s):
# "1", "2"
# 
# We denote here by DATA the data you are using in the Shiny plot.
# This was manually defined as an object of class data.frame
# (or derived from it, such a data.table) that was already in
# memory under the name 'med.events'.
# Assuming this object still exists with the same name, then:

DATA <- med.events;

# These data has 5 columns, and contains info for 100 patients.
# 
# To allow using data from other sources than a "data.frame"
# and other similar structures (for example, from a remote SQL
# database), we use a metchanism to request the data for the
# selected patients that uses a function called
# "get.data.for.patients.fnc()" which you may have redefined
# to better suit your case (chances are, however, that you are
# using its default version appropriate to the data source);
# in any case, the following is its definition:
get.data.for.patients.fnc <- function(patientid, d, idcol, cols=NA, maxrows=NA) d[ d[[idcol]] %in% patientid, ]
# Try to extract the data only for the selected patient ID(s):
.data.for.selected.patients. <- get.data.for.patients.fnc(
    c("1", "2"),
    DATA, ### don't forget to put here your REAL DATA! ###
    "PATIENT_ID"
);
# Compute the appropriate CMA:
cma <- CMA9(data=.data.for.selected.patients.,
            # (please note that even if some parameters are
            # not relevant for a particular CMA type, we
            # nevertheless pass them as they will be ignored)
            ID.colname="PATIENT_ID",
            event.date.colname="DATE",
            event.duration.colname="DURATION",
            event.daily.dose.colname="PERDAY",
            medication.class.colname="CATEGORY",
            carry.only.for.same.medication=FALSE,
            consider.dosage.change=FALSE,
            followup.window.start=0,
            followup.window.start.unit="days",
            followup.window.duration=730,
            followup.window.duration.unit="days",
            observation.window.start=0,
            observation.window.start.unit="days",
            observation.window.duration=730,
            observation.window.duration.unit="days",
            date.format="%m/%d/%Y"
           );

if( !is.null(cma) ) # if the CMA was computed ok
{
    # Try to plot it:
    plot(cma,
         # (same idea as for CMA: we send arguments even if
         # they aren't used in a particular case)
         align.all.patients=FALSE,
         align.first.event.at.zero=FALSE,
         show.legend=TRUE,
         legend.x="right",
         legend.y="bottom",
         legend.bkg.opacity=0.5,
         legend.cex=0.75,
         legend.cex.title=1,
         duration=NA,
         show.period="days",
         period.in.days=90,
         bw.plot=FALSE,
         col.na="#D3D3D3",
         unspecified.category.label="drug",
         col.cats=rainbow,
         lty.event="solid",
         lwd.event=2,
         pch.start.event=15,
         pch.end.event=16,
         col.continuation="#000000",
         lty.continuation="dotted",
         lwd.continuation=1,
         cex=1,
         cex.axis=1,
         cex.lab=1.25,
         highlight.followup.window=TRUE,
         followup.window.col="#00FF00",
         highlight.observation.window=TRUE,
         observation.window.col="#FFFF00",
         observation.window.density=35,
         observation.window.angle=-30,
         observation.window.opacity=0.3,
         show.real.obs.window.start=TRUE,
         real.obs.window.density=35,
         real.obs.window.angle=30,
         print.CMA=TRUE,
         CMA.cex=0.5,
         plot.CMA=TRUE,
         CMA.plot.ratio=0.1,
         CMA.plot.col="#90EE90",
         CMA.plot.border="#006400",
         CMA.plot.bkg="#7FFFD4",
         CMA.plot.text="#006400",
         plot.CMA.as.histogram=TRUE,
         show.event.intervals=TRUE,
         print.dose=TRUE,
         print.dose.outline.col="#FFFFFF",
         print.dose.centered=FALSE,
         plot.dose=FALSE,
         lwd.event.max.dose=8,
         plot.dose.lwd.across.medication.classes=FALSE,
         min.plot.size.in.characters.horiz=10,
         min.plot.size.in.characters.vert=0.5
    );
}

This code is pretty much ready to be run, except for some issues that might surround accessing the actual data used for plotting: the user is reminded of these through the yellow-on-red bold italic highlighting of DATA (not shown in the code listing above). In a nutshell (for details, see below), if (a) the user interactively uses the App to load or connect to a data source (such as an external file or an SQL database), then the identity of this data source is known (the file name or the database location), but if (b) the data source was passed to the plot_interactive_cma() function as the data argument, the App cannot know how this data source was named (and this “name” might not even exist if, for example, the data was created on-the-fly while calling the plot_interactive_cma() function). Wickedly, even in case (a), it is generally unsafe to assume that the data source will stay the same (or will be accessible in the same way) in the future. Thus, while we provide as much info about the data source used to produce the current plot as possible, we also warn the user to be careful when running this code!

Computing the CMA for several patients

By switching the UI element 8 (in the overview figure) Compute CMA for several patients… to ON, the user unlocks a new set of UI elements that allow the computation of the currently defined CMA for more patients and the export of the results to an external file.

First, it is important to highlight that the App is not intended for heavy computations, which explains why we are currently limiting this CMA computation to at most 100 patients, at most 5000 events across all patients (if more patients or events are selected, the computation will be done for only the first 100 and 5000, respectively) and for at most 5 minutes of running time (after which the computation is automatically stopped). If seriously heavy computation is needed, we recommend the use of the appropriate CMA() functions from and R session or script, which allow many types of parallel processing and the use of several types of data sources with very fine-grained control, as described in the vignettes AdhereR: Adherence to Medications and Using AdhereR with various database technologies for processing very large datasets. The R code needed to compute the current CMA can be accessed through the Show R code button (UI element 7).

The patients for which the computation of the CMA is to be performed can be done in two main ways:

  1. by individually selecting patients by their IDs, through a multiple selection dropdown list (element 11 in the figure below):
Selecting patients individually for the computation of CMA.

Selecting patients individually for the computation of CMA.

  1. by selecting a continuous range of patients using two sliders (element 15 in the figure below) which define a set of positions in a list (element 14 in the figure below); this list can contain the patient IDs in their original order in the data source, or can have them sorted in ascending or descending order by ID (element 13 in the figure below).
Selecting patients by range of positions in a list for the computation of CMA. In this example, we ordered the patients descreasingly by ID (using the combobox 13), resulting in a mapping of positions (#) to ID as shown in the list 14, where patient with ID “100” is on psition 1, patient “99” on position 2, etc. The sliders 15 define the range of positions (#) 3 to 24, which means that we selected patients with IDs “98”, “97”, “96” … “78” and “77”.

Selecting patients by range of positions in a list for the computation of CMA. In this example, we ordered the patients descreasingly by ID (using the combobox 13), resulting in a mapping of positions (#) to ID as shown in the list 14, where patient with ID “100” is on psition 1, patient “99” on position 2, etc. The sliders 15 define the range of positions (#) 3 to 24, which means that we selected patients with IDs “98”, “97”, “96” … “78” and “77”.

These two ways of selecting patients should be flexible enough for cover most cases of (semi-)interactive use; for more patients and/or the selection of patients based on more complex criteria, we suggest the use of the R code in a script.

After patients have been selected, the user can press the Compute CMA button (UI element 12) to access a specialized dialog box (see figure below) where the CMA computation can be started, its progress monitored, or stopped, and from where the results can be exported to file.

Starting, stopping and watchin the progress of CMA computation for several patients. The list of patients is given in UI element 16, and the progress of the computation is tracked by theprogress bar and individual success report (UI elements 17). The button Save results (as TSV) allows the user to select a file where the results will be exported as a TAB-separated CSV file.

Starting, stopping and watchin the progress of CMA computation for several patients. The list of patients is given in UI element 16, and the progress of the computation is tracked by theprogress bar and individual success report (UI elements 17). The button Save results (as TSV) allows the user to select a file where the results will be exported as a TAB-separated CSV file.

Setting parameters

The left-hand panel has two tabs: Params and Data, and we are focusing here on Params, which contains various parameters customizing the computed CMA and the plotting of the results. UI element 5 in the overview figure shows part of this panel, but the following principles apply:

Basic information about the current dataset. Here, med.events, showing also the five important columns.

Basic information about the current dataset. Here, med.events, showing also the five important columns.

Folding and unfolding the contents of a section. The section Follow-up window (FUW) with content folded (hidden) on the left, and unfolded (visible) on the right.

Folding and unfolding the contents of a section. The section Follow-up window (FUW) with content folded (hidden) on the left, and unfolded (visible) on the right.

We will now go through all sections one by one.

General settings

The General settings section is always visible and allows the selection of:

  • CMA type: the type of CMA to compute, which can be (please see Dima & Dediu, 2017, and the vignette AdhereR: Adherence to Medications for more details):

    • simple: one of the “simple” CMAs, currently CMA0 tot CMA9,
    • per episode: computes one of the “simple” CMAs repeatedly for each treatment episode,
    • sliding window: computes one of the “simple” CMAs repeatedly for a set of sliding windows
  • CMA to compute: the “simple” CMA to compute, either by itself (for CMA type == simple) or iteratively (for the other two “complex” types); please note that by definition CMA0 cannot be used with “complex” CMAs (which explains why it cannot be selected in these cases)

  • Patient(s) to plot: the list of patient IDs, selected from a drop-down list (which allows multiple selections) containing all the patient IDs in the current data source (at least one patient must be selected, otherwise an error is generated)

Depending on these selections the plot may change or various types of errors or warnings may be thrown.

Follow-up window (FUW) and Observation window (OW)

These two sections are very similar and allow the definition of the follow-up (FUW) and observation (OW) windows by specifying:

  • their start: this can be either:

    • the number of units (days, weeks, months or years) relative to the first event (for FUW) and to the start of the FUW (for OW), or
    • an absolute calendar date
  • and their duration as a number of units (days, weeks, months or years).

Carry over

This section is shown only for CMA5 to CMA9 and concerns the way carry over is considered:

  • For same treat.only: only for the treatment class or across classes
  • Consider dosage changes: should dosage changes be considered when computing the carry over?

Define episodes

This section is shown only for CMA per episodes and concerns the way treatment episodes are defined:

  • Treat. change starts new episode?: does changing the treatment class trigger a new episode?
  • Dose change starts new episode?: does changing the dose trigger a new episode?
  • Max. gap duration: the duration of a gap above which a new episode is triggered; the gap can be in units (Max. gap duration unit) of days, weeks, months or years, or as a percent of the last prescription
  • Plot CMA as histogram?: should the distribution of CMA estimates across episodes for a given participant be plotted as a histogram or as a barplot

Define sliding windows (SW)

This section is shown only for sliding windows and concerns the way this sequence of regularly spaced and uniform sliding windows is defined:

  • SW start: when is the first sliding window starting (relative to the start of the OW) in terms of units (SW start unit) that can be days, weeks, months or years
  • how long is one such sliding window SW duration in terms of SW duration unit (days, weeks, months or years)
  • the step between two consecutive sliding windows can be defined either in terms of:

    • SW number of steps: the total number of steps (i.e., sliding windows of the given duration covering the OW), or
    • the duration of a setp (i.e., one sliding window) in terms of how many (SW step duration) units (SW step unit; days, weeks, months or years) it lasts
  • Plot CMA as histogram?: should the distribution of CMA estimates across sliding windows for a given participant be plotted as a histogram or as a barplot