Build and evaluate a decision tree in R

Introduction

For this ThuRsday Tutorial, we're going to cover something a bit different. Instead of showing how to do fairly simple epidemiological calculations, this edition will cover the first of many Machine Learning and Artificial Intelligence topics: Decision Trees. First, we're going to cover what Decision Trees are, then do a quick step by step implementation guide in R, and lastly, leave you with a bit of homework should you wish. As usual, if you want to follow along, head over to Cody’s Github to download the article code and data.

What is a Decision Tree?

A decision tree is a type of supervised learning algorithm used for classification and regression tasks. To newcomers that previous sentence may have made very little sense, so let's break this down.  A supervising learning algorithm is a machine learning method that uses data that has at least in part already been reviewed by a human, so the algorithm can learn through reviewing previously tagged data, and test itself against similarly tagged data.Classification is the sorting of data into specific categories (for example, petting pictures of cats and dogs in a "pets" category, while pictures of flowers and trees may be put in a "plants" category), and Regression is a way to estimate future numerical data based on patterns seen in previous numerical data. A decision tree operates much like the process you might use to make a decision in everyday life—by asking a series of yes/no questions until a conclusion is reached. In the context of machine learning, these questions are based on data and are structured in a tree-like model of decisions.

The Structure of a Decision Tree

1. Nodes: Each point in the tree where a question is asked is called a node. The very first question, which starts the decision-making process, is known as the root node. From this point, the tree branches out based on possible answers to the questions posed.
2. Branches: Each answer to a question leads to a new branch in the tree. These branches represent the flow from one question to another, guiding the process through more specific inquiries until a final decision can be made.
3. Leaves: The leaves of the tree are the end points, representing the outcomes or final decisions. Once a leaf is reached, no further questions are necessary, and the decision tree has provided its prediction or classification based on the inputs given.

Decision trees are built using algorithms that identify the most significant questions to ask at each step. The goal is to select questions that best split the data into distinct categories, making it easier to predict or classify the data as you move deeper into the tree. This process involves statistical methods that measure how well each question separates the data according to the target variable (the outcome we want to predict). In technical terms, the algorithm selects the questions based on measures like Gini impurity or entropy, which quantify how much uncertainty or "messiness" exists in the categories formed at each step. By choosing questions that reduce this uncertainty the most, the decision tree can efficiently reach accurate decisions. One practical example of how one could imagine a decision tree is imagining having to sift through hospital visit data to find potential cases of the flu. Say that you know from previous experience what all the symptoms and test results may need to be to be certain of a flu diagnosis (the training section), and you're given a few records to skim through. You'd likely use a system of yes or no questions, arranged by relative relevance to sort these records. Does the record indicate a positive flu test? If so, then we can say it's a flu record, if not, it may not be the flu, and if we don't have results, we can move down a branch and onto the next node. Does the record indicate fever, aches, or have had recent contact with someone already sick with the flu? Depending on the answer we can use the algorithm to not only have a good idea of which records are flu or not, but also have a standard path to explain our decisions easily.   Now that we have a basic idea of how Decision Trees work, let's try building one in R.  

Building a Decision Tree in R

Project Outline

for this scenario, imagine you've been contacted to take a look at some healthcare expenditure data, and to fill in some potentially missing data so that other programs might be able to better estimate the benefits of prevention programs for various conditions prior to surgical intervention in terms of patients seen for said procedures. For this, we'll be using data from the Health Service Executive , sourced from Open EU Datasets Portal. For this tutorial, the data is also included in the Github repository where the tutorial code is also found. This dataset shows various surgical procedures for the last 20 years or so, how many of these procedures were performed each year, and if there was a break in the reporting series for the year being reported on. By nature of the data, the decision tree will be very simple, and while such a simple use case may not be ideal for this task, the limited variables make it easier to see how these work. In general, you'd want data with more variables that could help explain potential relationships between key factors.  

Setting Up in R: Libraries

First, ensure you have the necessary packages installed and loaded. We will use `readr` for data loading, `caret` for data partitioning, and `rpart` for creating the decision tree models.

Data Read-In

We start by loading our dataset and examining its structure. This step helps us understand the types of variables and data formats we are dealing with.

Here, we can see that we're dealing with character and numerical variables, which is fine to start with. But to be able to make our Decision Tree work out, we'll need to change a few of these data types around a bit.

 Data Preparation

Converting character variables to factors is crucial because many modeling techniques, including decision trees, handle factors differently from other data types. Namely, given that factor data is organized into distinct categories, it can help our Decision Tree organize the questions and answers into a proper tree. Here, we will want to see how the Year , Type of Surgical Procedure and Case Type influence our Value variable, so let's go ahead and change those to factors.

We also split the data into two subsets based on a flag indicating a break in series, where those with breaks are going to be serving as our implementation dataset, while the rest will be used for training and testing.

Creating Training and Testing Data

For training our model, we create a stratified sample to ensure each category of the surgical procedure is fairly represented in both training and testing sets.

Here we are setting a random seed to pick out what records are retained in training or testing, then breaking the data up based on surgical procedure as mentioned earlier, and creating a 70-30 split for training and testing respectively. We then break those sets up for later use.

Building and Evaluating the Decision Tree Model

We now build our decision tree model using the `rpart` function.

Within `rpart`, we set up an equation of sorts for the Decision Tree to work with where our Value variable is considered to be determined by a mix of our factored variables, where we can train on our predetermined data, and are using ANOVA regression tree models specifically. Next we can also take a look at how well it's handled the training data so far, either through various statistical measures using the summary function, or by visually inspecting the resulting tree using both plot for the basic shape, and text to show the variables and associated potential choices and weight. Even at this point we can see that the Mean Squared Error is very large, and can fairly comfortably be certain this method is less than ideal for this question, but we can still see how it performs against the testing dataset, and go from there.

 Improving the Decision Tree

These initial results are not great to say the least, so we might consider several strategies: –Cross-validation: Helps in understanding how the model performs across different subsets of the data, and can improve results with that understanding. –Parameter tuning: Adjusting the complexity of the model might help reduce overfitting or underfitting. As a quick note, overfitting refers to when a model can do extremely well on training data, but when introduced to new data which don't match the exact trends and patterns, tend to fail. Underfitting basically means a generally under-performing model both with training and other data.

With the above script we edit our model call slightly, using a slightly different method and then doing a 10x cross validation. We can then print the results of this validation and get a cp value for the best model. This cp value can then be used to control the specifics of the model and be ran instead of the base model. We can then go over what we've previously done, and finally implement a model on our missing data.Given the data isn't truly missing, we can look at the Predicted Value and examine it against our actual Value to see how well this model did in both base performance and how the model can explain data relationships.

GitHub Gist

Conclusion: Next Steps and Future Improvements

Based on the performance and the nature of data, we could consider: – Including more relevant features that might influence the outcomes. – Trying different types of models that may fair better for this question (Regression models, XGBoost, etc). – Revisiting data preprocessing steps to handle outliers, missing values, or transform variables. – If possible, changing the question to see if end values can be categorical instead of numerical (so under 5000, between 5000-10000, etc), which tends to perform better in tree models. Feel free to try your hand at improving performance, and use it as a bit of a testing ground. It's worth noting that with a lot of Machine Learning methods, practicing how and when to implement them is crucial to having a good understanding of methods and their steps.

Related reading: Exploratory data analysis in R; Sampling methods and sample size.

Build an SEIR model in Python

The Susceptible-Exposed-Infected-Recovered (SEIR) model is a natural extension of the SIR Model, accounting for a fourth category of disease state, Exposure. For this PyFriday Tutorial, we'll cover how to not only make a quick SEIR model but also how to graph the results. If you'd like a decent background on what all goes into the SIR model conceptually and mathematically, please check out our Epi Explained article for Understanding SIR Models , or to see the more simplistic model in action you can also check out the previous SIR model PyFriday Tutorial  article. As usual, if you'd like to play with the code on your own, you can download it through Cody’s Github.

Step 1: Installing and Loading Necessary Libraries

We have to start with setting up the Python environment by installing and importing the necessary libraries. For this tutorial, we'll utilize numpy for numerical operations,

scipy for solving differential equations, and plotly for interactive visualizations.

• numpy is crucial for handling arrays and mathematical functions.
• scipy offers powerful options for numerical integration and solving differential equations, playing a pivotal role in our model’s computation.
• plotly enables the creation of dynamic plots, allowing us to present the SIR model’s dynamics, as well as allow for users to see more minute details that otherwise wouldn’t be available.

Step 2: Defining the SEIR Model

Incorporating the Exposed compartment modifies our approach from the original SIR model slightly. Now, we have four states: Susceptible (S), Exposed (E), Infected (I), and Recovered (R). The transitions now include a movement from Susceptible to Exposed before individuals become Infectious. Here's how we set up the SEIR model in Python:

  Now on first look this might be a bit strange even to experienced R users as the function seems to do very little with what we're inputting, but in reality this is because we're actually going to use this function in yet another function in Step 3. For now though, let's do a quick run-through of what we're just set up. The function takes three arguments:

• time: The current time point. While not directly used in the differential equations, it is required by the odeint function from the scipy package.
• variables: A vector containing the current numbers of susceptible, infected, and recovered individuals (S, E, I, R).
• parameters: A vector of parameters for the model, specifically beta, gamma, sigma, and N (the total population).

Within the function, we use the with function to combine

variables and parameters into a single list, which simplifies accessing these values. The with function allows us to refer to elements of the list directly by their names (e.g.,S,

E,I,

R, beta,

gamma,sigma,

N) within its code block. The rates of change for each compartment are then calculated according to the SEIR model equations:

• dS: The rate of change of susceptible individuals is the negative product of the infection rate (beta), the number of susceptible individuals (S), and the number of infected individuals (I), divided by the total population (N). This represents the number of new infections per time unit.
• dE represents the rate of change for the Exposed population, transitioning from Susceptible due to infection and moving towards Infectious as the incubation period ends. The parameter sigma controls the rate at which Exposed individuals become Infectious, adding a new dynamic to the model.
• dI: The rate of change of infected individuals is the number of new infections (as calculated for dS) minus the number of recoveries, which is the product of the recovery rate (gamma) and the number of infected individuals (I).
• dR: The rate of change of recovered individuals is simply the number of recoveries, as calculated in the dI equation.

Finally, the function returns a list containing the rates of change for each compartment, which is then used by the odeint function to solve the model over time.

Step 3: Initial Conditions, Parameters, and Solving

Before solving the model, we need to specify the initial conditions (i.e., the starting numbers of susceptible, infected, and recovered individuals) and the parameters (beta,

sigma, and gamma) which control the disease transmission, incubation period and recovery rates, respectively. Here, we start with a population of 1000 individuals, with only one infected person as the outbreak begins. The parameter values are hypothetical and can be adjusted based on the disease's characteristics you're modeling (the incubation period is seen as an inverse, so we have an incubation period of five days, thus a parameter value of 1/5). With the function we already made and initial setup ready, we can now solve the differential equations using

scipy's odeint function, which will return the solutions to every step from day 0 to day 50.

Step 4: Visualizing the Outcome with Plotly

Finally, to understand the dynamics of the disease through time, we'll visualize the results using plotly. This step transforms the output into a dynamic and interactive plot that can is much easier to understand, and can often wow your desired audience.

This is a fairly simply plotly graph, but it's worth noting that for each class, we need to create a separate trace for each line, which allows them to act independently of one another. It should be noted that in Python, graphs often require initialization (hence the empty go.Figure() call). Otherwise, a lot of this code is actually very similar to how R handles graphing if you are familiar with that. Below is our interactive output:

Conclusion

Through this tutorial, we've navigated the implementation and visualization of the SEIR model using Python, showcasing the model's utility in understanding infectious disease dynamics. This model as mentioned earlier is a natural extension of the SIR compartmental model, though by no means should be seen as the last one. In fact, there are dozens of potential compartmental models that include things like vaccination, spatial relationships, and much more.

Build an SEIR model in R

The Susceptible-Exposed-Infected-Recovered (SEIR) model is a natural extension of the SIR Model, accounting for a fourth category of disease state, Exposure. For this ThuRsday Tutorial, we'll cover how to not only make a quick SEIR model but also how to graph the results. If you'd like a decent background on what all goes into the SIR model conceptually and mathematically, please check out our Epi Explained article for Understanding SIR Models , or to see the more simplistic model in action you can also check out the previous SIR model ThuRsday Tutorial  article. As usual, if you'd like to play with the code on your own, you can download it through Cody’s Github.

Step 1: Installing and Loading Necessary Packages

Just like last time, we'll be using two primary packages in this tutorial. The two packages we'll be using are deSolve and plotly.

• deSolve is a package for solving differential equations in R, which we’ll use to solve the SEIR model equations.
• plotly allows for creating interactive plots, which will enable us to visualize the dynamics of the SEIR model effectively.

Step 2: Defining the SEIR Model

Incorporating the Exposed compartment modifies our approach from the original SIR model slightly. Now, we have four states: Susceptible (S), Exposed (E), Infected (I), and Recovered (R). The transitions now include a movement from Susceptible to Exposed before individuals become Infectious. Here's how we set up the SEIR model in R:

Now on first look this might be a bit strange even to experienced R users as the function seems to do very little with what we're inputting, but in reality this is because we're actually going to use this function in yet another function in Step 3. For now though, let's do a quick run-through of what we're just set up. The function takes three arguments:

• time: The current time point. While not directly used in the differential equations, it is required by the ode function from the deSolve package.
• variables: A vector containing the current numbers of susceptible, infected, and recovered individuals (S, E, I, R).
• parameters: A vector of parameters for the model, specifically beta, gamma, sigma, and N (the total population).

Within the function, we use the with function to combine

variables and parameters into a single list, which simplifies accessing these values. The with function allows us to refer to elements of the list directly by their names (e.g.,S,

E,I,

R, beta,

gamma,sigma,

N) within its code block. The rates of change for each compartment are then calculated according to the SEIR model equations:

• dS: The rate of change of susceptible individuals is the negative product of the infection rate (beta), the number of susceptible individuals (S), and the number of infected individuals (I), divided by the total population (N). This represents the number of new infections per time unit.
• dE represents the rate of change for the Exposed population, transitioning from Susceptible due to infection and moving towards Infectious as the incubation period ends. The parameter sigma controls the rate at which Exposed individuals become Infectious, adding a new dynamic to the model.
• dI: The rate of change of infected individuals is the number of new infections (as calculated for dS) minus the number of recoveries, which is the product of the recovery rate (gamma) and the number of infected individuals (I).
• dR: The rate of change of recovered individuals is simply the number of recoveries, as calculated in the dI equation.

Finally, the function returns a list containing the rates of change for each compartment, which is then used by the ode function to solve the model over time.  

Step 3: Setting Initial Conditions and Parameters, Then Solving

Before solving the model, we need to specify the initial conditions (i.e., the starting numbers of susceptible, infected, and recovered individuals) and the parameters (beta,

sigma, and gamma) which control the disease transmission, incubation period and recovery rates, respectively. Here, we start with a population of 1000 individuals, with only one infected person as the outbreak begins. The parameter values are hypothetical and can be adjusted based on the disease's characteristics you're modeling (the incubation period is seen as an inverse, so we have an incubation period of five days, thus a parameter value of 1/5). With the function we already made and initial setup ready, we can now solve the differential equations using

deSolve's ode function, which will return the solutions to every step from day 0 to day 50.

Step 4: Visualizing the Results with Plotly

Finally, to understand the dynamics of the disease through time, we'll visualize the results using plotly. This step transforms the output into a dynamic and interactive plot that can is much easier to understand, and can often wow your desired audience.

This is a fairly simply plotly graph, but it's worth noting that for each class, we need to create a separate trace for each line, which allows them to act independently of one another. The output can be seen here:

Conclusion

Through this tutorial, you've learned how to implement and visualize the SEIR model in R using deSolve and plotly. This model as mentioned earlier is a natural extension of the SIR compartmental model, though by no means should be seen as the last one. In fact, there are dozens of potential compartmental models that include things like vaccination, spatial relationships, and much more.  

Build an SIR model in Python

The Susceptible-Infected-Recovered (SIR) model is a fundamental concept in epidemiology, offering insights into how diseases spread and recede in populations over time through a relatively simply set of functions. For this ThuRsday Tutorial, we’ll cover how to not only make a quick SIR model but also how to graph the results. If you’d like a decent background on what all goes into the SIR model conceptually and mathematically, please check out our Epi Explained article for Understanding SIR Models. Now, let’s get to coding. As usual, if you’d like to play with the code on your own, you can download it through Cody’s Github.

Step 1: Installing and Loading Necessary Libraries

We have to start with setting up the Python environment by installing and importing the necessary libraries. For this tutorial, we'll utilize numpy for numerical operations,

scipy for solving differential equations, and plotly for interactive visualizations.

• numpy is crucial for handling arrays and mathematical functions.
• scipy offers powerful options for numerical integration and solving differential equations, playing a pivotal role in our model’s computation.
• plotly enables the creation of dynamic plots, allowing us to present the SIR model’s dynamics, as well as allow for users to see more minute details that otherwise wouldn’t be available.

Step 2: Defining the SIR Model

The essence of the SIR model lies in its compartmental approach, segmenting the population into Susceptible (S), Infected (I), and Recovered (R) groups. The transitions between these states are governed by differential equations. Here's the Python function that does all the mathematical heavy lifting:

In this function:

• variables holds the current counts of Susceptible, Infected, and Recovered individuals.
• N, beta, and gamma represent the total population, infection rate, and recovery rate, respectively.

Step 3: Initial Conditions, Parameters, and Solving

Prior to solving the model, initial conditions (i.e., initial numbers of S, I, R individuals) and parameters (beta and gamma) need specification. In our scenario, we consider a population of 1000, with a single initial infection. These parameters are flexible and should reflect the specific disease dynamics being modeled, as well as your population of interest. The differential equations are solved using scipy.integrate.odeint, yielding the system's evolution over time.

Step 4: Visualizing the Outcome with Plotly

The culmination of our efforts is visualizing the disease's progression. By leveraging plotly, we transform our model's output into an interactive graph, offering a clear and engaging depiction of the epidemic's dynamics. Creating a plot in

plotly involves adding traces for each population compartment (S, I, R), each with its distinct color and style, facilitating an intuitive grasp of the model's implications.

It should be noted that in Python, graphs often require initialization (hence the empty go.Figure()  call). Otherwise, a lot of this code is actually very similar to how R handles graphing if you are familiar with that. Below is our interactive output:

Conclusion

Through this tutorial, we've navigated the implementation and visualization of the SIR model using Python, showcasing the model's utility in understanding infectious disease dynamics. While the SIR model provides a foundational perspective, it simplifies real-world complexities such as population heterogeneity and spatial dynamics. Future tutorials will explore more nuanced models like SEIR and network-based models, offering a deeper dive into epidemiological modeling.

Build an SIR model in R

The Susceptible-Infected-Recovered (SIR) model is a fundamental concept in epidemiology, offering insights into how diseases spread and recede in populations over time through a relatively simply set of functions. For this ThuRsday Tutorial, we'll cover how to not only make a quick SIR model but also how to graph the results. If you'd like a decent background on what all goes into the SIR model conceptually and mathematically, please check out our Epi Explained article for Understanding SIR Models. Now, let's get to coding. As usual, if you'd like to play with the code on your own, you can download it through Cody’s Github.

Step 1: Installing and Loading Necessary Packages

Before diving into the modeling process, you must ensure that the required R packages are installed and loaded into your workspace. The two packages we'll be using are deSolve and plotly.

• deSolve is a package for solving differential equations in R, which we’ll use to solve the SIR model equations.
• plotly allows for creating interactive plots, which will enable us to visualize the dynamics of the SIR model effectively.

Step 2: Defining the SIR Model

The SIR model is made up of three compartments: Susceptible (S), Infected (I), and Recovered (R). Individuals transition from being susceptible to infected and then to recovered over time, modeled through a set of differential equations. Here's how to define these equations in R:

Now on first look this might be a bit strange even to experienced R users as the function seems to do very little with what we're inputting, but in reality this is because we're actually going to use this function in yet another function in Step 3. For now though, let's do a quick run-through of what we're just set up. The function takes three arguments:

• time: The current time point. While not directly used in the differential equations, it is required by the ode function from the deSolve package.
• variables: A vector containing the current numbers of susceptible, infected, and recovered individuals (S, I, R).
• parameters: A vector of parameters for the model, specifically beta, gamma, and N (the total population).

Within the function, we use the with function to combine

variables and parameters into a single list, which simplifies accessing these values. The with function allows us to refer to elements of the list directly by their names (e.g.,S,

I,R,

beta,gamma,

N) within its code block. The rates of change for each compartment are then calculated according to the SIR model equations:

• dS: The rate of change of susceptible individuals is the negative product of the infection rate (beta), the number of susceptible individuals (S), and the number of infected individuals (I), divided by the total population (N). This represents the number of new infections per time unit.
• dI: The rate of change of infected individuals is the number of new infections (as calculated for dS) minus the number of recoveries, which is the product of the recovery rate (gamma) and the number of infected individuals (I).
• dR: The rate of change of recovered individuals is simply the number of recoveries, as calculated in the dI equation.

Finally, the function returns a list containing the rates of change for each compartment, which is then used by the ode function to solve the model over time.  

Step 3: Setting Initial Conditions and Parameters, Then Solving

Before solving the model, we need to specify the initial conditions (i.e., the starting numbers of susceptible, infected, and recovered individuals) and the parameters (beta and gamma) which control the disease transmission and recovery rates, respectively. Here, we start with a population of 1000 individuals, with only one infected person as the outbreak begins. The beta and gamma values are hypothetical and can be adjusted based on the disease's characteristics you're modeling. With the function we already made and initial setup ready, we can now solve the differential equations using deSolve's

ode function, which will return the solutions to every step from day 0 to day 50.

Step 4: Visualizing the Results with Plotly

Finally, to understand the dynamics of the disease through time, we'll visualize the results using plotly. This step transforms the output into a dynamic and interactive plot that can is much easier to understand, and can often wow your desired audience.

This is a fairly simply plotly graph, but it's worth noting that for each class, we need to create a separate trace for each line, which allows them to act independently of one another. The output can be seen here:

Conclusion

Through this tutorial, you've learned how to implement and visualize the SIR model in R using deSolve and plotly. This model serves as a basic framework for understanding disease dynamics in populations, providing a foundation for more complex epidemiological modeling. It's worth noting again that the SIR model is highly simplified; It assumes no population movement, random interactions, and other factors. Later on we'll cover things like SEIR and MEM models that are a bit more complex. But for now, thanks for reading this ThuRsday Tutorial!