Thesis/chapters/chap04/chap04.tex

% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Author:   Phillip Rothenbeck
% Title:    Investigating the Evolution of the COVID-19 Pandemic in Germany Using Physics-Informed Neural Networks
% File:     chap04/chap04.tex
% Part:     Experiments
% Description:
%         summary of the content in this chapter
% Version:  01.01.2012
% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\chapter{Experiments   10}
\label{chap:evaluation}
In the preceding chapters, we explained the methods (see~\Cref{chap:methods})
based the theoretical background, that we established in~\Cref{chap:background}.
In this chapter present the setups and results from the experiments and
simulations, we ran. First, we discuss the experiments dedicated to identify
the epidemiological parameters of $\beta$ and $\alpha$ in synthetic and
real-world data. Second, we examine the reproduction number in synthetic and
real-world data of Germany. Each section, is divided into a description of the
experimental setup and the results.

% -------------------------------------------------------------------

\section{Identifying the Transition Rates on Real-World and Synthetic Data  5}
\label{sec:sir}
In this section, we aim to identify the transmission rate $\beta$ and the
recovery rate $\alpha$ from either synthetic or preprocessed real-world data.
The methodology that we employ to identify the transition rates is described
in~\Cref{sec:pinn:sir}. Meanwhile, the methods we utilize to preprocess the
real-world data are detailed in~\Cref{sec:preprocessing:rq}.

% -------------------------------------------------------------------

\subsection{Setup   1}
\label{sec:sir:setup}

In this subsection, we present the configurations for the training of our
PINNs, which are designed to identify the transition parameters. This
encompasses the specific parameters for the preprocessing and the configuration
of the PINN themselves.\\

In order to validate our method, we first generate a dataset of synthetic data.
We achieve this by solving~\Cref{eq:modSIR} for a given set of parameters.
The parameters are set to $\alpha = \nicefrac{1}{3}$ and $\beta = \nicefrac{1}{2}$.
The size of the population is $N = \expnumber{7.6}{6}$ and the initial amount of
infectious individuals of is $I_0 = 10$. We conduct the simulation over 150
days, resulting in a dataset of the form of~\Cref{fig:synthetic_SIR}.\\ In
order to process the real-world RKI data, it is necessary to preprocess the raw
data for each state and Germany separately. This is achieved by utilizing a
recovery queue with a recovery period of 14 days. With regard to population
size of each state, we set it to the respective value counted at the end of
2019\footnote{\url{https://de.statista.com/statistik/kategorien/kategorie/8/themen/63/branche/demographie/\#overview}}.
The initial number of infectious individuals is set to the number of infected
people on March 09. 2020 from the dataset. The data we extract spans from
March 09. 2020 to June 22. 2023, encompassing a period of 1200 days and
representing the time span during which the COVID-19 disease was the most
active and severe.

\begin{figure}[h]
    %\centering
    \setlength{\unitlength}{1cm} % Set the unit length for coordinates
    \begin{picture}(12, 9.5) % Specify the size of the picture environment (width, height)
        \put(1.5, 4.5){
            \begin{subfigure}{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{SIR_synth.pdf}
                \label{fig:synthetic_SIR}
            \end{subfigure}
        }
        \put(8, 4.5){
            \begin{subfigure}{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{datasets_states/Germany_SIR_14.pdf}
                \label{fig:germany_sir}
            \end{subfigure}
        }
        \put(0, 0){
            \begin{subfigure}{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{datasets_states/Schleswig_Holstein_SIR_14.pdf}
                \label{fig:schleswig_holstein_sir}
            \end{subfigure}
        }
        \put(4.75, 0){
            \begin{subfigure}{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{datasets_states/Berlin_SIR_14.pdf}
                \label{fig:berlin_sir}
            \end{subfigure}
        }
        \put(9.5, 0){
            \begin{subfigure}{0.3\textwidth}
                \centering
                \includegraphics[width=\textwidth]{datasets_states/Thueringen_SIR_14.pdf}
                \label{fig:thüringen_sir}
            \end{subfigure}
        }

    \end{picture}
    \caption{Synthetic and real-world training data. The synthetic data is
        generated with $\alpha=\nicefrac{1}{3}$ and $\beta=\nicefrac{1}{2}$
        and~\Cref{eq:modSIR}. The Germany data is taken from the death case
        data set. Exemplatory we show illustrations of the datasets of Schleswig
        Holstein, Berlin, and Thuringia. For the other states see~\Cref{chap:appendix} }
    \label{fig:datasets_sir}
\end{figure}

The PINN that we utilize comprises of seven hidden layers with twenty neurons
each, and an activation function of ReLU. We employ the Adam optimizer and the
polynomial scheduler of the PyTorch library, for training, with a base learning rate
of $\expnumber{1}{-3}$. We train the model for 10000 epochs to extract the
parameters. For each set of parameters, we conduct five iterations to
demonstrate stability of the values. The configuration is similar to the
configuration, that Shaier \etal ~\cite{Shaier2021} use for their work aside
from the learning rate and the scheduler choice.\\

The following section presents the results of the simulations conducted with the
setups that we describe in this section.

% -------------------------------------------------------------------

\subsection{Results    4}
\label{sec:sir:results}

\begin{figure}[t]
    \centering
    \includegraphics[width=0.7\textwidth]{reproducability.pdf}
    \caption{Visualization of all 5 predictions for the synthetic dataset,
        compared to the true values of $\alpha = \nicefrac{1}{3}$ and $\beta = \nicefrac{1}{2}$}
    \label{fig:reprod}
\end{figure}

In this section, we present the results, that we obtain from the conducted
experiments, that we describe in the preceding section. We begin by examining
the results for the synthetic dataset, focusing the accuracy and
reproducibility. We then proceed to present and discuss the results for the
German states and Germany.\\

The results of the experiment regarding the synthetic data can be seen
in~\Cref{table:alpha_beta_synth} and in~\Cref{fig:reprod}.~\Cref{fig:reprod}
depicts the values of $\beta$ and $\alpha$ for each iteration in comparison to the true
values of $\beta=\nicefrac{1}{2}$ and $\alpha=\nicefrac{1}{3}$. In~\Cref{table:alpha_beta_synth}
we present the mean $\mu$ and standard deviation $\sigma$ of both values across
all five iterations.\\

\begin{table}[h]
    \begin{center}
        \begin{tabular}{ccc|ccc}
            true $\alpha$ & $\mu(\alpha)$ & $\sigma(\alpha)$ & true $\beta$ & $\mu(\beta)$ & $\sigma(\beta)$ \\
            \hline
            0.3333        & 0.3334        & 0.0011           & 0.5000       & 0.5000       & 0.0017          \\
        \end{tabular}
        \caption{The mean $\mu$ and standard deviation $\sigma$ across the 5
            independent iterations of training our PINNs with the synthetic dataset.}
        \label{table:alpha_beta_synth}
    \end{center}
\end{table}

The results demonstrate that the model is capable of approximating the correct
parameters for the small, synthetic dataset in each of the five iterations.
While the predicted value is not precisely accurate, the standard deviation is
sufficiently small, and taking the mean of multiple iterations produces an
almost perfect result.\\

In~\Cref{table:alpha_beta} we present the results of the training for the
real-world data. The results are presented from top to bottom, in the order of
the community identification number, with the last entry being Germany. Both
the mean $\mu$ and the standard deviation $\sigma$ are calculated across all
five iterations of our experiment. We can observe that the values of
\emph{Hamburg} have the highest standard deviation, while \emph{Mecklenburg Vorpommern}
has the lowest $\sigma$.\\

\begin{table}[h]
    \begin{center}
        \begin{tabular}{c|cc|cc}
                                   & $\mu(\alpha)$ & $\sigma(\alpha)$ & $\mu(\beta)$ & $\sigma(\beta)$ \\
            \hline
            Schleswig Holstein     & 0.0771        & 0.0010           & 0.0966       & 0.0013          \\
            Hamburg                & 0.0847        & 0.0035           & 0.1077       & 0.0037          \\
            Niedersachsen          & 0.0735        & 0.0014           & 0.0962       & 0.0018          \\
            Bremen                 & 0.0588        & 0.0018           & 0.0795       & 0.0025          \\
            Nordrhein-Westfalen    & 0.0780        & 0.0009           & 0.1001       & 0.0011          \\
            Hessen                 & 0.0653        & 0.0016           & 0.0854       & 0.0020          \\
            Rheinland-Pfalz        & 0.0808        & 0.0016           & 0.1036       & 0.0018          \\
            Baden-Württemberg      & 0.0862        & 0.0014           & 0.1132       & 0.0016          \\
            Bayern                 & 0.0809        & 0.0021           & 0.1106       & 0.0027          \\
            Saarland               & 0.0746        & 0.0021           & 0.0996       & 0.0024          \\
            Berlin                 & 0.0901        & 0.0008           & 0.1125       & 0.0008          \\
            Brandenburg            & 0.0861        & 0.0008           & 0.1091       & 0.0010          \\
            Mecklenburg Vorpommern & 0.0910        & 0.0007           & 0.1167       & 0.0008          \\
            Sachsen                & 0.0797        & 0.0017           & 0.1073       & 0.0022          \\
            Sachsen-Anhalt         & 0.0932        & 0.0019           & 0.1207       & 0.0027          \\
            Thüringen              & 0.0952        & 0.0011           & 0.1248       & 0.0016          \\
            Germany                & 0.0803        & 0.0012           & 0.1044       & 0.0014          \\
        \end{tabular}
        \caption{Mean and standard deviation across the 5 iterations, that we
            conducted for each German state and Germany as the whole country.}
        \label{table:alpha_beta}
    \end{center}
\end{table}

\begin{figure}[t]
    \centering
    \includegraphics[width=\textwidth]{mean_std_alpha_beta_res.pdf}
    \label{fig:alpha_beta_mean_std}
\end{figure}

In~\Cref{fig:alpha_beta_mean_std}, we present a visual representation of the
means and standard deviations in comparison to the national values. It is
noteworthy that the states of Saxony-Anhalt and Thuringia have the highest
transmission rates of all states, while Bremen and Hessen have the lowest
values for $\beta$. The transmission rates of Hamburg, Baden Württemberg,
Bavaria, and all eastern states lay above the national rate of transmission.
Similarly, the recovery rate yields comparable outcomes. For the recovery rate,
the same states that exhibit a transmission rate exceeding the national value,
have a higher recovery rate than the national standard, with the exception of
Saxony.It is noteworthy that the recovery rates of all states exhibit a
tendency to align with the recovery rate of $\alpha=\nicefrac{1}{14}$, which is
equivalent to a recovery period of 14 days.\\

It is evident that there is a correlation between the values of $\alpha$ and
$\beta$ for each state. States with a high transmission rate tend to have a
high recovery rate, and vice versa. The correlation between $\alpha$ and
$\beta$ can be explained by the implicate definition of $\alpha$ using a
recovery queue with a constant recovery period of 14 days. This might result to
the PINN not learning $\alpha$ as a standalone parameter but rather as a
function of the transmission rate $\beta$. This phenomenon occurs because the
transmission rate determines the number of individuals that get infected per
day, and the recovery queue moves a proportional number of people to the
removed compartment. Consequently, a number of people defined by $\beta$ move
to the $R$ compartment 14 days after they were infected.\\

This issue can be addressed by reducing the SIR model, thereby eliminating the
significance of the $R$ compartment size. In the following section, we present
our experiments for the reduced SIR model with time-independent parameters.

% -------------------------------------------------------------------

\section{Reduced SIR Model   5}
\label{sec:rsir}
In this section we describe the experiments we conduct to identify the
time-dependent reproduction number for both synthetic and real-world data.
Similar to the previous section, we first describe the setup of our experiments
and afterwards present the results. The methods we employ for the preprocessing
are described in~\Cref{sec:preprocessing:rq} and for the PINN, that we use,
are described in~\Cref{sec:pinn:rsir}.

% -------------------------------------------------------------------

\subsection{Setup    1}
\label{sec:rsir:setup}
This section outlines the selection of parameters and configuration for data
generation, preprocessing, and the neural networks. We employ these setups to
train the PINNs to identify the reproduction number on both synthetic and
real-world data.\\

For the purposes of validation, we create a synthetic dataset, by setting the parameter
of $\alpha$ and the reproduction value each to a specific values, and solving~\Cref{eq:reduced_sir_ODE}
for a given time interval. We set $\alpha=\nicefrac{1}{3}$ and $\Rt$ to the
values as can be seen in~\Cref{fig:synthetic_I_r_t} as well as the population
size $N=\expnumber{7.6}{6}$ and the initial amount of infected people to
$I_0=10$. Furthermore, we set our simulated time span to 150 days. We use this
dataset to demonstrate, that our method is working on a simple and minimal
dataset.\\ To obtain a dataset of the infectious group, consisting of the
real-world data, we we processed the data of the dataset
\emph{COVID-19-Todesfälle in Deutschland} to extract the number of infections
in Germany as a whole. For the German states, we use the data of \emph{SARS-CoV-2
    Infektionen in Deutschland}. In the preprocessing stage, we employ a constant
rate for $\alpha$ to move individuals into the removed compartment. For each
state we generate two datasets with a different recovery rate. First, we choose
$\alpha = \nicefrac{1}{14}$, which aligns with the time of recovery\footnote{\url{https://github.com/robert-koch-institut/SARS-CoV-2-Infektionen_in_Deutschland.git}}.
Second, we use $\alpha=\nicefrac{1}{5}$, as 5 days into the infection is the
point at which the infectiousness is at its peak\footnote{\url{https://www.infektionsschutz.de/coronavirus/fragen-und-antworten/ansteckung-uebertragung-und-krankheitsverlauf/}}.
As in~\Cref{sec:sir}, we set the population size $N$ of each state and Germany
to the corresponding size at the end of 2019. Furthermore, for the same reason
we restrict the data points to an interval of 1200 days, beginning on March 09.
2020.\\

\begin{figure}[t]
    \centering
    \begin{subfigure}{0.3\textwidth}
        \centering
        \includegraphics[width=\textwidth]{I_synth.pdf}
        \caption{Synthetic data}
        \label{fig:synthetic_I}
    \end{subfigure}
    \quad
    \begin{subfigure}{0.3\textwidth}
        \centering
        \includegraphics[width=\textwidth]{I_synth_r_t.pdf}
        \caption{Synthetic data}
        \label{fig:synthetic_I_r_t}
    \end{subfigure}
    \vskip\baselineskip
    \begin{subfigure}{0.67\textwidth}
        \centering
        \includegraphics[width=\textwidth]{datasets_states/Germany_datasets.pdf}
        \caption{}
        \label{fig:germany_I_14}
    \end{subfigure}

\end{figure}

In order to achieve the desired output, the selected neural network
architecture comprises of four hidden layers, each containing 100 neurons. The
activation function is the tangens hyperbolicus function. For the real-world
data, we weight the data loss by a factor of $\expnumber{1}{6}$, to the total
loss. The model is trained using a base learning rate of $\expnumber{1}{-3}$,
with the same scheduler and optimizer as we describe in~\Cref{sec:sir:setup}.
We train the model for 20000 epochs. To reduce the standard deviation, each
experiment is conducted 15 times.\\

% -------------------------------------------------------------------

\subsection{Results   4}
\label{sec:rsir:results}

In this section we provide the results for our experiments. First, we present
our findings for the synthetic dataset. Then, we provide and discuss the
results for the real-world data.\\

\begin{figure}
    \centering
    \includegraphics[width=\textwidth]{synthetic_R_t_statistics.pdf}
    \caption{text}
    \label{fig:synth_r_t_results}
\end{figure}


% -------------------------------------------------------------------