% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % Author: Phillip Rothenbeck % Title: Investigating the Evolution of the COVID-19 Pandemic in Germany Using Physics-Informed Neural Networks % File: chap04/chap04.tex % Part: Experiments % Description: % summary of the content in this chapter % Version: 01.01.2012 % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \chapter{Experiments 10} \label{chap:evaluation} In the preceding chapters, we explained the methods (see~\Cref{chap:methods}) based the theoretical background, that we established in~\Cref{chap:background}. In this chapter present the setups and results from the experiments and simulations, we ran. First, we discuss the experiments dedicated to identify the epidemiological parameters of $\beta$ and $\alpha$ in synthetic and real-world data. Second, we examine the reproduction number in synthetic and real-world data of Germany. Each section, is divided into a description of the experimental setup and the results. % ------------------------------------------------------------------- \section{Identifying the Transition Rates on Real-World and Synthetic Data 5} \label{sec:sir} In this section, we aim to identify the transmission rate $\beta$ and the recovery rate $\alpha$ from either synthetic or preprocessed real-world data. The methodology that we employ to identify the transition rates is described in~\Cref{sec:pinn:sir}. Meanwhile, the methods we utilize to preprocess the real-world data are detailed in~\Cref{sec:preprocessing:rq}. % ------------------------------------------------------------------- \subsection{Setup 1} \label{sec:sir:setup} In this subsection, we present the configurations for the training of our PINNs, which are designed to identify the transition parameters. This encompasses the specific parameters for the preprocessing and the configuration of the PINN themselves.\\ In order to validate our method, we first generate a dataset of synthetic data. We achieve this by solving~\Cref{eq:modSIR} for a given set of parameters. The parameters are set to $\alpha = \nicefrac{1}{3}$ and $\beta = \nicefrac{1}{2}$. The size of the population is $N = \expnumber{7.6}{6}$ and the initial amount of infectious individuals of is $I_0 = 10$. We conduct the simulation over 150 days, resulting in a dataset of the form of~\Cref{fig:synthetic_SIR}.\\ In order to process the real-world RKI data, it is necessary to preprocess the raw data for each state and Germany separately. This is achieved by utilizing a recovery queue with a recovery period of 14 days. With regard to population size of each state, we set it to the respective value counted at the end of 2019\footnote{\url{https://de.statista.com/statistik/kategorien/kategorie/8/themen/63/branche/demographie/\#overview}}. The initial number of infectious individuals is set to the number of infected people on March 09. 2020 from the dataset. The data we extract spans from March 09. 2020 to June 22. 2023, encompassing a period of 1200 days and representing the time span during which the COVID-19 disease was the most active and severe. \begin{figure}[h] %\centering \setlength{\unitlength}{1cm} % Set the unit length for coordinates \begin{picture}(12, 9.5) % Specify the size of the picture environment (width, height) \put(1.5, 4.5){ \begin{subfigure}{0.3\textwidth} \centering \includegraphics[width=\textwidth]{SIR_synth.pdf} \label{fig:synthetic_SIR} \end{subfigure} } \put(8, 4.5){ \begin{subfigure}{0.3\textwidth} \centering \includegraphics[width=\textwidth]{datasets_states/Germany_SIR_14.pdf} \label{fig:germany_sir} \end{subfigure} } \put(0, 0){ \begin{subfigure}{0.3\textwidth} \centering \includegraphics[width=\textwidth]{datasets_states/Schleswig_Holstein_SIR_14.pdf} \label{fig:schleswig_holstein_sir} \end{subfigure} } \put(4.75, 0){ \begin{subfigure}{0.3\textwidth} \centering \includegraphics[width=\textwidth]{datasets_states/Berlin_SIR_14.pdf} \label{fig:berlin_sir} \end{subfigure} } \put(9.5, 0){ \begin{subfigure}{0.3\textwidth} \centering \includegraphics[width=\textwidth]{datasets_states/Thueringen_SIR_14.pdf} \label{fig:thüringen_sir} \end{subfigure} } \end{picture} \caption{Synthetic and real-world training data. The synthetic data is generated with $\alpha=\nicefrac{1}{3}$ and $\beta=\nicefrac{1}{2}$ and~\Cref{eq:modSIR}. The Germany data is taken from the death case data set. Exemplatory we show illustrations of the datasets of Schleswig Holstein, Berlin, and Thuringia. For the other states see~\Cref{chap:appendix} } \label{fig:datasets_sir} \end{figure} The PINN that we utilize comprises of seven hidden layers with twenty neurons each, and an activation function of ReLU. We employ the Adam optimizer and the polynomial scheduler of the PyTorch library, for training, with a base learning rate of $\expnumber{1}{-3}$. We train the model for 10000 epochs to extract the parameters. For each set of parameters, we conduct five iterations to demonstrate stability of the values. The configuration is similar to the configuration, that Shaier \etal ~\cite{Shaier2021} use for their work aside from the learning rate and the scheduler choice.\\ The following section presents the results of the simulations conducted with the setups that we describe in this section. % ------------------------------------------------------------------- \subsection{Results 4} \label{sec:sir:results} \begin{figure}[t] \centering \includegraphics[width=0.7\textwidth]{reproducability.pdf} \caption{Visualization of all 5 predictions for the synthetic dataset, compared to the true values of $\alpha = \nicefrac{1}{3}$ and $\beta = \nicefrac{1}{2}$} \label{fig:reprod} \end{figure} In this section, we present the results, that we obtain from the conducted experiments, that we describe in the preceding section. We begin by examining the results for the synthetic dataset, focusing the accuracy and reproducibility. We then proceed to present and discuss the results for the German states and Germany.\\ The results of the experiment regarding the synthetic data can be seen in~\Cref{table:alpha_beta_synth} and in~\Cref{fig:reprod}.~\Cref{fig:reprod} depicts the values of $\beta$ and $\alpha$ for each iteration in comparison to the true values of $\beta=\nicefrac{1}{2}$ and $\alpha=\nicefrac{1}{3}$. In~\Cref{table:alpha_beta_synth} we present the mean $\mu$ and standard deviation $\sigma$ of both values across all five iterations.\\ \begin{table}[h] \begin{center} \begin{tabular}{ccc|ccc} true $\alpha$ & $\mu(\alpha)$ & $\sigma(\alpha)$ & true $\beta$ & $\mu(\beta)$ & $\sigma(\beta)$ \\ \hline 0.3333 & 0.3334 & 0.0011 & 0.5000 & 0.5000 & 0.0017 \\ \end{tabular} \caption{The mean $\mu$ and standard deviation $\sigma$ across the 5 independent iterations of training our PINNs with the synthetic dataset.} \label{table:alpha_beta_synth} \end{center} \end{table} The results demonstrate that the model is capable of approximating the correct parameters for the small, synthetic dataset in each of the five iterations. While the predicted value is not precisely accurate, the standard deviation is sufficiently small, and taking the mean of multiple iterations produces an almost perfect result.\\ In~\Cref{table:alpha_beta} we present the results of the training for the real-world data. The results are presented from top to bottom, in the order of the community identification number, with the last entry being Germany. Both the mean $\mu$ and the standard deviation $\sigma$ are calculated across all five iterations of our experiment. We can observe that the values of \emph{Hamburg} have the highest standard deviation, while \emph{Mecklenburg Vorpommern} has the lowest $\sigma$.\\ \begin{table}[h] \begin{center} \begin{tabular}{c|cc|cc} & $\mu(\alpha)$ & $\sigma(\alpha)$ & $\mu(\beta)$ & $\sigma(\beta)$ \\ \hline Schleswig Holstein & 0.0771 & 0.0010 & 0.0966 & 0.0013 \\ Hamburg & 0.0847 & 0.0035 & 0.1077 & 0.0037 \\ Niedersachsen & 0.0735 & 0.0014 & 0.0962 & 0.0018 \\ Bremen & 0.0588 & 0.0018 & 0.0795 & 0.0025 \\ Nordrhein-Westfalen & 0.0780 & 0.0009 & 0.1001 & 0.0011 \\ Hessen & 0.0653 & 0.0016 & 0.0854 & 0.0020 \\ Rheinland-Pfalz & 0.0808 & 0.0016 & 0.1036 & 0.0018 \\ Baden-Württemberg & 0.0862 & 0.0014 & 0.1132 & 0.0016 \\ Bayern & 0.0809 & 0.0021 & 0.1106 & 0.0027 \\ Saarland & 0.0746 & 0.0021 & 0.0996 & 0.0024 \\ Berlin & 0.0901 & 0.0008 & 0.1125 & 0.0008 \\ Brandenburg & 0.0861 & 0.0008 & 0.1091 & 0.0010 \\ Mecklenburg Vorpommern & 0.0910 & 0.0007 & 0.1167 & 0.0008 \\ Sachsen & 0.0797 & 0.0017 & 0.1073 & 0.0022 \\ Sachsen-Anhalt & 0.0932 & 0.0019 & 0.1207 & 0.0027 \\ Thüringen & 0.0952 & 0.0011 & 0.1248 & 0.0016 \\ Germany & 0.0803 & 0.0012 & 0.1044 & 0.0014 \\ \end{tabular} \caption{Mean and standard deviation across the 5 iterations, that we conducted for each German state and Germany as the whole country.} \label{table:alpha_beta} \end{center} \end{table} \begin{figure}[t] \centering \includegraphics[width=\textwidth]{mean_std_alpha_beta_res.pdf} \label{fig:alpha_beta_mean_std} \end{figure} In~\Cref{fig:alpha_beta_mean_std}, we present a visual representation of the means and standard deviations in comparison to the national values. It is noteworthy that the states of Saxony-Anhalt and Thuringia have the highest transmission rates of all states, while Bremen and Hessen have the lowest values for $\beta$. The transmission rates of Hamburg, Baden Württemberg, Bavaria, and all eastern states lay above the national rate of transmission. Similarly, the recovery rate yields comparable outcomes. For the recovery rate, the same states that exhibit a transmission rate exceeding the national value, have a higher recovery rate than the national standard, with the exception of Saxony.It is noteworthy that the recovery rates of all states exhibit a tendency to align with the recovery rate of $\alpha=\nicefrac{1}{14}$, which is equivalent to a recovery period of 14 days.\\ It is evident that there is a correlation between the values of $\alpha$ and $\beta$ for each state. States with a high transmission rate tend to have a high recovery rate, and vice versa. The correlation between $\alpha$ and $\beta$ can be explained by the implicate definition of $\alpha$ using a recovery queue with a constant recovery period of 14 days. This might result to the PINN not learning $\alpha$ as a standalone parameter but rather as a function of the transmission rate $\beta$. This phenomenon occurs because the transmission rate determines the number of individuals that get infected per day, and the recovery queue moves a proportional number of people to the removed compartment. Consequently, a number of people defined by $\beta$ move to the $R$ compartment 14 days after they were infected.\\ This issue can be addressed by reducing the SIR model, thereby eliminating the significance of the $R$ compartment size. In the following section, we present our experiments for the reduced SIR model with time-independent parameters. % ------------------------------------------------------------------- \section{Reduced SIR Model 5} \label{sec:rsir} In this section we describe the experiments we conduct to identify the time-dependent reproduction number for both synthetic and real-world data. Similar to the previous section, we first describe the setup of our experiments and afterwards present the results. The methods we employ for the preprocessing are described in~\Cref{sec:preprocessing:rq} and for the PINN, that we use, are described in~\Cref{sec:pinn:rsir}. % ------------------------------------------------------------------- \subsection{Setup 1} \label{sec:rsir:setup} This section outlines the selection of parameters and configuration for data generation, preprocessing, and the neural networks. We employ these setups to train the PINNs to identify the reproduction number on both synthetic and real-world data.\\ For the purposes of validation, we create a synthetic dataset, by setting the parameter of $\alpha$ and the reproduction value each to a specific values, and solving~\Cref{eq:reduced_sir_ODE} for a given time interval. We set $\alpha=\nicefrac{1}{3}$ and $\Rt$ to the values as can be seen in~\Cref{fig:synthetic_I_r_t} as well as the population size $N=\expnumber{7.6}{6}$ and the initial amount of infected people to $I_0=10$. Furthermore, we set our simulated time span to 150 days. We use this dataset to demonstrate, that our method is working on a simple and minimal dataset.\\ To obtain a dataset of the infectious group, consisting of the real-world data, we we processed the data of the dataset \emph{COVID-19-Todesfälle in Deutschland} to extract the number of infections in Germany as a whole. For the German states, we use the data of \emph{SARS-CoV-2 Infektionen in Deutschland}. In the preprocessing stage, we employ a constant rate for $\alpha$ to move individuals into the removed compartment. For each state we generate two datasets with a different recovery rate. First, we choose $\alpha = \nicefrac{1}{14}$, which aligns with the time of recovery\footnote{\url{https://github.com/robert-koch-institut/SARS-CoV-2-Infektionen_in_Deutschland.git}}. Second, we use $\alpha=\nicefrac{1}{5}$, as 5 days into the infection is the point at which the infectiousness is at its peak\footnote{\url{https://www.infektionsschutz.de/coronavirus/fragen-und-antworten/ansteckung-uebertragung-und-krankheitsverlauf/}}. As in~\Cref{sec:sir}, we set the population size $N$ of each state and Germany to the corresponding size at the end of 2019. Furthermore, for the same reason we restrict the data points to an interval of 1200 days, beginning on March 09. 2020.\\ \begin{figure}[t] \centering \begin{subfigure}{0.3\textwidth} \centering \includegraphics[width=\textwidth]{I_synth.pdf} \caption{Synthetic data} \label{fig:synthetic_I} \end{subfigure} \quad \begin{subfigure}{0.3\textwidth} \centering \includegraphics[width=\textwidth]{I_synth_r_t.pdf} \caption{Synthetic data} \label{fig:synthetic_I_r_t} \end{subfigure} \vskip\baselineskip \begin{subfigure}{0.67\textwidth} \centering \includegraphics[width=\textwidth]{datasets_states/Germany_datasets.pdf} \caption{} \label{fig:germany_I_14} \end{subfigure} \end{figure} In order to achieve the desired output, the selected neural network architecture comprises of four hidden layers, each containing 100 neurons. The activation function is the tangens hyperbolicus function. For the real-world data, we weight the data loss by a factor of $\expnumber{1}{6}$, to the total loss. The model is trained using a base learning rate of $\expnumber{1}{-3}$, with the same scheduler and optimizer as we describe in~\Cref{sec:sir:setup}. We train the model for 20000 epochs. To reduce the standard deviation, each experiment is conducted 15 times.\\ % ------------------------------------------------------------------- \subsection{Results 4} \label{sec:rsir:results} In this section we provide the results for our experiments. First, we present our findings for the synthetic dataset. Then, we provide and discuss the results for the real-world data.\\ \begin{figure} \centering \includegraphics[width=\textwidth]{synthetic_R_t_statistics.pdf} \caption{text} \label{fig:synth_r_t_results} \end{figure} % -------------------------------------------------------------------