added graphs and results, conclusion

fault_injection_async25.tex

@@ -14,6 +14,8 @@
 \usepackage{subcaption}
 \usepackage{pgfplots}
 \usepackage{pgfplotstable}
+\pgfplotsset{compat=1.18}
+\usepackage{tabularx}
 % Tikz because graphs are fun
 \usepackage{tikz}
 \usepackage{tikz-timing}
@@ -22,7 +24,6 @@
 \usetikzlibrary{positioning}
 
 \makeglossaries
-\renewcommand{\glstreepredesc}{\hspace{3em}}
 
 % Acronyms for the document
 \newacronym{dut}{DUT}{Design Under Test}
@@ -35,11 +36,22 @@
 \newacronym{prs}{PRS}{Production Rule Set}
 \newacronym{uvm}{UVM}{Universal Verification Method}
 \newacronym{eda}{EDA}{Electronic Design Automation}
+\newacronym{dims}{DIMS}{Delay-Insensitive Minterm Synthesis}
+\newacronym{ncl}{NCL}{Null Convention Logic}
+\newacronym{nclx}{NCLX}{\acs{ncl} Flow with Explicit Completeness}
 
 % Simple citation required command
 \newcommand{\citationneeded}{\textcolor{red}{[citation needed]}}
 \newcommand{\referenceneeded}{\textcolor{red}{[reference needed]}}
 
+\makeatletter
+\newcommand{\linebreakand}{%
+  \end{@IEEEauthorhalign}
+  \hfill\mbox{}\par
+  \mbox{}\hfill\begin{@IEEEauthorhalign}
+}
+\makeatother
+
 \def\BibTeX{{\rm B\kern-.05em{\sc i\kern-.025em b}\kern-.08em
     T\kern-.1667em\lower.7ex\hbox{E}\kern-.125emX}}
 \begin{document}
@@ -65,7 +77,7 @@ fhuemer@ecs.tuwien.ac.at}
 TU Wien\\
 Vienna, Austria \\
 steininger@ecs.tuwien.ac.at}
-\and
+\linebreakand
 \IEEEauthorblockN{Rajit Manohar}
 \IEEEauthorblockA{Computer Systems Lab\\
 Yale University\\
@@ -242,20 +254,29 @@ where $P_{hit}$ additionally describes the probability of an injection hitting a
     \label{fig:setup/testbench}
 \end{figure}
 
-To test our new tool, we ran our setup against previous results by Behal et.al \cite{behalExplainingFaultSensitivity2021}. We simulated the same multiplier circuit, using the same buffer styles as well. We however did not sweep over any form of pipeline load factor. We found the definition of the metric ambiguous, especially once the circuit contains non-linear pipelines. For this reason we have opted to exclude this metric from further testing.
+To test our new tool, we simulated the same multiplier circuit as Behal et.al \cite{behalExplainingFaultSensitivity2021}. We however did not sweep over any form of pipeline load factor. We found the definition of the metric ambiguous, especially once the circuit contains non-linear pipelines. For this reason we have opted to exclude this metric from further testing. For consistency, the multiplier was generated using \texttt{pypr} \cite{huemerContributionsEfficiencyRobustness2022}, which is able to generate \acs{prs} rules in an ACT container. We elected to simulate four different versions, a four bit multiplier with unit delays (10 time steps) in \acs{dims}, a four bit multiplier with randomized delays ($\pm 5\%$, \acs{prs} node delay between 95 and 105 time steps) in \acs{dims}, an 8 bit multiplier with unit delays (10 time steps) in \acs{dims}, and a 4 bit multiplier with unit delays (10 time steps) in \acs{nclx}.
 
 The \ac{dut} is wrapped in a \acs{uvm}-like testbench setup, which is provided by our new simulation library. Our future ambition is to enable a write once - use everywhere architecture, where wrapper code has to be written once and can then be reused arbitrarily for all tests and verification procedures. The overall architecture of the test setup can be seen in Figure \ref{fig:setup/testbench}. Since, like \acs{uvm}, asynchronous logic contrary to synchronous logic inherently uses a message passing abstraction, we do not require much additional logic in the way of sequencers or monitors to interface with the \ac{dut}. Input tokens are directly forwarded to the \ac{dut}, model, and scoreboard.
 
-Points to talk about
-
-\begin{itemize}
-    \item how much more detail should i mention about the target circuit and families? we're starting to run low on space
-    \item can I have the tikz graphics that were in the Behal paper for deadlocking, interlocking\dots
-\end{itemize}
-
-
-\section{Results}
-\label{sec:results}
+\begin{table}[ht]
+    \centering
+    \normalsize
+    \begin{tabularx}{0.4\textwidth}{|X|r|}
+        \hline
+        \textbf{Parameter} & \textbf{Default setting}\\
+        \hline
+        Hit probability & $0.8$ \\
+        \hline
+        Modes per fork & $1$ \\
+        \hline
+        Coverage certainty & $0.2$ \\
+        \hline
+        Victim coverage & $0.5$ \\
+        \hline
+    \end{tabularx}
+    \caption{Default generation engine configuration}
+    \label{tab:setup/config}
+\end{table}
 
 \begin{figure*}[htbp]
     \centering
@@ -282,16 +303,67 @@ Points to talk about
         \caption{varying the number of selected signals}
         \label{fig:res/deviation_sel_signals_dims}
     \end{subfigure}
-    \caption{Variation of failure type rates in percentage points when}
+    \caption{Variation of failure type rates in \acs{dims} in percentage points when}
     \label{fig:res/deviation_dims}
 \end{figure*}
 
 
+\begin{figure*}[htbp]
+    \centering
+    \begin{subfigure}{0.4\textwidth}
+        \begin{center}
+            \input{results/deviation_num_injections_nclx.tex}
+        \end{center}
+        \caption{varying the number of injections}
+        \label{fig:res/deviation_num_sims_nclx}
+    \end{subfigure}
+    %\hfill
+    \begin{subfigure}{0.4\textwidth}
+        \begin{center}
+            \input{results/deviation_num_signals_nclx.tex}
+        \end{center}
+        \caption{varying the number of selected signals}
+        \label{fig:res/deviation_sel_signals_nclx}
+    \end{subfigure}
+    \caption{Variation of failure type rates in \acs{nclx} in percentage points when}
+    \label{fig:res/deviation_nclx}
+\end{figure*}
+
+To determine the performance of both our simulation environment, as well as our fault-injection engine, we selected a configuration that would yield a low amount of simulations by default (see Table~\ref{tab:setup/config}) and vary configuration parameters individually from there. 
+
+\section{Results}
+\label{sec:results}
+
+\begin{figure}
+    \centering
+    \scalebox{0.85}{\input{results/sim_scaling.tex}}
+    \caption{Simulation scaling over configuration parameters}
+    \label{fig:res/sim_scaling}
+\end{figure}
+
+In our testing, this setup has shown itself quite capable as a cluster simulation tool. When running a batch of 13317 simulations, we measured a total execution time of 1 minute and 32 seconds, when executing on 4 nodes of 4 jobs each. This equates to almost exactly 9 simulations per second per core, which is in large part due to \texttt{actsim}'s high performance.
+
+Failure distributions for the examined circuits can be seen in Figure~\ref{fig:results/aggregated}. The simulation configuration shown in this graph was set to a lower assumed hit-probability ($0.1$, otherwise identical to Table~\ref{tab:setup/config}), to increase the number of simulations in hopes of establishing an accurate baseline. From there, varying hit-probability, assumed failure modes per fanout, and coverage certainty ultimately equates to a difference in injections per selected signal.
+
+Figure~\ref{fig:res/deviation_num_sims_dims} and Figure~\ref{fig:res/deviation_num_sims_nclx} show how observed failure mode distribution changes when the number of injections per signal is decreased. We observe that for both logic families, deviation is less than a single percentage point when going from over 20000 simulations down to about 3000.
+
+As a similar exercise, we established another baseline test for varying the percentage of signals to be targeted. For this, we configured the injection-engine to select all signals, then gradually lowered the percentage of selected signals (see Figure~\ref{fig:res/deviation_sel_signals_dims} and \ref{fig:res/deviation_sel_signals_nclx}). While not as stable as number of simulations, deviation still stayed within limits down to about 50\%, with the number of glitches observed in \acs{nclx} deviating by about $2.9$ points.
+
+An interesting side note here is that randomness definitely plays a factor here, as a signal coverage of $70\%$ has less deviation from reference for \acs{dims} than $90\%$. Further study would be needed to determine the variance for these values.
+
+Finally, Figure~\ref{fig:res/sim_scaling} shows how the configuration parameters for the injection engine influence the number of calculated required injections. Coverage certainty, hit probability, and victim coverage are in a value range of 0 to 1, while expected failure modes per fanout is in range 1 to 3.\\
+We see that hit probability, victim coverage, as well as expected failure modes change the number of simulations as we expected. While we expected victim coverage to grow the number of injections linearly, this was under the assumptions of all signals having identical fanout. However, since high fanout signals are selected first, we see strong initial growth, which then plateaus out as only low fanout signals are left to be selected.
 
 \section{Conclusion}
 \label{sec:conclusion}
 
+When developing or verifying hardware, certain tasks require more resources than one local node can provide. To address this, we have presented \texttt{action}, an extendible cluster computation tool meant to augment the ACT toolchain. In addition we have provided extensions to \texttt{actsim}, which enable \acs{set} injections into a circuit, as well as a simulation library, which enables \acs{uvm}-like testing setups. This customization on a source code level in conjunction with \texttt{actsim}'s major improvements over previous simulators, like mixed-fidelity simulation, have provided exceptional simulation throughput of 9 simulations per core per second.
+
+We have then used \texttt{action} in conjunction with \texttt{actsim} to develop a novel fault-injection engine, which we also presented in this paper. Through this engine, we were able to show low variance in failure mode distribution for our \acs{dims} and \acs{nclx} implementation of a 4 bit multiplier when reducing the number of injections. We were also able to show useful performance scaling characteristics when increasing circuit size.
+
+\renewcommand{\IEEEiedlistdecl}{\IEEEsetlabelwidth{SONET}}
 \printacronyms
+\renewcommand{\IEEEiedlistdecl}{\relax}
 
 \printbibliography
 

results/aggregated.tex

@@ -23,13 +23,13 @@
         nodes near coords, % Option to show values near the bars
     ]
     
-        \addplot table[x=Circuit, y=Timing deviation] {\aggregateddata};
-        \addplot table[x=Circuit, y=Value failure] {\aggregateddata};
-        \addplot table[x=Circuit, y=Coding failure] {\aggregateddata};
-        \addplot table[x=Circuit, y=Glitch] {\aggregateddata};
-        \addplot table[x=Circuit, y=Deadlock] {\aggregateddata};
-        \addplot table[x=Circuit, y=Token count failure] {\aggregateddata};
-        \addplot table[x=Circuit, y=Nothing] {\aggregateddata};
+        \addplot[fill=blue] table[x=Circuit, y=Timing deviation] {\aggregateddata};
+        \addplot[fill=red] table[x=Circuit, y=Value failure] {\aggregateddata};
+        \addplot[fill=brown] table[x=Circuit, y=Coding failure] {\aggregateddata};
+        \addplot[fill=gray] table[x=Circuit, y=Glitch] {\aggregateddata};
+        \addplot[fill=purple] table[x=Circuit, y=Deadlock] {\aggregateddata};
+        \addplot[fill=green] table[x=Circuit, y=Token count failure] {\aggregateddata};
+        \addplot[fill=cyan] table[x=Circuit, y=Nothing] {\aggregateddata};
     
         \legend{Timing deviation, Value failure, Coding failure, Glitch, Deadlock, Token count failure, Nothing}
     

results/deviation_num_injections_dims.csv

@@ -1,9 +1,9 @@
-Total simulations,Timing deviation,Value failure,Coding failure,Glitch,Deadlock,Token count failure,Nothing
-2338,0.37,-0.81,-0.02,-1.01,0.02,-0.33,-0.50
-3149,0.87,0.27,-0.40,0.14,-0.29,-0.04,-0.47
-4773,0.10,-0.71,-0.53,-0.58,-0.07,-0.46,0.12
-5015,-0.16,-0.01,0.54,-0.39,-0.08,-0.13,0.29
-7780,0.04,-0.06,0.89,-0.37,-0.05,-0.27,-0.08
-9583,0.34,0.00,0.93,-0.07,-0.09,-0.46,-0.29
-13317,0.25,0.21,1.12,-0.06,-0.03,0.01,-0.42
-25757,0.00,0.00,0.00,0.00,0.00,0.00,0.00
+Total simulations;Timing deviation;Value failure;Coding failure;Glitch;Deadlock;Token count failure;Nothing
+2338;0.37;-0.81;-0.02;-1.01;0.02;-0.33;-0.50
+3149;0.87;0.27;-0.40;0.14;-0.29;-0.04;-0.47
+4773;0.10;-0.71;-0.53;-0.58;-0.07;-0.46;0.12
+5015;-0.16;-0.01;0.54;-0.39;-0.08;-0.13;0.29
+7780;0.04;-0.06;0.89;-0.37;-0.05;-0.27;-0.08
+9583;0.34;0.00;0.93;-0.07;-0.09;-0.46;-0.29
+13317;0.25;0.21;1.12;-0.06;-0.03;0.01;-0.42
+25757;0.00;0.00;0.00;0.00;0.00;0.00;0.00

results/deviation_num_injections_dims.tex

@@ -1,5 +1,5 @@
 
-\pgfplotstableread[col sep=comma]{results/deviation_num_injections_dims.csv}\deviationsnuminjectionsdims
+\pgfplotstableread[col sep=semicolon]{results/deviation_num_injections_dims.csv}\deviationsnuminjectionsdims
 
 \begin{tikzpicture}
     \begin{axis}[
@@ -13,12 +13,12 @@
         grid=both,  % Add a grid for better readability
     ]
     
-        \addplot table[x=Timing deviation, y=Total simulations] {\deviationsnuminjectionsdims};
-        \addplot table[x=Value failure, y=Total simulations] {\deviationsnuminjectionsdims};
-        \addplot table[x=Coding failure, y=Total simulations] {\deviationsnuminjectionsdims};
-        \addplot table[x=Glitch, y=Total simulations] {\deviationsnuminjectionsdims};
-        \addplot table[x=Deadlock, y=Total simulations] {\deviationsnuminjectionsdims};
-        \addplot table[x=Token count failure, y=Total simulations] {\deviationsnuminjectionsdims};
-        \addplot table[x=Nothing, y=Total simulations] {\deviationsnuminjectionsdims};
+        \addplot[blue, mark=*] table[x=Timing deviation, y=Total simulations] {\deviationsnuminjectionsdims};
+        \addplot[red, mark=square*] table[x=Value failure, y=Total simulations] {\deviationsnuminjectionsdims};
+        \addplot[brown, mark=triangle*] table[x=Coding failure, y=Total simulations] {\deviationsnuminjectionsdims};
+        \addplot[gray, mark=diamond*] table[x=Glitch, y=Total simulations] {\deviationsnuminjectionsdims};
+        \addplot[purple, mark=+] table[x=Deadlock, y=Total simulations] {\deviationsnuminjectionsdims};
+        \addplot[green, mark=x] table[x=Token count failure, y=Total simulations] {\deviationsnuminjectionsdims};
+        \addplot[cyan, mark=halfcircle] table[x=Nothing, y=Total simulations] {\deviationsnuminjectionsdims};
     \end{axis}
 \end{tikzpicture}

results/deviation_num_injections_nclx.csv

@@ -0,0 +1,9 @@
+Total simulations;Timing deviation;Value failure;Coding failure;Glitch;Deadlock;Token count failure;Nothing
+2445;0.19;-0.24;0.00;-0.37;-0.64;-0.07;-0.22
+3263;0.35;0.25;0.00;0.87;-0.01;-0.09;-0.12
+4943;-0.21;-0.47;0.00;-0.29;0.22;-0.24;0.29
+5206;0.11;0.61;0.00;1.07;0.02;0.77;-0.05
+8060;0.23;0.12;0.00;0.67;0.12;0.06;-0.23
+9941;0.30;0.11;0.00;0.42;0.08;-0.04;-0.29
+13765;0.00;0.12;0.00;0.63;0.06;0.00;-0.13
+26807;0.00;0.00;0.00;0.00;0.00;0.00;0.00

results/deviation_num_injections_nclx.tex

@@ -0,0 +1,24 @@
+
+\pgfplotstableread[col sep=semicolon]{results/deviation_num_injections_nclx.csv}\deviationsnuminjectionsnclx
+
+\begin{tikzpicture}
+    \begin{axis}[
+        width=.95\textwidth,
+        xlabel={Deviation from max runs (\% points)},
+        ylabel={Number Simulations},
+        ylabel near ticks,
+        xmin=-2, xmax=2,
+        ymin=2000, ymax=26000,
+        scaled y ticks=false,
+        grid=both,  % Add a grid for better readability
+    ]
+    
+        \addplot[blue, mark=*] table[x=Timing deviation, y=Total simulations] {\deviationsnuminjectionsnclx};
+        \addplot[red, mark=square*] table[x=Value failure, y=Total simulations] {\deviationsnuminjectionsnclx};
+        \addplot[brown, mark=triangle*] table[x=Coding failure, y=Total simulations] {\deviationsnuminjectionsnclx};
+        \addplot[gray, mark=diamond*] table[x=Glitch, y=Total simulations] {\deviationsnuminjectionsnclx};
+        \addplot[purple, mark=+] table[x=Deadlock, y=Total simulations] {\deviationsnuminjectionsnclx};
+        \addplot[green, mark=x] table[x=Token count failure, y=Total simulations] {\deviationsnuminjectionsnclx};
+        \addplot[cyan, mark=halfcircle] table[x=Nothing, y=Total simulations] {\deviationsnuminjectionsnclx};
+    \end{axis}
+\end{tikzpicture}

results/deviation_num_signals_dims.csv

@@ -1,7 +1,8 @@
-Selected signals,Timing deviation,Value failure,Coding failure,Glitch,Deadlock,Token count failure,Nothing
-100,0.00,0.00,0.00,0.00,0.00,0.00,0.00
-90,-0.61,-1.00,-1.37,0.09,-0.38,-0.37,0.73
-70,0.37,-0.81,-0.02,-1.01,0.02,-0.33,-0.50
-50,0.00,0.13,-1.00,1.11,-0.19,0.16,0.46
-30,0.00,0.13,-1.00,1.11,-0.19,0.16,0.46
-10,0.00,0.13,-1.00,1.11,-0.19,0.16,0.46
+Selected signals;Timing deviation;Value failure;Coding failure;Glitch;Deadlock;Token count failure;Nothing
+100;0.00;0.00;0.00;0.00;0.00;0.00;0.00
+90;-0.61;-1.00;-1.37;0.09;-0.38;-0.37;0.73
+70;0.37;-0.81;-0.02;-1.01;0.02;-0.33;-0.50
+50;0.00;0.13;-1.00;1.11;-0.19;0.16;0.46
+30;0.14;-0.50;-0.32;1.43;0.39;0.37;-0.14
+10;-1.48;-0.36;-4.48;4.55;1.70;1.58;0.41
+5;1.56;0.19;-5.24;1.74;1.21;1.36;-1.94

results/deviation_num_signals_dims.tex

@@ -1,24 +1,24 @@
 
-\pgfplotstableread[col sep=comma]{results/deviation_num_signals_dims.csv}\deviationsnumsignalsdims
+\pgfplotstableread[col sep=semicolon]{results/deviation_num_signals_dims.csv}\deviationsnumsignalsdims
 
 \begin{tikzpicture}
     \begin{axis}[
         width=.95\textwidth,
         xlabel={Deviation from max runs (\% points)},
-        ylabel={Number Simulations},
+        ylabel={Signal Coverage (\%)},
         ylabel near ticks,
-        xmin=-2, xmax=2,
+        xmin=-3, xmax=3,
         ymin=0, ymax=100,
         scaled y ticks=false,
         grid=both,  % Add a grid for better readability
     ]
         
-        \addplot table[x=Timing deviation, y=Selected signals] {\deviationsnumsignalsdims};
-        \addplot table[x=Value failure, y=Selected signals] {\deviationsnumsignalsdims};
-        \addplot table[x=Coding failure, y=Selected signals] {\deviationsnumsignalsdims};
-        \addplot table[x=Glitch, y=Selected signals] {\deviationsnumsignalsdims};
-        \addplot table[x=Deadlock, y=Selected signals] {\deviationsnumsignalsdims};
-        \addplot table[x=Token count failure, y=Selected signals] {\deviationsnumsignalsdims};
-        \addplot table[x=Nothing, y=Selected signals] {\deviationsnumsignalsdims};
+        \addplot[blue, mark=*] table[x=Timing deviation, y=Selected signals] {\deviationsnumsignalsdims};
+        \addplot[red, mark=square*] table[x=Value failure, y=Selected signals] {\deviationsnumsignalsdims};
+        \addplot[brown, mark=triangle*] table[x=Coding failure, y=Selected signals] {\deviationsnumsignalsdims};
+        \addplot[gray, mark=diamond*] table[x=Glitch, y=Selected signals] {\deviationsnumsignalsdims};
+        \addplot[purple, mark=+] table[x=Deadlock, y=Selected signals] {\deviationsnumsignalsdims};
+        \addplot[green, mark=x] table[x=Token count failure, y=Selected signals] {\deviationsnumsignalsdims};
+        \addplot[cyan, mark=halfcircle] table[x=Nothing, y=Selected signals] {\deviationsnumsignalsdims};
     \end{axis}
 \end{tikzpicture}

results/deviation_num_signals_nclx.csv

@@ -0,0 +1,8 @@
+Selected signals;Timing deviation;Value failure;Coding failure;Glitch;Deadlock;Token count failure;Nothing
+100;0.00;0.00;0.00;0.00;0.00;0.00;0.00
+90;-0.12;0.40;0.00;1.03;-0.09;0.49;0.16
+70;-0.08;0.26;0.00;1.47;-0.29;0.30;0.20
+50;0.19;0.73;0.00;2.92;0.05;0.28;-0.03
+30;0.37;1.57;0.00;3.10;-0.36;1.41;-0.58
+10;0.21;-0.33;0.00;4.98;0.31;1.02;-0.22
+5;-0.02;2.61;0.00;9.49;0.09;5.99;-0.36

results/deviation_num_signals_nclx.tex

@@ -0,0 +1,24 @@
+
+\pgfplotstableread[col sep=semicolon]{results/deviation_num_signals_nclx.csv}\deviationsnumsignalsnclx
+
+\begin{tikzpicture}
+    \begin{axis}[
+        width=.95\textwidth,
+        xlabel={Deviation from max runs (\% points)},
+        ylabel={Signal Coverage (\%)},
+        ylabel near ticks,
+        xmin=-3, xmax=3,
+        ymin=0, ymax=100,
+        scaled y ticks=false,
+        grid=both,  % Add a grid for better readability
+    ]
+        
+        \addplot[blue, mark=*] table[x=Timing deviation, y=Selected signals] {\deviationsnumsignalsnclx};
+        \addplot[red, mark=square*] table[x=Value failure, y=Selected signals] {\deviationsnumsignalsnclx};
+        \addplot[brown, mark=triangle*] table[x=Coding failure, y=Selected signals] {\deviationsnumsignalsnclx};
+        \addplot[gray, mark=diamond*] table[x=Glitch, y=Selected signals] {\deviationsnumsignalsnclx};
+        \addplot[purple, mark=+] table[x=Deadlock, y=Selected signals] {\deviationsnumsignalsnclx};
+        \addplot[green, mark=x] table[x=Token count failure, y=Selected signals] {\deviationsnumsignalsnclx};
+        \addplot[cyan, mark=halfcircle] table[x=Nothing, y=Selected signals] {\deviationsnumsignalsnclx};
+    \end{axis}
+\end{tikzpicture}

results/sim_scaling.tex

@@ -0,0 +1,40 @@
+\pgfplotsset{width=7cm,compat=1.3}
+
+\begin{tikzpicture}
+    \pgfplotsset{
+        scale only axis,
+        ymin=0, ymax=26000,
+        scaled y ticks=false,
+    }
+    
+    \begin{axis}[
+        xmin=0, xmax=1,
+        axis x line*=bottom,
+        % xlabel=x-axis,
+        ylabel=Number of simulations,
+    ]
+
+        \addplot[mark=x,red] table[x=Coverage certainty, y=Simulations, col sep=semicolon] {results/sim_scaling_dims_coverage_certainty.csv}; \label{scaling-plot-cov-cert}
+
+        \addplot[mark=o,green] table[x=Hit probability, y=Simulations, col sep=semicolon] {results/sim_scaling_dims_hit_prob.csv}; \label{scaling-plot-hit-prob}
+
+        \addplot[mark=x,blue] table[x=Victim coverage, y=Simulations, col sep=semicolon] {results/sim_scaling_dims_victim_coverage.csv}; \label{scaling-plot-vic-cov}
+
+    \end{axis}
+
+    \begin{axis}[
+      axis x line*=top,
+      axis y line=none,
+      xmin=0, xmax=4,
+    %   xlabel=x-axis 2
+    ]
+
+        \addlegendimage{/pgfplots/refstyle=scaling-plot-cov-cert}\addlegendentry{Coverage certainty}
+        \addlegendimage{/pgfplots/refstyle=scaling-plot-hit-prob}\addlegendentry{Hit probability}
+        \addlegendimage{/pgfplots/refstyle=scaling-plot-vic-cov}\addlegendentry{Victim coverage}
+
+        \addplot[smooth,mark=*,brown] table[x=Modes, y=Simulations, col sep=semicolon] {results/sim_scaling_dims_modes.csv};
+        \addlegendentry{Failure modes}
+    \end{axis}
+
+\end{tikzpicture}

results/sim_scaling_dims_coverage_certainty.csv

@@ -0,0 +1,5 @@
+Coverage certainty;Simulations
+0.2;2338
+0.4;3149
+0.6;4773
+0.8;9583

results/sim_scaling_dims_hit_prob.csv

@@ -0,0 +1,4 @@
+Hit probability;Simulations
+0.1;25757
+0.5;5015
+0.8;3149

results/sim_scaling_dims_modes.csv

@@ -0,0 +1,4 @@
+Modes;Simulations
+1;3149
+2;7780
+3;13317

results/sim_scaling_dims_victim_coverage.csv

@@ -0,0 +1,8 @@
+Victim coverage;Simulations
+1;4025
+0.9;3849
+0.7;3497
+0.5;3149
+0.3;2510
+0.1;1314
+0.05;609

test-circuit/injection_test.yml

@@ -16,7 +16,7 @@ prepare:
     hit-probability: 0.8
     modes-per-fork: 1
     coverage-certainty: 0.4
-    victim-coverage: 1
+    victim-coverage: 0.05
     injection-windows:
       - begin: 240
         end: 1200
@@ -35,5 +35,5 @@ deploy:
       sim_outputs: sim_result
     top: tb
 
-# select count (*) from sim_outputs where (fault_flags & B'100000') = B'100000';
+# select count (*) from sim_outputs where (fault_flags & B'000001') = B'000001'; select count (*) from sim_outputs where (fault_flags & B'000010') = B'000010'; select count (*) from sim_outputs where (fault_flags & B'000100') = B'000100'; select count (*) from sim_outputs where (fault_flags & B'001000') = B'001000'; select count (*) from sim_outputs where (fault_flags & B'010000') = B'010000'; select count (*) from sim_outputs where (fault_flags & B'100000') = B'100000'; select count (*) from sim_outputs where (fault_flags) = B'000000'; 
 # update sim_configs set part_status = 'not_started' where part_status = 'finished' and not exists (select 1 from sim_outputs so where so.sim_config = sim_configs.id);