2004 · 18 citations · 21 references
ComputingComputer ScienceComputer SystemsFault ToleranceFault-tolerant ComputingOperating SystemsBenchmarkingDependability AnalysisDependability MetricsSoftware Fault ToleranceWorst-case Execution Time AnalysisDependabilityDependability BenchmarkingError CodesOperating System BenchmarkingRobustnessSystem DependabilityWorkloadWorkload Characterization
We propose a dependability benchmark suitable for a general purpose operating system (OS). The specifications of the benchmark components are presented and illustrated on a benchmark prototype dedicated to Windows 2000. The important novelty, as regards OS dependability benchmarking, is threefold. First, it lies on a comprehensive and structured set of measures: outcomes are considered both at the OS level and at the application level. Second, these measures include not only robustness measures (e.g., the distribution among the observed outcomes for the OS and the application: error codes, exceptions, workload correct or erroneous completion, OS and application hang), but also the related temporal measures in the presence of faults (e.g., system call and workload execution times, as well as operating system restart time). Finally, we are considering a realistic workload (namely, TPC-C client), instead of a synthetic workload.
21
lmbench: portable tools for performance analysis
Larry McVoy, Carl Staelin · 1996 · 838 citations
Comparing the robustness of POSIX operating systems
Philip Koopman, J. DeVale · 2003 · 167 citations
Comparing operating systems using robustness benchmarks
Philip Koopman, Jongmo Sung, C.P. Dingman et al. · 2002 · 149 citations