=== 12:14:15 suspend saturn and jupiter === 12:14:15 L0 baselines L0:btrfs-cow run 1: fdatasync p50 1.29 ms p99 4.75 ms p99.9 5.08 ms max 11.89 ms nvme 55->71C throttle+0 L0:btrfs-cow run 2: fdatasync p50 1.29 ms p99 1.99 ms p99.9 5.14 ms max 36.59 ms nvme 56->70C throttle+1 L0:btrfs-cow run 3: fdatasync p50 1.29 ms p99 1.55 ms p99.9 5.08 ms max 11.23 ms nvme 57->70C throttle+0 L0:btrfs-cow run 4: fdatasync p50 1.27 ms p99 1.63 ms p99.9 5.08 ms max 17.20 ms nvme 56->69C throttle+0 L0:btrfs-cow run 5: fdatasync p50 1.29 ms p99 1.79 ms p99.9 5.08 ms max 9.55 ms nvme 58->70C throttle+0 L0:btrfs-nocow run 1: fdatasync p50 1.17 ms p99 1.38 ms p99.9 4.95 ms max 9.47 ms nvme 54->67C throttle+0 L0:btrfs-nocow run 2: fdatasync p50 1.17 ms p99 1.40 ms p99.9 4.95 ms max 9.54 ms nvme 56->67C throttle+0 L0:btrfs-nocow run 3: fdatasync p50 1.17 ms p99 1.40 ms p99.9 4.88 ms max 9.53 ms nvme 55->67C throttle+0 L0:btrfs-nocow run 4: fdatasync p50 1.17 ms p99 1.38 ms p99.9 4.95 ms max 11.99 ms nvme 54->67C throttle+0 L0:btrfs-nocow run 5: fdatasync p50 1.17 ms p99 1.40 ms p99.9 5.01 ms max 11.99 ms nvme 54->66C throttle+0 === 12:20:48 partition round === host: raw partition as the VM disk L1:P-rawpart-none-native/rawdev run 1: fdatasync p50 0.53 ms p99 0.69 ms p99.9 3.95 ms max 5.91 ms nvme 53->64C throttle+0 (+0s) L1:P-rawpart-none-native/rawdev run 2: fdatasync p50 0.53 ms p99 0.69 ms p99.9 1.65 ms max 5.47 ms nvme 53->63C throttle+0 (+0s) L1:P-rawpart-none-native/rawdev run 3: fdatasync p50 0.53 ms p99 0.68 ms p99.9 4.15 ms max 5.59 ms nvme 51->61C throttle+0 (+0s) L1:P-rawpart-none-native/rawdev run 4: fdatasync p50 0.53 ms p99 0.70 ms p99.9 4.08 ms max 5.47 ms nvme 50->61C throttle+0 (+0s) L1:P-rawpart-none-native/rawdev run 5: fdatasync p50 0.53 ms p99 0.68 ms p99.9 2.61 ms max 12.48 ms nvme 50->60C throttle+0 (+0s) L1:P-rawpart-none-native/ext4 run 1: fdatasync p50 1.22 ms p99 1.81 ms p99.9 4.95 ms max 9.99 ms nvme 51->63C throttle+0 (+0s) L1:P-rawpart-none-native/ext4 run 2: fdatasync p50 1.11 ms p99 1.47 ms p99.9 4.82 ms max 8.79 ms nvme 52->63C throttle+0 (+0s) L1:P-rawpart-none-native/ext4 run 3: fdatasync p50 1.11 ms p99 1.48 ms p99.9 4.88 ms max 9.89 ms nvme 51->63C throttle+0 (+0s) L1:P-rawpart-none-native/ext4 run 4: fdatasync p50 1.14 ms p99 1.50 ms p99.9 4.88 ms max 10.48 ms nvme 51->63C throttle+0 (+0s) L1:P-rawpart-none-native/ext4 run 5: fdatasync p50 1.12 ms p99 1.48 ms p99.9 5.01 ms max 9.90 ms nvme 52->63C throttle+0 (+0s) L1:P-rawpart-none-native/xfs run 1: fdatasync p50 1.17 ms p99 1.45 ms p99.9 4.62 ms max 9.87 ms nvme 51->62C throttle+0 (+0s) L1:P-rawpart-none-native/xfs run 2: fdatasync p50 1.17 ms p99 1.47 ms p99.9 4.88 ms max 9.92 ms nvme 51->62C throttle+0 (+0s) L1:P-rawpart-none-native/xfs run 3: fdatasync p50 1.17 ms p99 1.45 ms p99.9 4.75 ms max 9.90 ms nvme 55->65C throttle+0 (+0s) L1:P-rawpart-none-native/xfs run 4: fdatasync p50 1.17 ms p99 1.47 ms p99.9 4.42 ms max 10.91 ms nvme 52->63C throttle+0 (+0s) L1:P-rawpart-none-native/xfs run 5: fdatasync p50 1.17 ms p99 1.45 ms p99.9 4.69 ms max 9.97 ms nvme 51->62C throttle+0 (+0s) === host: xfs on the partition L0:xfs run 1: fdatasync p50 1.06 ms p99 1.61 ms p99.9 5.73 ms max 14.85 ms nvme 59->66C throttle+0 (+0s) L0:xfs run 2: fdatasync p50 1.06 ms p99 1.25 ms p99.9 4.82 ms max 9.46 ms nvme 53->64C throttle+0 (+0s) L0:xfs run 3: fdatasync p50 1.06 ms p99 1.25 ms p99.9 4.82 ms max 10.61 ms nvme 52->63C throttle+0 (+0s) L0:xfs run 4: fdatasync p50 1.06 ms p99 1.25 ms p99.9 4.75 ms max 7.20 ms nvme 51->63C throttle+0 (+0s) L0:xfs run 5: fdatasync p50 1.06 ms p99 1.25 ms p99.9 4.82 ms max 9.35 ms nvme 51->63C throttle+0 (+0s) L1:P-xfs-raw-none-native/rawdev run 1: fdatasync p50 1.16 ms p99 1.38 ms p99.9 4.88 ms max 9.90 ms nvme 51->64C throttle+0 (+0s) L1:P-xfs-raw-none-native/rawdev run 2: fdatasync p50 0.54 ms p99 0.71 ms p99.9 2.67 ms max 7.78 ms nvme 51->61C throttle+0 (+0s) L1:P-xfs-raw-none-native/rawdev run 3: fdatasync p50 0.54 ms p99 0.69 ms p99.9 2.61 ms max 4.37 ms nvme 58->63C throttle+0 (+0s) L1:P-xfs-raw-none-native/rawdev run 4: fdatasync p50 0.54 ms p99 0.71 ms p99.9 1.63 ms max 5.63 ms nvme 50->61C throttle+0 (+0s) L1:P-xfs-raw-none-native/rawdev run 5: fdatasync p50 0.53 ms p99 0.69 ms p99.9 1.35 ms max 6.89 ms nvme 49->59C throttle+0 (+0s) L1:P-xfs-raw-none-native/ext4 run 1: fdatasync p50 2.54 ms p99 3.03 ms p99.9 6.46 ms max 11.49 ms nvme 56->67C throttle+0 (+0s) L1:P-xfs-raw-none-native/ext4 run 2: fdatasync p50 1.22 ms p99 1.89 ms p99.9 5.41 ms max 10.80 ms nvme 54->65C throttle+0 (+0s) L1:P-xfs-raw-none-native/ext4 run 3: fdatasync p50 1.19 ms p99 1.86 ms p99.9 5.08 ms max 9.88 ms nvme 51->64C throttle+0 (+0s) L1:P-xfs-raw-none-native/ext4 run 4: fdatasync p50 1.17 ms p99 1.53 ms p99.9 4.95 ms max 10.01 ms nvme 52->64C throttle+0 (+0s) L1:P-xfs-raw-none-native/ext4 run 5: fdatasync p50 1.19 ms p99 1.50 ms p99.9 4.88 ms max 9.84 ms nvme 51->64C throttle+0 (+0s) L1:P-xfs-raw-none-native/xfs run 1: fdatasync p50 1.83 ms p99 2.77 ms p99.9 5.80 ms max 8.96 ms nvme 50->65C throttle+0 (+0s) L1:P-xfs-raw-none-native/xfs run 2: fdatasync p50 1.25 ms p99 2.74 ms p99.9 6.06 ms max 11.28 ms nvme 53->65C throttle+0 (+0s) L1:P-xfs-raw-none-native/xfs run 3: fdatasync p50 1.20 ms p99 2.04 ms p99.9 5.28 ms max 10.68 ms nvme 52->65C throttle+0 (+0s) L1:P-xfs-raw-none-native/xfs run 4: fdatasync p50 1.20 ms p99 2.06 ms p99.9 5.54 ms max 10.45 ms nvme 52->65C throttle+0 (+0s) L1:P-xfs-raw-none-native/xfs run 5: fdatasync p50 1.22 ms p99 2.06 ms p99.9 5.41 ms max 9.91 ms nvme 52->65C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/rawdev run 1: fdatasync p50 1.17 ms p99 1.38 ms p99.9 4.88 ms max 9.68 ms nvme 60->67C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/rawdev run 2: fdatasync p50 0.54 ms p99 0.71 ms p99.9 1.32 ms max 5.51 ms nvme 54->64C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/rawdev run 3: fdatasync p50 0.54 ms p99 0.72 ms p99.9 1.65 ms max 5.61 ms nvme 51->61C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/rawdev run 4: fdatasync p50 0.54 ms p99 0.72 ms p99.9 1.63 ms max 5.54 ms nvme 49->60C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/rawdev run 5: fdatasync p50 0.54 ms p99 0.72 ms p99.9 4.11 ms max 6.63 ms nvme 60->64C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/ext4 run 1: fdatasync p50 2.57 ms p99 3.65 ms p99.9 6.91 ms max 12.48 ms nvme 50->67C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/ext4 run 2: fdatasync p50 1.32 ms p99 2.01 ms p99.9 5.34 ms max 6.67 ms nvme 60->67C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/ext4 run 3: fdatasync p50 1.20 ms p99 1.55 ms p99.9 5.14 ms max 11.50 ms nvme 52->65C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/ext4 run 4: fdatasync p50 1.22 ms p99 1.55 ms p99.9 5.01 ms max 10.04 ms nvme 51->64C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/ext4 run 5: fdatasync p50 1.16 ms p99 1.52 ms p99.9 5.01 ms max 10.65 ms nvme 51->63C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/xfs run 1: fdatasync p50 1.47 ms p99 3.49 ms p99.9 6.85 ms max 10.39 ms nvme 51->65C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/xfs run 2: fdatasync p50 1.99 ms p99 8.16 ms p99.9 11.08 ms max 19.82 ms nvme 53->68C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/xfs run 3: fdatasync p50 1.94 ms p99 8.98 ms p99.9 10.94 ms max 11.98 ms nvme 55->68C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/xfs run 4: fdatasync p50 1.94 ms p99 8.03 ms p99.9 10.94 ms max 16.17 ms nvme 54->69C throttle+0 (+0s) L1:P-xfs-qcow2-falloc-none-native/xfs run 5: fdatasync p50 1.81 ms p99 6.98 ms p99.9 9.37 ms max 13.22 ms nvme 56->69C throttle+0 (+0s) === host: ext4 on the partition L0:ext4 run 1: fdatasync p50 1.07 ms p99 4.75 ms p99.9 6.00 ms max 10.33 ms nvme 55->65C throttle+0 (+0s) L0:ext4 run 2: fdatasync p50 1.03 ms p99 4.75 ms p99.9 5.73 ms max 8.07 ms nvme 55->67C throttle+0 (+0s) L0:ext4 run 3: fdatasync p50 1.06 ms p99 4.75 ms p99.9 6.52 ms max 10.20 ms nvme 54->67C throttle+0 (+0s) L0:ext4 run 4: fdatasync p50 1.11 ms p99 4.69 ms p99.9 6.19 ms max 12.68 ms nvme 54->68C throttle+0 (+0s) L0:ext4 run 5: fdatasync p50 1.11 ms p99 1.30 ms p99.9 4.75 ms max 9.34 ms nvme 55->65C throttle+0 (+0s) L1:P-ext4-raw-none-native/rawdev run 1: fdatasync p50 1.17 ms p99 1.40 ms p99.9 4.82 ms max 9.07 ms nvme 54->65C throttle+0 (+0s) L1:P-ext4-raw-none-native/rawdev run 2: fdatasync p50 0.53 ms p99 0.68 ms p99.9 2.28 ms max 5.59 ms nvme 55->65C throttle+0 (+0s) L1:P-ext4-raw-none-native/rawdev run 3: fdatasync p50 0.53 ms p99 0.70 ms p99.9 3.62 ms max 6.91 ms nvme 53->63C throttle+0 (+0s) L1:P-ext4-raw-none-native/rawdev run 4: fdatasync p50 0.53 ms p99 0.69 ms p99.9 4.15 ms max 5.58 ms nvme 52->62C throttle+0 (+0s) L1:P-ext4-raw-none-native/rawdev run 5: fdatasync p50 0.54 ms p99 0.76 ms p99.9 3.72 ms max 5.59 ms nvme 51->62C throttle+0 (+1s) L1:P-ext4-raw-none-native/ext4 run 1: fdatasync p50 2.51 ms p99 3.00 ms p99.9 6.52 ms max 18.32 ms nvme 52->66C throttle+0 (+0s) L1:P-ext4-raw-none-native/ext4 run 2: fdatasync p50 1.14 ms p99 1.47 ms p99.9 4.95 ms max 13.22 ms nvme 57->65C throttle+0 (+0s) L1:P-ext4-raw-none-native/ext4 run 3: fdatasync p50 1.19 ms p99 1.50 ms p99.9 4.95 ms max 12.23 ms nvme 54->65C throttle+0 (+0s) L1:P-ext4-raw-none-native/ext4 run 4: fdatasync p50 1.14 ms p99 1.47 ms p99.9 4.88 ms max 13.37 ms nvme 54->65C throttle+0 (+0s) L1:P-ext4-raw-none-native/ext4 run 5: fdatasync p50 1.22 ms p99 1.48 ms p99.9 5.01 ms max 9.88 ms nvme 52->65C throttle+0 (+0s) L1:P-ext4-raw-none-native/xfs run 1: fdatasync p50 1.81 ms p99 2.77 ms p99.9 6.13 ms max 14.56 ms nvme 52->65C throttle+0 (+0s) L1:P-ext4-raw-none-native/xfs run 2: fdatasync p50 1.25 ms p99 2.77 ms p99.9 6.06 ms max 11.27 ms nvme 53->65C throttle+0 (+0s) L1:P-ext4-raw-none-native/xfs run 3: fdatasync p50 1.20 ms p99 2.09 ms p99.9 5.01 ms max 10.74 ms nvme 53->65C throttle+0 (+0s) L1:P-ext4-raw-none-native/xfs run 4: fdatasync p50 1.20 ms p99 2.07 ms p99.9 5.47 ms max 11.12 ms nvme 52->65C throttle+0 (+0s) L1:P-ext4-raw-none-native/xfs run 5: fdatasync p50 1.20 ms p99 2.07 ms p99.9 5.28 ms max 19.92 ms nvme 52->65C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/rawdev run 1: fdatasync p50 1.17 ms p99 1.65 ms p99.9 4.82 ms max 9.71 ms nvme 51->64C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/rawdev run 2: fdatasync p50 0.54 ms p99 0.73 ms p99.9 4.15 ms max 13.82 ms nvme 51->62C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/rawdev run 3: fdatasync p50 0.54 ms p99 0.72 ms p99.9 1.73 ms max 5.31 ms nvme 51->61C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/rawdev run 4: fdatasync p50 0.54 ms p99 0.72 ms p99.9 4.15 ms max 6.59 ms nvme 49->60C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/rawdev run 5: fdatasync p50 0.54 ms p99 0.72 ms p99.9 4.11 ms max 10.15 ms nvme 60->64C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/ext4 run 1: fdatasync p50 2.54 ms p99 3.52 ms p99.9 6.91 ms max 17.95 ms nvme 51->66C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/ext4 run 2: fdatasync p50 1.25 ms p99 1.55 ms p99.9 4.95 ms max 8.08 ms nvme 53->65C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/ext4 run 3: fdatasync p50 1.20 ms p99 1.55 ms p99.9 5.08 ms max 15.60 ms nvme 52->65C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/ext4 run 4: fdatasync p50 1.24 ms p99 1.55 ms p99.9 5.08 ms max 10.77 ms nvme 52->64C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/ext4 run 5: fdatasync p50 1.24 ms p99 1.71 ms p99.9 5.08 ms max 16.79 ms nvme 54->65C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/xfs run 1: fdatasync p50 1.60 ms p99 3.42 ms p99.9 6.32 ms max 16.16 ms nvme 52->66C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/xfs run 2: fdatasync p50 1.29 ms p99 3.13 ms p99.9 6.26 ms max 12.52 ms nvme 53->65C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/xfs run 3: fdatasync p50 1.24 ms p99 2.61 ms p99.9 5.54 ms max 12.55 ms nvme 54->66C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/xfs run 4: fdatasync p50 1.24 ms p99 2.57 ms p99.9 5.14 ms max 9.11 ms nvme 55->65C throttle+0 (+0s) L1:P-ext4-qcow2-falloc-none-native/xfs run 5: fdatasync p50 1.22 ms p99 2.61 ms p99.9 5.01 ms max 18.78 ms nvme 54->65C throttle+0 (+0s) === host: ext2 on the partition L0:ext2 run 1: fdatasync p50 0.46 ms p99 0.59 ms p99.9 4.15 ms max 8.70 ms nvme 53->63C throttle+0 (+0s) L0:ext2 run 2: fdatasync p50 0.46 ms p99 0.58 ms p99.9 4.08 ms max 11.50 ms nvme 60->65C throttle+0 (+0s) L0:ext2 run 3: fdatasync p50 0.47 ms p99 0.59 ms p99.9 4.15 ms max 8.69 ms nvme 53->63C throttle+0 (+0s) L0:ext2 run 4: fdatasync p50 0.47 ms p99 0.59 ms p99.9 4.18 ms max 11.51 ms nvme 56->64C throttle+0 (+0s) L0:ext2 run 5: fdatasync p50 0.46 ms p99 0.58 ms p99.9 4.05 ms max 12.82 ms nvme 52->62C throttle+0 (+0s) L1:P-ext2-raw-none-native/rawdev run 1: fdatasync p50 0.54 ms p99 0.71 ms p99.9 4.15 ms max 10.27 ms nvme 53->63C throttle+0 (+0s) L1:P-ext2-raw-none-native/rawdev run 2: fdatasync p50 0.54 ms p99 0.72 ms p99.9 4.11 ms max 10.53 ms nvme 56->65C throttle+0 (+0s) L1:P-ext2-raw-none-native/rawdev run 3: fdatasync p50 0.54 ms p99 0.72 ms p99.9 4.23 ms max 9.25 ms nvme 54->64C throttle+0 (+0s) L1:P-ext2-raw-none-native/rawdev run 4: fdatasync p50 0.54 ms p99 0.72 ms p99.9 4.23 ms max 10.49 ms nvme 53->63C throttle+0 (+0s) L1:P-ext2-raw-none-native/rawdev run 5: fdatasync p50 0.54 ms p99 0.72 ms p99.9 4.23 ms max 10.42 ms nvme 54->63C throttle+0 (+0s) L1:P-ext2-raw-none-native/ext4 run 1: fdatasync p50 1.16 ms p99 1.55 ms p99.9 5.14 ms max 14.34 ms nvme 52->65C throttle+0 (+0s) L1:P-ext2-raw-none-native/ext4 run 2: fdatasync p50 1.24 ms p99 1.63 ms p99.9 5.01 ms max 11.25 ms nvme 57->67C throttle+0 (+0s) L1:P-ext2-raw-none-native/ext4 run 3: fdatasync p50 1.19 ms p99 1.53 ms p99.9 5.08 ms max 15.00 ms nvme 56->66C throttle+0 (+0s) L1:P-ext2-raw-none-native/ext4 run 4: fdatasync p50 1.25 ms p99 1.89 ms p99.9 5.28 ms max 10.61 ms nvme 54->66C throttle+0 (+0s) L1:P-ext2-raw-none-native/ext4 run 5: fdatasync p50 1.16 ms p99 1.52 ms p99.9 4.95 ms max 15.61 ms nvme 55->66C throttle+0 (+0s) L1:P-ext2-raw-none-native/xfs run 1: fdatasync p50 1.17 ms p99 1.48 ms p99.9 5.08 ms max 10.02 ms nvme 55->66C throttle+0 (+0s) L1:P-ext2-raw-none-native/xfs run 2: fdatasync p50 1.17 ms p99 1.50 ms p99.9 5.01 ms max 17.91 ms nvme 53->64C throttle+0 (+0s) L1:P-ext2-raw-none-native/xfs run 3: fdatasync p50 1.17 ms p99 1.50 ms p99.9 4.95 ms max 18.02 ms nvme 54->65C throttle+0 (+0s) L1:P-ext2-raw-none-native/xfs run 4: fdatasync p50 1.19 ms p99 1.84 ms p99.9 5.34 ms max 9.95 ms nvme 53->65C throttle+0 (+0s) L1:P-ext2-raw-none-native/xfs run 5: fdatasync p50 1.17 ms p99 1.48 ms p99.9 5.01 ms max 11.02 ms nvme 52->65C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/rawdev run 1: fdatasync p50 0.55 ms p99 0.74 ms p99.9 4.15 ms max 9.11 ms nvme 54->64C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/rawdev run 2: fdatasync p50 0.55 ms p99 0.75 ms p99.9 4.29 ms max 12.70 ms nvme 52->62C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/rawdev run 3: fdatasync p50 0.55 ms p99 0.75 ms p99.9 4.18 ms max 10.54 ms nvme 51->61C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/rawdev run 4: fdatasync p50 0.55 ms p99 0.76 ms p99.9 4.29 ms max 10.44 ms nvme 50->61C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/rawdev run 5: fdatasync p50 0.55 ms p99 0.76 ms p99.9 4.18 ms max 10.33 ms nvme 50->61C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/ext4 run 1: fdatasync p50 1.19 ms p99 2.15 ms p99.9 5.73 ms max 24.43 ms nvme 49->62C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/ext4 run 2: fdatasync p50 1.19 ms p99 1.55 ms p99.9 5.14 ms max 14.32 ms nvme 53->65C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/ext4 run 3: fdatasync p50 1.22 ms p99 1.56 ms p99.9 5.08 ms max 21.28 ms nvme 53->65C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/ext4 run 4: fdatasync p50 1.17 ms p99 1.56 ms p99.9 5.14 ms max 20.01 ms nvme 52->64C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/ext4 run 5: fdatasync p50 1.27 ms p99 1.94 ms p99.9 5.41 ms max 10.60 ms nvme 51->64C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/xfs run 1: fdatasync p50 1.19 ms p99 2.04 ms p99.9 5.01 ms max 10.67 ms nvme 52->64C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/xfs run 2: fdatasync p50 1.19 ms p99 1.93 ms p99.9 4.95 ms max 9.83 ms nvme 52->64C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/xfs run 3: fdatasync p50 1.19 ms p99 2.24 ms p99.9 5.08 ms max 16.32 ms nvme 52->63C throttle+0 (+1s) L1:P-ext2-qcow2-falloc-none-native/xfs run 4: fdatasync p50 1.17 ms p99 1.91 ms p99.9 4.95 ms max 9.91 ms nvme 53->65C throttle+0 (+0s) L1:P-ext2-qcow2-falloc-none-native/xfs run 5: fdatasync p50 1.19 ms p99 2.02 ms p99.9 5.01 ms max 9.87 ms nvme 52->64C throttle+0 (+0s) === 13:12:44 L1 on the root btrfs: current backing and nodatacow raw L1:C-cow-qcow2-default/rawdev run 1: fdatasync p50 1.43 ms p99 3.19 ms p99.9 5.21 ms max 31.64 ms nvme 58->68C throttle+0 (+0s) L1:C-cow-qcow2-default/rawdev run 2: fdatasync p50 1.42 ms p99 1.86 ms p99.9 5.28 ms max 19.74 ms nvme 55->68C throttle+0 (+0s) L1:C-cow-qcow2-default/rawdev run 3: fdatasync p50 1.43 ms p99 1.78 ms p99.9 5.28 ms max 13.94 ms nvme 56->69C throttle+0 (+0s) L1:C-cow-qcow2-default/rawdev run 4: fdatasync p50 1.42 ms p99 1.86 ms p99.9 5.34 ms max 26.58 ms nvme 57->69C throttle+0 (+0s) L1:C-cow-qcow2-default/rawdev run 5: fdatasync p50 1.43 ms p99 2.09 ms p99.9 5.93 ms max 32.60 ms nvme 56->69C throttle+0 (+0s) L1:C-cow-qcow2-default/ext4 run 1: fdatasync p50 2.93 ms p99 5.08 ms p99.9 8.59 ms max 25.23 ms nvme 56->72C throttle+0 (+0s) L1:C-cow-qcow2-default/ext4 run 2: fdatasync p50 2.93 ms p99 4.05 ms p99.9 7.11 ms max 17.81 ms nvme 57->72C throttle+0 (+0s) L1:C-cow-qcow2-default/ext4 run 3: fdatasync p50 2.90 ms p99 3.72 ms p99.9 7.11 ms max 18.78 ms nvme 58->72C throttle+0 (+0s) L1:C-cow-qcow2-default/ext4 run 4: fdatasync p50 2.93 ms p99 3.88 ms p99.9 7.05 ms max 21.65 ms nvme 61->74C throttle+0 (+0s) L1:C-cow-qcow2-default/ext4 run 5: fdatasync p50 2.90 ms p99 4.05 ms p99.9 7.24 ms max 21.50 ms nvme 60->74C throttle+0 (+0s) L1:C-cow-qcow2-default/xfs run 1: fdatasync p50 2.80 ms p99 4.75 ms p99.9 7.24 ms max 31.42 ms nvme 58->73C throttle+0 (+0s) L1:C-cow-qcow2-default/xfs run 2: fdatasync p50 2.80 ms p99 4.82 ms p99.9 8.59 ms max 29.96 ms nvme 59->73C throttle+0 (+0s) L1:C-cow-qcow2-default/xfs run 3: fdatasync p50 2.80 ms p99 4.69 ms p99.9 6.85 ms max 28.02 ms nvme 61->74C throttle+0 (+0s) L1:C-cow-qcow2-default/xfs run 4: fdatasync p50 2.80 ms p99 4.69 ms p99.9 6.91 ms max 19.32 ms nvme 58->73C throttle+0 (+0s) L1:C-cow-qcow2-default/xfs run 5: fdatasync p50 2.80 ms p99 4.82 ms p99.9 8.36 ms max 14.45 ms nvme 59->72C throttle+0 (+0s) L1:C-cow-qcow2-default/btrfs run 1: fdatasync p50 3.59 ms p99 7.18 ms p99.9 9.63 ms max 48.20 ms nvme 60->76C throttle+0 (+0s) L1:C-cow-qcow2-default/btrfs run 2: fdatasync p50 3.39 ms p99 6.78 ms p99.9 8.36 ms max 69.19 ms nvme 55->74C throttle+0 (+0s) L1:C-cow-qcow2-default/btrfs run 3: fdatasync p50 3.42 ms p99 6.98 ms p99.9 8.85 ms max 75.38 ms nvme 56->73C throttle+0 (+0s) L1:C-cow-qcow2-default/btrfs run 4: fdatasync p50 3.42 ms p99 6.98 ms p99.9 9.63 ms max 78.47 ms nvme 55->74C throttle+0 (+0s) L1:C-cow-qcow2-default/btrfs run 5: fdatasync p50 3.42 ms p99 6.98 ms p99.9 9.63 ms max 79.07 ms nvme 56->75C throttle+0 (+0s) L1:C-nocow-raw-none-native/rawdev run 1: fdatasync p50 1.35 ms p99 1.97 ms p99.9 5.54 ms max 6.42 ms nvme 54->69C throttle+0 (+0s) L1:C-nocow-raw-none-native/rawdev run 2: fdatasync p50 1.24 ms p99 1.86 ms p99.9 5.01 ms max 8.23 ms nvme 57->69C throttle+0 (+0s) L1:C-nocow-raw-none-native/rawdev run 3: fdatasync p50 1.24 ms p99 1.50 ms p99.9 5.01 ms max 5.49 ms nvme 58->69C throttle+0 (+0s) L1:C-nocow-raw-none-native/rawdev run 4: fdatasync p50 1.24 ms p99 1.52 ms p99.9 5.01 ms max 7.68 ms nvme 57->69C throttle+0 (+0s) L1:C-nocow-raw-none-native/rawdev run 5: fdatasync p50 1.24 ms p99 1.50 ms p99.9 5.08 ms max 14.31 ms nvme 56->67C throttle+0 (+0s) L1:C-nocow-raw-none-native/ext4 run 1: fdatasync p50 2.83 ms p99 4.08 ms p99.9 7.90 ms max 30.79 ms nvme 58->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/ext4 run 2: fdatasync p50 2.54 ms p99 3.16 ms p99.9 6.52 ms max 11.44 ms nvme 54->69C throttle+0 (+0s) L1:C-nocow-raw-none-native/ext4 run 3: fdatasync p50 2.57 ms p99 3.16 ms p99.9 6.65 ms max 24.92 ms nvme 59->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/ext4 run 4: fdatasync p50 2.54 ms p99 3.10 ms p99.9 6.59 ms max 12.50 ms nvme 55->70C throttle+0 (+0s) L1:C-nocow-raw-none-native/ext4 run 5: fdatasync p50 2.54 ms p99 3.16 ms p99.9 6.52 ms max 23.07 ms nvme 59->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/ext4-nojournal run 1: fdatasync p50 1.35 ms p99 1.68 ms p99.9 5.14 ms max 23.84 ms nvme 58->69C throttle+0 (+0s) L1:C-nocow-raw-none-native/ext4-nojournal run 2: fdatasync p50 1.35 ms p99 1.68 ms p99.9 5.14 ms max 27.31 ms nvme 56->67C throttle+0 (+0s) L1:C-nocow-raw-none-native/ext4-nojournal run 3: fdatasync p50 1.35 ms p99 1.71 ms p99.9 5.14 ms max 25.06 ms nvme 55->67C throttle+0 (+0s) L1:C-nocow-raw-none-native/ext4-nojournal run 4: fdatasync p50 1.35 ms p99 1.68 ms p99.9 5.14 ms max 16.97 ms nvme 57->68C throttle+0 (+0s) L1:C-nocow-raw-none-native/ext4-nojournal run 5: fdatasync p50 1.35 ms p99 1.96 ms p99.9 5.21 ms max 16.20 ms nvme 55->68C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs run 1: fdatasync p50 2.70 ms p99 3.46 ms p99.9 6.72 ms max 24.34 ms nvme 55->71C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs run 2: fdatasync p50 2.70 ms p99 3.33 ms p99.9 6.78 ms max 23.85 ms nvme 59->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs run 3: fdatasync p50 2.67 ms p99 3.23 ms p99.9 6.72 ms max 14.22 ms nvme 58->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs run 4: fdatasync p50 2.64 ms p99 3.29 ms p99.9 6.72 ms max 23.87 ms nvme 57->71C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs run 5: fdatasync p50 2.67 ms p99 3.69 ms p99.9 7.18 ms max 20.45 ms nvme 57->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs-logbsize run 1: fdatasync p50 2.67 ms p99 3.26 ms p99.9 6.72 ms max 12.89 ms nvme 58->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs-logbsize run 2: fdatasync p50 2.74 ms p99 3.29 ms p99.9 6.72 ms max 15.49 ms nvme 59->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs-logbsize run 3: fdatasync p50 2.67 ms p99 3.46 ms p99.9 6.78 ms max 24.41 ms nvme 60->73C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs-logbsize run 4: fdatasync p50 2.57 ms p99 3.39 ms p99.9 6.52 ms max 11.83 ms nvme 55->70C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs-logbsize run 5: fdatasync p50 2.57 ms p99 3.23 ms p99.9 6.59 ms max 22.03 ms nvme 57->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs-extlog run 1: fdatasync p50 2.70 ms p99 3.42 ms p99.9 6.85 ms max 23.85 ms nvme 56->71C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs-extlog run 2: fdatasync p50 2.74 ms p99 3.33 ms p99.9 6.72 ms max 27.30 ms nvme 56->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs-extlog run 3: fdatasync p50 2.74 ms p99 3.33 ms p99.9 6.72 ms max 14.65 ms nvme 60->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs-extlog run 4: fdatasync p50 2.74 ms p99 3.36 ms p99.9 6.78 ms max 22.80 ms nvme 59->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/xfs-extlog run 5: fdatasync p50 2.74 ms p99 3.85 ms p99.9 7.44 ms max 17.94 ms nvme 57->71C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs run 1: fdatasync p50 3.00 ms p99 4.62 ms p99.9 7.83 ms max 30.64 ms nvme 58->74C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs run 2: fdatasync p50 2.87 ms p99 4.36 ms p99.9 8.03 ms max 14.62 ms nvme 53->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs run 3: fdatasync p50 2.83 ms p99 3.95 ms p99.9 7.70 ms max 28.60 ms nvme 55->73C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs run 4: fdatasync p50 2.93 ms p99 4.49 ms p99.9 7.37 ms max 23.34 ms nvme 57->74C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs run 5: fdatasync p50 2.97 ms p99 7.83 ms p99.9 9.76 ms max 23.44 ms nvme 55->73C throttle+0 (+1s) L1:C-nocow-raw-none-native/btrfs-nocow run 1: fdatasync p50 2.77 ms p99 6.39 ms p99.9 9.76 ms max 33.36 ms nvme 55->72C throttle+0 (+1s) L1:C-nocow-raw-none-native/btrfs-nocow run 2: fdatasync p50 2.77 ms p99 3.98 ms p99.9 9.63 ms max 20.64 ms nvme 54->72C throttle+0 (+1s) L1:C-nocow-raw-none-native/btrfs-nocow run 3: fdatasync p50 2.77 ms p99 3.82 ms p99.9 7.11 ms max 26.38 ms nvme 56->73C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs-nocow run 4: fdatasync p50 2.77 ms p99 3.82 ms p99.9 7.63 ms max 23.81 ms nvme 55->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs-nocow run 5: fdatasync p50 2.77 ms p99 3.92 ms p99.9 7.05 ms max 13.64 ms nvme 55->72C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs-notreelog run 1: fdatasync p50 3.19 ms p99 4.82 ms p99.9 8.09 ms max 22.77 ms nvme 55->74C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs-notreelog run 2: fdatasync p50 3.23 ms p99 5.14 ms p99.9 8.16 ms max 19.19 ms nvme 58->75C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs-notreelog run 3: fdatasync p50 3.26 ms p99 5.01 ms p99.9 8.45 ms max 17.13 ms nvme 59->75C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs-notreelog run 4: fdatasync p50 3.26 ms p99 4.95 ms p99.9 7.57 ms max 19.71 ms nvme 58->75C throttle+0 (+0s) L1:C-nocow-raw-none-native/btrfs-notreelog run 5: fdatasync p50 3.23 ms p99 4.75 ms p99.9 7.44 ms max 16.76 ms nvme 57->74C throttle+0 (+0s) === 14:04:00 L2: resume saturn, stop its kinc nodes L2:L2-best/rawdev run 1: fdatasync p50 1.45 ms p99 2.21 ms p99.9 5.21 ms max 9.80 ms nvme 60->66C throttle+0 (+0s) L2:L2-best/rawdev run 2: fdatasync p50 0.85 ms p99 1.30 ms p99.9 4.42 ms max 5.99 ms nvme 57->67C throttle+0 (+0s) L2:L2-best/rawdev run 3: fdatasync p50 0.82 ms p99 1.24 ms p99.9 3.39 ms max 5.02 ms nvme 55->66C throttle+0 (+0s) L2:L2-best/rawdev run 4: fdatasync p50 0.83 ms p99 1.27 ms p99.9 3.26 ms max 4.77 ms nvme 55->65C throttle+0 (+0s) L2:L2-best/rawdev run 5: fdatasync p50 0.86 ms p99 1.35 ms p99.9 4.42 ms max 6.06 ms nvme 54->65C throttle+0 (+0s) L2:L2-best/xfs run 1: fdatasync p50 1.74 ms p99 2.93 ms p99.9 5.67 ms max 8.20 ms nvme 61->70C throttle+0 (+0s) L2:L2-best/xfs run 2: fdatasync p50 1.71 ms p99 2.83 ms p99.9 6.19 ms max 14.89 ms nvme 56->67C throttle+0 (+0s) L2:L2-best/xfs run 3: fdatasync p50 1.74 ms p99 3.00 ms p99.9 6.06 ms max 12.66 ms nvme 56->66C throttle+0 (+0s) L2:L2-best/xfs run 4: fdatasync p50 1.79 ms p99 3.13 ms p99.9 5.54 ms max 14.15 ms nvme 57->66C throttle+0 (+0s) L2:L2-best/xfs run 5: fdatasync p50 1.70 ms p99 2.87 ms p99.9 6.13 ms max 11.92 ms nvme 56->68C throttle+0 (+0s) L2:L2-best/ext4 run 1: fdatasync p50 3.29 ms p99 5.01 ms p99.9 8.45 ms max 17.39 ms nvme 56->71C throttle+0 (+0s) L2:L2-best/ext4 run 2: fdatasync p50 1.89 ms p99 2.90 ms p99.9 5.93 ms max 10.17 ms nvme 58->70C throttle+0 (+0s) L2:L2-best/ext4 run 3: fdatasync p50 1.84 ms p99 3.13 ms p99.9 6.00 ms max 20.38 ms nvme 57->70C throttle+0 (+0s) L2:L2-best/ext4 run 4: fdatasync p50 1.79 ms p99 2.67 ms p99.9 5.93 ms max 11.83 ms nvme 58->69C throttle+0 (+0s) L2:L2-best/ext4 run 5: fdatasync p50 1.81 ms p99 2.83 ms p99.9 6.00 ms max 22.08 ms nvme 57->69C throttle+0 (+0s) L2:L2-current/rawdev run 1: fdatasync p50 3.39 ms p99 7.37 ms p99.9 9.76 ms max 37.45 ms nvme 58->73C throttle+0 (+0s) L2:L2-current/rawdev run 2: fdatasync p50 1.88 ms p99 3.06 ms p99.9 6.39 ms max 36.94 ms nvme 59->72C throttle+0 (+0s) L2:L2-current/rawdev run 3: fdatasync p50 2.02 ms p99 3.03 ms p99.9 6.19 ms max 15.10 ms nvme 59->72C throttle+0 (+0s) L2:L2-current/rawdev run 4: fdatasync p50 2.01 ms p99 3.03 ms p99.9 6.19 ms max 9.28 ms nvme 59->72C throttle+0 (+0s) L2:L2-current/rawdev run 5: fdatasync p50 2.04 ms p99 3.03 ms p99.9 6.39 ms max 42.38 ms nvme 59->68C throttle+0 (+0s) L2:L2-current/btrfs run 1: fdatasync p50 7.24 ms p99 15.01 ms p99.9 19.53 ms max 58.29 ms nvme 57->78C throttle+0 (+0s) L2:L2-current/btrfs run 2: fdatasync p50 4.95 ms p99 8.85 ms p99.9 11.99 ms max 59.54 ms nvme 59->76C throttle+0 (+0s) L2:L2-current/btrfs run 3: fdatasync p50 5.01 ms p99 8.85 ms p99.9 11.86 ms max 70.61 ms nvme 56->75C throttle+0 (+0s) L2:L2-current/btrfs run 4: fdatasync p50 4.95 ms p99 8.85 ms p99.9 12.26 ms max 64.72 ms nvme 57->77C throttle+0 (+0s) L2:L2-current/btrfs run 5: fdatasync p50 4.95 ms p99 8.98 ms p99.9 12.26 ms max 47.16 ms nvme 58->77C throttle+0 (+0s) L2:L2-current/ext4 run 1: fdatasync p50 7.90 ms p99 13.17 ms p99.9 17.69 ms max 82.03 ms nvme 58->76C throttle+0 (+0s) L2:L2-current/ext4 run 2: fdatasync p50 4.23 ms p99 7.05 ms p99.9 9.76 ms max 57.52 ms nvme 59->74C throttle+0 (+0s) L2:L2-current/ext4 run 3: fdatasync p50 4.23 ms p99 6.32 ms p99.9 9.50 ms max 60.91 ms nvme 59->74C throttle+0 (+0s) L2:L2-current/ext4 run 4: fdatasync p50 4.23 ms p99 6.52 ms p99.9 9.24 ms max 61.48 ms nvme 55->74C throttle+0 (+0s) L2:L2-current/ext4 run 5: fdatasync p50 4.29 ms p99 6.72 ms p99.9 10.55 ms max 34.70 ms nvme 60->75C throttle+0 (+0s) L2:L2-nocow/rawdev run 1: fdatasync p50 3.36 ms p99 4.82 ms p99.9 7.70 ms max 17.93 ms nvme 56->73C throttle+0 (+0s) L2:L2-nocow/rawdev run 2: fdatasync p50 1.74 ms p99 2.83 ms p99.9 5.87 ms max 21.46 ms nvme 55->70C throttle+0 (+0s) L2:L2-nocow/rawdev run 3: fdatasync p50 1.74 ms p99 2.74 ms p99.9 5.93 ms max 26.81 ms nvme 54->69C throttle+0 (+0s) L2:L2-nocow/rawdev run 4: fdatasync p50 1.73 ms p99 2.87 ms p99.9 5.87 ms max 30.30 ms nvme 58->71C throttle+0 (+0s) L2:L2-nocow/rawdev run 5: fdatasync p50 1.73 ms p99 2.90 ms p99.9 6.26 ms max 14.90 ms nvme 58->71C throttle+0 (+0s) L2:L2-nocow/xfs run 1: fdatasync p50 3.69 ms p99 6.65 ms p99.9 9.76 ms max 44.81 ms nvme 58->74C throttle+0 (+0s) L2:L2-nocow/xfs run 2: fdatasync p50 3.72 ms p99 6.65 ms p99.9 9.90 ms max 33.22 ms nvme 56->74C throttle+0 (+0s) L2:L2-nocow/xfs run 3: fdatasync p50 3.72 ms p99 6.46 ms p99.9 10.03 ms max 37.60 ms nvme 58->74C throttle+0 (+0s) L2:L2-nocow/xfs run 4: fdatasync p50 3.69 ms p99 6.52 ms p99.9 9.76 ms max 45.56 ms nvme 55->73C throttle+0 (+0s) L2:L2-nocow/xfs run 5: fdatasync p50 3.69 ms p99 6.72 ms p99.9 9.76 ms max 27.32 ms nvme 56->73C throttle+0 (+0s) === 14:37:30 restore: saturn kinc, resume jupiter === 14:37:34 CLEAN ROUND DONE