ETS Decentralized Counters Benchmark Results (64 hardware threads)

Results

The benchmark has been run with two different scheduler threads pinning strategies (the +sbt s strategy and the +sbt tnnps strategy). Please see http://erlang.org/doc/man/erl.html for the documentation of +sbt s and +sbt tnnps.

Machine Configuration

Machine:
Microsoft Azure VM instance: Standard D64s v3 (64 vcpus, 256 GB memory):

2 * Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz (16 cores with hyper-threading)
2 NUMA nodes
32 cores
64 hardware threads
256GB ram

Operating System:

Description:	Ubuntu 18.04.2 LTS
Linux version:	4.18.0-1014-azure

Erlang Compilation Options

The option --with-ets-write-concurrency-locks=256 was passed to the Erlang configure script before compiling. This will result in that the terms that are stored in ETS tables of type set, bag and duplicate_bag are divided between 256 locks (the default value for this parameter is currently 64). This option was changed from the default to make the impact of decentralized counters larger.

      
  ./configure --with-ets-write-concurrency-locks=256

Benchmark configuration

The benchmark is implemented in the function ets_SUITE:throughput_benchmark/1 (otp/lib/stdlib/test/ets_SUITE.erl). The following benchmark configuration was passed to the function ets_SUITE:throughput_benchmark/1 to get the results:

      
    throughput_benchmark(
      #ets_throughput_bench_config{
         benchmark_duration_ms = 18000,
         recover_time_ms = 1000,
         thread_counts = [1,2,4,8,16,32,64],
         key_ranges = [1000000],
         table_types =
             [
              [set, public, {decentralized_counters, false}, {write_concurrency, true}, {read_concurrency, true}],
              [set, public, {decentralized_counters, true}, {write_concurrency, true}, {read_concurrency, true}],
	      [ordered_set, public, {decentralized_counters, false}, {write_concurrency, true}, {read_concurrency, true}],
              [ordered_set, public, {decentralized_counters, true}, {write_concurrency, true}, {read_concurrency, true}]
             ],
         scenarios =
              [
               [
                {0.5, insert},
                {0.5, delete}
               ],
               [
                {0.1, insert},
                {0.1, delete},
                {0.8, lookup}
               ],
               [
                {0.01, insert},
                {0.01, delete},
                {0.98, lookup}
               ],
               [
                {1.0, lookup}
               ],
               [
                {0.1, insert},
                {0.1, delete},
                {0.4, lookup},
                {0.4, nextseq10}
               ],
               [
                {0.1, insert},
                {0.1, delete},
                {0.4, lookup},
                {0.4, nextseq1000}
               ],
               [
                {0.1, insert},
                {0.1, delete},
                {0.79, lookup},
                {0.01, selectAll}
               ],
               [
                {0.1, insert},
                {0.1, delete},
                {0.79, lookup},
                {0.01, partial_select1000}
               ]
              ],
         print_result_paths_fun = fun stdout_notify_res/2})

Run the Benchmark Again

      
        git clone https://github.com/kjellwinblad/otp
        cd otp
        git checkout origin/kjell/stdlib/ets_hash_decentralized_counters_benchmark_prep_2 -b kjell/stdlib/ets_hash_decentralized_counters_benchmark_prep_2
        ./run_bench.sh

The ./run_bench.sh script assumes that the system runs Ubuntu 18.04 and that the user has sudo rights.