HDDS-15550 Added an option to bypass bufferpool usage for large writes

[yandrey321]

What changes were proposed in this pull request?

Added an option to bypass bufferpool usage for large writes: if write is 1+MB in size the buffer pool usage might overweight the benefit of larger writes, instead the client can send the client's buffer to netty directly and wait for async send completion. With a constant cost of the send/processing writes on the backend, the approach without pool shows better throughput and better scaling with increasing number of threads.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15550

How was this patch tested?

Run synthetic benchmark that trying to mimic 'ozone dreon dfsg' workflow.

Results for running on ARM based mac with 14 CPUs:

--- scaling summary writeSize=4096KB ---

threads	pooled MB/s	ref MB/s	ref/pooled	pool eff.	ref eff.

   1 |      7,005.2 |      7,075.5 |      1.01x |      1.00 |      1.00
   2 |     13,754.9 |     14,253.6 |      1.04x |      0.98 |      1.01
   7 |     37,531.7 |     43,381.8 |      1.16x |      0.77 |      0.88
  14 |     32,056.2 |     36,420.4 |      1.14x |      0.33 |      0.37
  28 |     27,205.8 |     47,208.2 |      1.74x |      0.14 |      0.24

--- scaling summary writeSize=3072KB ---

threads	pooled MB/s	ref MB/s	ref/pooled	pool eff.	ref eff.

   1 |      7,158.3 |      7,434.5 |      1.04x |      1.00 |      1.00
   2 |     13,550.2 |     10,898.3 |      0.80x |      0.95 |      0.73
   7 |     44,841.0 |     47,059.0 |      1.05x |      0.89 |      0.90
  14 |     45,307.1 |     63,610.4 |      1.40x |      0.45 |      0.61
  28 |     43,916.4 |     50,250.6 |      1.14x |      0.22 |      0.24

--- scaling summary writeSize=2048KB ---

threads	pooled MB/s	ref MB/s	ref/pooled	pool eff.	ref eff.

   1 |      7,174.5 |      7,374.9 |      1.03x |      1.00 |      1.00
   2 |     13,896.8 |     14,049.3 |      1.01x |      0.97 |      0.95
   7 |     40,723.9 |     46,376.9 |      1.14x |      0.81 |      0.90
  14 |     39,079.7 |     66,200.4 |      1.69x |      0.39 |      0.64
  28 |     35,713.0 |     66,226.4 |      1.85x |      0.18 |      0.32

--- scaling summary writeSize=1024KB ---

threads	pooled MB/s	ref MB/s	ref/pooled	pool eff.	ref eff.

   1 |      7,081.8 |      7,038.9 |      0.99x |      1.00 |      1.00
   2 |     13,808.5 |     13,238.7 |      0.96x |      0.97 |      0.94
   7 |     42,239.6 |     43,045.3 |      1.02x |      0.85 |      0.87
  14 |     41,534.1 |     64,251.4 |      1.55x |      0.42 |      0.65
  28 |     41,600.6 |     60,394.1 |      1.45x |      0.21 |      0.31