<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/haproxy.git/include/haproxy/queue.h, branch master</title>
<subtitle>github.com: haproxy/haproxy.git
</subtitle>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/haproxy.git/'/>
<entry>
<title>CLEANUP: assorted typo fixes in the code and comments</title>
<updated>2022-10-30T16:17:56+00:00</updated>
<author>
<name>Ilya Shipitsin</name>
<email>chipitsine@gmail.com</email>
</author>
<published>2022-10-29T04:34:32+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/haproxy.git/commit/?id=4a689dad0398b103949d7ffee3476dfcaf21e6fc'/>
<id>4a689dad0398b103949d7ffee3476dfcaf21e6fc</id>
<content type='text'>
This is 32nd iteration of typo fixes
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This is 32nd iteration of typo fixes
</pre>
</div>
</content>
</entry>
<entry>
<title>MINOR: queue: add a pointer to the server and the proxy in the queue</title>
<updated>2021-06-24T08:52:31+00:00</updated>
<author>
<name>Willy Tarreau</name>
<email>w@1wt.eu</email>
</author>
<published>2021-06-23T14:11:02+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/haproxy.git/commit/?id=cdc83e01922fa9bf4a9442dbaca9a2fd9e881c12'/>
<id>cdc83e01922fa9bf4a9442dbaca9a2fd9e881c12</id>
<content type='text'>
A queue is specific to a server or a proxy, so we don't need to place
this distinction inside all pendconns, it can be in the queue itself.
This commit adds the relevant fields "px" and "sv" into the struct
queue, and initializes them accordingly.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
A queue is specific to a server or a proxy, so we don't need to place
this distinction inside all pendconns, it can be in the queue itself.
This commit adds the relevant fields "px" and "sv" into the struct
queue, and initializes them accordingly.
</pre>
</div>
</content>
</entry>
<entry>
<title>MINOR: queue: add queue_init() to initialize a queue</title>
<updated>2021-06-24T08:52:31+00:00</updated>
<author>
<name>Willy Tarreau</name>
<email>w@1wt.eu</email>
</author>
<published>2021-06-23T13:08:06+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/haproxy.git/commit/?id=df3b0cbe31bdb15495a040437496cbdff5d494d4'/>
<id>df3b0cbe31bdb15495a040437496cbdff5d494d4</id>
<content type='text'>
This is better and cleaner than open-coding this in the server and
proxy code, where it has all chances of becoming wrong once forgotten.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This is better and cleaner than open-coding this in the server and
proxy code, where it has all chances of becoming wrong once forgotten.
</pre>
</div>
</content>
</entry>
<entry>
<title>MEDIUM: queue: simplify again the process_srv_queue() API (v2)</title>
<updated>2021-06-24T08:52:31+00:00</updated>
<author>
<name>Willy Tarreau</name>
<email>w@1wt.eu</email>
</author>
<published>2021-06-22T16:47:51+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/haproxy.git/commit/?id=9ab78293bfc76a35e0d8ad02c4b777b6ede666f9'/>
<id>9ab78293bfc76a35e0d8ad02c4b777b6ede666f9</id>
<content type='text'>
This basically undoes the API changes that were performed by commit
0274286dd ("BUG/MAJOR: server: fix deadlock when changing maxconn via
agent-check") to address the deadlock issue: since process_srv_queue()
doesn't use the server lock anymore, it doesn't need the "server_locked"
argument, so let's get rid of it before it gets used again.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This basically undoes the API changes that were performed by commit
0274286dd ("BUG/MAJOR: server: fix deadlock when changing maxconn via
agent-check") to address the deadlock issue: since process_srv_queue()
doesn't use the server lock anymore, it doesn't need the "server_locked"
argument, so let's get rid of it before it gets used again.
</pre>
</div>
</content>
</entry>
<entry>
<title>Revert "MEDIUM: queue: simplify again the process_srv_queue() API"</title>
<updated>2021-06-24T05:22:18+00:00</updated>
<author>
<name>Willy Tarreau</name>
<email>w@1wt.eu</email>
</author>
<published>2021-06-24T05:22:18+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/haproxy.git/commit/?id=ccd85a3e08fdd260e15c1dd8380543a1e47d5c9f'/>
<id>ccd85a3e08fdd260e15c1dd8380543a1e47d5c9f</id>
<content type='text'>
This reverts commit c83e45e9b001591633188a480a896c935d3c9625.

The recent changes since 5304669e1 MEDIUM: queue: make
pendconn_process_next_strm() only return the pendconn opened a tiny race
condition between stream_free() and process_srv_queue(), as the pendconn
is accessed outside of the lock, possibly while it's being freed. A
different approach is required.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This reverts commit c83e45e9b001591633188a480a896c935d3c9625.

The recent changes since 5304669e1 MEDIUM: queue: make
pendconn_process_next_strm() only return the pendconn opened a tiny race
condition between stream_free() and process_srv_queue(), as the pendconn
is accessed outside of the lock, possibly while it's being freed. A
different approach is required.
</pre>
</div>
</content>
</entry>
<entry>
<title>MEDIUM: queue: simplify again the process_srv_queue() API</title>
<updated>2021-06-22T16:57:15+00:00</updated>
<author>
<name>Willy Tarreau</name>
<email>w@1wt.eu</email>
</author>
<published>2021-06-22T16:47:51+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/haproxy.git/commit/?id=c83e45e9b001591633188a480a896c935d3c9625'/>
<id>c83e45e9b001591633188a480a896c935d3c9625</id>
<content type='text'>
This basically undoes the API changes that were performed by commit
0274286dd ("BUG/MAJOR: server: fix deadlock when changing maxconn via
agent-check") to address the deadlock issue: since process_srv_queue()
doesn't use the server lock anymore, it doesn't need the "server_locked"
argument, so let's get rid of it before it gets used again.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This basically undoes the API changes that were performed by commit
0274286dd ("BUG/MAJOR: server: fix deadlock when changing maxconn via
agent-check") to address the deadlock issue: since process_srv_queue()
doesn't use the server lock anymore, it doesn't need the "server_locked"
argument, so let's get rid of it before it gets used again.
</pre>
</div>
</content>
</entry>
<entry>
<title>MINOR: server: replace the pendconns-related stuff with a struct queue</title>
<updated>2021-06-22T16:43:14+00:00</updated>
<author>
<name>Willy Tarreau</name>
<email>w@1wt.eu</email>
</author>
<published>2021-06-18T07:30:30+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/haproxy.git/commit/?id=a05704582cbec36029fb3d7d70387c27c54841bf'/>
<id>a05704582cbec36029fb3d7d70387c27c54841bf</id>
<content type='text'>
Just like for proxies, all three elements (pendconns, nbpend, queue_idx)
were moved to struct queue.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Just like for proxies, all three elements (pendconns, nbpend, queue_idx)
were moved to struct queue.
</pre>
</div>
</content>
</entry>
<entry>
<title>MINOR: proxy: replace the pendconns-related stuff with a struct queue</title>
<updated>2021-06-22T16:43:14+00:00</updated>
<author>
<name>Willy Tarreau</name>
<email>w@1wt.eu</email>
</author>
<published>2021-06-18T07:22:21+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/haproxy.git/commit/?id=7f3c1df248af20eeb70a93a5df39665ae3df62dc'/>
<id>7f3c1df248af20eeb70a93a5df39665ae3df62dc</id>
<content type='text'>
All three elements (pendconns, nbpend, queue_idx) were moved to struct
queue.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
All three elements (pendconns, nbpend, queue_idx) were moved to struct
queue.
</pre>
</div>
</content>
</entry>
<entry>
<title>BUG/MAJOR: server: fix deadlock when changing maxconn via agent-check</title>
<updated>2021-06-22T09:39:20+00:00</updated>
<author>
<name>Amaury Denoyelle</name>
<email>adenoyelle@haproxy.com</email>
</author>
<published>2021-06-18T09:11:36+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/haproxy.git/commit/?id=0274286dd3768c0d5e58588a1cb7e7e710fbc9d4'/>
<id>0274286dd3768c0d5e58588a1cb7e7e710fbc9d4</id>
<content type='text'>
The server_parse_maxconn_change_request locks the server lock. However,
this function can be called via agent-checks or lua code which already
lock it. This bug has been introduced by the following commit :

  commit 79a88ba3d09f7e2b73ae27cb5d24cc087a548fa6
  BUG/MAJOR: server: prevent deadlock when using 'set maxconn server'

This commit tried to fix another deadlock with can occur because
previoulsy server_parse_maxconn_change_request requires the server lock
to be held. However, it may call internally process_srv_queue which also
locks the server lock. The locking policy has thus been updated. The fix
is functional for the CLI 'set maxconn' but fails to address the
agent-check / lua counterparts.

This new issue is fixed in two steps :
- changes from the above commit have been reverted. This means that
  server_parse_maxconn_change_request must again be called with the
  server lock.

- to counter the deadlock fixed by the above commit, process_srv_queue
  now takes an argument to render the server locking optional if the
  caller already held it. This is only used by
  server_parse_maxconn_change_request.

The above commit was subject to backport up to 1.8. Thus this commit
must be backported in every release where it is already present.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The server_parse_maxconn_change_request locks the server lock. However,
this function can be called via agent-checks or lua code which already
lock it. This bug has been introduced by the following commit :

  commit 79a88ba3d09f7e2b73ae27cb5d24cc087a548fa6
  BUG/MAJOR: server: prevent deadlock when using 'set maxconn server'

This commit tried to fix another deadlock with can occur because
previoulsy server_parse_maxconn_change_request requires the server lock
to be held. However, it may call internally process_srv_queue which also
locks the server lock. The locking policy has thus been updated. The fix
is functional for the CLI 'set maxconn' but fails to address the
agent-check / lua counterparts.

This new issue is fixed in two steps :
- changes from the above commit have been reverted. This means that
  server_parse_maxconn_change_request must again be called with the
  server lock.

- to counter the deadlock fixed by the above commit, process_srv_queue
  now takes an argument to render the server locking optional if the
  caller already held it. This is only used by
  server_parse_maxconn_change_request.

The above commit was subject to backport up to 1.8. Thus this commit
must be backported in every release where it is already present.
</pre>
</div>
</content>
</entry>
<entry>
<title>Revert "OPTIM: queue: don't call pendconn_unlink() when the pendconn is not queued"</title>
<updated>2020-10-23T07:21:55+00:00</updated>
<author>
<name>Willy Tarreau</name>
<email>w@1wt.eu</email>
</author>
<published>2020-10-23T06:57:33+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/haproxy.git/commit/?id=670119955bde36907a0695d1682eae22ed19815e'/>
<id>670119955bde36907a0695d1682eae22ed19815e</id>
<content type='text'>
This reverts commit b7ba1d901174cb1193033f7d967987ef74e89856. Actually
this test had already been removed in the past by commit fac0f645d
("BUG/MEDIUM: queue: make pendconn_cond_unlink() really thread-safe"),
but the condition to reproduce the bug mentioned there was not clear.

Now after analysis and a certain dose of code cleanup, things start to
appear more obvious. what happens is that if we check the presence of
the node in the tree without taking the lock, we can see the NULL at
the instant the node is being unlinked by another thread in
pendconn_process_next_strm() as part of __pendconn_unlink_prx() or
__pendconn_unlink_srv(). Till now there is no issue except that the
pendconn is not removed from the queue during this operation and that
the task is scheduled to be woken up by pendconn_process_next_strm()
with the stream being added to the list of the server's active
connections by __stream_add_srv_conn(). The first thread finishes
faster and gets back to stream_free() faster than the second one
sets the srv_conn on the stream, so stream_free() skips the s-&gt;srv_conn
test and doesn't try to dequeue the freshly queued entry. At the
very least a barrier would be needed there but we can't afford to
free the stream while it's being queued. So there's no other solution
than making sure that either __pendconn_unlink_prx() or
pendconn_cond_unlink() get the entry but never both, which is why the
lock is required around the test. A possible solution would be to set
p-&gt;target before unlinking the entry and using it to complete the test.
This would leave no dead period where the pendconn is not seen as
attached.

It is possible, yet extremely difficult, to reproduce this bug, which
was first noticed in bug #880. Running 100 servers with maxconn 1 and
maxqueue 1 on leastconn and a connect timeout of 30ms under 16 threads
with DEBUG_UAF, with a traffic making the backend's queue oscillate
around zero (typically using 250 connections with a local httpterm
server) may rarely manage to trigger a use-after-free.

No backport is needed.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This reverts commit b7ba1d901174cb1193033f7d967987ef74e89856. Actually
this test had already been removed in the past by commit fac0f645d
("BUG/MEDIUM: queue: make pendconn_cond_unlink() really thread-safe"),
but the condition to reproduce the bug mentioned there was not clear.

Now after analysis and a certain dose of code cleanup, things start to
appear more obvious. what happens is that if we check the presence of
the node in the tree without taking the lock, we can see the NULL at
the instant the node is being unlinked by another thread in
pendconn_process_next_strm() as part of __pendconn_unlink_prx() or
__pendconn_unlink_srv(). Till now there is no issue except that the
pendconn is not removed from the queue during this operation and that
the task is scheduled to be woken up by pendconn_process_next_strm()
with the stream being added to the list of the server's active
connections by __stream_add_srv_conn(). The first thread finishes
faster and gets back to stream_free() faster than the second one
sets the srv_conn on the stream, so stream_free() skips the s-&gt;srv_conn
test and doesn't try to dequeue the freshly queued entry. At the
very least a barrier would be needed there but we can't afford to
free the stream while it's being queued. So there's no other solution
than making sure that either __pendconn_unlink_prx() or
pendconn_cond_unlink() get the entry but never both, which is why the
lock is required around the test. A possible solution would be to set
p-&gt;target before unlinking the entry and using it to complete the test.
This would leave no dead period where the pendconn is not seen as
attached.

It is possible, yet extremely difficult, to reproduce this bug, which
was first noticed in bug #880. Running 100 servers with maxconn 1 and
maxqueue 1 on leastconn and a connect timeout of 30ms under 16 threads
with DEBUG_UAF, with a traffic making the backend's queue oscillate
around zero (typically using 250 connections with a local httpterm
server) may rarely manage to trigger a use-after-free.

No backport is needed.
</pre>
</div>
</content>
</entry>
</feed>
