<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/openstack/python-swiftclient.git/swiftclient/client.py, branch stable/rocky</title>
<subtitle>opendev.org: openstack/python-swiftclient.git
</subtitle>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/openstack/python-swiftclient.git/'/>
<entry>
<title>Remove unnecessary calls to parse_header_string().</title>
<updated>2018-07-24T18:53:25+00:00</updated>
<author>
<name>Timur Alperovich</name>
<email>timur@timuralp.com</email>
</author>
<published>2018-07-24T18:53:25+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/openstack/python-swiftclient.git/commit/?id=e28b12a1f2156b2e8a790290ae6da08adde3875f'/>
<id>e28b12a1f2156b2e8a790290ae6da08adde3875f</id>
<content type='text'>
Since we define the getheader() method on the response from
HTTPConnection, we don't have to call parse_header_string, as the values
will already be converted properly.

Change-Id: Ia81e8674b828b3ff1f014454126b469e41adfc23
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Since we define the getheader() method on the response from
HTTPConnection, we don't have to call parse_header_string, as the values
will already be converted properly.

Change-Id: Ia81e8674b828b3ff1f014454126b469e41adfc23
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge "Properly handle unicode headers."</title>
<updated>2018-07-24T06:52:54+00:00</updated>
<author>
<name>Zuul</name>
<email>zuul@review.openstack.org</email>
</author>
<published>2018-07-24T06:52:54+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/openstack/python-swiftclient.git/commit/?id=6872b95560ab77b200fde9e322af30bba99c99eb'/>
<id>6872b95560ab77b200fde9e322af30bba99c99eb</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Properly handle unicode headers.</title>
<updated>2018-07-23T21:38:40+00:00</updated>
<author>
<name>Timur Alperovich</name>
<email>timur@timuralp.com</email>
</author>
<published>2018-04-17T21:36:57+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/openstack/python-swiftclient.git/commit/?id=f4a2b16c2cc65410765abdff7a45532305a4548f'/>
<id>f4a2b16c2cc65410765abdff7a45532305a4548f</id>
<content type='text'>
Fix unicode handling in Python 3 and Python 2. There are currently two
failure modes. In python 2, swiftclient fails to log in debug mode if
the account name has a non-ASCII character. This is because the account
name will appear in the storage URL, which we attempt to pass to the
logger as a byte string (whereas it should be a unicode string). This
patch changes the behavior to convert the path strings into unicode by
calling the parse_header_string() function.

The second failure mode is with Python 3, where http_lib returns headers
that are latin-1 encoded, but swiftclient expects UTF-8. The patch
automatically converts headers from latin-1 (iso-8859-1) to UTF-8, so
that we can properly handle non-ASCII headers in responses.

Change-Id: Ifa7f3d5af71bde8127129f1f8603772d80d063c1
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Fix unicode handling in Python 3 and Python 2. There are currently two
failure modes. In python 2, swiftclient fails to log in debug mode if
the account name has a non-ASCII character. This is because the account
name will appear in the storage URL, which we attempt to pass to the
logger as a byte string (whereas it should be a unicode string). This
patch changes the behavior to convert the path strings into unicode by
calling the parse_header_string() function.

The second failure mode is with Python 3, where http_lib returns headers
that are latin-1 encoded, but swiftclient expects UTF-8. The patch
automatically converts headers from latin-1 (iso-8859-1) to UTF-8, so
that we can properly handle non-ASCII headers in responses.

Change-Id: Ifa7f3d5af71bde8127129f1f8603772d80d063c1
</pre>
</div>
</content>
</entry>
<entry>
<title>Add close() to _RetryBody.</title>
<updated>2018-07-20T19:31:06+00:00</updated>
<author>
<name>Timur Alperovich</name>
<email>timuralp@swiftstack.com</email>
</author>
<published>2018-01-23T02:22:04+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/openstack/python-swiftclient.git/commit/?id=02b08aaa10b3655aee95e6069b5116db0524a268'/>
<id>02b08aaa10b3655aee95e6069b5116db0524a268</id>
<content type='text'>
Allows clients to give up on reading the rest of the server response,
if they so choose.

Change-Id: Iccc95b1b5e7d066470966ee0c62a3beb260846e5
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Allows clients to give up on reading the rest of the server response,
if they so choose.

Change-Id: Iccc95b1b5e7d066470966ee0c62a3beb260846e5
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge "Stop mutating header dicts"</title>
<updated>2018-07-17T01:05:14+00:00</updated>
<author>
<name>Zuul</name>
<email>zuul@review.openstack.org</email>
</author>
<published>2018-07-17T01:05:14+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/openstack/python-swiftclient.git/commit/?id=23d29eda8d03785c22d67930e21a0ba4098ba23c'/>
<id>23d29eda8d03785c22d67930e21a0ba4098ba23c</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Make OS_AUTH_URL work in DevStack by default</title>
<updated>2018-06-20T13:25:46+00:00</updated>
<author>
<name>Clay Gerrard</name>
<email>clay.gerrard@gmail.com</email>
</author>
<published>2017-06-13T18:14:52+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/openstack/python-swiftclient.git/commit/?id=11908250549fed549169519d411394296fb4a57b'/>
<id>11908250549fed549169519d411394296fb4a57b</id>
<content type='text'>
An earlier change added support for versionless authurls, but the
huristic to detect them didn't work for some configurations I've
encountered.

Now we use a little bit tighter pattern matching and support auth_url
values with more than one path component.

Change-Id: I5a99c7b4e957ee7c8a5b5470477db49ab2ddba4b
Related-Change-Id: If7ecb67776cb77828f93ad8278cc5040015216b7
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
An earlier change added support for versionless authurls, but the
huristic to detect them didn't work for some configurations I've
encountered.

Now we use a little bit tighter pattern matching and support auth_url
values with more than one path component.

Change-Id: I5a99c7b4e957ee7c8a5b5470477db49ab2ddba4b
Related-Change-Id: If7ecb67776cb77828f93ad8278cc5040015216b7
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge "Use a valid default for auth_version"</title>
<updated>2018-06-06T04:11:31+00:00</updated>
<author>
<name>Zuul</name>
<email>zuul@review.openstack.org</email>
</author>
<published>2018-06-06T04:11:31+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/openstack/python-swiftclient.git/commit/?id=11c5bb887fb4ffff24eb7ad60bec7be79148775e'/>
<id>11c5bb887fb4ffff24eb7ad60bec7be79148775e</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>Make swiftclient respect region_name when using sessions</title>
<updated>2018-05-16T23:04:48+00:00</updated>
<author>
<name>Erik Olof Gunnar Andersson</name>
<email>eandersson@blizzard.com</email>
</author>
<published>2018-05-16T23:03:57+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/openstack/python-swiftclient.git/commit/?id=1971ef880ff225379d4a91f00f89f323a1605eeb'/>
<id>1971ef880ff225379d4a91f00f89f323a1605eeb</id>
<content type='text'>
Change-Id: I94aca6f1120c34616562be7345f0e5aa51a69499
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Change-Id: I94aca6f1120c34616562be7345f0e5aa51a69499
</pre>
</div>
</content>
</entry>
<entry>
<title>Use a valid default for auth_version</title>
<updated>2018-05-04T18:31:03+00:00</updated>
<author>
<name>Pete Zaitcev</name>
<email>zaitcev@kotori.zaitcev.us</email>
</author>
<published>2018-05-04T18:31:03+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/openstack/python-swiftclient.git/commit/?id=7a13754eebb15a3c7fa507ad1d58e3c4460e302c'/>
<id>7a13754eebb15a3c7fa507ad1d58e3c4460e302c</id>
<content type='text'>
The valid set of values for auth_version does not include
values starting with the 'v'.

In this particular function, the auth_version variable is
only used for comparisons with v3. So, the code worked
correctly. However, let's clean this up in order to reduce
review confusion and defuse possible future landmine in case
of code changes.

Change-Id: I671016d7992a1922b786b4eb8876b3fbb2532e15
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The valid set of values for auth_version does not include
values starting with the 'v'.

In this particular function, the auth_version variable is
only used for comparisons with v3. So, the code worked
correctly. However, let's clean this up in order to reduce
review confusion and defuse possible future landmine in case
of code changes.

Change-Id: I671016d7992a1922b786b4eb8876b3fbb2532e15
</pre>
</div>
</content>
</entry>
<entry>
<title>Add force auth retry mode in swiftclient</title>
<updated>2018-03-13T03:29:48+00:00</updated>
<author>
<name>Kota Tsuyuzaki</name>
<email>tsuyuzaki.kota@lab.ntt.co.jp</email>
</author>
<published>2018-03-12T08:54:17+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/openstack/python-swiftclient.git/commit/?id=e65070964c7b1e04119c87e5f344d39358780d18'/>
<id>e65070964c7b1e04119c87e5f344d39358780d18</id>
<content type='text'>
This patch attemps to add an option to force get_auth call while retrying
an operation even if it gets errors other than 401 Unauthorized.

Why we need this:
The main reason why we need this is current python-swiftclient requests could
never get succeeded under certion situation using third party proxies/load balancers
between the client and swift-proxy server. I think, it would be general situation
of the use case.

Specifically describing nginx case, the nginx can close the socket from the client
when the response code from swift is not 2xx series. In default, nginx can wait the
buffers from the client for a while (default 30s)[1] but after the time past, nginx
will close the socket immediately. Unfortunately, if python-swiftclient has still been
sending the data into the socket, python-swiftclient will get socket error (EPIPE,
BrokenPipe). From the swiftclient perspective, this is absolutely not an auth error,
so current python-swiftclient will continue to retry without re-auth.
However, if the root cause is sort of 401 (i.e. nginx got 401 unauthorized from the
swift-proxy because of token expiration), swiftclient will loop 401 -&gt; EPIPE -&gt; 401...
until it consume the max retry times.

In particlar, less time to live of the token and multipart object upload with large
segments could not get succeeded as below:

Connection Model:

python-swiftclient -&gt; nginx -&gt; swift-proxy -&gt; swift-backend

Case: Try to create slo with large segments and the auth token expired with 1 hour

1. client create a connection to nginx with successful response from swift-proxy and its auth
2. client continue to put large segment objects
   (e.g. 1~5GB for each and the total would 20~30GB, i.e. 20~30 segments)
3. after some of segments uploaded, 1 hour past but client is still trying to
   send remaining segment objects.
4. nginx got 401 from swift-proxy for a request and wait that the connection is closed
   from the client but timeout past because the python-swiftclient is still sending much data
   into the socket before reading the 401 response.
5. client got socket error because nginx closed the connection during sending the buffer.
6. client retries a new connection to nginx without re-auth...

&lt;loop 4-6&gt;

7. finally python-swiftclient failed with socket error (Broken Pipe)

In operational perspective, setting longer timeout for lingering close would be an option but
it's not complete solution because any other proxy/LB may not support the options.

If we actually do THE RIGHT THING in python-swiftclient, we should send expects: 100-continue
header and handle the first response to re-auth correctly.

HOWEVER, the current python's httplib and requests module used by python-swiftclient doesn't
support expects: 100-continue header [2] and the thread proposed a fix [3] is not super active.
And we know the reason we depends on the library is to fix a security issue that existed
in older python-swiftclient [4] so that we should touch around it super carefully.

In the reality, as the hot fix, this patch try to mitigate the unfortunate situation
described above WITHOUT 100-continue fix, just users can force to re-auth when any errors
occurred during the retries that can be accepted in the upstream.

1: http://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_close
2: https://github.com/requests/requests/issues/713
3: https://bugs.python.org/issue1346874
4: https://review.openstack.org/#/c/69187/

Change-Id: I3470b56e3f9cf9cdb8c2fc2a94b2c551927a3440
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This patch attemps to add an option to force get_auth call while retrying
an operation even if it gets errors other than 401 Unauthorized.

Why we need this:
The main reason why we need this is current python-swiftclient requests could
never get succeeded under certion situation using third party proxies/load balancers
between the client and swift-proxy server. I think, it would be general situation
of the use case.

Specifically describing nginx case, the nginx can close the socket from the client
when the response code from swift is not 2xx series. In default, nginx can wait the
buffers from the client for a while (default 30s)[1] but after the time past, nginx
will close the socket immediately. Unfortunately, if python-swiftclient has still been
sending the data into the socket, python-swiftclient will get socket error (EPIPE,
BrokenPipe). From the swiftclient perspective, this is absolutely not an auth error,
so current python-swiftclient will continue to retry without re-auth.
However, if the root cause is sort of 401 (i.e. nginx got 401 unauthorized from the
swift-proxy because of token expiration), swiftclient will loop 401 -&gt; EPIPE -&gt; 401...
until it consume the max retry times.

In particlar, less time to live of the token and multipart object upload with large
segments could not get succeeded as below:

Connection Model:

python-swiftclient -&gt; nginx -&gt; swift-proxy -&gt; swift-backend

Case: Try to create slo with large segments and the auth token expired with 1 hour

1. client create a connection to nginx with successful response from swift-proxy and its auth
2. client continue to put large segment objects
   (e.g. 1~5GB for each and the total would 20~30GB, i.e. 20~30 segments)
3. after some of segments uploaded, 1 hour past but client is still trying to
   send remaining segment objects.
4. nginx got 401 from swift-proxy for a request and wait that the connection is closed
   from the client but timeout past because the python-swiftclient is still sending much data
   into the socket before reading the 401 response.
5. client got socket error because nginx closed the connection during sending the buffer.
6. client retries a new connection to nginx without re-auth...

&lt;loop 4-6&gt;

7. finally python-swiftclient failed with socket error (Broken Pipe)

In operational perspective, setting longer timeout for lingering close would be an option but
it's not complete solution because any other proxy/LB may not support the options.

If we actually do THE RIGHT THING in python-swiftclient, we should send expects: 100-continue
header and handle the first response to re-auth correctly.

HOWEVER, the current python's httplib and requests module used by python-swiftclient doesn't
support expects: 100-continue header [2] and the thread proposed a fix [3] is not super active.
And we know the reason we depends on the library is to fix a security issue that existed
in older python-swiftclient [4] so that we should touch around it super carefully.

In the reality, as the hot fix, this patch try to mitigate the unfortunate situation
described above WITHOUT 100-continue fix, just users can force to re-auth when any errors
occurred during the retries that can be accepted in the upstream.

1: http://nginx.org/en/docs/http/ngx_http_core_module.html#lingering_close
2: https://github.com/requests/requests/issues/713
3: https://bugs.python.org/issue1346874
4: https://review.openstack.org/#/c/69187/

Change-Id: I3470b56e3f9cf9cdb8c2fc2a94b2c551927a3440
</pre>
</div>
</content>
</entry>
</feed>
