<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/libgit2.git, branch cmn/static-archive-later</title>
<subtitle>github.com: libgit2/libgit2.git
</subtitle>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/libgit2.git/'/>
<entry>
<title>cmake: move the rule to find static archives close to building clar</title>
<updated>2018-01-07T12:51:26+00:00</updated>
<author>
<name>Carlos Martín Nieto</name>
<email>cmn@dwim.me</email>
</author>
<published>2018-01-07T12:51:26+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/libgit2.git/commit/?id=85e40bbf0938cac5668e810864f6be9619b9e60e'/>
<id>85e40bbf0938cac5668e810864f6be9619b9e60e</id>
<content type='text'>
If we're building static libraries, we want to use that for building our clar
binary. This is done in 49551254 (2017-09-22; cmake: use static dependencies
when building static libgit2) but that commit included the rule too early,
making it affect the search for iconv, meaning we did not find it when we were
building a static libgit2.

Move the rule to just before building clar, after we've included the rules for
building the library itself. This lets us find and link to the dynamic libiconv.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
If we're building static libraries, we want to use that for building our clar
binary. This is done in 49551254 (2017-09-22; cmake: use static dependencies
when building static libgit2) but that commit included the rule too early,
making it affect the search for iconv, meaning we did not find it when we were
building a static libgit2.

Move the rule to just before building clar, after we've included the rules for
building the library itself. This lets us find and link to the dynamic libiconv.
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge pull request #4257 from pks-t/pks/stale-test</title>
<updated>2018-01-03T21:15:16+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@edwardthomson.com</email>
</author>
<published>2018-01-03T21:15:16+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/libgit2.git/commit/?id=eebc5e0d72c86634e6cd6fcc1306c309c02f87af'/>
<id>eebc5e0d72c86634e6cd6fcc1306c309c02f87af</id>
<content type='text'>
Execute stale tests</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Execute stale tests</pre>
</div>
</content>
</entry>
<entry>
<title>Merge pull request #4437 from pks-t/pks/openssl-hash-errors</title>
<updated>2018-01-03T20:57:25+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@edwardthomson.com</email>
</author>
<published>2018-01-03T20:57:25+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/libgit2.git/commit/?id=a223bae5cf7629fb77b2af0b0df9d02c634520c0'/>
<id>a223bae5cf7629fb77b2af0b0df9d02c634520c0</id>
<content type='text'>
hash: openssl: check return values of SHA1_* functions</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
hash: openssl: check return values of SHA1_* functions</pre>
</div>
</content>
</entry>
<entry>
<title>Merge pull request #4462 from pks-t/pks/diff-generated-excessive-stats</title>
<updated>2018-01-03T20:55:06+00:00</updated>
<author>
<name>Edward Thomson</name>
<email>ethomson@edwardthomson.com</email>
</author>
<published>2018-01-03T20:55:06+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/libgit2.git/commit/?id=399c0b194045be1a7f6440347ce3e08ffc1b8584'/>
<id>399c0b194045be1a7f6440347ce3e08ffc1b8584</id>
<content type='text'>
diff_generate: avoid excessive stats of .gitattribute files</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
diff_generate: avoid excessive stats of .gitattribute files</pre>
</div>
</content>
</entry>
<entry>
<title>diff_generate: avoid excessive stats of .gitattribute files</title>
<updated>2018-01-03T16:28:24+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2018-01-03T16:07:36+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/libgit2.git/commit/?id=d8896bda5c43616f3c755242703fce7c2a97ad67'/>
<id>d8896bda5c43616f3c755242703fce7c2a97ad67</id>
<content type='text'>
When generating a diff between two trees, for each file that is to be
diffed we have to determine whether it shall be treated as text or as
binary files. While git has heuristics to determine which kind of diff
to generate, users can also that default behaviour by setting or
unsetting the 'diff' attribute for specific files.

Because of that, we have to query gitattributes in order to determine
how to diff the current files. Instead of hitting the '.gitattributes'
file every time we need to query an attribute, which can get expensive
especially on networked file systems, we try to cache them instead. This
works perfectly fine for every '.gitattributes' file that is found, but
we hit cache invalidation problems when we determine that an attribuse
file is _not_ existing. We do create an entry in the cache for missing
'.gitattributes' files, but as soon as we hit that file again we
invalidate it and stat it again to see if it has now appeared.

In the case of diffing large trees with each other, this behaviour is
very suboptimal. For each pair of files that is to be diffed, we will
repeatedly query every directory component leading towards their
respective location for an attributes file. This leads to thousands or
even hundreds of thousands of wasted syscalls.

The attributes cache already has a mechanism to help in that scenario in
form of the `git_attr_session`. As long as the same attributes session
is still active, we will not try to re-query the gitmodules files at all
but simply retain our currently cached results. To fix our problem, we
can create a session at the top-most level, which is the initialization
of the `git_diff` structure, and use it in order to look up the correct
diff driver. As the `git_diff` structure is used to generate patches for
multiple files at once, this neatly solves our problem by retaining the
session until patches for all files have been generated.

The fix has been tested with linux.git by calling
`git_diff_tree_to_tree` and `git_diff_to_buf` with v4.10^{tree} and
v4.14^{tree}.

                | time    | .gitattributes stats
    without fix | 33.201s | 844614
    with fix    | 30.327s | 4441

While execution only improved by roughly 10%, the stat(3) syscalls for
.gitattributes files decreased by 99.5%. The benchmarks were quite
simple with best-of-three timings on Linux ext4 systems. One can assume
that for network based file systems the performance gain will be a lot
larger due to a much higher latency.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
When generating a diff between two trees, for each file that is to be
diffed we have to determine whether it shall be treated as text or as
binary files. While git has heuristics to determine which kind of diff
to generate, users can also that default behaviour by setting or
unsetting the 'diff' attribute for specific files.

Because of that, we have to query gitattributes in order to determine
how to diff the current files. Instead of hitting the '.gitattributes'
file every time we need to query an attribute, which can get expensive
especially on networked file systems, we try to cache them instead. This
works perfectly fine for every '.gitattributes' file that is found, but
we hit cache invalidation problems when we determine that an attribuse
file is _not_ existing. We do create an entry in the cache for missing
'.gitattributes' files, but as soon as we hit that file again we
invalidate it and stat it again to see if it has now appeared.

In the case of diffing large trees with each other, this behaviour is
very suboptimal. For each pair of files that is to be diffed, we will
repeatedly query every directory component leading towards their
respective location for an attributes file. This leads to thousands or
even hundreds of thousands of wasted syscalls.

The attributes cache already has a mechanism to help in that scenario in
form of the `git_attr_session`. As long as the same attributes session
is still active, we will not try to re-query the gitmodules files at all
but simply retain our currently cached results. To fix our problem, we
can create a session at the top-most level, which is the initialization
of the `git_diff` structure, and use it in order to look up the correct
diff driver. As the `git_diff` structure is used to generate patches for
multiple files at once, this neatly solves our problem by retaining the
session until patches for all files have been generated.

The fix has been tested with linux.git by calling
`git_diff_tree_to_tree` and `git_diff_to_buf` with v4.10^{tree} and
v4.14^{tree}.

                | time    | .gitattributes stats
    without fix | 33.201s | 844614
    with fix    | 30.327s | 4441

While execution only improved by roughly 10%, the stat(3) syscalls for
.gitattributes files decreased by 99.5%. The benchmarks were quite
simple with best-of-three timings on Linux ext4 systems. One can assume
that for network based file systems the performance gain will be a lot
larger due to a much higher latency.
</pre>
</div>
</content>
</entry>
<entry>
<title>Merge pull request #4439 from tiennou/fix/4352</title>
<updated>2018-01-03T13:09:21+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2018-01-03T13:09:21+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/libgit2.git/commit/?id=30455a56506d3dc086cf8efb831d8b1513dd3027'/>
<id>30455a56506d3dc086cf8efb831d8b1513dd3027</id>
<content type='text'>
cmake: create a dummy file for Xcode</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
cmake: create a dummy file for Xcode</pre>
</div>
</content>
</entry>
<entry>
<title>streams: openssl: fix thread-safety for OpenSSL error messages</title>
<updated>2018-01-03T12:54:42+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2018-01-03T12:54:42+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/libgit2.git/commit/?id=ba56f781a91487ad657e1a72888c914b1cec5de9'/>
<id>ba56f781a91487ad657e1a72888c914b1cec5de9</id>
<content type='text'>
The function `ERR_error_string` can be invoked without providing a
buffer, in which case OpenSSL will simply return a string printed into a
static buffer. Obviously and as documented in ERR_error_string(3), this
is not thread-safe at all. As libgit2 is a library, though, it is easily
possible that other threads may be using OpenSSL at the same time, which
might lead to clobbered error strings.

Fix the issue by instead using a stack-allocated buffer. According to
the documentation, the caller has to provide a buffer of at least 256
bytes of size. While we do so, make sure that the buffer will never get
overflown by switching to `ERR_error_string_n` to specify the buffer's
size.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The function `ERR_error_string` can be invoked without providing a
buffer, in which case OpenSSL will simply return a string printed into a
static buffer. Obviously and as documented in ERR_error_string(3), this
is not thread-safe at all. As libgit2 is a library, though, it is easily
possible that other threads may be using OpenSSL at the same time, which
might lead to clobbered error strings.

Fix the issue by instead using a stack-allocated buffer. According to
the documentation, the caller has to provide a buffer of at least 256
bytes of size. While we do so, make sure that the buffer will never get
overflown by switching to `ERR_error_string_n` to specify the buffer's
size.
</pre>
</div>
</content>
</entry>
<entry>
<title>hash: openssl: check return values of SHA1_* functions</title>
<updated>2018-01-03T12:49:11+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2017-12-08T10:10:19+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/libgit2.git/commit/?id=75e1737a5173709b14b669bf08dae78232c6389d'/>
<id>75e1737a5173709b14b669bf08dae78232c6389d</id>
<content type='text'>
The OpenSSL functions `SHA1_Init`, `SHA1_Update` and `SHA1_Final` all
return 1 for success and 0 otherwise, but we never check their return
values. Do so.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The OpenSSL functions `SHA1_Init`, `SHA1_Update` and `SHA1_Final` all
return 1 for success and 0 otherwise, but we never check their return
values. Do so.
</pre>
</div>
</content>
</entry>
<entry>
<title>tests: create new test target for all SSH-based tests</title>
<updated>2018-01-03T11:50:39+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2017-11-20T13:26:33+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/libgit2.git/commit/?id=5874e151d7b10de84fc1ca168339fdc622292219'/>
<id>5874e151d7b10de84fc1ca168339fdc622292219</id>
<content type='text'>
Some tests shall be run against our own SSH server we spin up in Travis.
As those need to be run separate from our previous tests which run
against git-daemon, we have to do this in a separate step. Instead of
bundling all that knowledge in the CI script, move it into the test
build instructions by creating a new test target.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Some tests shall be run against our own SSH server we spin up in Travis.
As those need to be run separate from our previous tests which run
against git-daemon, we have to do this in a separate step. Instead of
bundling all that knowledge in the CI script, move it into the test
build instructions by creating a new test target.
</pre>
</div>
</content>
</entry>
<entry>
<title>tests: online::clone: inline creds-test with nonexistent URL</title>
<updated>2018-01-03T11:50:39+00:00</updated>
<author>
<name>Patrick Steinhardt</name>
<email>ps@pks.im</email>
</author>
<published>2017-06-07T11:06:53+00:00</published>
<link rel='alternate' type='text/html' href='http://91.123.203.49/cgit/delta/libgit2.git/commit/?id=54a1bf057a1123cf55ac3447c79761c817382f47'/>
<id>54a1bf057a1123cf55ac3447c79761c817382f47</id>
<content type='text'>
Right now, we test our credential callback code twice, once via SSH on
localhost and once via a non-existent GitHub repository. While the first
URL makes sense to be configurable, it does not make sense to hard-code
the non-existing repository, which requires us to call tests multiple
times. Instead, we can just inline the URL into another set of tests.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Right now, we test our credential callback code twice, once via SSH on
localhost and once via a non-existent GitHub repository. While the first
URL makes sense to be configurable, it does not make sense to hard-code
the non-existing repository, which requires us to call tests multiple
times. Instead, we can just inline the URL into another set of tests.
</pre>
</div>
</content>
</entry>
</feed>
