Start up compat and encoding files

There were a few things which were being reused consistently for
compatibility purposes, such as the `buffer`/`memoryview` functions
as well as the `izip` method which needed to be aliased for
Python 3.  The `buffer` function was taken from `smmap` [1] and
reworked slightly to handle the optional third parameter.

This also adds a compatibility file dedicated entirely to encoding
issues, which seem to be the biggest problem.  The main functions
were taken in part from the Django project [2] and rewritten
slightly because our needs are a bit more narrow.

A constants file has been added to consistently handle the
constants which are required for the gitdb project in the core
and the tests.  This is part of a greater plan to reorganize
the `util.py` file included in this project.

This points the async extension back at the original repository
and points it to the latest commit.

[1]: https://github.com/Byron/smmap/blob/1af4b42a2354acbb53c7956d647655922658fd80/smmap/util.py#L20-L26
[2]: https://github.com/django/django/blob/b8d255071ead897cf68120cd2fae7c91326ca2cc/django/utils/encoding.py

1 files changed, 35 insertions, 0 deletions

diff --git a/gitdb/utils/encoding.py b/gitdb/utils/encoding.py
new file mode 100644
index 0000000..12164e7
--- /dev/null
+++ b/gitdb/utils/encoding.py
@@ -0,0 +1,35 @@
+from gitdb.utils import compat
+
+if compat.PY3:
+    string_types = (str, )
+    text_type = str
+else:
+    string_types = (basestring, )
+    text_type = unicode
+
+def force_bytes(data, encoding="utf-8"):
+    if isinstance(data, bytes):
+        return data
+
+    if isinstance(data, compat.memoryview):
+        return bytes(data)
+
+    if isinstance(data, string_types):
+        return data.encode(encoding)
+
+    return data
+
+def force_text(data, encoding="utf-8"):
+    if isinstance(data, text_type):
+        return data
+
+    if isinstance(data, string_types):
+        return data.decode(encoding)
+
+    if not isinstance(data, bytes):
+        data = force_bytes(data, encoding)
+
+    if compat.PY3:
+        return text_type(data, encoding)
+    else:
+        return text_type(data)