bugfixes and tweaks to documentation

10 files changed, 242 insertions, 119 deletions

diff --git a/docs/contents.rst b/docs/contents.rst
index 87b5e27..0ed0b32 100644
--- a/docs/contents.rst
+++ b/docs/contents.rst
@@ -6,14 +6,17 @@ Table Of Contents
 
     Front Page <index>
     install
+    quickstart
     overview
 
+    lib/passlib.base
     lib/passlib.hash
+    lib/passlib.sqldb
     lib/passlib.unix
     lib/passlib.utils
 
     password_hash_api
-    
+
     history
     copyright
 
diff --git a/docs/index.rst b/docs/index.rst
index 8ffcbc4..d54ef98 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -10,10 +10,6 @@ as stored in mysql and postgres, and various other contexts.
 
 A quick sample of some of the more frequently used modules:
 
-    * :doc:`bps.fs <lib/bps.fs/filepath>` -- object-oriented filesystem access
-    * :mod:`bps.host` -- desktop and host resources
-    * :mod:`bps.logs` -- enhancements to Python's logging system
-    * :mod:`bps.text` -- text parsing and formatting
     * :mod:`passlib` -- password hashing algorithms
 
     ... see the :doc:`library overview <overview>` for a complete list.
diff --git a/docs/lib/_scratch1.rst b/docs/lib/_scratch1.rst
deleted file mode 100644
index 4c00293..0000000
--- a/docs/lib/_scratch1.rst
+++ /dev/null
@@ -1,49 +0,0 @@
-=============================================
-:mod:`passlib` - Crypt Algorithms
-=============================================
-
-.. currentmodule:: passlib
-
-All of the crypt algorithms must inherit from :class:`CryptHandler`,
-which defines a common interface all algorithms must support.
-You may use the algorithms directly, by creating
-an instance and calling it as described in :doc:`Implementing a Crypt Algorithm <implementation>`.
-However, you will normally will not need to deal with the internals of the algorithms
-directly, but rather take advantage of one of the predefined algorithms,
-through the :doc:`frontend functions <quickstart>` or a
-custom :doc:`crypt context <contexts>`.
-
-Standard Algorithms
-===================
-The following algorithms are all standard password hashing algorithms
-used by various Posix operating systems over the years.
-
-.. note::
-    BPS tries to use external accelaration for these classes when possible,
-    but provides a pure-python fallback so that these algorithms will
-    ALWAYS be available for use.
-
-.. autoclass:: UnixCrypt
-.. autoclass:: Md5Crypt
-.. autoclass:: Sha256Crypt
-.. autoclass:: Sha512Crypt
-.. autoclass:: BCrypt
-
-Database Algorithms
-===================
-BPS also provides implementations of the hash
-algorithms used by MySql and PostgreSQL.
-
-.. autoclass:: Mysql10Crypt
-.. autoclass:: Mysql41Crypt
-.. autoclass:: PostgresMd5Crypt
-
-.. data:: mysql_context
-
-    This context object contains the algorithms used by MySql 4.1 and newer
-    for storing user passwords.
-
-.. data:: postgres_context
-
-    This context object should be able to read/write/verify
-    the values found in the password field of the pg_shadow table in Postgres.
diff --git a/docs/lib/_scratch2.rst b/docs/lib/_scratch2.rst
deleted file mode 100644
index d80ac56..0000000
--- a/docs/lib/_scratch2.rst
+++ /dev/null
@@ -1,17 +0,0 @@
-===================================================================
-:mod:`passlib` - Implementing a Custom Crypt Algorithm
-===================================================================
-
-.. currentmodule:: passlib
-
-New password algorithms can be implemented
-by subclassing :class:`CryptHandler`,
-which provides the underlying framework used
-for all the password algorithms.
-
-To create a new one,
-you simple subclass CryptHandler,
-and implement the identify, encrypt, and verify methods
-(at the very least).
-
-.. autoclass:: CryptHandler
diff --git a/docs/lib/passlib.base.rst b/docs/lib/passlib.base.rst
index c690327..1838b37 100644
--- a/docs/lib/passlib.base.rst
+++ b/docs/lib/passlib.base.rst
@@ -5,7 +5,7 @@
 .. currentmodule:: passlib.base
 
 For more complex deployment scenarios than
-the frontend functions described in :doc:`Quick Start <quickstart>`,
+the frontend functions described in :doc:`Quick Start </quickstart>`,
 the CryptContext class exists...
 
 .. autoclass:: CryptContext
@@ -153,13 +153,14 @@ A sample policy file::
 
     Stores configuration options for a CryptContext object.
 
-    Construction
-    ------------
     Policy objects can be constructed by the following methods:
 
-    .. automethod:: from_file
+    .. automethod:: from_path
+    .. automethod:: from_string
+    .. automethod:: from_source
+    .. automethod:: from_sources
 
-    .. method:: CryptPolicy
+    .. method:: (constructor)
 
         You can specify options directly to the constructor.
         This accepts dot-seperated keywords such as found in the config file format,
diff --git a/docs/lib/passlib.hash.rst b/docs/lib/passlib.hash.rst
index 5b59854..944debf 100644
--- a/docs/lib/passlib.hash.rst
+++ b/docs/lib/passlib.hash.rst
@@ -50,11 +50,16 @@ the modular crypt format.
     passlib.hash.phpass
     passlib.hash.nthash
 
+.. toctree::
+    :hidden:
+
+    passlib.hash.sun_md5_crypt
+
 .. todo::
 
     These aren't fully implemented / tested yet:
 
-    * :mod:`~passlib.hash.sun_md5_crypt` - MD5-based crypt descendant used by Solaris 10 (NOT related to md5-crypt above).
+    * :mod:`~passlib.hash.sun_md5_crypt` - MD5-based scheme used by Solaris 10 (NOT related to md5-crypt above).
 
 Other Schemes
 -------------
diff --git a/docs/lib/passlib.hash.sun_md5_crypt.rst b/docs/lib/passlib.hash.sun_md5_crypt.rst
index e4f91b6..393c9c2 100644
--- a/docs/lib/passlib.hash.sun_md5_crypt.rst
+++ b/docs/lib/passlib.hash.sun_md5_crypt.rst
@@ -17,7 +17,9 @@ This algorithm is used by Solaris, as a replacement for the aging des-crypt.
 It is mainly used on later versions of Solaris, and is not found many other
 places. While based on the MD5 message digest, it has very little at all
 in common with the :mod:`~passlib.hash.md5_crypt` algorithm. It supports
-32 bit variable rounds and an 8 character salt.
+32 bit variable rounds and an 8 character salt. Due to a theoretic pre-image
+attacks on the MD5 message digest, this algorithm should probably not
+be used in new deploys.
 
 Usage
 =====
@@ -47,15 +49,15 @@ A sun-md5-crypt hash string has the format ``$md5,rounds={rounds}${salt}${checks
 An alternate format, ``$md5${salt}${checksum}`` is used when the rounds value is 0.
 
 .. note::
-    Solaris seems to deviates from the :ref:`modular-crypt-format` in that
+    Solaris seems to deviate from the :ref:`modular-crypt-format` in that
     it considers ``$`` *or* ``,`` to indicate the end of the identifier.
 
 .. warning::
 
     One of the remaining issues with this implementation is that some
-    existing hashes found on the web use a ``$`` where this uses ``,``.
-    It is unclear whether this is an accepted alternate format or not,
-    nor whether this affects the resulting hash.
+    existing sun-md5-crypt hashes found on the web use a ``$`` in place of the ``,``.
+    It is unclear whether this is an accepted alternate format or just a typo,
+    nor whether this is supposed to affect the checksum in the resulting hash string.
 
 Algorithm
 =========
@@ -67,20 +69,27 @@ by one of the creators). Given a password, the number of rounds, and a salt...
 
 * for rounds+4096 iterations, a new digest is created:
     - ``MuffetCoinToss(rounds, previous digest)`` is called, resulting in a 0 or 1.
-    - if a 1, the next digest is the MD5 of: the last digest concatenated with a magic constant
+    - if a 1, the next digest is the MD5 of: the last digest concatenated with a constant
       data string, along with the current iteration number as an ascii string.
     - if a 0, the same as 1, except that magic constant data is not included.
 
-* The magic constant data string is an 1517 byte ascii string - an excerpt from Hamlet,
-  starting with ``To be, or not to be`` and ending with ``all my sins remember'd.\n``,
-  with a null character appended (exact Project Gutenberg source linked to below).
-
 * The final checksum is then encoded into :mod:`hash64 <~passlib.hash.h64>` using the same
-  transposed indexes that :mod:`~passlib.hash.md5_crypt` uses.
+  transposed byte order that :mod:`~passlib.hash.md5_crypt` uses.
+
+The constant data string is referenced above is a 1517 byte ascii string... an excerpt from Hamlet,
+starting with ``To be, or not to be...`` and ending with ``...all my sins remember'd.\n``,
+with a null character appended (exact Project Gutenberg source linked to below).
+
+.. warning::
+
+    Note that this has a weakness in that the per-round operation appends data
+    which is known to the attacker, the coin flip algorithm only serves to
+    frustrate brute-force attacks. Reversing this hash is dependant
+    on MD5's general pre-image attack resistance (which is currently theoretically vulnerable).
 
-Coin Flip
----------
-The MuffetCoinToss algorithm is as follows:
+Muffer Coin Toss
+----------------
+The Muffet Coin Toss algorithm is as follows:
 Given the current round number, and a 16 byte MD5 digest, it returns a 0 or 1,
 using the following formula:
 
@@ -89,29 +98,30 @@ using the following formula:
     All references below to a specific bit of the digest should be interpreted mod 128.
     All references below to a specific byte of the digest should be interpreted mod 16.
 
-the coinflip generates two 8 bit integers X & Y as follows:
+the coinflip generates two 8 bit integers ``X`` & ``Y`` as follows:
+
+* ``X`` is generated from the following formula:
 
-* X is generated from the following formula:
-  for each I in 0..7 inclusive:
+  for each ``i`` in 0..7 inclusive:
 
-    - let A be the I'th byte of the digest as an 8-bit int.
-    - let B be the I+3'th byte of the digest as an 8-bit int.
+    - let ``A`` be the ``i``'th byte of the digest, as an 8-bit int.
+    - let ``B`` be the ``i+3``'th byte of the digest, as an 8-bit int.
 
-    - let R be A shifted right by (B mod 5) bits.
+    - let ``R`` be ``A`` shifted right by ``B % 5`` bits.
 
-    - let V be the R'th byte of the digest.
-    - if the (A mod 8)'th bit of B is 1, divide V by 2.
+    - let ``V`` be the ``R``'th byte of the digest.
+    - if the ``A % 8``'th bit of ``B`` is 1, divide ``V`` by 2.
 
-    - use the V'th bit of the digest as the I'th bit of X.
+    - use the ``V``'th bit of the digest as the ``i``'th bit of ``X``.
 
-* Y is generated exactly the same as X, except that
-  A is the I+8'th byte of the digest,
-  and B is the I+11'th byte of the digest.
+* ``Y`` is generated exactly the same as ``X``, except that
+  ``A`` is the ``i+8``'th byte of the digest,
+  and ``B`` is the ``i+11``'th byte of the digest.
 
-* if bit ``round`` of the digest is 1, X is divided by 2.
-* if bit ``round+64`` of the digest is 1, Y is divided by 2.
+* if bit ``round`` of the digest is 1, ``X`` is divided by 2.
+* if bit ``round+64`` of the digest is 1, ``Y`` is divided by 2.
 
-* the final result is X'th bit of the digest XORed against Y'th bit of the digest.
+* the final result is ``X``'th bit of the digest XORed against ``Y``'th bit of the digest.
 
 References
 ==========
diff --git a/docs/lib/passlib.sqldb.rst b/docs/lib/passlib.sqldb.rst
index baf3797..0520f30 100644
--- a/docs/lib/passlib.sqldb.rst
+++ b/docs/lib/passlib.sqldb.rst
@@ -2,7 +2,7 @@
 :mod:`passlib.sqldb` - SQL Database Helpers
 ============================================
 
-.. module:: passlib.unix
+.. module:: passlib.sqldb
     :synopsis: frontend for encrypting & verifying passwords used in various sql databases
 
 PostgreSQL
diff --git a/docs/password_hash_api.rst b/docs/password_hash_api.rst
index 1f0c815..7bd9c05 100644
--- a/docs/password_hash_api.rst
+++ b/docs/password_hash_api.rst
@@ -103,9 +103,83 @@ to provide an easy interface for applications to encrypt new passwords
 and verify existing passwords, without having to deal with details such
 as salt formats.
 
-.. autofunction:: encrypt
-.. autofunction:: identify
-.. autofunction:: verify
+.. function:: encrypt(secret, \*\*settings_and_context)
+
+    encrypt secret, returning resulting hash string.
+
+    :arg secret:
+        A string containing the secret to encode.
+
+        Unicode behavior is specified on a per-hash basis,
+        but the common case is to encode into utf-8
+        before processing.
+
+    :param kwds:
+        All other keywords are algorithm-specified,
+        and should be listed in :attr:`setting_kwds`
+        and :attr:`context_kwds`.
+
+        Common keywords include ``salt`` and ``rounds``.
+
+    :raises ValueError:
+        * if settings are invalid and not correctable.
+          (eg: provided salt contains invalid characters / length).
+
+        * if a context kwd contains an invalid value, or was required
+          but omitted.
+
+        * if secret contains forbidden characters (e.g: des-crypt forbids null characters).
+          this should rarely occur, since most modern algorithms have no limitations
+          on the types of characters.
+
+    :returns:
+        Hash encoded in algorithm-specified format.
+
+.. function:: identify(hash)
+
+    identify if a hash string belongs to this algorithm.
+
+    :arg hash:
+        the candidate hash string to check
+
+    :returns:
+        * ``True`` if input appears to be a hash string belonging to this algorithm.
+        * ``True`` if input appears to be a configuration string belonging to this algorithm.
+        * ``False`` if no input is specified
+        * ``False`` if none of the above conditions was met.
+
+    .. note::
+        Some handlers may or may not return ``True`` for malformed hashes.
+        Those that do will raise a ValueError once the hash is passed to :func:`verify`.
+        Most handlers, however, will just return ``False``.
+
+.. function:: verify(secret, hash, \*\*context)
+
+    verify a secret against an existing hash.
+
+    This checks if a secret matches against the one stored
+    inside the specified hash.
+
+    :param secret:
+        A string containing the secret to check.
+    :param hash:
+        A string containing the hash to check against.
+
+    :param context:
+        Any additional keywords will be passed to the encrypt
+        method. These should be limited to those listed
+        in :attr:`context_kwds`.
+
+    :raises TypeError:
+        * if the secret is not a string.
+
+    :raises ValueError:
+        * if the hash not specified
+        * if the hash does not match this algorithm's hash format
+        * if the provided secret contains forbidden chars (see :func:`encrypt`)
+
+    :returns:
+        ``True`` if the secret matches, otherwise ``False``.
 
 Secondary Interface
 ===================
@@ -116,16 +190,97 @@ for *implementing* new password schemes. It also happens to match
 the tradition unix crypt interface, and consists of two functions:
 ``genconfig()`` and ``genhash``.
 
-.. autofunction:: genconfig
-.. autofunction:: genhash
 
-Other Methods
-=============
+.. function:: genconfig(\*\*settings)
+
+    returns configuration string encoding settings for hash generation
+
+    Many hashes have configuration options,  and support a format
+    which encodes them into a single configuration string.
+    (This configuration string is usually an abbreviated version of their
+    encoded hash format, sans the actual checksum, and is commonly
+    referred to as a ``salt string``, though it may contain much more
+    than just a salt).
+
+    This function takes in optional configuration options (a complete list
+    of which should be found in :attr:`setting_kwds`), validates
+    the inputs, fills in defaults where appropriate, and returns
+    a configuration string.
+
+    For algorithms which do not have any configuration options,
+    this function should always return ``None``.
+
+    While each algorithm may have it's own configuration options,
+    the following keywords (if supported) should always have a consistent
+    meaning:
+
+    * ``salt`` - algorithm uses a salt. if passed into genconfig,
+      should contain an encoded salt string of length and character set
+      required by the specific handler.
+
+      salt strings which are too small or have invalid characters
+      should cause an error, salt strings which are too large
+      should be truncated but accepted.
+
+    * ``rounds`` - algorithm uses a variable number of rounds. if passed
+      into genconfig, should contain an integer number of rounds
+      (this may represent logarithmic rounds, eg bcrypt, or linear, eg sha-crypt).
+      if the number of rounds is too small or too large, it should
+      be clipped but accepted.
+
+    :param settings:
+        this function takes in keywords as specified in :attr:`setting_kwds`.
+        commonly supported keywords include ``salt`` and ``rounds``.
+
+    :raises ValueError:
+        * if any configuration options are required, missing, AND
+          a default value cannot be autogenerated.
+          (for example: salt strings should be autogenerated if not specified).
+        * if any configuration options are invalid, and cannot be
+          normalized in a reasonble manner (eg: salt strings clipped to maximum size).
+
+    :returns:
+        the configuration string, or ``None`` if the algorithm does not support any configuration options.
+
+.. function:: genhash(secret, config, \*\*context)
+
+    encrypt secret to hash
+
+    takes in a password, optional configuration string,
+    and any required contextual information the algorithm needs,
+    and returns the encoded hash strings.
+
+    :arg secret: string containing the password to be encrypted
+    :arg config:
+        configuration string to use when encrypting secret.
+        this can either be an existing hash that was previously
+        returned by :meth:`genhash`, or a configuration string
+        that was previously created by :meth:`genconfig`.
+
+    :param context:
+        All other keywords must be external contextual information
+        required by the algorithm to create the hash. If any,
+        these kwds must be specified in :attr:`context_kwds`.
+
+    :raises TypeError:
+        * if the configuration string is not provided
+        * if required contextual information is not provided
+
+    :raises ValueError:
+        * if the configuration string is not in a recognized format.
+        * if the secret contains a forbidden character (rare, but some algorithms have limitations, eg: forbidding null characters)
+        * if the contextual information is invalid
+
+    :returns:
+        encoded hash matching specified secret, config, and context.
+
+Optional Parse Methods
+======================
 Some of the handlers in passlib expose some additional function and attributes,
 which may be useful, but whose behavior varies between handlers (if present at all),
 and may not conform exactly to the following summary:
 
-.. autofunction:: parse
+.. function:: parse(hash)
 
     This method usually takes in a hash or configuration string
     belonging to the scheme, and parses it into a dictionary
@@ -138,7 +293,7 @@ and may not conform exactly to the following summary:
     Most implementations of ``parse()`` do very little sanity checking,
     leaving that job to ``genconfig``.
 
-.. autofunction:: render
+.. function:: render(checksum=None, \*\*settings)
 
     This method is the inverse of :func:`parse`:
     it takes in a dictionary such as returned by :func:`parse`,
@@ -147,6 +302,12 @@ and may not conform exactly to the following summary:
     Most implementations of ``render()`` do very little sanity checking,
     and may be willing to form strings which are malformed.
 
+Optional Informational Attributes
+=================================
+Many of the handlers in passlib expose the following informational
+attributes, though their presence and meaning is not uniform
+across all handlers in passlib.
+
 For schemes which support a variable number of rounds,
 the following attributes are usually exposed:
 
@@ -166,3 +327,16 @@ the following attributes are usually exposed:
     The maximum number of rounds the scheme allows.
     Specifying values above this will generally result
     in a warning, and ``max_rounds`` will be used instead.
+
+For schemes which support a salt,
+the following attributes are usually exposed:
+
+.. attribute:: min_salt_chars
+
+    minimum number of characters required in salt string,
+    if provided to :func:`genconfig` or :func:`encrypt`.
+
+.. attribute:: max_salt_chars
+
+    maximum number of characters which will be *used*
+    if a salt string is provided to :func:`genconfig` or :func:`encrypt`.
diff --git a/docs/quickstart.rst b/docs/quickstart.rst
index c6128f8..3ea0295 100644
--- a/docs/quickstart.rst
+++ b/docs/quickstart.rst
@@ -41,6 +41,6 @@ without having to delve too deeply into this module::
 
 Frontend Functions
 ==================
-.. autofunction:: encrypt
-.. autofunction:: verify
-.. autofunction:: identify
+.. function:: encrypt
+.. function:: verify
+.. function:: identify