supporting hashes added

=======================
* added unix_fallback scheme, for detecting wildcard/disabled passwords in /etc/shadow files
* added plaintext scheme, for migrating existing application
* added hex md4/md5/sha1/sha256/sha512 schemes, for migrating existing applications
* docs & UTs added for above schemes

13 files changed, 401 insertions, 55 deletions

diff --git a/docs/lib/passlib.hash.hex_digests.rst b/docs/lib/passlib.hash.hex_digests.rst
new file mode 100644
index 0000000..7224069
--- /dev/null
+++ b/docs/lib/passlib.hash.hex_digests.rst
@@ -0,0 +1,49 @@
+==================================================================
+:samp:`passlib.hash.hex_{digest}` - Hexdecimal Standard Digests
+==================================================================
+
+.. currentmodule:: passlib.hash
+
+Some existing applications store passwords by storing them using
+hexidecimal-encoded message digests, such as MD5 or SHA1.
+Such schemes are *extremely* vulnerable to pre-computed brute-force attacks,
+and should not be used in new applications. However, for the sake
+of backwards compatibility when converting existing applications,
+Passlib provides wrappers for few of the common hashes.
+
+Usage
+=====
+These classes all wrap the underlying hashlib implementations,
+and are mainly useful only for plugging them into a :class:`passlib.base.CryptContext`.
+However, they can be used directly as follows::
+
+    >>> from passlib.hash import hex_sha1 as hs
+
+    >>> #encrypt password
+    >>> h = hs.encrypt("password")
+    >>> h
+    '5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8'
+
+    >>> hs.identify(h) #check if hash is recognized
+    True
+    >>> hs.identify('JQMuyS6H.AGMo') #check if some other hash is recognized
+    False
+
+    >>> hs.verify("password", h) #verify correct password
+    True
+    >>> hs.verify("secret", h) #verify incorrect password
+    False
+
+Interface
+=========
+.. autoclass:: hex_md4()
+.. autoclass:: hex_md5()
+.. autoclass:: hex_sha1()
+.. autoclass:: hex_sha256()
+.. autoclass:: hex_sha512()
+
+Format & Algorithm
+==================
+All of these classes just report the result of the specified digest,
+encoded as a series of lowercase hexidecimal characters;
+though upper case is accepted as input.
diff --git a/docs/lib/passlib.hash.plaintext.rst b/docs/lib/passlib.hash.plaintext.rst
new file mode 100644
index 0000000..d0ad62d
--- /dev/null
+++ b/docs/lib/passlib.hash.plaintext.rst
@@ -0,0 +1,35 @@
+==================================================================
+:class:`passlib.hash.plaintext` - Plaintext
+==================================================================
+
+.. currentmodule:: passlib.hash
+
+This class stores passwords in plaintext.
+This is, of course, ridiculously insecure;
+it is provided for backwards compatibility when migrating
+existing applications. *It should not be used* for any other purpose.
+
+Usage
+=====
+This class is mainly useful only for plugging into a :class:`passlib.base.CryptContext`.
+When used, it should always be the last scheme in the list,
+as it will recognize all hashes.
+It can be used directly as follows::
+
+    >>> from passlib.hash import plaintext as pt
+
+    >>> #"encrypt" password
+    >>> pt.encrypt("password")
+    'password'
+
+    >>> nt.identify('password') #check if hash is recognized (all hashes are recognized)
+    True
+
+    >>> nt.verify("password", "password") #verify correct password
+    True
+    >>> nt.verify("secret", "password") #verify incorrect password
+    False
+
+Interface
+=========
+.. autoclass:: plaintext
diff --git a/docs/lib/passlib.hash.rst b/docs/lib/passlib.hash.rst
index d0dbf0b..2cece32 100644
--- a/docs/lib/passlib.hash.rst
+++ b/docs/lib/passlib.hash.rst
@@ -75,10 +75,11 @@ the modular crypt format.
     passlib.hash.phpass
     passlib.hash.nthash
 
-Other Schemes
--------------
-The following schemes are used in very specified contexts,
-and have encoding schemes and other requirements
+Database Schemes
+----------------
+The following schemes are used by various SQL databases
+to encode their own user accounts.
+These schemes have encoding and contextual requirements
 not seen outside those specific contexts:
 
 .. toctree::
@@ -87,3 +88,16 @@ not seen outside those specific contexts:
     passlib.hash.mysql323
     passlib.hash.mysql41
     passlib.hash.postgres_md5
+
+
+Other Schemes
+-------------
+The following schemes are used in various contexts,
+mainly for legacy compatibility purposes.
+
+.. toctree::
+    :maxdepth: 1
+
+    passlib.hash.hex_digests
+    passlib.hash.plaintext
+    passlib.hash.unix_fallback
diff --git a/docs/lib/passlib.hash.unix_fallback.rst b/docs/lib/passlib.hash.unix_fallback.rst
new file mode 100644
index 0000000..d534546
--- /dev/null
+++ b/docs/lib/passlib.hash.unix_fallback.rst
@@ -0,0 +1,36 @@
+==================================================================
+:class:`passlib.hash.unix_fallback` - Unix Fallback Helper
+==================================================================
+
+.. currentmodule:: passlib.hash
+
+This class does not provide an encryption scheme,
+but instead provides a helper for handling disabled / wildcard
+password fields as found in unix ``/etc/shadow`` files.
+
+Usage
+=====
+This class is mainly useful only for plugging into a :class:`passlib.base.CryptContext`.
+When used, it should always be the last scheme in the list,
+as it is designed to provide a fallback behavior.
+It can be used directly as follows::
+
+    >>> from passlib.hash import unix_fallback as uf
+
+    >>> #'encrypting' a password always results in "!", the default reject hash.
+    >>> uf.encrypt("password")
+    '!'
+
+    >>> uf.identify('!') #check if hash is recognized (all hashes are recognized)
+    True
+    >>> uf.identify('')
+    True
+
+    >>> uf.verify("password", "") #verify against empty string - all password allowed
+    True
+    >>> uf.verify("password", "!") #verify against non-empty string - no passwords allowed
+    False
+
+Interface
+=========
+.. autoclass:: unix_fallback
diff --git a/passlib/base.py b/passlib/base.py
index db76da7..fc8e8ee 100644
--- a/passlib/base.py
+++ b/passlib/base.py
@@ -93,17 +93,24 @@ _driver_locations = {
     "bsdi_crypt":       ("passlib.drivers.des_crypt",   "bsdi_crypt"),
     "crypt16":          ("passlib.drivers.des_crypt",   "crypt16"),
     "des_crypt":        ("passlib.drivers.des_crypt",   "des_crypt"),
+    "hex_md4":          ("passlib.drivers.digests",     "hex_md4"),
+    "hex_md5":          ("passlib.drivers.digests",     "hex_md5"),
+    "hex_sha1":         ("passlib.drivers.digests",     "hex_sha1"),
+    "hex_sha256":       ("passlib.drivers.digests",     "hex_sha256"),
+    "hex_sha512":       ("passlib.drivers.digests",     "hex_sha512"),
     "md5_crypt":        ("passlib.drivers.md5_crypt",   "md5_crypt"),
     "mysql323":         ("passlib.drivers.mysql",       "mysql323"),
     "mysql41":          ("passlib.drivers.mysql",       "mysql41"),
     "nthash":           ("passlib.drivers.nthash",      "nthash"),
     "phpass":           ("passlib.drivers.phpass",      "phpass"),
+    "plaintext":        ("passlib.drivers.misc",        "plaintext"),
     "postgres_md5":     ("passlib.drivers.postgres",    "postgres_md5"),
     "postgres_plaintext":("passlib.drivers.postgres",   "postgres_plaintext"),
     "sha1_crypt":       ("passlib.drivers.sha1_crypt",  "sha1_crypt"),
     "sha256_crypt":     ("passlib.drivers.sha2_crypt",  "sha256_crypt"),
     "sha512_crypt":     ("passlib.drivers.sha2_crypt",  "sha512_crypt"),
     "sun_md5_crypt":    ("passlib.drivers.sun_md5_crypt","sun_md5_crypt"),
+    "unix_fallback":    ("passlib.drivers.misc",        "unix_fallback"),
 }
 
 def register_crypt_location(name, path):
diff --git a/passlib/drivers/des_crypt.py b/passlib/drivers/des_crypt.py
index d79eaaf..15db66c 100644
--- a/passlib/drivers/des_crypt.py
+++ b/passlib/drivers/des_crypt.py
@@ -1,8 +1,8 @@
-"""passlib.drivers.des_crypt - traditional unix (DES) crypt
+"""passlib.drivers.des_crypt - traditional unix (DES) crypt and variants
 
 .. note::
 
-    passlib restricts salt characters to just the hash64 charset,
+    for des-crypt, passlib restricts salt characters to just the hash64 charset,
     and salt string size to >= 2 chars; since implementations of des-crypt
     vary in how they handle other characters / sizes...
 
diff --git a/passlib/drivers/digests.py b/passlib/drivers/digests.py
new file mode 100644
index 0000000..482bb31
--- /dev/null
+++ b/passlib/drivers/digests.py
@@ -0,0 +1,82 @@
+"""passlib.drivers.digests - plain hash digests
+"""
+#=========================================================
+#imports
+#=========================================================
+#core
+import hashlib
+import logging; log = logging.getLogger(__name__)
+from warnings import warn
+#site
+#libs
+from passlib.utils.md4 import md4
+from passlib.utils.drivers import BaseHash
+#pkg
+#local
+__all__ = [
+    "create_hex_hash",
+    "hex_md4",
+    "hex_md5",
+    "hex_sha1",
+    "hex_sha256",
+    "hex_sha512",
+]
+
+#=========================================================
+#helpers for hexidecimal hashes
+#=========================================================
+class HexDigestHash(BaseHash):
+    "this provides a template for supporting passwords stored as plain hexidecimal hashes"
+    setting_kwds = ()
+    context_kwds = ()
+
+    _hash = None
+    checksum_chars = None
+    checksum_charset = "0123456789abcdef"
+
+    @classmethod
+    def identify(cls, hash):
+        cc = cls.checksum_charset
+        return bool(hash) and len(hash) == cls.checksum_chars and all(c in cc for c in hash)
+
+    @classmethod
+    def genhash(cls, secret, hash):
+        if secret is None:
+            raise TypeError, "no secret provided"
+        if isinstance(secret, unicode):
+            secret = secret.encode("utf-8")
+        if hash is not None and not cls.identify(hash):
+            raise ValueError, "not a %s hash" % (cls.name,)
+        return cls._hash(secret).hexdigest()
+
+    @classmethod
+    def verify(cls, secret, hash):
+        if hash is None:
+            raise ValueError, "no hash specified"
+        return cls.genhash(secret, hash) == hash.lower()
+
+def create_hex_hash(hash):
+    h = hash()
+    name = 'hex_' + h.name
+    return type(name, (HexDigestHash,), dict(
+        name=name,
+        _hash=hash,
+        checksum_chars=h.digest_size*2,
+        __doc__="""This class implements a plain hexidecimal %s hash, and follows the :ref:`password-hash-api`.
+
+It supports no optional or contextual keywords.
+""" % (h.name,)
+    ))
+
+#=========================================================
+#predefined handlers
+#=========================================================
+hex_md4 = create_hex_hash(md4)
+hex_md5 = create_hex_hash(hashlib.md5)
+hex_sha1 = create_hex_hash(hashlib.sha1)
+hex_sha256 = create_hex_hash(hashlib.sha256)
+hex_sha512 = create_hex_hash(hashlib.sha512)
+
+#=========================================================
+#eof
+#=========================================================
diff --git a/passlib/drivers/misc.py b/passlib/drivers/misc.py
new file mode 100644
index 0000000..a55c752
--- /dev/null
+++ b/passlib/drivers/misc.py
@@ -0,0 +1,89 @@
+"""passlib.drivers.misc - misc generic drivers
+"""
+#=========================================================
+#imports
+#=========================================================
+#core
+import logging; log = logging.getLogger(__name__)
+from warnings import warn
+#site
+#libs
+from passlib.utils.drivers import BaseHash
+#pkg
+#local
+__all__ = [
+    "unix_fallback",
+    "plaintext",
+]
+
+#=========================================================
+#handler
+#=========================================================
+class unix_fallback(BaseHash):
+    """This class fallback behavior for unix shadow files, and follows the :ref:`password-hash-api`.
+
+    This class does not implement a hash, but instead provides fallback
+    behavior as found in /etc/shadow on most unix variants.
+    If used, should be the last scheme in the context.
+
+    * this class recognizes all hash strings.
+    * it accepts all passwords if the hash is an empty string.
+    * it rejects all passwords if the hash is NOT an empty string (``!`` or ``*`` are frequently used).
+    * for security, newly encrypted passwords will hash to ``!``.
+    """
+    name = "unix_fallback"
+    setting_kwds = ()
+    context_kwds = ()
+
+    @classmethod
+    def identify(cls, hash):
+        return hash is not None
+
+    @classmethod
+    def genconfig(cls):
+        return "!"
+
+    @classmethod
+    def genhash(cls, secret, hash):
+        if secret is None:
+            raise TypeError, "secret must be string"
+        if hash is None:
+            raise ValueError, "no hash provided"
+        return hash
+
+    @classmethod
+    def verify(cls, secret, hash):
+        if hash is None:
+            raise ValueError, "no hash provided"
+        return not hash
+
+class plaintext(BaseHash):
+    """This class stores passwords in plaintext, and follows the :ref:`password-hash-api`.
+
+    Unicode passwords will be encoded using utf-8.
+    """
+    name = "plaintext"
+    setting_kwds = ()
+    context_kwds = ()
+
+    @classmethod
+    def identify(cls, hash):
+        return hash is not None
+
+    @classmethod
+    def genhash(cls, secret, hash):
+        if secret is None:
+            raise TypeError, "secret must be string"
+        if isinstance(secret, unicode):
+            secret = secret.encode("utf-8")
+        return secret
+
+    @classmethod
+    def verify(cls, secret, hash):
+        if hash is None:
+            raise ValueError, "no hash specified"
+        return hash == cls.genhash(secret, hash)
+
+#=========================================================
+#eof
+#=========================================================
diff --git a/passlib/tests/test_drivers.py b/passlib/tests/test_drivers.py
index bf577a8..93c229f 100644
--- a/passlib/tests/test_drivers.py
+++ b/passlib/tests/test_drivers.py
@@ -154,6 +154,31 @@ class DesCryptTest(HandlerCase):
 BuiltinDesCryptTest = create_backend_case(DesCryptTest, "builtin")
 
 #=========================================================
+#hex digests
+#=========================================================
+from passlib.drivers import digests
+
+class HexMd4Test(HandlerCase):
+    handler = digests.hex_md4
+    known_correct_hashes = [ ("password", '8a9d093f14f8701df17732b2bb182c74')]
+
+class HexMd5Test(HandlerCase):
+    handler = digests.hex_md5
+    known_correct_hashes = [ ("password", '5f4dcc3b5aa765d61d8327deb882cf99')]
+
+class HexSha1Test(HandlerCase):
+    handler = digests.hex_sha1
+    known_correct_hashes = [ ("password", '5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8')]
+
+class HexSha256Test(HandlerCase):
+    handler = digests.hex_sha256
+    known_correct_hashes = [ ("password", '5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8')]
+
+class HexSha512Test(HandlerCase):
+    handler = digests.hex_sha512
+    known_correct_hashes = [ ("password", 'b109f3bbbc244eb82441917ed06d618b9008dd09b3befd1b5e07394c706a8bb980b1d7785e5976ec049b46df5f1326af5a2ea6d103fd07c95385ffab0cacbc86')]
+
+#=========================================================
 #md5 crypt
 #=========================================================
 from passlib.drivers.md5_crypt import md5_crypt
@@ -246,6 +271,23 @@ class PHPassTest(HandlerCase):
         ]
 
 #=========================================================
+#plaintext
+#=========================================================
+from passlib.drivers.misc import plaintext
+
+class PlaintextTest(HandlerCase):
+    handler = plaintext
+
+    known_correct_hashes = (
+        ('',''),
+        ('password', 'password'),
+    )
+
+    known_other_hashes = [] #all strings are identified as belonging to this scheme
+
+    accepts_empty_hash = True
+
+#=========================================================
 #postgres_md5
 #=========================================================
 from passlib.drivers.postgres import postgres_md5, postgres_plaintext
@@ -467,5 +509,30 @@ class SunMD5CryptTest(HandlerCase):
         ]
 
 #=========================================================
+#unix fallback
+#=========================================================
+from passlib.drivers.misc import unix_fallback
+
+class UnixFallbackTest(HandlerCase):
+    #NOTE: this class behaves VERY differently from a normal password hash,
+    #so we subclass & disable a number of the default tests.
+
+    handler = unix_fallback
+
+    known_correct_hashes = [ ("password",""), ]
+    known_other_hashes = []
+    accepts_empty_hash = True
+
+    def test_50_encrypt_plain(self):
+        "test encrypt() basic behavior"
+        if self.supports_unicode:
+            secret = u"unic\u00D6de"
+        else:
+            secret = "too many secrets"
+        result = self.do_encrypt(secret)
+        self.assertEquals(result, "!")
+        self.assert_(not self.do_verify(secret, result))
+
+#=========================================================
 #EOF
 #=========================================================
diff --git a/passlib/tests/utils.py b/passlib/tests/utils.py
index 6db10ab..9daaff9 100644
--- a/passlib/tests/utils.py
+++ b/passlib/tests/utils.py
@@ -238,6 +238,9 @@ class HandlerCase(TestCase):
             "elCjmw2kSYu.Ec6ycULevoBK25fs2xXgMNrCzIMVcgEJAstJeonj1"),
     ]
 
+    #flag if scheme accepts empty string as hash (rare)
+    accepts_empty_hash = False
+
     #=========================================================
     #alg interface helpers - allows subclass to overide how
     # default tests invoke the handler (eg for context_kwds)
@@ -366,7 +369,7 @@ class HandlerCase(TestCase):
     def test_15_identify_none(self):
         "test identify() against None / empty string"
         self.assertEqual(self.do_identify(None), False)
-        self.assertEqual(self.do_identify(''), False)
+        self.assertEqual(self.do_identify(''), self.accepts_empty_hash)
 
     #=========================================================
     #verify()
@@ -404,7 +407,10 @@ class HandlerCase(TestCase):
         "test verify() throws error against hash=None/empty string"
         #find valid hash so that doesn't mask error
         self.assertRaises(ValueError, self.do_verify, 'stub', None, __msg__="hash=None:")
-        self.assertRaises(ValueError, self.do_verify, 'stub', '', __msg__="hash='':")
+        if self.accepts_empty_hash:
+            self.do_verify("stub", "")
+        else:
+            self.assertRaises(ValueError, self.do_verify, 'stub', '', __msg__="hash='':")
 
     #=========================================================
     #genconfig()
diff --git a/passlib/unix.py b/passlib/unix.py
index e837af1..30bf955 100644
--- a/passlib/unix.py
+++ b/passlib/unix.py
@@ -15,47 +15,7 @@ __all__ = [
         "netbsd_context",
         "freebsd_context",
 
-    "UnixDisabledHandler",
 ]
-#=========================================================
-#helpers
-#=========================================================
-
-#TODO: replace this with a "generic-reject" (also add a "generic-allow")
-
-class UnixDisabledHandler(CryptHandler):
-    """fake crypt handler which handles special (non-hash) strings found in /etc/shadow
-
-    unix shadow files sometimes have "!" or "*" characters indicating logins are disabled.
-    linux also prepends "!" to valid hashes to indicate a password is disabled.
-
-    this is a fake password hash, designed to recognize those values,
-    and return False for all verify attempts.
-    """
-    name = "unix_disabled"
-    setting_kwds = ()
-    context_kwds = ()
-
-    @classmethod
-    def genconfig(cls):
-        return None
-
-    @classmethod
-    def genhash(cls, secret, config):
-        return "!"
-
-    @classmethod
-    def identify(cls, hash):
-        return not hash or hash == "*" or hash.startswith("!")
-
-    @classmethod
-    def verify(cls, secret, hash):
-        return False
-
-register_crypt_handler(UnixDisabledHandler)
-
-#TODO: UnknownCryptHandler - given hash, detect if system crypt recognizes it,
-# allowing for pass-through for unknown ones.
 
 #=========================================================
 #build default context objects
@@ -73,16 +33,16 @@ register_crypt_handler(UnixDisabledHandler)
 #referencing linux shadow...
 # linux - des,md5, sha256, sha512
 
-linux_context = CryptContext([ "sha512_crypt", "sha256_crypt", "md5_crypt", "des_crypt", "unix_disabled" ])
+linux_context = CryptContext([ "sha512_crypt", "sha256_crypt", "md5_crypt", "des_crypt", "unix_fallback" ])
 
 #referencing source via -http://fxr.googlebit.com
 # freebsd 6,7,8 - des, md5, bcrypt, nthash
 # netbsd - des, ext, md5, bcrypt, sha1
 # openbsd - des, ext, md5, bcrypt
-bsd_context = CryptContext(["bcrypt",  "md5_crypt", "bsdi_crypt", "des_crypt", "nthash", "unix_disabled" ])
-freebsd_context = CryptContext([  "bcrypt", "md5_crypt", "nthash", "des_crypt", "unix_disabled" ])
-openbsd_context = CryptContext([ "bcrypt", "md5_crypt", "bsdi_crypt", "des_crypt", "unix_disabled" ])
-netbsd_context = CryptContext([  "bcrypt", "sha1_crypt", "md5_crypt", "bsdi_crypt", "des_crypt", "unix_disabled" ])
+bsd_context = CryptContext(["bcrypt",  "md5_crypt", "bsdi_crypt", "des_crypt", "nthash", "unix_fallback" ])
+freebsd_context = CryptContext([ "bcrypt", "md5_crypt", "nthash", "des_crypt", "unix_fallback" ])
+openbsd_context = CryptContext([ "bcrypt", "md5_crypt", "bsdi_crypt", "des_crypt", "unix_fallback" ])
+netbsd_context = CryptContext([ "bcrypt", "sha1_crypt", "md5_crypt", "bsdi_crypt", "des_crypt", "unix_fallback" ])
 
 
 #aix3
diff --git a/passlib/utils/drivers.py b/passlib/utils/drivers.py
index d2dc248..9443063 100644
--- a/passlib/utils/drivers.py
+++ b/passlib/utils/drivers.py
@@ -81,7 +81,7 @@ class BaseHash(object):
         "attr for checking if class has ANY settings, memoizes itself on first use"
         if cls.name is None:
             #otherwise this would optimize itself away prematurely
-            raise RuntimeError, "_has_settings must be called on subclass only: %r" % (cls,)
+            raise RuntimeError, "_has_settings must only be called on subclass: %r" % (cls,)
         value = cls._has_settings = bool(cls.setting_kwds)
         return value
 
diff --git a/passlib/utils/md4.py b/passlib/utils/md4.py
index f01ecb0..9821a16 100644
--- a/passlib/utils/md4.py
+++ b/passlib/utils/md4.py
@@ -25,7 +25,7 @@ def new(content=None):
 
 class md4(object):
     """pep-247 compatible implementation of MD4 hash algorithm
-    
+
     .. attribute:: digest_size
 
         size of md4 digest in bytes (16 bytes)
@@ -50,6 +50,7 @@ class md4(object):
     #FIXME: this isn't threadsafe
     #XXX: should we monkeypatch ourselves into hashlib for general use? probably wouldn't be nice.
 
+    name = "md4"
     digest_size = digestsize = 16
 
     _count = 0 #number of 64-byte blocks processed so far (not including _buf)