diff options
Diffstat (limited to 'docs/lib')
34 files changed, 1768 insertions, 0 deletions
diff --git a/docs/lib/bps.basic.rst b/docs/lib/bps.basic.rst new file mode 100644 index 0000000..1698f9d --- /dev/null +++ b/docs/lib/bps.basic.rst @@ -0,0 +1,35 @@ +======================================================== +:mod:`bps.basic` -- Manipulation of basic Python objects +======================================================== + +.. module:: bps.basic + :synopsis: tools for manipulating basic python datatypes + +This module contains utilities for manipulating the basic python +datatypes, like :class:`dict` or :class:`list`. It also +contains functions such as would be found in :mod:`functools` +and :mod:`itertools`, under the rationale that functions +and generators can also be considered basic python objects. + +Dictionary Helpers +================== +.. autofunction:: invert_dict +.. autofunction:: zip_dict +.. autofunction:: unzip_dict +.. autofunction:: pop_from_dict +.. autofunction:: update_dict_defaults +.. autofunction:: prefix_from_dict + +Iterator and Functional Helpers +=============================== +.. autofunction:: iter_unique +.. autofunction:: unique + +Set and Sequence Helpers +======================== +.. autofunction:: intersects +.. autofunction:: sameset + +.. + not documented: + .. autofunction:: revpartial diff --git a/docs/lib/bps.cache.rst b/docs/lib/bps.cache.rst new file mode 100644 index 0000000..aba0e80 --- /dev/null +++ b/docs/lib/bps.cache.rst @@ -0,0 +1,26 @@ +================================= +:mod:`bps.cache` -- Caching Tools +================================= + +.. module:: bps.cache + :synopsis: caching tools + +This module defines a number of function decorators, +most of which come in function- and method- specific +variants, and aid in caching. + +Caching Decorators +================== +These decorators allow for quick "memoization" of a function. + +.. autofunction:: cached_function +.. autofunction:: cached_method + +Stateful Decorators +=================== +These decorators allow for quick and easy setup of callbacks, +allowing the decorated method to alert listeners that a value has changed. + +.. autofunction:: stateful_function +.. autofunction:: stateful_method +.. autofunction:: is_stateful diff --git a/docs/lib/bps.error.types.rst b/docs/lib/bps.error.types.rst new file mode 100644 index 0000000..7cf9baa --- /dev/null +++ b/docs/lib/bps.error.types.rst @@ -0,0 +1,73 @@ +=========================================== +:mod:`bps.error.types` -- BPS Error Classes +=========================================== + +.. module:: bps.error.types + :synopsis: All the BPS Error Classes + +This modules contains all the exceptions classes which BPS +defines, stored in one location for easy access. +Some of these are errors raised by various parts of BPS, +while others are helpers designed to be used inside your own code. +Many of them exist mainly to act as pretty-printed helpers +for specific cases of more generic Python exceptions. + +Attribute Errors +========================= +Helpers for creating explicit :exc:`AttributeError` messages. + +.. autoexception:: MissingAttributeError +.. autoexception:: ReadonlyAttributeError +.. autoexception:: PermanentAttributeError +.. autoexception:: UnsetAttributeError + +Function Errors +=============== +These errors are useful when implementing complicated +python functions. + +.. autoexception:: ParamError +.. autoexception:: NormError +.. autoexception:: RangeError + +.. note:: + BPS 3.x used to define an :exc:`InvariantError` which could be raised + when an internal invariant was violated in an application. + However, the common Python practice seems to be + to raise :exc:`AssertionError`. + Unlike ``assert`` statements, raising this error + directly will not be disabling when in optimized mode. + This, ``InvariantError`` was removed, in favor of :exc:`AssertionError`. + +Reference Errors +================ +These errors will be raised by the :mod:`bps.refs` module. + +.. autoexception:: ProxyEmptyError +.. autoexception:: ProxyNestError + +Meta Errors +=========== + +.. autoexception:: AbstractMethodError + +.. + Command Line Errors: + + These errors are useful when implemented code that's + acting as a command line frontend. They are designed + to integrate well with the :mod:`bps.app.command` framework, + see it for more details. + + .. autoexception:: CommandError + .. autoexception:: ParseError + .. autoexception:: InputError + + + Command Class Errors: + + These errors are useful mainly for :mod:`bps.app.command`, + and will not be needed otherwise. + + .. autoexception:: DistTypeError + .. autoexception:: EnvTypeError diff --git a/docs/lib/bps.error.utils.rst b/docs/lib/bps.error.utils.rst new file mode 100644 index 0000000..b733a06 --- /dev/null +++ b/docs/lib/bps.error.utils.rst @@ -0,0 +1,17 @@ +============================================= +:mod:`bps.error.utils` -- BPS Error Utilities +============================================= + +.. module:: bps.error.utils + :synopsis: Utilties for dealing with errors + +This module contains a few utilties used by BPS +for handling errors. :func:`format_exception` in particular +is used by the :class:`bps.logs.formatters.FancyFormatter` to print tracebacks. + +.. autofunction:: format_exception +.. autofunction:: get_sysexit_rc + +.. seealso:: + :func:`bps.develop.trap` + diff --git a/docs/lib/bps.fs.rst b/docs/lib/bps.fs.rst new file mode 100644 index 0000000..8b5bb55 --- /dev/null +++ b/docs/lib/bps.fs.rst @@ -0,0 +1,20 @@ +======================================== +:mod:`bps.fs` -- Filesystem Interaction +======================================== + +.. module:: bps.fs + :synopsis: filesystem interaction + +This module provides a clean object-oriented interface +to the host filesystem, in the form of the :class:`FilePath` object, +as well as a number of additional utilities for accessing the filesystem +and managing permissions: + +* :doc:`The filepath object <bps.fs/filepath>` +* :doc:`Other filesystem utilities <bps.fs/utils>` + +.. toctree:: + :hidden: + + bps.fs/filepath + bps.fs/utils diff --git a/docs/lib/bps.fs/filepath.rst b/docs/lib/bps.fs/filepath.rst new file mode 100644 index 0000000..2ce129f --- /dev/null +++ b/docs/lib/bps.fs/filepath.rst @@ -0,0 +1,91 @@ +==================================== +:mod:`bps.fs` -- The filepath object +==================================== + +.. module:: bps.fs + :synopsis: filesystem interaction + +Overview +======== + +This module provides a clean object-oriented interface +to the host filesystem, in the form of the :class:`FilePath` object. +Objects of this class act just like strings (they are in fact a subclass), +but they contain additional attributes and methods for manipulating +them as paths and interacting with the local filesystem. +The methods and attributes wrap functionality available in the :mod:`os`, +:mod:`os.path`, and :mod:`shutils` modules, and while the full contents +of those modules is not directly available, the common ones are, +and more are added frequently. + +Usage +===== + +Usage is very simple, just call the :func:`filepath` function +with a string, and a new :class:`FilePath` object +will be returned which will act exactly like the original +string, but with additional methods for manipulating the filesystem. + +Some examples using the filepath object:: + + >>> #this imports the default bps3 objects, + >>> #you can alternately use "from bps.fs import filepath" + >>> from bps import * + >>> #this example code assumes the current directory is the bps3 source dir + >>> path = filepath(".") + >>> path #it looks like a string + '.' + >>> type(path) #but it's not + <class 'bps.fs.FilePath'> + >>> #get the absolute path (your exact path will vary) + >>> path.abspath + '/home/elic/dev/libs/bps' + >>> #get a directory listing + >>> path.listdir() + [ '.svn', 'bps', 'docs', 'tests', 'setup.cfg', 'setup.py', 'bps.e4p' ] + >>> #join paths together, equivalent of os.path.join... + >>> #note that this will always use the host-specific path separator + >>> docs = path / "docs" / "_static" + >>> docs + './docs/_static' + >>> #note that under windows, this would appear as '.\\docs\\_static' + >>> #get the absolute version of the path (your result will vary) + >>> docs.abspath + '/home/elic/dev/libs/bps/docs/_static' + >>> #check the filetype of a path + >>> docs.ftype + 'dir' + >>> #touch a path (updating it's mtime & atime) + >>> docs.touch() + + +Creating Filepaths +================== +.. autofunction:: filepath + +Using Filepaths +=============== +.. autoclass:: FilePath + + .. warning:: + + Relative paths will always be relative to the current working directory + *at the time of the method call*, so changing the cwd will usually + result in a different outcome when the instance in question + references a relative path. + +.. note:: + + :class:`FilePath` will probably *never* be extended to include urls + and other similar resources: that was tried in an earlier iteration + of this library, and it was determined there was so little + commonality between the two (filepaths and urls), both in terms + of interface, code, and use cases, that tieing them together + would confusing and without much benefit. A similar-but-separate + UrlPath may be added in the future, however. + +.. todo:: + + :class:`FilePath` needs to support unicode. + Currently waiting for the Python 3.x design team to provide + some guidance-by-example on how to handle differing OS encoding policies. diff --git a/docs/lib/bps.fs/utils.rst b/docs/lib/bps.fs/utils.rst new file mode 100644 index 0000000..a58c4f3 --- /dev/null +++ b/docs/lib/bps.fs/utils.rst @@ -0,0 +1,47 @@ +=========================================== +:mod:`bps.fs` -- Other Filesystem Utilities +=========================================== + +.. currentmodule:: bps.fs + +In addition to :func:`filepath`, this module also provides +some additional utility functions for manipulating the filesystem. + +Permissions +=========== +The following functions deal with file access permissions +(mainly unix-centric, though they are functional under windows). +While they are ostensibly wrappers for similar functions +available under :mod:`os`, these versions provide some enhanced capabilities: + +.. autofunction:: chmod +.. autofunction:: setumask +.. autofunction:: getumask + +Mode Parsing +------------ +To help manipulate symbolic mode strings, +the following helper functions are available: + +.. autofunction:: parse_mode_mask +.. autofunction:: repr_mode_mask + +Other Functions +=============== +This module provides some additional functions for interacting with the filesystem: + +.. autofunction:: is_filepath +.. autofunction:: posix_to_local +.. autofunction:: local_to_posix +.. autofunction:: splitsep +.. autofunction:: is_shortcut +.. autofunction:: read_shortcut + +.. data:: os_has_symlinks + + This is a module-level constant set to ``True`` if the os supports symlinks. + +.. data:: os_has_shortcuts + + This is a module-level constant set to ``True`` if the os supports windows shortcuts (aka LNK files). + This will only be true under windows, though :func:`read_shortcut` will work cross-platform. diff --git a/docs/lib/bps.host.posix.rst b/docs/lib/bps.host.posix.rst new file mode 100644 index 0000000..7f616fb --- /dev/null +++ b/docs/lib/bps.host.posix.rst @@ -0,0 +1,23 @@ +================================================= +:mod:`bps.host.posix` -- Posix-specific Utilties +================================================= + +.. module:: bps.host.posix + :platform: posix + :synopsis: posix-specific utilities + +This contains a number of posix-specific helper functions. +They are either very posix-specific, or simply haven't +been rolled into a common function in :mod:`bps.host` +along with compatriots from other OS modules... + +.. autofunction:: resolve_uid + +.. autofunction:: resolve_gid + +.. autofunction:: resolve_user + +.. autofunction:: resolve_group + +.. autofunction:: chown + diff --git a/docs/lib/bps.host.rst b/docs/lib/bps.host.rst new file mode 100644 index 0000000..63cea1f --- /dev/null +++ b/docs/lib/bps.host.rst @@ -0,0 +1,128 @@ +========================================== +:mod:`bps.host` -- Locating Host Resources +========================================== + +.. module:: bps.host + :synopsis: host resource discovery & desktop interaction + + +This package provides methods for accessing various host resources, +much like stdlib's ``os`` package. In fact, this package +mainly exists to provide routines which ``os`` does not provide, +for one reason or another. + +The this module is broken into the following sections: + +* `Process Management`_ -- OS-agnostic signaling & pid management +* `System Interaction`_ -- finding installed applications +* `Desktop Interaction`_ -- opening, printing, executing files via desktop environment +* `Resource Paths`_ -- helpers for locating home dir, desktop, user config directory, and more. +* `User Accounts`_ -- retrieve basic information about the user accounts on the host system. + +.. toctree:: + :maxdepth: 2 + + bps.host.posix + bps.host.windows + bps.host.utils + +.. note:: + + The main two reasons many of these functions probably are not included in the stdlib + is that this module relies on `pywin32 <http://sourceforge.net/projects/pywin32/>`_ under Windows, + and the fact that this module makes some arbitrary decisions + about path locations which work 90% of cases, but not the 100% that the stdlib requires. + +Usage +===== +The typical use of this module's core functions is to import ``bps.host`` into +your package, and then access it's various methods from the imported object:: + + >>> #note that while this example was written under linux, the host module interface + >>> #is designed to be uniform, so that you can use the *exact same calls* + >>> #to acheive the same effect under windows, without changing your code. + >>> from bps import host + >>> + >>> #check what desktop environment you're running under + >>> host.get_desktop_name() + 'gnome' + >>> + >>> #find location of an executable + >>> host.find_exe("meld") + '/usr/bin/meld' + >>> + >>> #tell desktop to open a file + >>> host.desktop_open("myfile.txt") + >>> + >>> #get current pid + >>> host.get_pid() + 12984 + >>> + >>> #check if a pid is running + >>> host.has_pid(12984) + True + >>> + >>> #kill a pid + >>> host.term_pid(12984) + >>> + +Process Management +================== + +.. function:: get_pid + + Returns current PID. + Alias for ``os.getpid()``, included just for symmetry with the other pid functions. + +.. autofunction:: has_pid +.. autofunction:: term_pid +.. autofunction:: kill_pid + +System Interaction +================== +.. autofunction:: find_exe + +.. attribute:: exe_exts + + This should be a tuple of all the extensions that will be searched + when trying to find an exe. For example, under posix, the list will be ``('',)``, + but under windows the tuple will contain ``('.exe','.bat')``. + +.. todo:: + Would like to add database for detecting & locating applications via windows registry, + or other methods. + +Desktop Interaction +=================== + +.. autofunction:: get_desktop_name +.. autofunction:: desktop_open +.. autofunction:: desktop_compose_email + +Resource Paths +============== +All the resource path functions are designed to quickly +locate the directories that are important to a cross-platform +desktop application, without having to know os-specific details... + +.. autofunction:: get_env_path + +.. autoclass:: EnvPaths + +---- + +The following functions return a :class:`ProgPaths` instance, +with various resource paths chosen according to the default conventions +of the OS you are currently running on, allowing quick and easy +creation of applications which store their config in the right place +no matter what OS you run them on... + +.. autofunction:: get_app_path +.. autofunction:: get_service_path +.. autoclass:: ProgPaths + +User Accounts +============= +.. autofunction:: find_user + +.. autoclass:: UserProfile diff --git a/docs/lib/bps.host.utils.rst b/docs/lib/bps.host.utils.rst new file mode 100644 index 0000000..b3c40b6 --- /dev/null +++ b/docs/lib/bps.host.utils.rst @@ -0,0 +1,20 @@ +=========================================== +:mod:`bps.host.utils` -- General Utilties +=========================================== + +.. module:: bps.host.utils + +Signals +======= +The signal functions provide a enhanced interface +to stdlib's :mod:`signal` module. Much like :mod:`atexit` +enhances the ``sys.exitfunc``, these utilties +allow multiple handlers to be chained to a given unix signal. + +.. autofunction:: has_signal + +.. autofunction:: add_signal_handler + +.. autofunction:: remove_signal_handler + +.. autofunction:: adapt_sig_term diff --git a/docs/lib/bps.host.windows.rst b/docs/lib/bps.host.windows.rst new file mode 100644 index 0000000..120d3a8 --- /dev/null +++ b/docs/lib/bps.host.windows.rst @@ -0,0 +1,23 @@ +====================================================== +:mod:`bps.host.windows` -- Windows-specific Utilities +====================================================== + +.. module:: bps.host.windows + :platform: nt + :synopsis: windows-specific utilities + +This contains a number of windows-specific helper functions. +They are either very windows-specific, or simply haven't +been rolled into a common function in :mod:`bps.host` +along with compatriots from other OS modules... + +.. autofunction:: regpath + +.. autoclass:: RegistryPath + +.. autofunction:: reghandle + +.. autoclass:: RegistryHandle + +.. autofunction:: detect_office_app + diff --git a/docs/lib/bps.logs-config.rst b/docs/lib/bps.logs-config.rst new file mode 100644 index 0000000..521f027 --- /dev/null +++ b/docs/lib/bps.logs-config.rst @@ -0,0 +1,73 @@ +Overview +======== +Python's logging system offers two levels +of interaction when you want to configure the +loggers: you can either interact with +the low-level logger, handler, and formatter objects; +or you can hand it a filepath to a separate file +containing a monolithic configuration of the entire +system. + +The :mod:`bps.logs` package attempts to fill in the +programmatic "middle ground" between these two +styles, through it's :func:`parse_config` +and :func:`config_logging` functions. +Take a large number of input styles, +included external files or strings +containing a full configuration file, +or fragments, all of which are normalized +into a standard dictionary-based data structure. +This may then be manipuled programmatically, re-normalized, +or passed on to the the configuration function, +allowing for complex configuration needs +to be accomplished with a few short commands. + +Normalized Configuration Structure +================================== +The data structure which BPS uses +to represent a set of changes to be applied +to the logging system's configuration is a dictionary +which contains certain predefined keys (none are required unless otherwise noted). +The value attached to each key has a "normalized" format, +which will be the format it is in as returned by :func:`parse_config`, +but there are also other "input" formats, which will be accepted +by :func`parse_config` and returned normalized. +The following keys are recognized: + + ``"levels"`` + If present, this should be a dictionary whose keys + are the names of logger objects, and the corresponding + values the level that logger should be set to. + + formatters + [Optional] + This should be a dictionary mapping formatter names to dicts of formatter options, + to be passed to compile_formatter(). The names may be referred to by the handlers. + handlers + [Optional] + This should be a dictionary mapping handlers names to dicts of handlers options, + to be passed to compile_handler(). The names may be referred to be the output section. + outputs + [Optional] + This should be a dictionary mapping loggers to lists of handler names, + as specified in the handler section, or in the default handler presets. + +The following keywords are accepted by :func:`parse_config`, +and will be merged into one of the above keys during normalization: + + ``"level"`` + This keyword specifies the logging level used by the root logger. + This is a shortcut allowing the master level to be set quickly, + without needing to create a dictionary. + + It will be used as the default value for the "<root>" key + inside the "levels" dictionary (above). + + ``"default_handler"`` + This is a shortcut which allows you to specify just a keywords + for creating a handler, but which will result in all the commands + needed to create the handler and attach it as the sole output + for the root logger. For example, setting ``default_handler=opts`` + will result in the following normalized options: + ``output="<root>=default only", handlers=dict(default=opts)``. + diff --git a/docs/lib/bps.logs-config_format.rst b/docs/lib/bps.logs-config_format.rst new file mode 100755 index 0000000..5744702 --- /dev/null +++ b/docs/lib/bps.logs-config_format.rst @@ -0,0 +1,283 @@ +======================= +BPS Logging File Format +======================= + +The BPS Logging Format is an alternate ini-based file format +for configuring python's builtin logging system. Both this format +and the stdlib format are accepted (and auto-detected) by :func:`bps3.logs.config_logging`. + +.. warning:: + + This documentation currently assumes you are familiar with + the python logging package, it's standard format, + and it's object system. There may eventually be a rewrite to + correct this. + +Why another format? +=================== +Python's builtin logging system specifies a format for configuring the logging +system [#stdfmt]_. While this format offers the ability to configure every +aspect of the logging system, the manner in which it does this is somewhat +verbose, makes some simple tasks much more time consuming than they need to be, +and deciphering an existing config file is not the trivial task it should be. + +A prime example of this issue is configuring the logging levels of a number +of loggers at once. Under the stdlib logging format, you would need to do +the following: + +.. code-block:: cfg + + [loggers] + keys=root,app,app.model,mylib + + [logger_root] + level = WARNING + + [logger_app] + level = DEBUG + + [logger_app.model] + level = INFO + + [logger_mylib] + level = DEBUG + +For doing development work, where various loggers may need to be added and +removed frequently, this format becomes incredibly cumbersome. This +was the main motivation for creating a new format. Under the BPS Logging Format, +the equivalent commands to acheive the above would be: + +.. code-block:: cfg + + [logging:levels] + <root> = WARNING + app = DEBUG + app.model = INFO + mylib = DEBUG + +While a couple of rare features of the stdlib format have not been replicated +in the new format, work is ongoing, and the majority of the features have been +converted over into what is hoped to be a more consise, understandable, and +easily editable format. + +Format Overview +=============== +The BPS Logging Format is based around the ConfigParser's file format. +It defines the following section names, all of which begin with the prefix +``logging:``, and sections lacking this prefix will be ignored. +None of the following sections are required, except where interdepedant +references exist. The sections are as follows: + + `logging:levels`_ + This section lets you configure the logging levels for any logger + in the logging system. + + `logging:options`_ + This section lets you set various global logging system options, + including some custom extensions provided by BPS. + + `logging:output`_ + This section maps loggers to handlers, + allowing you to control where the output of the logging system + is going. + + `logging:handler:$NAME`_ + Sections of this type (eg `logging:handler:myhandler`) define + the configuration to be used when a handler name is referenced + in the `logging:output`_ section. + + `logging:formatter:$NAME`_ + Sections of this type (eg `logging:formatter:myformatter`) define + the configuration to be used when a formatter name is referenced + in a `logging:handler:* <logging:handler:$NAME>`_ section. + +logging:levels +-------------- +This section lets you configure the logging levels for any logger +in the logging system. + +The keys in this section correspond to logger names, +and the values to a predefined logging level. This logging level can +be a predefined name (eg ``NOTSET``, ``DEBUG``, etc), or an integer value ( ``0``, ``10``, etc). +Spaces in the logging level will be ignored, as will any text following a ``#`` symbol, +allowing in-line comments. + +The logger name of ``<root>`` is interpreted as a convenient alias for the empty string, +which corresponds to the root logger of python's logging system. All other logger names +which start with ``<``, contain a series of letters, and end with ``>``, +are considered reserved by this format, for use in an grouping/alias system which is still under development. + +A very verbose example of the ``logging:levels`` section, showing off the various options: + +.. code-block:: cfg + + [logging:levels] + + #this is an example of a full-line comment + + #this will set the root logger level + <root> = WARNING + + app = DEBUG #this is an example of a in-line comment + + #note that "#WARNING" below will be ignored + app.model = INFO #WARNING + + #this uses an integer level + mylib = 10 + +A more compact example, without all the comments: + +.. code-block:: cfg + + [logging:levels] + <root> = WARNING + app = DEBUG + app.model = INFO + mylib = 10 + +.. note:: + If a undefined textual logging level is specified, + a :exc:`KeyError` will be raised at the time this file is loaded. + +logging:options +--------------- + +This section controls for the python logging system. +The following keys are currently recognized (unrecognized +keys will be ignored): + + ``capture_stdout`` + This is a boolean keyword. If set to ``true``, + standard output will be captured, and re-routed to + a logger object named ``sys.stdout``. + If set to ``false``, and stdout is currently being + captured by BPS, the capturing of stdou will be stopped. + + See :mod:`bps3.log.capture` for details. + + ``capture_stderr`` + This functions identically to ``capture_stdout``, + except that it operates on standard error. + + ``capture_warnings`` + This functions similarly to ``capture_stdout``, + except that it captures the warning issued by the + python :mod:`warning` module, and sends such messages + to the logger appropriate for the module which issued + the warning. + + *Setting this option is HIGHLY recommended*, as it will + integrate the warnings module into the logging system + (how python should have had it to begin with). + + ``warning_fmt`` + When used with ``capture_warnings``, this option + allows you to specify a custom warning format string. + See :func:`capture_warnings` for details about the format + of this string, which correponds to the ``fmt`` keyword. + + ``warning_target`` + When used with ``capture_warnings``, this options + allows you to specify a custom target for any warnings + sent to the logging system. + See :func:`capture_warnings` for details about the format + of this string, which correponds to the ``target`` keyword. + +As an example, the following configuration snippet captures +everything from stdout and warnings, and leaves stderr alone: + +.. code-block:: cfg + + [logging:options] + capture_warnings = true + #if no warning_fmt is specified, the default will be used: + #warning_fmt = %(category)s:\n\t message: %(message)s\n\tfilename: %(filename)s\n\t lineno: %(lineno)s + + capture_stderr = true + + #uncomment this next to explicitly release stdout + #capture_stdout = false + +logging:output +-------------- +This section maps loggers to handlers, allowing you to control where the output of the logging system +is going. It consists of "name = handler1, handler2, ..." entries, +which have the effect of attaching (one or more) handlers to the named logger. +If a given entry ends with ``" only"``, any existing handlers attached to the logger +will be removed before adding the specified handlers, and messages +will not propagate past this logger. + +.. todo:: + give examples + +logging:handler:$NAME +--------------------- +If a handler is specified by name in `logging:output`_, +the configuration loader will look for a section with +the corresponding name to determine the handler's class +and configuration. If a handler entry is present, +but not referenced by the `logging:output`_ section +of the that file, it will be ignored. + +It consists of keyword arguments passed to the +:func:`compile_handler` function, which has pretty much +the same syntax as the `fileConfig` format. + +.. todo:: + document keywords, give examples + +logging:formatter:$NAME +----------------------- +This section configures a named formatter, +and must be present for all formatters +referenced in a ``[logging:handler:$NAME]`` section. +It consists of keyword arguments passed to the +`create_formatter` function, which has pretty much +the same syntax as the `fileConfig` format. + + +Example Files +============= + +An example of a full-featured logging config file, +which is probably overkill for a typical application: + +.. code-block:: cfg + + [logging:options] + capture_stdout = false + capture_warnings = true + warning_fmt = %(category)s: %(message)s + + [logging:levels] + <root> = INFO + myapp = DEBUG + pylons = WARNING + + [logging:output] + <root> = console + myapp = syslog + + [logging:handler:console] + class = StreamHandler + args = (sys.stderr,) + level = NOTSET + formatter = generic + startup_msg = True + + [logging:handler:syslog] + class=handlers.SysLogHandler + level=ERROR + formatter=generic + args=(('localhost', handlers.SYSLOG_UDP_PORT), handlers.SysLogHandler.LOG_USER) + + [logging:formatter:generic] + format = %(asctime)s,%(msecs)03d %(levelname)-5.5s [%(name)s] %(message)s + datefmt = %H:%M:%S + +============= + +.. rubric:: Footnotes + +.. [#stdfmt] `<http://docs.python.org/library/logging.html#configuration-file-format>`_ diff --git a/docs/lib/bps.logs.rst b/docs/lib/bps.logs.rst new file mode 100644 index 0000000..add9fad --- /dev/null +++ b/docs/lib/bps.logs.rst @@ -0,0 +1,34 @@ +===================================== +:mod:`bps.logs` -- Logging Utilities +===================================== + +.. module:: bps.logs + :synopsis: logging utilities + +This module provides a number of extensions to the standard python logging module. +Features include: + + * `setup_std_logging`: a replacement for ``basicConfig()`` for initializing the logging system, + This features many additional features, such as improved default heuristics, + the ability to capture stdout, stderr, and python warnings and rerouting them + through the logging system, etc. + * `config_logging`: a replacement for ``fileConfig()`` which supports a more compact file format. + * `FancyFormatter`: a formatter which supports numerous formatting options + * `log`: a intelligent proxy Logger, which uses module inspection to determine which logger it should invoke + +General usage: + + Call setupLogging() to initialize logging system, + and/or call configLogging(path) to load configuration from a file. + Most of the components (log, the formatters, etc) are designed + to be used individually, they don't require use of any of the other + bps3.logs components. + +.. toctree:: + + bps.logs-config_format + +.. todo:: + + document this module + diff --git a/docs/lib/bps.meta.rst b/docs/lib/bps.meta.rst new file mode 100755 index 0000000..6d11846 --- /dev/null +++ b/docs/lib/bps.meta.rst @@ -0,0 +1,42 @@ +=============================================================== +:mod:`bps.meta` -- Introspection & Monkeypatching +=============================================================== + +.. module:: bps.meta + :synopsis: introspection & monkeypatching + +This module contains various utilities introspection +utilities, to enhance what is already provided through +python's :mod:`inspect` module, + +Interface Tests +=============== +.. autofunction:: is_class +.. autofunction:: is_num +.. autofunction:: is_seq +.. autofunction:: is_oseq +.. autofunction:: is_str + +Class Inspection +================ +.. autofunction:: is_overridden +.. autofunction:: find_attribute + +Module Inspection +================= +.. autofunction:: get_module_exports + +Decorators +=========== +.. autofunction:: abstractmethod +.. autofunction:: decorate_per_instance + +Autosuper +========= +.. autofunction:: instrument_super + +Monkeypatching +============== +.. autofunction:: monkeypatch +.. autofunction:: monkeypatch_mixin + diff --git a/docs/lib/bps.misc.rst b/docs/lib/bps.misc.rst new file mode 100644 index 0000000..e39affb --- /dev/null +++ b/docs/lib/bps.misc.rst @@ -0,0 +1,23 @@ +=============================================== +:mod:`bps.misc` -- Miscellanous Utilities +=============================================== + +.. module:: bps.misc + :synopsis: utilities which fit in no other category + +This module contains everything which didn't naturally +fit in any category handled by an existing module. + +Entries in this module will occasionally be moved +into other modules after an appropriate category is created. +That is done on an as-needed basis when enough related +functions are collected in this module. + +Properties +========== +.. autofunction:: indirect_property +.. autofunction:: constructor_property + +Functions +========= +.. autofunction:: stepped_delay diff --git a/docs/lib/bps.numeric.rst b/docs/lib/bps.numeric.rst new file mode 100644 index 0000000..1fff928 --- /dev/null +++ b/docs/lib/bps.numeric.rst @@ -0,0 +1,55 @@ +================================================= +:mod:`bps.numeric` -- Numeric Tools +================================================= + +.. module:: bps.numeric + :synopsis: mathematical and numeric tools + +Number Theory +============= +.. autofunction:: factors +.. autofunction:: gcd +.. autofunction:: lcm + +Primality Testing +================= +.. autofunction:: is_prime +.. autofunction:: next_prime +.. autofunction:: prev_prime +.. autofunction:: iter_primes + +Numeric Formats +=============== +.. autofunction:: int_to_base +.. autofunction:: float_to_base +.. autofunction:: int_to_roman +.. autofunction:: roman_to_int + +Miscellaneous Functions +======================= +.. autofunction:: sdivmod +.. autofunction:: splitfrac +.. autofunction:: avgsd +.. autofunction:: digits +.. autofunction:: limit + +Bytes Strings +============= +The following functions manipulate strings +as if they were binary data, not characters. +They allow for doing bit-wise boolean operations +on strings, converting them to integers, etc. + +.. note:: + When this module is converted to Python 3.0, + these will all be operations on ``bytes``, not ``str``. + +.. autofunction:: int_to_bytes +.. autofunction:: bytes_to_int +.. autofunction:: list_to_bytes +.. autofunction:: bytes_to_list +.. autofunction:: xor_bytes +.. autofunction:: or_bytes +.. autofunction:: and_bytes +.. autofunction:: invert_bytes +.. autofunction:: binop_bytes diff --git a/docs/lib/bps.refs.rst b/docs/lib/bps.refs.rst new file mode 100644 index 0000000..f23aa8a --- /dev/null +++ b/docs/lib/bps.refs.rst @@ -0,0 +1,22 @@ +================================================= +:mod:`bps.refs` -- Weak Reference and Proxy Tools +================================================= + +.. module:: bps.refs + :synopsis: weak reference and proxy tools + +Properties +========== +.. autofunction:: weakref_property + +Classes +======= +.. autoclass:: SoftValueDict +.. autoclass:: WeakSet + +Proxies +======= +.. autoclass:: ProxyObject + +.. autofunction:: is_proxy_active +.. autofunction:: proxy_using_object diff --git a/docs/lib/bps.rng.rst b/docs/lib/bps.rng.rst new file mode 100644 index 0000000..5869ff4 --- /dev/null +++ b/docs/lib/bps.rng.rst @@ -0,0 +1,71 @@ +================================================= +:mod:`bps.rng` -- Random Number Generation +================================================= + +.. module:: bps.rng + :synopsis: random number generation + +This module is essentially just a wrapper for stdlib's +random module. It provides a few additional +methods for managing & getting random numbers, +but also provides a more useful interface +for the *type* of randomness you want. + +Random Number Generators +======================== +The following random number generator +instances are always available from this module: + +.. data:: random + + This will be an instance of the best pseudo random number generator + available (currently the python builtin prng), with as good + an entropic source as is available for seeding via + the seed() and reseed() methods. + Use this for most non-cryptographic purposes. + +.. data:: srandom + + This will be an instance of the strongest random number generator + available on your system. It will use python's SystemRandom + if os.urandom support is available, otherwise it will fall back + to the same generator as prandom. This should be used + for cryptographic purposes over the normal prng. + + .. warning:: + If urandom is present, this is dependant on the strength + of your system's urandom implementation. If urandom is missing, + the fallback (normal) may not have enough entropy to defend + from attackers. To help this somewhat, it is recommended + to call ``strong.reseed()`` before calls which will consume + randomness for critical purposes, just to help scramble things + as best as possible (reseed is a no-op if urandom is being used). + +.. data:: drandom + + This is a variant of the *random* generator, + except that all outside entropic + sources are disabled, so that it's state is completely + deteremined by the value passed into seed(). + + This is mainly useful in unitests, when you need + to reliably repeat the same results over an over. + +Extra Methods +============= +In addition to the methods provided by stdlib's random module, +all the above rngs will contain the following extra methods: + +.. function:: reseed() + + Unlike seed(), which attempts to set the random number generator's + state explicitly, this method attempts to pull in outside + entropy sources (current rng state, time, etc) to help + randomize the state of your prng as much as possible. + + .. todo:: + In the future, need a way for app to add entropy to the system. + +.. function:: getrandbytes() + +.. function:: weightedchoice() diff --git a/docs/lib/bps.rst b/docs/lib/bps.rst new file mode 100644 index 0000000..6956ec4 --- /dev/null +++ b/docs/lib/bps.rst @@ -0,0 +1,79 @@ +============== +Global Exports +============== + +This page deals lists the objects that are imported +when using the command ``from bps import *``. + +While the BPS package is generally accessed by importing +one one of the submodules, it does offer a limited list +of exports, which are designed to be dropped into your +module's global namespace directly. + +Since populating the global namespace like this usually +causes havoc due to it's implict nature, the objects +exported by default have been limited only to ones +which the BPS authors felt day in and day out were +going to be needed so often, and so unpredictably, +that it would be nice if they were available almost like builtins. +Thus, much of our code begins with the stanza:: + + >>> #import from the bps global namespace + >>> from bps import * + +This ensures a number of very useful objects +are always available. But since this import can be abused, +objects are very rarely added to this list. + +Exported Objects +================ +The following objects will be exported by ``from bps import *``. +While they are documented more fully elsewhere, here is a quick description: + + :func:`abstractmethod() <bps.meta.abstractmethod>` + + This is a very useful decorator to have around if you do a lot + of interface-style class creation. + + .. note:: + A native version has been introduced + in Python 2.6, but that is not yet used by BPS. + + :class:`BaseClass <bps.types.BaseClass>` + + This class can be used as a drop-in replacement for ``object``, + it provides features such as an intelligent ``self.__super`` method, + and a ``cls.__initsubclass__`` method for performing actions + based on the created of inherited classes. + + :func:`filepath() <bps.fs.filepath>` + + This is the constructor for BPS's all-singing-all-dancing filepath object. + It's so useful, this was the first global export added. + Never use `os.path` again! + + :func:`log <bps.logs.log>` + + This is a magic logger object. + Import it into your module and call it, + and through introspection, it will act just like ``logging.getLogger(__name__)``, + and log all messages to the name of the module it was called from. + + :func:`partial` + This is an export of stdlib's `functools.partial <http://docs.python.org/library/functools.html#functools.partial>`_, + since it is used a lot (at least, by the BPS developers it is). + An implementation of this has been exported by BPS since it's inception, + it was only when Python 2.5 was set as a minimum requirement + that BPS started using the stdlib version. + + :data:`Undef <bps.types.Undef>` + + A companion to ``None`` which represents an undefined/missing value. + Same as javascript's "undefined". + + :func:`warn` + + This is an export of stdlib's `warnings.warn <http://docs.python.org/library/warnings.html#warnings.warn>`_. + Warnings should be used much more often then they are, + and that would be encouraged if not for the inconvience + of having to add an import stanza at the top. diff --git a/docs/lib/bps.security.policy.rst b/docs/lib/bps.security.policy.rst new file mode 100644 index 0000000..e6135de --- /dev/null +++ b/docs/lib/bps.security.policy.rst @@ -0,0 +1,106 @@ +================================================================== +:mod:`bps.security.policy` -- Lightweight Access Control Framework +================================================================== + +.. module:: bps.security.policy + :synopsis: lightweight access control framework + +Overview +======== +This module provides a framework for applications +to build complex permission and security policies, +centered around the common "user -> role -> permission" pattern. +This framework is derived from one deployed in a few web applications, +which in turn was inspired by Roundup's `access control mechanism <http://www.roundup-tracker.org/docs/design.html#access-control>`_. +Never-the-less, it's generic enough that it should be suitable for use +by gui and command line applications as well. + +An application can make use of this framework by: + + * creating a :class:`Policy` instance for the application. + * registering all potential roles with the policy + * registering all permissions, as expressed + in terms of actions, roles, and optional guard functions. + * querying the policy either to enumerate a given user's permissions, + or check if the user has permission to perform a specific action. + +.. _permission-question: + +Framing a Permission Question +============================= + +.. todo:: finish write up the structure of the "permission question" + +When an application needs to test whether a user has permission +to perform a given action, the first thing that must be done +to use any policy framework is to encode the question in a format +the permission system understands. This module encodes +permission questions using the following 5 parameters: + + * ``action`` - a string, usually a verb such as ``"update"``, + which represents the action permission is being requested for. + + * ``klass`` - optional string, usually a noun such as ``"BlogEntry"`` + which the action will be acting upon. (Some actions + act globally, and won't have a class specified). + + * ``item`` - optional object, usually an instance of the class identified by ``klass``. + This is generally used when the permission applies to only certain instances + of the class, which must be decided on a case-by-case basis. + + * ``attr`` - optional string, usually a attribute of ``klass`` such as ``"date"``. + This is typically used when the action is restricted on a per-attribute basis. + + * ``scope`` - optional object, usually the owner of the instance + being acted on, or a composite object which the action is being + performed inside of. This is needed very rarely, but there are + some cases, such as when requesting permission to create + a new instance of class which will be stored inside a particular + object, and that object affects the outcome of the permission check. + +Combinations of 1 or more of these parameters can be put together +in order to encode the following questions: + + 1. ``Does {user} have permission to {action}?`` + + 2. ``Does {user} have permission to {action} an object of type {klass}?``. + + 3. ``Does {user} have permission to {action} the object {item} of type {klass}?`` + + 4. ``Does {user} have permission to {action} the attribute {attr} of an object of type {klass}?`` + + 5. ``Does {user} have permission to {action} the attribute {attr} of the object {item} of type {klass}?`` + + 6. ``Does {user} have permission to {action} an object of type {klass} as part of {scope}?``. + As an exmaple: does the user have permission to *create* an object of type + *entry* as part of *<a specific journal instance>*? + +Usage Example +============= + +.. todo:: write usage example for sec policy + +The Policy Class +================ + +.. autoclass:: Policy + +The Support Classes +=================== +The following classes are used internally by :class:`Policy`, +and generally the programmer will not need to create them directly +(though they may need to examine them if preparing a list of +the user's permissions for display). + +.. autoclass:: Role + +.. autoclass:: Permission + +.. + Not documenting these right now, the system is usuable without + knowledge of this bit, although they could be usuable by the guard func, + but no use-case has needed this just yet: + + .. _permissions-constants: + + .. autoclass:: PERM diff --git a/docs/lib/bps.security.pwgen.rst b/docs/lib/bps.security.pwgen.rst new file mode 100644 index 0000000..c657b12 --- /dev/null +++ b/docs/lib/bps.security.pwgen.rst @@ -0,0 +1,15 @@ +================================================ +:mod:`bps.security.pwgen` -- Password Generation +================================================ + +.. module:: bps.security.pwgen + :synopsis: password generation algorithms + +The following single function allows +easy password generation in a number of styles: + +.. autofunction:: generate_secret + +.. todo:: + document internal classes + diff --git a/docs/lib/bps.security.pwhash.rst b/docs/lib/bps.security.pwhash.rst new file mode 100644 index 0000000..b2f0aa1 --- /dev/null +++ b/docs/lib/bps.security.pwhash.rst @@ -0,0 +1,44 @@ +============================================= +:mod:`bps.security.pwhash` - Password Hashing +============================================= + +.. module:: bps.security.pwhash + :synopsis: password hashing (unix-crypt, md5-crypt, etc) + +Overview +======== +This module handles encrypting and verifying password hashes +(such as from unix shadow files). This module contains implementations of most +of the modern password hashing algorithms, +as well as a complex framework for implementing +new algorithms, managing hashes generated +within different contexts with different supported +algorithms, and other features. + +The algorithms currently supported by default in BPS: + + * Unix-Crypt + * MD5-Crypt + * BCrypt + * SHA-Crypt (256 & 512 bit modes) + + * PostgreSQL & MySQL password hashes + +Sections +======== +The documentation for the pwhash module is broken into the following sections: + +* :doc:`Quick Start <bps.security.pwhash/quickstart>` -- frontend funcs for quickly creating / validating hashes +* :doc:`Crypt Contexts <bps.security.pwhash/contexts>` -- for using just the algorithms your application needs +* :doc:`Crypt Algorithms <bps.security.pwhash/algorithms>` -- details of the algorithms BPS implements +* :doc:`Implementing a Custom Crypt Algorithm <bps.security.pwhash/implementation>` -- Roll your own +* :doc:`Helper Functions <bps.security.pwhash/utils>` + +.. toctree:: + :hidden: + + bps.security.pwhash/quickstart + bps.security.pwhash/contexts + bps.security.pwhash/algorithms + bps.security.pwhash/implementation + bps.security.pwhash/utils diff --git a/docs/lib/bps.security.pwhash/algorithms.rst b/docs/lib/bps.security.pwhash/algorithms.rst new file mode 100644 index 0000000..6973313 --- /dev/null +++ b/docs/lib/bps.security.pwhash/algorithms.rst @@ -0,0 +1,49 @@ +============================================= +:mod:`bps.security.pwhash` - Crypt Algorithms +============================================= + +.. currentmodule:: bps.security.pwhash + +All of the crypt algorithms must inherit from :class:`CryptAlgorithm`, +which defines a common interface all algorithms must support. +You may use the algorithms directly, by creating +an instance and calling it as described in :doc:`Implementing a Crypt Algorithm <implementation>`. +However, you will normally will not need to deal with the internals of the algorithms +directly, but rather take advantage of one of the predefined algorithms, +through the :doc:`frontend functions <quickstart>` or a +custom :doc:`crypt context <contexts>`. + +Standard Algorithms +=================== +The following algorithms are all standard password hashing algorithms +used by various Posix operating systems over the years. + +.. note:: + BPS tries to use external accelaration for these classes when possible, + but provides a pure-python fallback so that these algorithms will + ALWAYS be available for use. + +.. autoclass:: UnixCrypt +.. autoclass:: Md5Crypt +.. autoclass:: Sha256Crypt +.. autoclass:: Sha512Crypt +.. autoclass:: BCrypt + +Database Algorithms +=================== +BPS also provides implementations of the hash +algorithms used by MySql and PostgreSQL. + +.. autoclass:: Mysql10Crypt +.. autoclass:: Mysql41Crypt +.. autoclass:: PostgresMd5Crypt + +.. data:: mysql_context + + This context object contains the algorithms used by MySql 4.1 and newer + for storing user passwords. + +.. data:: postgres_context + + This context object should be able to read/write/verify + the values found in the password field of the pg_shadow table in Postgres. diff --git a/docs/lib/bps.security.pwhash/contexts.rst b/docs/lib/bps.security.pwhash/contexts.rst new file mode 100644 index 0000000..3312355 --- /dev/null +++ b/docs/lib/bps.security.pwhash/contexts.rst @@ -0,0 +1,35 @@ +============================================= +:mod:`bps.security.pwhash` - Crypt Contexts +============================================= + +.. currentmodule:: bps.security.pwhash + +For more complex deployment scenarios than +the frontend functions described in :doc:`Quick Start <quickstart>`, +the CryptContext class exists... + +.. autoclass:: CryptContext + +Predefined Contexts +=================== +The following context objects are predefined by BPS: + +.. data:: default_context + + This context object contains all the algorithms + supported by BPS, listed (mostly) in order of strength. + :func:`identify`, :func:`verify`, and :func:`encrypt` + are all merely wrappers for this object's methods + of the same name. + +.. data:: linux_context + + This context object contains only the algorithms + in use on modern linux systems (namely: + unix-crypt, md5-crypt, sha512-crypt). + +.. data:: bsd_context + + This context object contains only the algorithms + in use on modern BSD systems (namely: + unix-crypt, md5-crypt, bcrypt). diff --git a/docs/lib/bps.security.pwhash/implementation.rst b/docs/lib/bps.security.pwhash/implementation.rst new file mode 100644 index 0000000..2bf1661 --- /dev/null +++ b/docs/lib/bps.security.pwhash/implementation.rst @@ -0,0 +1,17 @@ +=================================================================== +:mod:`bps.security.pwhash` - Implementing a Custom Crypt Algorithm +=================================================================== + +.. currentmodule:: bps.security.pwhash + +New password algorithms can be implemented +by subclassing :class:`CryptAlgorithm`, +which provides the underlying framework used +for all the password algorithms. + +To create a new one, +you simple subclass CryptAlgorithm, +and implement the identify, encrypt, and verify methods +(at the very least). + +.. autoclass:: CryptAlgorithm diff --git a/docs/lib/bps.security.pwhash/quickstart.rst b/docs/lib/bps.security.pwhash/quickstart.rst new file mode 100644 index 0000000..44e3fa3 --- /dev/null +++ b/docs/lib/bps.security.pwhash/quickstart.rst @@ -0,0 +1,46 @@ +======================================== +:mod:`bps.security.pwhash` - Quick Start +======================================== + +.. currentmodule:: bps.security.pwhash + +Usage Example +============= +In order to get off the ground quickly, here's an +example of how to quickly encrypt and verify passwords +without having to delve too deeply into this module:: + + >>> from bps.security import pwhash + + >>> #encrypt password using strongest algorithm defined by this module + >>> hash = pwhash.encrypt("too many secrets") + >>> hash + $6$rounds=39000$DNnCxm85LEP1WXUh$IVkALQeSuhr2hcUV90Tv8forzli3K.XwX.1JzPjgwltgvCAgllN3x1jNpG9E1C8IQPm0gEIesqATDyKh/nEnh0' + + >>> #verify password against hash + >>> pwhash.verify("mypass", hash) + False + >>> pwhash.verify("too many secrets", hash) + True + + >>> #identify the algorithm used in a hash + >>> pwhash.identify(hash) + 'sha512-crypt' + + >>> #choose a specific algorithm to use (instead of the default) + >>> hash2 = pwhash.encrypt("too many secrets", alg="bcrypt") + '$2a$11$unZuTsMEjeo5mqFX6rmRduQPBDx9t3djd2voi9W.oFhUDQu1NNMcW' + + >>> #check if we used right algorithm + >>> pwhash.identify(hash2) + 'bcrypt' + + >>> #the hash type is autodetected by verify + >>> pwhash.verify("too many secrets", hash2) + True + +Frontend Functions +================== +.. autofunction:: encrypt +.. autofunction:: verify +.. autofunction:: identify diff --git a/docs/lib/bps.security.pwhash/utils.rst b/docs/lib/bps.security.pwhash/utils.rst new file mode 100644 index 0000000..897349d --- /dev/null +++ b/docs/lib/bps.security.pwhash/utils.rst @@ -0,0 +1,19 @@ +============================================= +:mod:`bps.security.pwhash` - Helper Functions +============================================= + +.. currentmodule:: bps.security.pwhash + +A couple of utility functions are available, +mainly useful when writing custom password hash algorithms. +The ``h64_*`` series of functions all provide +utilities for encoding & decoding strings +under the modified base64 system used by most +of the standard unix hash algorithms. + +.. autofunction:: h64_encode +.. autofunction:: h64_decode +.. autofunction:: h64_gen_salt + +.. autofunction:: is_crypt_context +.. autofunction:: is_crypt_alg diff --git a/docs/lib/bps.security.rst b/docs/lib/bps.security.rst new file mode 100644 index 0000000..c84bc8b --- /dev/null +++ b/docs/lib/bps.security.rst @@ -0,0 +1,17 @@ +====================================== +:mod:`bps.security` -- Security Tools +====================================== + +.. module:: bps.security + :synopsis: security related modules + +This package provides nothing on it's own, +but instead is a collection of a number +of security-related subpackages: + +.. toctree:: + :maxdepth: 1 + + bps.security.pwgen + bps.security.pwhash + bps.security.policy diff --git a/docs/lib/bps.stream.rst b/docs/lib/bps.stream.rst new file mode 100644 index 0000000..9955abd --- /dev/null +++ b/docs/lib/bps.stream.rst @@ -0,0 +1,24 @@ +=============================================== +:mod:`bps.stream` -- Stream & Buffer Utilities +=============================================== + +.. module:: bps.stream + :synopsis: stream (file, StringIO) helpers + +This module contain various stream & buffer related utilities. + +Non-Blocking Reads +================== + +.. autofunction:: nb_read +.. autofunction:: nb_readline_iter +.. autoclass:: nb_readline_list + +Other Functions +=============== +.. autofunction:: get_stream_size + +.. + not listing this one till it's heuristic or use-case is better defined: + + .. autofunction:: get_input_type diff --git a/docs/lib/bps.text.rst b/docs/lib/bps.text.rst new file mode 100755 index 0000000..2dabd11 --- /dev/null +++ b/docs/lib/bps.text.rst @@ -0,0 +1,105 @@ +================================================= +:mod:`bps.text` -- Text parsing & formatting +================================================= + +.. module:: bps.text + :synopsis: text parsing & formatting + +This module provides various methods for manipulating +various types of strings. It includes helpers +for cleaning user input, inflecting english words, +and some other features. + +String Parsing +=============== + +.. autofunction:: asbool +.. autofunction:: condense +.. autofunction:: split_condense + +Filename Sanitizing +=================== +.. autofunction:: clean_filename + +Extending :func:`clean_filename` +-------------------------------- +The clean_filename function is designed to be extended +to suite your own requirements, and yet still perform +optimally. If you have a preset configuration +which you frequently use, simply create an instance +of :class:`FileCleaner`, passing in the appropriate +options for your preset, or clone an existing preset +using ``preset.copy()``. These instances can be called +directly, just like the `clean_filename` function proper. +Or, you may insert it into ``bps3.text.cfg_presets`` +under a custom name, so that it will be globally available +through :func:`clean_filename`. See the source code for more. + +Language Inflection +=================== +BPS implements a language inflector class based off of +the one implemented in Ruby On Rails. Current only English +is supported (but see note below). While the system +is class based, the following public functions +are offered up for easy access: + +.. autofunction:: pluralize +.. autofunction:: singularize +.. autofunction:: countof +.. autofunction:: oneof +.. autofunction:: ordinal + +.. note:: + Currently, there only exists an (American) English language inflector, + but if and when more Inflector subclasses are written for other languages, + this system will be expanded as the use cases require. + +.. + Variable Renaming + ================= + BPS has only the beginnings of support for variable name mangling, + such as converting from ``CamelCase`` to ``lower_case_with_underlines``. + This will hopefully be fleshed out more in the future. + + .. autofunction:: lu_to_cc + +Format String Backport +====================== +Python 2.6 introduced a new formatting system. +BPS contains a pure-python implementation of this system, +so that it is available to Python 2.5 deployments. +Thus, the following methods are aliases for the native +python implementations when available; otherwise +they are backed by a pure-python implementation. + +.. autofunction:: render_format +.. autofunction:: format +.. autoclass:: Formatter + +.. note:: + For Python 2.5 users who *really* want to have ``str.format()`` + available to them directly, they may import :mod:`bps.text.patch_format` + somewhere in their application. By importing this module, + the native strings types of Python 2.5 will be monkeypatched to include + a format method which should be a compatible with the real thing. + This is not imported by default, as it's a somewhat evil thing to do. + +Format String Parsing +===================== +The following functions are available for examining +format strings. They are rarely needed, +but occasionally code has the need to inspect a format string template: + +.. autofunction:: fmt_has_field +.. autofunction:: get_fmt_fields + +.. + these are present, but not documented yet + .. autofunction:: parse_fmt_string + .. autofunction:: parse_fmt_field + + +.. + agent string parsing: + .. autofunction:: parse_agent_string + .. autofunction:: agent_string_has_product diff --git a/docs/lib/bps.types.rst b/docs/lib/bps.types.rst new file mode 100644 index 0000000..6375e6b --- /dev/null +++ b/docs/lib/bps.types.rst @@ -0,0 +1,40 @@ +================================================= +:mod:`bps.types` -- Useful Classes and Types +================================================= + +.. module:: bps.types + :synopsis: useful classes and types + +This module contains most of the classes defined by BPS: + + * `base classes`_ + * `simple data structures`_ + * `dictionary classes`_ + * `other classes`_ + +Base Classes +============ +.. autoclass:: BaseClass +.. autoclass:: BaseMetaClass + +Simple Data Structures +====================== +.. autoclass:: stub + +.. class:: namedtuple + + Returns a new subclass with named tuple fields + + This class is just a backport from Python 2.6. + When BPS is loaded under 2.6 or higher, + the native implementation will be used instead. + +Dictionary Classes +================== +.. autoclass:: CustomDict +.. autoclass:: OrderedDict + +Other Classes +============= +.. autoclass:: CloseableClass + diff --git a/docs/lib/bps.undef.rst b/docs/lib/bps.undef.rst new file mode 100644 index 0000000..0f8441b --- /dev/null +++ b/docs/lib/bps.undef.rst @@ -0,0 +1,37 @@ +========================================== +:mod:`bps.undef` -- The "Undefined" Object +========================================== + +.. module:: bps.undef + :synopsis: provides an "Undef" singleton (ala Javascript) + +Other languages like javascript (and frequently other python libraries) +have the recurring need for a "undefined" singleton, representing +that a value is not specified; this is opposed to ``None`` +which technically represents "no value present", but does double duty +as meaning "undefined" as well. But sometimes, that double duty just doesn't +cut it. BPS provides the following Undef object. + +.. data:: Undef + + The Undef object signals that the value is not defined. + It has the unique property that is it never equal to anything (in a boolean sense), + including itself, much like the sql "NULL" object. + +.. function:: defined(value) + + Helper for checking if a value is or is not the :data:`Undef` object. + This just for completeness, it's equivalent to ``value is not Undef``, + which is typically faster. + +.. function:: undefined(value) + + Inverse of :func:`defined`. + +.. caution:: + Mako's "Undefined" and peak's "NOT_GIVEN" objects are other examples + of this singleton. Hopefully a way will be found to unify these objects + before it becomes a problem. Because of this, it's generally + useful to use Undef as an internal value inside your code, + usually as a default value for a function keyword, + and never use it as a return value. diff --git a/docs/lib/bps.warndep.rst b/docs/lib/bps.warndep.rst new file mode 100644 index 0000000..a234f1b --- /dev/null +++ b/docs/lib/bps.warndep.rst @@ -0,0 +1,29 @@ +======================================================= +:mod:`bps.warndep` -- Warning and deprecation Utilities +======================================================= + +.. module:: bps.warndep + :synopsis: warning & deprecation utilities + +This module contains some helpful functions for +issuing deprecation warnings about methods, +functions, and properties which are about to +be relocated or removed entirely. + +Deprecation Decorators +====================== +These decorators automatically issue +a deprecation warning when the decorated +object is accessed: + +.. autofunction:: deprecated_function +.. autofunction:: deprecated_method + +Deprecation Constructors +======================== +These functions create an entirely new object, +usually wrapping the old object in some manner. + +.. autofunction:: deprecated_property +.. autofunction:: relocated_function + |