move doc; rephrase it a bit

2 files changed, 364 insertions, 0 deletions

diff --git a/docs/DEVGUIDE.rst b/docs/DEVGUIDE.rst
new file mode 100644
index 00000000..df031bde
--- /dev/null
+++ b/docs/DEVGUIDE.rst
@@ -0,0 +1,188 @@
+Setup and running tests
+=======================
+
+If you plan on hacking on psutil this is what you're supposed to do first:
+
+- clone the GIT repository:
+
+.. code-block:: bash
+
+  $ git clone git@github.com:giampaolo/psutil.git
+
+- install test deps and GIT hooks:
+
+.. code-block:: bash
+
+  make setup-dev-env
+
+- run tests:
+
+.. code-block:: bash
+
+  make test
+
+- bear in mind that ``make``
+  (see `Makefile <https://github.com/giampaolo/psutil/blob/master/Makefile>`_)
+  is the designated tool to run tests, build, install etc. and that it is also
+  available on Windows
+  (see `make.bat <https://github.com/giampaolo/psutil/blob/master/make.bat>`_).
+- do not use ``sudo``; ``make install`` installs psutil as a limited user in
+  "edit" mode; also ``make setup-dev-env`` installs deps as a limited user.
+- use `make help` to see the list of available commands.
+
+Coding style
+============
+
+- python code strictly follows `PEP 8 <https://www.python.org/dev/peps/pep-0008/>`_
+  styling guides and this is enforced by ``make install-git-hooks``.
+- C code strictly follows `PEP 7 <https://www.python.org/dev/peps/pep-0007/>`_
+  styling guides.
+
+Makefile
+========
+
+Some useful make commands:
+
+.. code-block:: bash
+
+    make install        # install
+    make setup-dev-env  # install useful dev libs (fkale8, unittest2, etc.)
+    make test           # run unit tests
+    make test-memleaks  # run memory leak tests
+    make test-coverage  # run test coverage
+    make flake8         # run PEP8 linter
+
+There are some differences between ``make`` on UNIX and Windows.
+For instance, to run a specific Python version. On UNIX:
+
+.. code-block:: bash
+
+    make test PYTHON=python3.5
+
+On Windows:
+
+.. code-block:: bat
+
+    set PYTHON=C:\python35\python.exe && make test
+
+...or:
+
+.. code-block:: bat
+
+    make -p 35 test
+
+If you want to modify psutil and run a script on the fly which uses it do
+(on UNIX):
+
+.. code-block:: bash
+
+    make test TSCRIPT=foo.py
+
+On Windows:
+
+.. code-block:: bat
+
+    make test foo.py
+
+Adding a new feature
+====================
+
+Usually the files involved when adding a new functionality are:
+
+.. code-block:: bash
+
+    psutil/__init__.py                   # main psutil namespace
+    psutil/_ps{platform}.py              # python platform wrapper
+    psutil/_psutil_{platform}.c          # C platform extension
+    psutil/_psutil_{platform}.h          # C header file
+    psutil/tests/test_process|system.py  # main test suite
+    psutil/tests/test_{platform}.py      # platform specific test suite
+
+Typical process occurring when adding a new functionality (API):
+
+- define the new function in ``psutil/__init__.py``.
+- write the platform specific implementation in ``psutil/_ps{platform}.py``
+  (e.g. ``psutil/_pslinux.py``).
+- if the change requires C, write the C implementation in
+  ``psutil/_psutil_{platform}.c`` (e.g. ``psutil/_psutil_linux.c``).
+- write a generic test in ``psutil/tests/test_system.py`` or
+  ``psutil/tests/test_process.py``.
+- if possible, write a platform specific test in
+  ``psutil/tests/test_{platform}.py`` (e.g. ``test_linux.py``).
+  This usually means testing the return value of the new feature against
+  a system CLI tool.
+- update doc in ``doc/index.py``.
+- update ``HISTORY.rst``.
+- update ``README.rst`` (if necessary).
+- make a pull request.
+
+Make a pull request
+===================
+
+- fork psutil
+- create your feature branch (``git checkout -b my-new-feature``)
+- commit your changes (``git commit -am 'add some feature'``)
+- push to the branch (``git push origin my-new-feature``)
+- create a new pull request
+
+Continuous integration
+======================
+
+All of the services listed below are automatically run on ``git push``.
+
+Unit tests
+----------
+
+Tests are automatically run for every GIT push on **Linux**, **macOS** and
+**Windows** by using:
+
+- `Travis <https://travis-ci.org/giampaolo/psutil>`_ (Linux, macOS)
+- `Appveyor <https://ci.appveyor.com/project/giampaolo/psutil>`_ (Windows)
+
+Test files controlling these are
+`.travis.yml <https://github.com/giampaolo/psutil/blob/master/.travis.yml>`_
+and
+`appveyor.yml <https://github.com/giampaolo/psutil/blob/master/appveyor.yml>`_.
+Both services run psutil test suite against all supported python version
+(2.6 - 3.6).
+Two icons in the home page (README) always show the build status:
+
+.. image:: https://img.shields.io/travis/giampaolo/psutil/master.svg?maxAge=3600&label=Linux%20/%20macOS
+    :target: https://travis-ci.org/giampaolo/psutil
+    :alt: Linux and macOS tests (Travis)
+
+.. image:: https://img.shields.io/appveyor/ci/giampaolo/psutil/master.svg?maxAge=3600&label=Windows
+    :target: https://ci.appveyor.com/project/giampaolo/psutil
+    :alt: Windows tests (Appveyor)
+
+BSD, AIX and Solaris are currently tested manually.
+
+Test coverage
+-------------
+
+Test coverage is provided by `coveralls.io <https://coveralls.io/github/giampaolo/psutil>`_,
+it is controlled via `.travis.yml <https://github.com/giampaolo/psutil/blob/master/.travis.yml>`_
+and it is updated on every git push.
+An icon in the home page (README) always shows the last coverage percentage:
+
+.. image:: https://coveralls.io/repos/giampaolo/psutil/badge.svg?branch=master&service=github
+    :target: https://coveralls.io/github/giampaolo/psutil?branch=master
+    :alt: Test coverage (coverall.io)
+
+Documentation
+=============
+
+- doc source code is written in a single file: `/docs/index.rst <https://raw.githubusercontent.com/giampaolo/psutil/master/docs/index.rst>`_.
+- it uses `RsT syntax <http://docutils.sourceforge.net/docs/user/rst/quickref.html>`_
+  and it's built with `sphinx <http://sphinx-doc.org/>`_.
+- doc can be built with ``make setup-dev-env; cd docs; make html``.
+- public doc is hosted on http://psutil.readthedocs.io/
+
+Releasing a new version
+=======================
+
+These are notes for myself (Giampaolo):
+
+- ``make release``
+- post announce (``make print-announce``) on psutil and python-announce mailing
+  lists, twitter, g+, blog.
diff --git a/docs/DEVNOTES b/docs/DEVNOTES
new file mode 100644
index 00000000..046044a3
--- /dev/null
+++ b/docs/DEVNOTES
@@ -0,0 +1,176 @@
+TODO
+====
+
+A collection of ideas and notes about stuff to implement in future versions.
+"#NNN" occurrences refer to bug tracker issues at:
+https://github.com/giampaolo/psutil/issues
+
+PLATFORMS
+=========
+
+- #355: Android (with patch)
+- #82: Cygwin (PR at #998)
+- #276: GNU/Hurd
+- #693: Windows Nano
+- #1251: Windows bash
+- DragonFlyBSD
+- HP-UX
+
+FEATURES
+========
+
+- set process name/title
+
+- #1115: users() idle time.
+
+- #1102: Process.is64bit().
+
+- #371: sensors_temperatures() at least for macOS.
+
+- #669: Windows / net_if_addrs(): return broadcast addr.
+
+- #550: CPU info (frequency, architecture, threads per core, cores per socket,
+  sockets, ...)
+
+- #772: extended net_io_counters() metrics.
+
+- #900: wheels for macOS and Linux.
+
+- #922: extended net_io_stats() info.
+
+- #914: extended platform specific process info.
+
+- #898: wifi stats
+
+- #893: (BSD) process environ
+
+- #809: (BSD) per-process resource limits (rlimit()).
+
+- (UNIX) process root (different from cwd)
+
+- #782: (UNIX) process num of signals received.
+
+- (Linux) locked files via /proc/locks:
+  https://www.centos.org/docs/5/html/5.2/Deployment_Guide/s2-proc-locks.html
+
+- #269: NIC rx/tx queue. This should probably go into net_if_stats().
+  Figure out on what platforms this is supported:
+  Linux: yes
+  Others: ?
+
+- Process.threads(): thread names; patch for macOS available at:
+  https://code.google.com/p/plcrashreporter/issues/detail?id=65
+  Sample code:
+  https://github.com/janmojzis/pstree/blob/master/proc_kvm.c
+
+- Asynchronous psutil.Popen (see http://bugs.python.org/issue1191964)
+
+- (Windows) fall back on using WMIC for Process methods returning AccessDenied
+
+- #613: thread names.
+
+- #604: emulate os.getloadavg() on Windows
+
+- scripts/taskmgr-gui.py (using tk).
+
+- system-wide number of open file descriptors:
+  - https://jira.hyperic.com/browse/SIGAR-30
+
+- Number of system threads.
+  - Windows: http://msdn.microsoft.com/en-us/library/windows/desktop/ms684824(v=vs.85).aspx
+
+- Doc / wiki which compares similarities between UNIX cli tools and psutil.
+  Example:
+  ```
+  df -a  ->  psutil.disk_partitions
+  lsof  ->  psutil.Process.open_files() and psutil.Process.open_connections()
+  killall->  (actual script)
+  tty  ->  psutil.Process.terminal()
+  who  ->  psutil.users()
+  ```
+
+- psutil.proc_tree() something which obtains a {pid:ppid, ...} dict for
+  all running processes in one shot. This can be factored out from
+  Process.children() and exposed as a first class function.
+  PROS: on Windows we can take advantage of _psutil_windows.ppid_map()
+  which is faster than iterating over all pids and calling ppid().
+  CONS: scripts/pstree.py shows this can be easily done in the user code
+  so maybe it's not worth the addition.
+
+- advanced cmdline interface exposing the whole API and providing different
+  kind of outputs (e.g. pprinted, colorized, json).
+
+- [Linux]: process cgroups (http://en.wikipedia.org/wiki/Cgroups). They look
+  similar to prlimit() in terms of functionality but uglier (they should allow
+  limiting per-process network IO resources though, which is great). Needs
+  further reading.
+
+- Python 3.3. exposed different sched.h functions:
+  http://docs.python.org/dev/whatsnew/3.3.html#os
+  http://bugs.python.org/issue12655
+  http://docs.python.org/dev/library/os.html#interface-to-the-scheduler
+  It might be worth to take a look and figure out whether we can include some
+  of those in psutil.
+  Also, we can probably reimplement wait_pid() on POSIX which is currently
+  implemented as a busy-loop.
+
+- os.times() provides 'elapsed' times (cpu_times() might).
+
+- ...also guest_time and cguest_time on Linux.
+
+- Enrich exception classes hierarchy on Python >= 3.3 / post PEP-3151 so that:
+   - NoSuchProcess inherits from ProcessLookupError
+   - AccessDenied inherits from PermissionError
+   - TimeoutExpired inherits from TimeoutError (debatable)
+   See: http://docs.python.org/3/library/exceptions.html#os-exceptions
+
+- Process.threads() might grow an extra "id" parameter so that it can be
+   used as such:
+  ```
+  >>> p = psutil.Process(os.getpid())
+  >>> p.threads(id=psutil.current_thread_id())
+  thread(id=2539, user_time=0.03, system_time=0.02)
+  >>>
+  ```
+  Note: this leads to questions such as "should we have a custom NoSuchThread
+  exception? Also see issue #418.
+  Note #2: this would work with os.getpid() only.
+  psutil.current_thread_id() might be desirable as per issue #418 though.
+
+- should psutil.TimeoutExpired exception have a 'msg' kwarg similar to
+  NoSuchProcess and AccessDenied? Not that we need it, but currently we
+  cannot raise a TimeoutExpired exception with a specific error string.
+
+- process_iter() might grow an "attrs" parameter similar to Process.as_dict()
+  invoke the necessary methods and include the results into a "cache"
+  attribute attached to the returned Process instances so that one can avoid
+  catching NSP and AccessDenied:
+  for p in process_iter(attrs=['cpu_percent']):
+  print(p.cache['cpu_percent'])
+  This also leads questions as whether we should introduce a sorting order.
+
+- round Process.memory_percent() result?
+
+- #550: number of threads per core.
+
+- cpu_percent() and cpu_times_percent() use global vars so are not thread safe.
+
+BUGFIXES
+========
+
+- #600: windows / open_files(): support network file handles.
+
+REJECTED
+========
+
+- #550: threads per core
+
+RESOURCES
+=========
+
+- sigar: https://github.com/hyperic/sigar (Java)
+- zabbix: https://zabbix.org/wiki/Get_Zabbix
+- libstatgrab: http://www.i-scream.org/libstatgrab/
+- top: http://www.unixtop.org/
+- oshi: https://github.com/oshi/oshi
+- netdata: https://github.com/netdata/netdata