Move most of shell scripts to Erlang code

A large part of the rabbitmq-server(8) and CLI scripts, both
Bourne-shell and Windows Batch versions, was moved to Erlang code and
the RabbitMQ startup procedure was reorganized to be closer to a regular
Erlang application.

A new application called `rabbitmq_prelaunch` is responsible for:

  1. Querying the environment variables to initialize important
     variables (using the new `rabbit_env` module in rabbitmq-common).
  2. Checking the compatibility with the Erlang/OTP runtime.
  3. Configuring Erlang distribution.
  5. Writing the PID file.

The application is started early (i.e. it is started before `rabbit`).

The `rabbit` application runs the second half of the prelaunch sequence
at the beginning of the application `start()` function. This second
phase is responsible for the following steps:

  1. Preparing the feature flags registry.
  2. Reading and validating the configuration.
  3. Configuring logging.
  4. Running the various cluster checks.

In addition to this prelaunch sequence, the `rabbit` application start
procedure ends with a "postlaunch" sequence which takes care of
starting enabled plugins.

Thanks to this, RabbitMQ can be started with `application:start(rabbit)`
as any other Erlang application. The only caveats are:

  * Mnesia must be stopped at the time `rabbit_prelaunch` is started,
    and must remain stopped when `rabbit` is started, to allow the
    Erlang distribution setup and cluster checks. `rabbit` takes care of
    starting Mnesia.

  * Likewise for Ra, because it relies on the `ra` application
    environment to be configured.

Transitioning from scripts to Erlang code has the following benefits:

  * RabbitMQ start behavior should be identical between Unix and
    Windows. Also, features should be on par now. For instance, RabbitMQ
    now writes a PID file on Windows, like it always did on Unix-based
    systems.

  * The difference between published packages and a development
    environment are greatly reduced. In fact, we removed all the "if
    this is a dev working copy, then ..." blocks.

    As part of that, the `rabbit` application is now treated like its
    plugins: it is packaged as an `.ez` archive and written to the
    `plugins` directory (even though it is not technically a plugin).

    Also in a development copy, the CLI is copied to the top-level
    project. So when testing a plugin for instance, the CLI to use is
    `sbin/rabbitmqctl` in the current directory, not the master copy in
    `rabbit/scripts`.

  * As a consequence of the previous two points, maintaining and testing
    on Windows is now made easy. It should even be possible to setup CI
    on Windows.

  * There are less issues with paths containing non-US-ASCII characters,
    which can happen on Windows because RabbitMQ stores its data in user
    directories by default.

This process brings at least one more benefit: we now have early logging
during this prelaunch phase, which eases diagnostics and debugging.

There are also behavior changes:

  * The new format configuration files used to be converted to an
    Erlang-term-based file by the Cuttlefish CLI. To do that,
    configuration schemas were copied to a temporary directory and the
    generated configuration file was written to RabbitMQ data directory.

    Now, Cuttlefish is used as a library: everything happens in memory.
    No schemas are copied, no generated configuration is written to
    disk.

  * The PID file is removed when the Erlang VM exits.

  * The `rabbit_config` module was trimmed significantly because most of
    the configuration handling is done in `rabbit_prelaunch_conf` now.

  * The RabbitMQ nodename does not appear on the command line, therefore
    it is missing from ps(1) and top(1) output.

  * The `rabbit:start()` function will probably behave differently in
    some ways because it defers everything to the Erlang application
    controller (instead of reimplementing it).

1 files changed, 13 insertions, 18 deletions

diff --git a/Makefile b/Makefile
index 19ce494387..30a2066e38 100644
--- a/Makefile
+++ b/Makefile
@@ -133,11 +133,19 @@ define PROJECT_ENV
 	  ]
 endef
 
-LOCAL_DEPS = sasl mnesia os_mon inets compiler public_key crypto ssl syntax_tools xmerl
+# With Erlang.mk default behavior, the value of `$(APPS_DIR)` is always
+# relative to the top-level executed Makefile. In our case, it could be
+# a plugin for instance. However, the rabbitmq_prelaunch application is
+# in this repository, not the plugin's. That's why we need to override
+# this value here.
+APPS_DIR := $(CURDIR)/apps
+
+LOCAL_DEPS = sasl rabbitmq_prelaunch os_mon inets compiler public_key crypto ssl syntax_tools xmerl
 BUILD_DEPS = rabbitmq_cli syslog
-DEPS = ranch lager rabbit_common ra sysmon_handler stdout_formatter recon observer_cli
+DEPS = cuttlefish ranch lager rabbit_common ra sysmon_handler stdout_formatter recon observer_cli
 TEST_DEPS = rabbitmq_ct_helpers rabbitmq_ct_client_helpers amqp_client meck proper
 
+dep_cuttlefish = hex 2.2.0
 dep_syslog = git https://github.com/schlagert/syslog 3.4.5
 
 define usage_xml_to_erl
@@ -164,6 +172,9 @@ ERLANG_MK_COMMIT = rabbitmq-tmp
 include rabbitmq-components.mk
 include erlang.mk
 
+# See above why we mess with `$(APPS_DIR)`.
+unexport APPS_DIR
+
 ifeq ($(strip $(BATS)),)
 BATS := $(ERLANG_MK_TMP)/bats/bin/bats
 endif
@@ -240,20 +251,6 @@ USE_PROPER_QC := $(shell $(ERL) -eval 'io:format({module, proper} =:= code:ensur
 RMQ_ERLC_OPTS += $(if $(filter true,$(USE_PROPER_QC)),-Duse_proper_qc)
 endif
 
-.PHONY: copy-escripts clean-extra-sources clean-escripts
-
-CLI_ESCRIPTS_DIR = escript
-
-copy-escripts:
-	$(gen_verbose) $(MAKE) -C $(DEPS_DIR)/rabbitmq_cli install \
-		PREFIX="$(abspath $(CLI_ESCRIPTS_DIR))" \
-		DESTDIR=
-
-clean:: clean-escripts
-
-clean-escripts:
-	$(gen_verbose) rm -rf "$(CLI_ESCRIPTS_DIR)"
-
 # --------------------------------------------------------------------
 # Documentation.
 # --------------------------------------------------------------------
@@ -297,5 +294,3 @@ distclean:: distclean-manpages
 
 distclean-manpages::
 	$(gen_verbose) rm -f $(WEB_MANPAGES)
-
-app-build: copy-escripts