diff options
Diffstat (limited to 'deps/rabbitmq_cli/DESIGN.md')
| -rw-r--r-- | deps/rabbitmq_cli/DESIGN.md | 483 |
1 files changed, 483 insertions, 0 deletions
diff --git a/deps/rabbitmq_cli/DESIGN.md b/deps/rabbitmq_cli/DESIGN.md new file mode 100644 index 0000000000..4179048159 --- /dev/null +++ b/deps/rabbitmq_cli/DESIGN.md @@ -0,0 +1,483 @@ +# New (3.7.0+) RabbitMQ CLI Tools + +## Summary + +RabbitMQ version 3.7 comes with brave new CLI tool to replace `rabbitmqctl`. + +Some of the issues in the older tool suite we wanted to address: + + * Built-in into RabbitMQ server code + * Home-grown argument parser + * Started with `erl` with a lot of installation-dependent parameters + * Too many commands in a single tool + * All commands in resided the same module (function clauses) + +All this made it hard to maintain and extend the tools. + +The new CLI is different in a number of ways that address +the above issues: + + * Standalone [repository on GitHub](https://github.com/rabbitmq/rabbitmq-cli). + * Implemented in Elixir (using Elixir's standard CLI parser) + * Each command is its own module + * Extensible + * Commands can support more output formats (e.g. JSON and CSV) + * A single executable file that bundles all dependencies but the Erlang runtime + * Command scopes associate commands with tools (e.g. `rabbitmq-diagnostics` reuses relevant commands from `rabbitmqctl`) + +## Architecture + +Each command is defined in its own module and implements an Elixir (Erlang) behaviour. +(See [Command behaviour](#command-behaviour)) + +Output is processed by a formatter and printer which formats command output +and render the output (the default being the standard I/O output). +(see [Output formatting](#output-formatting)) + +CLI core consists of several modules implementing command execution process: + + * `RabbitMQCtl`: entry point. Generic execution logic. + * `Parser`: responsible for command line argument parsing (drives Elixir's `OptionParser`) + * `CommandModules`: responsible for command module discovery and loading + * `Config`: responsible for config unification: merges environment variable and command argument values + * `Output`: responsible for output formatting + * `Helpers`: self-explanatory + +### Command Execution Process + +#### Arguments parsing + +Command line arguments are parsed with [OptionParser](https://elixir-lang.org/docs/stable/elixir/OptionParser.html) +Parser returns a list of unnamed arguments and a map of options (named arguments) +First unnamed argument is a command name. +Named arguments can be global or command specific. +Command specific argument names and types are specified in the `switches/0` callback. +Global argument names are described in [Global arguments] + +#### Command discovery + +If arguments list is not empty, its first element is considered a command name. +Command name is converted to CamelCase and a module with +`RabbitMQ.CLI.*.Commands.<CommandName>Command` name is selected as a command module. + +List of available command modules depend on current tool scope +(see [Command scopes](#command-scopes)) + +#### Defaults and validation + +After the command module is found, effective command arguments are calculated by + merging global defaults and command specific defaults for both unnamed and named arguments. + +A command specifies defaults using the `merge_defaults/2` callback + (see [Command behaviour](#command-behaviour)) + +Arguments are then validated using the `validate/2` callback. `validate_execution_environment/2` is +another (optional) validation function with the same signature as `validate/2`. It is meant to +validate everything that is not CLI arguments, for example, whether a file exists or RabbitMQ is running +or stopped on the target node. + +##### Command Aliases + +It is possible to define aliases for commands using an aliases file. The file name can be +specified using `RABBITMQ_CLI_ALIASES_FILE` environment variable or the `--aliases-file` +command lineargument. + +Aliases can be specified using `alias = command [options]` format. +For example: + +``` +lq = list_queues +lq_vhost1 = list_queues -p vhost1 +lq_off = list_queues --offline +``` + +with such aliases config file running + +``` +RABBITMQ_CLI_ALIASES_FILE=/path/to/aliases.conf rabbitmqctl lq +``` + +will be equivalent to executing + +``` +RABBITMQ_CLI_ALIASES_FILE=/path/to/aliases.conf rabbitmqctl list_queues +``` + +while + +``` +RABBITMQ_CLI_ALIASES_FILE=/path/to/aliases.conf rabbitmqctl lq_off +``` + +is the same as running + +``` +RABBITMQ_CLI_ALIASES_FILE=/path/to/aliases.conf rabbitmqctl list_queues --offline +``` + +Builtin or plugin-provided commands are looked up first, if that yields no result, +aliases are inspected. Therefore it's not possible to override a command by +configuring an alias. + +Just like command lookups, alias expansion happens in the `RabbitMQ.CLI.Core.Parser` +module. + + +##### Command Aliases with Variables + +Aliases can also contain arguments. Command name must be the first word after the `=`. +Arguments specified in an alias will precede those passed from the command line. + +For example, if you specify the alias `passwd_user1 = change_password user1`, +you can call it with `rabbitmqctl passwd_user1 new_password`. + +Combined with the `eval` command, aliases can be a powerful tool for users who cannot or don't want +to develop, distribute and deploy their own commands via plugins. + +For instance, the following alias deletes all queues in a vhost: + +``` +delete_vhost_queues = eval '[rabbit_amqqueue:delete(Q, false, false, <<"rabbit-cli">>) || Q <- rabbit_amqqueue:list(_1)]' +``` + +This command will require a single positional argument for the vhost: + +``` +rabbitmqctl delete_vhost_queues vhost1 +``` + +It is also possible to use a named vhost argument by specifying an underscore +variable that's not an integer: + +``` +delete_vhost_queues = eval '[rabbit_amqqueue:delete(Q, false, false, <<"rabbit-cli">>) || Q <- rabbit_amqqueue:list(_vhost)]' +``` + +Then the alias can be called like this: + +``` +rabbitmqctl delete_vhost_queues -p vhost1 +# or +rabbitmqctl delete_vhost_queues ---vhost <vhost>1 +``` + +Keep in mind that `eval` command can accept only [global arguments](#global-arguments) as named, +and it's advised to use positional arguments instead. + +Numbered arguments will be passed to the eval'd code as Elixir strings (or Erlang binaries). +The code that relies on them should perform type conversion as necessary, e.g. by +using functions from the [`rabbit_data_coercion` module](https://github.com/rabbitmq/rabbitmq-common/blob/stable/src/rabbit_data_coercion.erl). + +#### Command execution + + Command is executed with the `run/2` callback, which contains main command logic. + This callback can return any value. + + Output from the `run/2` callback is being processed with the `output/2` callback, + which should format the output to a specific type. + (see [Output formatting](#output-formatting)) + + Output callback can return an error, with a specific exit code. + +#### Printing and formatting + + The `output/2` callback return value is being processed in `output.ex` module by + by the appropriate formatter and printer. + + A formatter translates the output value to a sequence of strings or error value. + Example formatters are `json`, `csv` and `erlang` + + A printer renders formatted strings to an output device (e.g. stdout, file) + +#### Return + + Errors during execution (e.g. validation failures, command errors) are being printed + to `stderr`. + + If command has failed to execute, a non-zero code is returned. + + +## Usage + +CLI tool is an Elixir Mix application. It is compiled into an escript +executable file. + +This file embeds Elixir, rabbit_common, and rabbitmqcli applications and can be executed anywhere +where `escript` is installed (it comes as a part of `erlang`). + +Although, some commands require rabbitmq broker code and data directory to be known. +For example, commands in `rabbitmq-plugins` tool and those controlling clustering +(e.g. `forget_cluster_node`, `rename_cluster_node` ...) + +Those directories can be defined using environment options or rabbitmq environment variables. + +In the broker distribution the escript file is called from a shell/cmd script, which loads +broker environment and exports it into the script. + +Environment variables also specify the locations of the enabled plugins file +and the plugins directory. + +All enabled plugins will be searched for commands. Commands from enabled plugins will +be shown in usage and available for execution. + +### Global Arguments + +#### Commonly Used Arguments + + * node (n): atom, broker node name, defaults to `rabbit@<current host>` + * quiet (q): boolean, if set to `true`, command banner is not shown, defaults to `false` + * timeout (t): integer, timeout value in **seconds** (used in some commands), defaults to `infinity` + * vhost (p): string, vhost to talk to, defaults to `/` + * formatter: string, formatter to use to format command output. (see [Output formatting](#output-formatting)) + * printer: string, printer to render output (see [Output formatting](#output-formatting)) + * dry-run: boolean, if specified the command will not run, but print banner only + +#### Environment Arguments + + * script-name: atom, configurable tool name (`rabbitmq-plugins`, `rabbitmqctl`) to select command scope (see [Command scopes](#command-scopes)) + * rabbitmq-home: string, broker install directory + * mnesia-dir: string, broker mnesia data directory + * plugins-dir: string, broker plugins directory + * enabled-plugins-file: string, broker enabled plugins file + * longnames (l): boolean, use longnames to communicate with broker erlang node. Should be set to `true` only if broker is started with longnames. + * aliases-file: string, a file name to load aliases from + * erlang-cookie: atom, an [erlang distribution cookie](http://erlang.org/doc/reference_manual/distributed.html) + +Environment argument defaults are loaded from rabbitmq environment variables (see [Environment configuration](#environment-configuration)). + +Some named arguments have single-letter aliases (in parenthesis). + +Boolean options without a value are parsed as `true` + +For example, parsing command + + rabbitmqctl list_queues --vhost my_vhost -t 10 --formatter=json name pid --quiet + +Will result with unnamed arguments list `["list_queues", "name", "pid"]` +and named options map `%{vhost: "my_vhost", timeout: 10, quiet: true}` + + +### Usage (Listing Commands in `help`) + +Usage is shown when the CLI is called without any arguments, or if there are some +problems parsing arguments. + +In that cases exit code is `64`. + +If you want to show usage and return `0` exit code run `help` command + + rabbitmqctl help + +Each tool (`rabbitmqctl`, `rabbitmq-plugins`) shows its scope of commands (see [Command scopes](#command-scopes)) + +## Command Behaviour + +Each command is implemented as an Elixir (or Erlang) module. Command module should +implement `RabbitMQ.CLI.CommandBehaviour` behaviour. + +Behaviour summary: + +Following functions MUST be implemented in a command module: + + usage() :: String.t | [String.t] + +Command usage string, or several strings (one per line) to print in command listing in usage. +Typically looks like `command_name [arg] --option=opt_value` + + banner(arguments :: List.t, options :: Map.t) :: String.t + +Banner to print before the command execution. +Ignored if argument `--quiet` is specified. +If `--dry-run` argument is specified, th CLI will only print the banner. + + merge_defaults(arguments :: List.t, options :: Map.t) :: {List.t, Map.t} + +Merge default values for arguments and options (named arguments). +Returns a tuple with effective arguments and options, that will be passed to `validate/2` and `run/2` + + validate(arguments :: List.t, options :: Map.t) :: :ok | {:validation_failure, Atom.t | {Atom.t, String.t}} + +Validate effective arguments and options. + +If function returns `{:validation_failure, err}` +CLI will print usage to `stderr` and exit with non-zero exit code (typically 64). + + run(arguments :: List.t, options :: Map.t) :: run_result :: any + +Run command. This function usually calls RPC on broker. + + output(run_result :: any, options :: Map.t) :: :ok | {:ok, output :: any} | {:stream, Enum.t} | {:error, ExitCodes.exit_code, [String.t]} + +Cast the return value of `run/2` command to a formattable value and an exit code. + +- `:ok` - return `0` exit code and won't print anything + +- `{:ok, output}` - return `exit` code and print output with `format_output/2` callback in formatter + +- `{:stream, stream}` - format with `format_stream/2` callback in formatter, iterating over enumerable elements. +Can return non-zero code, if error occurs during stream processing +(stream element for error should be `{:error, Message}`). + +- `{:error, exit_code, strings}` - print error messages to `stderr` and return `exit_code` code + +There is a default implementation for this callback in `DefaultOutput` module + +Most of the standard commands use the default implementation via `use RabbitMQ.CLI.DefaultOutput` + +Following functions are optional: + + switches() :: Keyword.t + +Keyword list of switches (argument names and types) for the command. + +For example: `def switches(), do: [offline: :boolean, time: :integer]` will +parse `--offline --time=100` arguments to `%{offline: true, time: 100}` options. + +This switches are added to global switches (see [Arguments parsing](#arguments-parsing)) + + aliases() :: Keyword.t + +Keyword list of argument names one-letter aliases. +For example: `[o: :offline, t: :timeout]` +(see [Arguments parsing](#arguments-parsing)) + + usage_additional() :: String.t | [String.t] + +Additional usage strings to print after all commands basic usage. +Used to explain additional arguments and not interfere with command listing. + + formatter() :: Atom.t + +Default formatter for the command. +Should be a module name of a module implementing `RabbitMQ.CLI.FormatterBehaviour` (see [Output formatting](#output-formatting)) + + scopes() :: [Atom.t] + +List of scopes to include command in. (see [Command scopes](#command-scopes)) + +More information about command development can be found in [the command tutorial](COMMAND_TUTORIAL.md) + +## Command Scopes + +Commands can be organized in scopes to be used in different tools +like `rabbitmq-diagnostics` or `rabbitmq-plugins`. + +One command can belong to multiple scopes. Scopes for a command can be +defined in the `scopes/0` callback in the command module. +Each scope is defined as an atom value. + +By default a command scope is selected using naming convention. +If command module is called `RabbitMQ.CLI.MyScope.Commands.DoSomethingCommand`, it will belong to +`my_scope` scope. A scope should be defined in snake_case. Namespace for the scope will be translated to CamelCase. + +When CLI is run, a scope is selected by a script name, which is the escript file name +or the `--script-name` argument passed. + +A script name is associated with a single scope in the application environment: + +Script names for scopes: + + * `rabbitmqctl` - `:ctl` + * `rabbitmq-plugins` - `:plugins` + * `rabbitmq-diagnostics` - `:diagnostics` + +This environment is extended by plugins `:scopes` environment variables, +but cannot be overridden. Plugins scopes can override each other, +so should be used with caution. + +So all the commands in the `RabbitMQ.CLI.Ctl.Commands` namespace will be available +for `rabbitmqctl` script. +All the commands in the `RabbitMQ.CLI.Plugins.Commands` namespace will be available +for `rabbitmq-plugins` script. + +To add a command to `rabbitmqctl`, one should either name it with +the `RabbitMQ.CLI.Ctl.Commands` prefix or add the `scopes()` callback, +returning a list with `:ctl` element + +## Output Formatting + +The CLI supports extensible output formatting. Formatting consists of two stages: + + * formatting - translating command output to a sequence of lines + * printing - outputting the lines to some device (e.g. stdout or filesystem) + +A formatter module performs formatting. +Formatter is a module, implementing the `RabbitMQ.CLI.FormatterBehaviour` behaviour: + + format_output(output :: any, options :: Map.t) :: String.t | [String.t] + +Format a single value, returned. It accepts output from command and named arguments (options) +and returns a list of strings, that should be printed. + + format_stream(output_stream :: Enumerable.t, options :: Map.t) :: Enumerable.t + +Format a stream of return values. This function uses elixir +Stream [https://elixir-lang.org/docs/stable/elixir/Stream.html] abstraction +to define processing of continuous data, so the CLI can output data in realtime. + +Used in `list_*` commands, that emit data asynchronously. + +`DefaultOutput` will return all enumerable data as stream, +so it will be formatted with this function. + +A printer module performs printing. Printer module should implement +the `RabbitMQ.CLI.PrinterBehaviour` behaviour: + + init(options :: Map.t) :: {:ok, printer_state :: any} | {:error, error :: any} + +Init the internal printer state (e.g. open a file handler). + + finish(printer_state :: any) :: :ok + +Finalize the internal printer state. + + print_output(output :: String.t | [String.t], printer_state :: any) :: :ok + +Print the output lines in the printer state context. +Is called for `{:ok, val}` command output after formatting `val` using formatter, +and for each enumerable element of `{:stream, enum}` enumerable + + print_ok(printer_state :: any) :: :ok + +Print an output without any values. Is called for `:ok` command return value. + +Output formatting logic is defined in `output.ex` module. + +Following rules apply for a value, returned from `output/2` callback: + + * `:ok` - initializes a printer, calls `print_ok` and finishes the printer. Exit code is `0`. + * `{:ok, value}` - calls `format_output/2` from formatter, then passes the value to printer. Exit code is `0`. + * `{:stream, enum}` - calls `format_stream/2` to augment the stream with formatting logic, +initializes a printer, then calls `print_output/2` for each successfully formatted stream +element (which is not `{:error, msg}`). Exit code is `0` if there were no errors. +In case of an error element, stream processing stops, error is printed to `stderr` +and the CLI exits with nonzero exit code. + * `{:error, exit_code, msg}` - prints `msg` to `stderr` and exits with `exit_code` + + +## Environment Configuration + +Some commands require information about the server environment to run correctly. +Such information is: + + * rabbitmq code directory + * rabbitmq mnesia directory + * enabled plugins file + * plugins directory + +Enabled plugins file and plugins directory are also used to locate plugins commands. + +This information can be provided using command line arguments (see [Environment arguments](#environment-arguments)) + +By default it will be loaded from environment variables, same as used in rabbitmq broker: + +| Argument name | Environment variable | +|----------------------|-------------------------------| +| rabbitmq-home | RABBITMQ_HOME | +| mnesia-dir | RABBITMQ_MNESIA_DIR | +| plugins-dir | RABBITMQ_PLUGINS_DIR | +| enabled-plugins-file | RABBITMQ_ENABLED_PLUGINS_FILE | +| longnames | RABBITMQ_USE_LONGNAME | +| node | RABBITMQ_NODENAME | +| aliases-file | RABBITMQ_CLI_ALIASES_FILE | +| erlang-cookie | RABBITMQ_ERLANG_COOKIE | |