Cinder recently removed their v2 API [1] which is causing the functional
tests to fail. Improve our 'is_service_enabled' test helper to use the
'versions show' command, which queries the service catalog and can give
us information about the service version as well as answer the more
general "is this service available" question. We also resolve a
long-standing TODO in the process.

[1] https://review.opendev.org/c/openstack/cinder/+/792299

Change-Id: I381069357aa008344e15327adf3a863c0c2e1f04
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

Neutron has got CRUD API for L3 conntrack helper since some time.
This patch adds support for it in the OSC.
OpenStack SDK supports that since [1]

This patch also bumps minimum OpenStack SDK version to
the 0.56.0 as that version introduced support for the
Neutron's L3 conntrack helper.

[1] https://review.opendev.org/c/openstack/openstacksdk/+/782870

Change-Id: I55604182ae50b6ad70c8bc1f7efad8859f191269

Devstack recently switched default Neutron's backend from ML2/OVS to
ML2/OVN. As OVN backend has some parity gaps and differences in some
APIs, functional tests job was failing with ML2/OVN as some tests
weren't properly skipped in case of missing some Neutron API extensions.

This patch fixes that by doing some small changes in the functional
tests:
- skip DHCP/L3 agent tests when dhcp/l3 agent scheduler extensions
aren't available,
- skip updating neutron agent as OVN agents don't allows that,
- skip service providers tests when there is no Neutron L3 agent
available,
- skip setting router as distributed as OVN backend don't supports that
router's attribute at all.

Depends-On: https://review.opendev.org/c/openstack/neutron/+/793141
Change-Id: I29a8db202086b0b49fed865409fa8ca244b98439

This mostly reuses the existing tests for '--block-device-mapping',
which can hopefully be removed at some point in the future.

This highlights two issues with the implementation of this option.
Firstly, the 'boot_index' parameter is not required so don't mandate it.
Secondly, and more significantly, we were defaulting the destination
type for the 'image' source type to 'local'. Nova only allows you to
attach a single image to local mapping [1], which means this default
would only make sense if you were expecting users to use the
'--block-device' option exclusively and omit the '--image' option. This
is the *less common* case so this is a bad default. Default instead to a
destination type of 'volume' like everything else, and require users
specifying '--block-device' alone to pass 'destination_type=local'
explicitly.

[1] https://github.com/openstack/nova/blob/c8a6f8d2e/nova/block_device.py#L193-L206

Change-Id: I1718be965f57c3bbdb8a14f3cfac967dd4c55b4d
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

The '--offload' option allows us to explicitly request that the server
be offloaded once shelved or if already shelved.

The '--wait' option allows us to wait for the shelve and/or offload
operations to complete before returning. It is implied when attempting
to offload a server than is not yet shelved.

Change-Id: Id226831e3c09bc95c34b222151b27391a844b073
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

Make use of 'FormattableColumn'-derived formatters, which provide better
output than what we were using before, particularly for the YAML output
format.

Change-Id: Id6d25a0a348596d5a0430ff7afbf87b049a76bc8
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

Make use of 'FormattableColumn'-derived formatters, which provide better
output than what we were using before, particularly for the YAML output
format. For example, compare before for the 'server show' command:

  $ openstack --os-compute-api-version 2.79 server show test-server -f yaml
  ...
  addresses: private=fdff:77e3:9bb4:0:f816:3eff:fe6d:a944, 10.0.0.44
  flavor: disk='1', ephemeral='0', extra_specs.hw_rng:allowed='True', original_name='m1.tiny',
    ram='512', swap='0', vcpus='1'
  ...

To after:

  $ openstack --os-compute-api-version 2.79 server show test-server -f yaml
  ...
  addresses:
    private:
    - fdff:77e3:9bb4:0:f816:3eff:fe6d:a944
    - 10.0.0.44
  flavor:
    disk: 1
    ephemeral: 0
    extra_specs:
      hw_rng:allowed: 'True'
    original_name: m1.tiny
    ram: 512
    swap: 0
    vcpus: 1
  ...

Similarly, compare before for 'server list':

  $ openstack --os-compute-api-version 2.79 server list -f yaml
  - ...
    Networks: private=fdff:77e3:9bb4:0:f816:3eff:fe6d:a944, 10.0.0.44
    Power State: Running
    Properties: ''
    ...

To after:

  $ openstack --os-compute-api-version 2.79 server list -f yaml
  - ...
    Networks:
      private:
      - fdff:77e3:9bb4:0:f816:3eff:fe6d:a944
      - 10.0.0.44
    Power State: 1
    Properties: {}
    ...

We also fix the human-readable output for the 'tags' field.

Before:

  $ openstack --os-compute-api-version 2.79 server list
  ...
  | tags   | ['bar', 'foo']  |

After:

  $ openstack --os-compute-api-version 2.79 server list
  ...
  | tags   | bar, foo  |

Change-Id: I7a8349106e211c57c4577b75326b39b88bd9ac1e
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>

Add support for Neutron address group CRUD operations. Subsequent patches
will be added to use address groups in security group rules.

Change-Id: I3c313fc9329837dde67815901528a34dca98ebcc
Implements: blueprint address-groups-in-sg-rules
Depends-On: https://review.opendev.org/738274
Depends-On: https://review.opendev.org/745594