summaryrefslogtreecommitdiff
path: root/ironic_python_agent
Commit message (Collapse)AuthorAgeFilesLines
* Gather details about bond interfaces if presentDerek Higgins2022-07-132-11/+61
| | | | | | | | | | If present gather information about bonded interfaces. Story: #2010093 Task: #45637 Change-Id: I394187640b4788ebec21c3391d33ed728fb72ffa (cherry picked from commit 7e4fe3bf6a2ae41656b7923796f9c2d056a2ed04)
* Fix discovering WWN/serial for devicemapper devicesDmitry Tantsur2022-06-222-16/+27
| | | | | | | | UDev prefix is DM_ not ID_ for them. On top of that, they don't have short serials (or at least don't always have). Change-Id: I5b6075fbff72201a2fd620f789978acceafc417b (cherry picked from commit 69e22545033f544d628f9c4ecd5a665ba0b5b85e)
* Multipath Hardware path handlingJulia Kreger2022-06-087-54/+730
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Removes multipath base devices from consideration by default, and instead allows the device-mapper device managed by multipath to be picked up and utilized instead. In effect, allowing us to ignore standby paths *and* leverage multiple concurrent IO paths if so offered via ALUA. In reality, anyone who has previously built IPA with multipath tooling might not have encountered issues previously because they used Active/Active SAN storage environments. They would have worked because the IO lock would have been exchanged between controllers and paths. However, Active/Passive environments will block passive paths from access, ultimately preventing new locks from being established without proper negotiation. Ultimately requiring multipathing *and* the agent to be smart enough to know to disqualify underlying paths to backend storage volumes. An additional benefit of this is active/active MPIO devices will, as long as ``multipath`` is present inside the ramdisk, no longer possibly result in duplicate IO wipes occuring accross numerous devices. Story: #2010003 Task: #45108 Resolves: rhbz#2076622 Resolves: rhbz#2070519 Change-Id: I0fd6356f036d5ff17510fb838eaf418164cdfc92 (cherry picked from commit 014d37743a3b5694e0e2a3cabfafe885417172d5) (cherry picked from commit 2c95ee45339dc910a6b29cb5e0f24a1f48898165) (cherry picked from commit 3c9b1131320fb97e81af38fc7471f5dbc968ca12)
* Do not try to guess EFI partition path by its numberDmitry Tantsur2022-06-083-47/+107
| | | | | | | | | | | | | | | The logic of adding a partition number to the device path does not work for devicemapper devices (e.g. a multipath storage device). Conflicts: ironic_python_agent/efi_utils.py ironic_python_agent/extensions/image.py ironic_python_agent/tests/unit/extensions/test_image.py ironic_python_agent/tests/unit/test_efi_utils.py Change-Id: I9a445e847d282c50adfa4bad5e7136776861005d (cherry picked from commit f09f6c9f1a09c7062d0450b3e0a4d3164fd53f7f) (cherry picked from commit 12e03698874d3d51884fd0de3174fa7b56076852)
* Add `mount` and `parted -l` to the collected commandsDmitry Tantsur2022-06-072-2/+12
| | | | | | | | | Conflicts: ironic_python_agent/tests/unit/test_utils.py Change-Id: I1c759552220291890704d0002a62ea3f51701691 (cherry picked from commit f1ee454a0ee9a8f18fbfd504d081ce3aeeb0ffa3) (cherry picked from commit 54cbfa1f3fe0b8935e39044baee8c4996b2a3ced)
* Collect a full lsblk output in the ramdisk logsDmitry Tantsur2022-06-072-6/+3
| | | | | | | | | | | The existing lsblk call is very handy for an overview, but there a lot more useful pairs to collect. Collect them in a machine-readable format to be able to use in debugging and further development. Change-Id: Ib27843524421944ee93de975d275e93276a5597a (cherry picked from commit 424e649bed3db5d1129b18b7ea4dfba88d552537) (cherry picked from commit bc74df8bfe89f703880f8a545ce939e9b1cd8651) (cherry picked from commit c674e567fc3fbf0976b1310763be88ad5a11f7e4)
* Create fstab entry with appropriate labelJulia Kreger2022-06-075-10/+125
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Depending on the how the stars align with partition images being written to a remote system, we *may* end up with *either* a Partition UUID value, or a Partition's UUID value. Which are distinctly different. This is becasue the value, when collected as a result of writing an image to disk *falls* back and passes the value to enable partition discovery and matching. Later on, when we realized we ought to create an fstab entry, we blindly re-used the value thinking it was, indeed, always a Partition's UUID and not the Partition UUID. Obviously, the label type is quite explicit, either UUID or PARTUUID respectively, when initial ramdisk utilities such as dracut are searching and mounting filesystems. Adds capability to identify the correct label to utilize based upon the current state of the block devices on disk. Granted, we are likely only exposed to this because of IO race conditions under high concurrecy load operations. Normally this would only be seen on test VMs, but systems being backed by a Storage Area Network *can* exibit the same IO race conditions as virtual machines. Change-Id: I953c936cbf8fad889108cbf4e50b1a15f511b38c Resolves: rhbz#2058717 Story: #2009881 Task: 44623 (cherry picked from commit 99ca1086dbfc7b6e41cf800b0bd899565e2e8922) (cherry picked from commit c69ea032fe8ab81e459fb44f846f440e7a2c8922)
* Use a pre-defined partition UUID to detect configdrive on GPTDmitry Tantsur2022-05-102-39/+42
| | | | | | | | | | | | Using partition numbers is currently broken for devicemapper devices. Fortunately, GPT has partition UUIDs, so we can just generate one and use it for lookup. NOTE on backport: _get_partition is imported from the image extension. In Yoga it was moved to partition_utils. Change-Id: I41ffe4f8e4c6e43182090b5aa2a2b4b34f32efd5 (cherry picked from commit 65c4de903a2d8059b1520b0102235b6700287f87)
* Merge "Re-read the partition table with partx -a, part 2" into bugfix/8.1Zuul2021-11-232-43/+43
|\
| * Re-read the partition table with partx -a, part 2Riccardo Pittau2021-11-102-43/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | Use add instead of update to re-read the partition table with partx. See [1] for more details. Co-authored-by: Arne Wiebalck <arne.wiebalck@cern.ch> [1] https: //opendev.org/openstack/ironic-python-agent/commit/dc8c1f16f9a00e2bff21612d1a9cf0ea0f3addf0 Change-Id: I2336e22dadc790cfbde87904612fcaa3b8c501db (cherry picked from commit 23e67b5fea58295d0320f99d4513a66d720f09ee)
* | Fix UEFI record regexJulia Kreger2021-11-162-6/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | | I accidently put colons on the test data and remembered taking the colon character out of the regex I was working on, but apparently left it in, and accounted for the active entry indicator flag which appears to have inconsistent support across vendors. The regex has been fixed, and a test added from a Lenovo SR650 which has some additional string entry data in the UEFI output which may separate entries. Change-Id: I1f67b0fb1f645fa82e98bd7c7bba3ffc7755cc74 (cherry picked from commit e10f052c06c03016b0ff4d9c1f3191c79fc50a1a)
* | Merge "Delete EFI boot entry duplicate labels first" into bugfix/8.1Zuul2021-11-152-24/+45
|\ \
| * | Delete EFI boot entry duplicate labels firstJulia Kreger2021-11-102-24/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some firmware seems to take an objection with EFI nvram entries being deleted after one is added, resulting in the entire entry table being reset to the last known good state. This is problematic, as ultimately deployments can time out if we previously booted with Networking, and the machine, while commanded to do other wise, reboots back to networking regardless. We will now delete entries first, before proceeding. Additionally, for general use, this pattern may serve the community better by avoiding cases where we would have previously just relied upon efibootmgr[0] to warn us of duplicate entries. [0]: https://github.com/rhboot/efibootmgr/blob/103aa22ece98f09fe3ea2a0c83988f0ee2d0e5a8/src/efibootmgr.c#L228 Change-Id: Ib61a7100a059e79a8b0901fd8f46b9bc41d657dc Story: 2009649 Task: 43808 (cherry picked from commit 67eddfa7e3fedbb530045f5b43a2c89db832fa2a) (cherry picked from commit 33b39705a50513c5af411216b48e2a6f6ac9ab14)
* | | Merge "Output verbose info from efibootmgr" into bugfix/8.1Zuul2021-11-152-21/+21
|\ \ \ | |/ / | | / | |/ |/|
| * Output verbose info from efibootmgrDerek Higgins2021-11-102-21/+21
| | | | | | | | | | | | | | | | When debugging boot manager problems it can be advantageous to see all the full entries rather then just their labels. Change-Id: I6a1bb78acaf5a4284727bdf533d4be6db2099f50 (cherry picked from commit caf695f70ab366498b46cb6f07f6751369c67e30)
* | Re-read the partition table with partx -aArne Wiebalck2021-11-092-16/+16
|/ | | | | | | | | | | | Re-read the partition table with 'partx -a', rather than 'partx -u'. This should fix an timing issue where the bootloader installation fails to mount the EFI partition from a whole disk image since it is not yet aware of the new partitions (observed with both, the iscsi and the direct deploy interface). Change-Id: If5da3075e813ae01df3decf8f0647aba111b0515 (cherry picked from commit dc8c1f16f9a00e2bff21612d1a9cf0ea0f3addf0)
* Assert EFI part UUID is not None before editing fstabArne Wiebalck2021-10-211-1/+1
| | | | | | | | | | | The EFI partition UUID may be None and this will break the fstab editing. While this is not necessarily fatal when instantiating a node, it creates an exception at the end of bootloader installation, so only attempt to add a line to fstab when the UUID is not None. Change-Id: I68799980e67c05afe4ca68ca9733605dd166d54d (cherry picked from commit 333ed70c94e366f16d8f2633f74a5ef05aa5fadb)
* Software RAID: Call udev_settle before creationArne Wiebalck2021-10-131-0/+7
| | | | | | | | | | | | | This patch fixes a race during software RAID creation: we create the partition with parted, the kernel then notifies udev, but we need to wait for udevd to create the device files before calling mdadm to create the md device. Credits to jcosmao for finding this. Change-Id: I642f28acc351cf50263e37dfbc8468bf59de2cc5 (cherry picked from commit 9d707e9f4bab40109b7e29df2136e86d65325ea3)
* Merge "Force immediate NTP time sync with chronyd at IPA startup"8.1.0Zuul2021-07-203-46/+13
|\
| * Force immediate NTP time sync with chronyd at IPA startupArne Wiebalck2021-07-163-46/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | In order to make sure we have the correct time early, e.g. by the time we create a TLS certificate, this patch proposes to force an immediate NTP update when using chronyd. While the previous approach uses the passed NTP server as well, the update may happen only after chronyd has performed measurements (which may be too late). Story: #2009058 Task: #42843 Change-Id: I6edafe8edeb8549f324959e7a1ec175c3049a515
* | Merge "Catch ismount not being handled"Zuul2021-07-162-9/+153
|\ \ | |/ |/|
| * Catch ismount not being handledJulia Kreger2021-06-292-9/+153
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While investigating another grub issue, I was confused by the path taken in the logs reported, and noticed that on a ramdisk, we might not actually have a valid response to os.path.ismount, I'm guessing depending on what in memory filesystem is in use while also coupled with attempting to check a filesystem. Adds a test to validate that exceptions raised on these commands where this issue can be encountered, are properly bypassed, and also adds additional logging to make it easier to figure out what is going on in the entire bootloader setup sequence. Change-Id: Ibd3060bef2e56468ada6b1a5c1cc1632a42803c3
* | Burn-in: Add network stepArne Wiebalck2021-07-134-0/+170
|/ | | | | | | | | | Add a clean step for network burn-in via fio. Get basic run parameters from the node's driver_info. Story: #2007523 Task: #42385 Change-Id: I2861696740b2de9ec38f7e9fc2c5e448c009d0bf
* Merge "Only mount the ESP if not yet mounted"Zuul2021-06-252-7/+13
|\
| * Only mount the ESP if not yet mountedArne Wiebalck2021-06-212-7/+13
| | | | | | | | | | | | | | Check if the ESP is already mounted before attempting to mount it for the bootloader installation. Change-Id: Ifd738b2c5663f1a211d7e13b5ba386be631d8db1
* | Coalesce heartbeatsDmitry Tantsur2021-06-182-9/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The IPA sends heartbeats to the conductor periodically and when requested, e.g. at the end of asynchronous commands. In order to avoid to send such notifications in too quick succession, e.g. when two asynchronous commands finish at the same time or when the periodic heartbeat was just sent right before a command ended, this patch proposes to coalesce heartbeats which are close together timewise and send only one for all of them in a time interval of 5 seconds. Co-Authored-By: Arne Wiebalck <arne.wiebalck@cern.ch> Story: #2008983 Task: 42633 Change-Id: Idfbce44065e1e5a8b730b94741b2604c51f0ab14
* | Reduce logging verbosity when collecting logsDmitry Tantsur2021-06-111-2/+1
|/ | | | | | | | It's not uncommon that some commands fail when collecting logs. We already log all failures in utils.execute, no need to duplicate them with a non-fatal ERROR logging. Change-Id: If151b3a3be979bd2b3ce01030e5d6242ad74eaa3
* Utilize CSV file for EFI loader selectionJulia Kreger2021-06-102-14/+115
| | | | | | | | | | | | | | | | | | | | | | Adds support to identify and utilize a CSV file to signal which bootloader to utilize, and set it when the OS is running as opposed to when EFI is running. This works around EFI loader potentially crashing some vendors hardware types when entry stored in the image does not match the EFI loader record which was utilzied to boot. Grub2+shim specifically specifically needs the CSV file name and entry label to match what the system was booted with in order to prevent the machine from potentially crashing. See https://storyboard.openstack.org/#!/story/2008962 and https://bugzilla.redhat.com/show_bug.cgi?id=1966129#c37 for more information. Change-Id: Ibf1ef4fe0764c0a6f1a39cb7eebc23ecc0ee177d Story: 2008962 Task: 42598 Co-Authored-By: Bob Fournier <bfournie@redhat.com>
* Merge "Burn-in: Add disk step"Zuul2021-06-074-0/+114
|\
| * Burn-in: Add disk stepArne Wiebalck2021-05-214-0/+114
| | | | | | | | | | | | | | | | | | | | Add a clean step for disk burn-in via fio. Get basic run parameters from the node's driver_info. Story: #2007523 Task: #42384 Change-Id: I5f5e336bd629846b3d779fd0fc7a2060b385b035
* | Merge "Ignore efi grub2-install failure"Zuul2021-06-072-29/+51
|\ \
| * | Ignore efi grub2-install failureSteve Baker2021-06-042-29/+51
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recent releases of redhat grub2 will always fail when installing to EFI paths, to encourage a transition to the signed shim bootloader. Partition image deploys avoid calling grub2-install with the preserve-efi-assets functions. Deploying whole disk images doesn't require grub2-install. This leaves whole disk images installed onto softraid devices, which still attempts to call grub2-install. This change will still attempt to run grub2-install in this one remaining case, but will ignore any failure. A future enhancement can avoid calling grub2-install entirely so that non-redhat secure-boot capable images can keep their signed bootloaders. Story: 2008923 Task: 42521 Change-Id: If432ef795d64d76442d739eb4f7d155ff847041e
* | | Merge "Refactor: use convert_image from ironic_lib"Zuul2021-06-042-27/+13
|\ \ \
| * | | Refactor: use convert_image from ironic_libDmitry Tantsur2021-06-012-27/+13
| |/ / | | | | | | | | | Change-Id: If890baf3545cff6cef7c645c42e7f9d9038c9aa7
* | | Merge "Stop accepting duplicated configdrive"Zuul2021-06-022-39/+71
|\ \ \ | |/ / |/| |
| * | Stop accepting duplicated configdriveDmitry Tantsur2021-06-022-39/+71
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We're currently requiring it twice: in image_info and in a separate configdrive argument. I think we should eventually settle on separate arguments for separate entities, so this change makes the value in image_info optional with a goal to stop accepting it. We could probably just remove the handling in image_info, but a deprecation is safer. The (unused in ironic) cache_image call is updated with an optional configdrive arguments. Story: #2008904 Task: #42480 Change-Id: I1e2efa28efa3ea7e389774cb7633d916757bc6ed
* | | Merge "Make _get_efi_bootloaders return relative paths"Zuul2021-05-272-17/+27
|\ \ \
| * | | Make _get_efi_bootloaders return relative pathsSteve Baker2021-05-262-17/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To make this function useful for purposes other than efibootmgr entries, this change moves the path manipulation to _run_efibootmgr. This change also adds boot*.efi entries to BOOTLOADERS_EFI so that it includes every entry in the UEFI Spec 2.9[1] Table 3-2 UEFI Image Types. [1] https://uefi.org/sites/default/files/resources/UEFI_Spec_2_9_2021_03_18.pdf Story: 2008923 Task: 42521 Change-Id: Ibe02786609aa0de65115897d8f4a9b4f36c8aed2
* | | | Limit qemu-img execution arenasJulia Kreger2021-05-262-2/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | qemu-img attempts to launch multiple threads by default *and* attempts to have multiple memory allocation arenas to operate from. While multithreading can be good for performance, this pattern and the memory footprint for process launch and dependencies can turn the memory footprint for a cirros image conversion (16MB) into 1.2GB of memory being asked for by the qemu-img tool. In order to limit this impact, as the default number of arenas is governed by the number of CPUs times the number 8, it seems reasonable to lower this to a more reasonable number which also helps keep our possible memory footprint from being exceeded. Change-Id: I71a28ec59ec31c691205eb34d9fcab63a2ccb682 Story: 2008928 Task: 42528
* | | | Merge "Rewrite write_image.sh in Python"Zuul2021-05-263-78/+42
|\ \ \ \ | |/ / / |/| | |
| * | | Rewrite write_image.sh in PythonDmitry Tantsur2021-05-183-78/+42
| | | | | | | | | | | | | | | | Change-Id: I0caa65561948f4e0934943a7a0d3a209701b5a59
* | | | Merge "Enable out-of-order writes when writing whole disk images"Zuul2021-05-251-1/+1
|\ \ \ \ | |/ / / | | | / | |_|/ |/| |
| * | Enable out-of-order writes when writing whole disk imagesDmitry Tantsur2021-05-181-1/+1
| | | | | | | | | | | | | | | | | | Per documentation it improves performance when using -O host_device. Change-Id: Ic6a97af9f865d07c9cb4257397a320475a28f88b
* | | Merge "Do not serialize command_params"Zuul2021-05-182-7/+1
|\ \ \ | |/ / |/| |
| * | Do not serialize command_paramsDmitry Tantsur2021-05-182-7/+1
| |/ | | | | | | | | | | | | | | | | The command params can be huge when configdrive is used. There is no point in sending them back, Ironic does not use them anyhow. Story: #2008904 Task: #42479 Change-Id: I6e3db5db2042ca3fb5dafacfacf036fd7fc2fc4c
* | Merge "Remove the iscsi extension"Zuul2021-05-126-697/+9
|\ \
| * | Remove the iscsi extensionDmitry Tantsur2021-05-106-697/+9
| |/ | | | | | | Change-Id: I2f0e581575112d6c7ba0d211661cab3e0b6caca6
* | Merge "Migrate functional tests for work_on_disk from ironic-lib"Zuul2021-05-121-0/+82
|\ \
| * | Migrate functional tests for work_on_disk from ironic-libDmitry Tantsur2021-05-101-0/+82
| |/ | | | | | | | | | | Missed in commit 24951b1029170840484a50fdd38d2a57858a578c. Change-Id: Iad5e8f161ac69b96b9332d83fe22b5e0b9192258
* | Merge "Burn-in: Add memory step"Zuul2021-05-114-0/+89
|\ \
View Lorry Controller Status