delta/openstack/ironic-python-agent.git/ironic_python_agent, branch stable/ussuri opendev.org: openstack/ironic-python-agent.git Use utf-16-le if BOM not present 2022-10-13T15:32:56+00:00 Riccardo Pittau elfosardo@gmail.com 2022-02-21T17:18:29+00:00 b42a747eaa7690aed6314f54e9407aabaa105a9c In case no BOM is present in the CSV file the utf-16 codec won't work. We fail over to utf-16-le as Little Endian is commonly used. NOTE: The original change landed this fix in efi_utils.py, however that was introduced after the Xena development cycle, so this backport moves the original change to where the code originally came from to populate the efi_utils.py file. Change-Id: I3e25ce4997f5dd3df87caba753daced65838f85a (cherry picked from commit 697fa6f3b6db10408eaadd57450456de87f13519) (cherry picked from commit 99b9d1403cacd3bbd489a6cf6913f32746ef6083) 
In case no BOM is present in the CSV file the utf-16 codec won't work.
We fail over to utf-16-le as Little Endian is commonly used.

NOTE: The original change landed this fix in efi_utils.py, however
      that was introduced after the Xena development cycle, so this
      backport moves the original change to where the code originally
      came from to populate the efi_utils.py file.

Change-Id: I3e25ce4997f5dd3df87caba753daced65838f85a
(cherry picked from commit 697fa6f3b6db10408eaadd57450456de87f13519)
(cherry picked from commit 99b9d1403cacd3bbd489a6cf6913f32746ef6083)
Re-read the partition table with partx -a, part 2 2021-12-15T01:45:40+00:00 Riccardo Pittau elfosardo@gmail.com 2021-11-09T11:50:46+00:00 dc62636c1720e595491989167feefc6cd5ff24e1 Use add instead of update to re-read the partition table with partx. See [1] for more details. Co-authored-by: Arne Wiebalck <arne.wiebalck@cern.ch> [1] https: //opendev.org/openstack/ironic-python-agent/commit/dc8c1f16f9a00e2bff21612d1a9cf0ea0f3addf0 Change-Id: I2336e22dadc790cfbde87904612fcaa3b8c501db (cherry picked from commit 23e67b5fea58295d0320f99d4513a66d720f09ee) (cherry picked from commit 56e63ce36a72448e128c01605d98492c80dca38f) 
Use add instead of update to re-read the partition table with partx.

See [1] for more details.

Co-authored-by: Arne Wiebalck <arne.wiebalck@cern.ch>

[1] https: //opendev.org/openstack/ironic-python-agent/commit/dc8c1f16f9a00e2bff21612d1a9cf0ea0f3addf0

Change-Id: I2336e22dadc790cfbde87904612fcaa3b8c501db
(cherry picked from commit 23e67b5fea58295d0320f99d4513a66d720f09ee)
(cherry picked from commit 56e63ce36a72448e128c01605d98492c80dca38f)
Re-read the partition table with partx -a 2021-12-15T01:45:40+00:00 Arne Wiebalck Arne.Wiebalck@cern.ch 2021-11-06T12:28:06+00:00 74389394001d3fd436ef963d60c93c2b3ee09f0d Re-read the partition table with 'partx -a', rather than 'partx -u'. This should fix an timing issue where the bootloader installation fails to mount the EFI partition from a whole disk image since it is not yet aware of the new partitions (observed with both, the iscsi and the direct deploy interface). Change-Id: If5da3075e813ae01df3decf8f0647aba111b0515 (cherry picked from commit dc8c1f16f9a00e2bff21612d1a9cf0ea0f3addf0) (cherry picked from commit ab3e8a75badf2719602b2416af9e3c19d7511c79) 
Re-read the partition table with 'partx -a', rather than 'partx -u'.

This should fix an timing issue where the bootloader installation
fails to mount the EFI partition from a whole disk image since it
is not yet aware of the new partitions (observed with both, the
iscsi and the direct deploy interface).

Change-Id: If5da3075e813ae01df3decf8f0647aba111b0515
(cherry picked from commit dc8c1f16f9a00e2bff21612d1a9cf0ea0f3addf0)
(cherry picked from commit ab3e8a75badf2719602b2416af9e3c19d7511c79)
Fix UEFI record regex 2021-11-19T19:35:11+00:00 Julia Kreger juliaashleykreger@gmail.com 2021-11-04T16:42:04+00:00 e5b75d4c0a9887d2b6fa43c748ccee483dd066f8 I accidently put colons on the test data and remembered taking the colon character out of the regex I was working on, but apparently left it in, and accounted for the active entry indicator flag which appears to have inconsistent support across vendors. The regex has been fixed, and a test added from a Lenovo SR650 which has some additional string entry data in the UEFI output which may separate entries. Change-Id: I1f67b0fb1f645fa82e98bd7c7bba3ffc7755cc74 (cherry picked from commit e10f052c06c03016b0ff4d9c1f3191c79fc50a1a) (cherry picked from commit 87a30ff4fa93e710efa2f9eeb227c779d40c16af) (cherry picked from commit 25f89e4d8859ec1a3e811c3d916cf49d603754fc) 
I accidently put colons on the test data and remembered taking the
colon character out of the regex I was working on, but apparently
left it in, and accounted for the active entry indicator flag
which appears to have inconsistent support across vendors.

The regex has been fixed, and a test added from a Lenovo SR650
which has some additional string entry data in the UEFI output
which may separate entries.

Change-Id: I1f67b0fb1f645fa82e98bd7c7bba3ffc7755cc74
(cherry picked from commit e10f052c06c03016b0ff4d9c1f3191c79fc50a1a)
(cherry picked from commit 87a30ff4fa93e710efa2f9eeb227c779d40c16af)
(cherry picked from commit 25f89e4d8859ec1a3e811c3d916cf49d603754fc)
Delete EFI boot entry duplicate labels first 2021-11-19T19:34:59+00:00 Julia Kreger juliaashleykreger@gmail.com 2021-10-28T18:41:00+00:00 81190407f3952275b2e05280c54a36dbcab28275 Some firmware seems to take an objection with EFI nvram entries being deleted after one is added, resulting in the entire entry table being reset to the last known good state. This is problematic, as ultimately deployments can time out if we previously booted with Networking, and the machine, while commanded to do other wise, reboots back to networking regardless. We will now delete entries first, before proceeding. Additionally, for general use, this pattern may serve the community better by avoiding cases where we would have previously just relied upon efibootmgr[0] to warn us of duplicate entries. [0]: https://github.com/rhboot/efibootmgr/blob/103aa22ece98f09fe3ea2a0c83988f0ee2d0e5a8/src/efibootmgr.c#L228 Change-Id: Ib61a7100a059e79a8b0901fd8f46b9bc41d657dc Story: 2009649 Task: 43808 (cherry picked from commit 67eddfa7e3fedbb530045f5b43a2c89db832fa2a) (cherry picked from commit 33b39705a50513c5af411216b48e2a6f6ac9ab14) (cherry picked from commit 8fca1457399ac7892427fac1b9ab74a4ac653f05) (cherry picked from commit 47ac40a7f943d0ce6e58f43330a73ac144c23aa1) 
Some firmware seems to take an objection with EFI nvram
entries being deleted after one is added, resulting in the
entire entry table being reset to the last known good state.

This is problematic, as ultimately deployments can time out
if we previously booted with Networking, and the machine, while
commanded to do other wise, reboots back to networking regardless.

We will now delete entries first, before proceeding.

Additionally, for general use, this pattern may serve the
community better by avoiding cases where we would have
previously just relied upon efibootmgr[0] to warn us of duplicate
entries.

[0]: https://github.com/rhboot/efibootmgr/blob/103aa22ece98f09fe3ea2a0c83988f0ee2d0e5a8/src/efibootmgr.c#L228

Change-Id: Ib61a7100a059e79a8b0901fd8f46b9bc41d657dc
Story: 2009649
Task: 43808
(cherry picked from commit 67eddfa7e3fedbb530045f5b43a2c89db832fa2a)
(cherry picked from commit 33b39705a50513c5af411216b48e2a6f6ac9ab14)
(cherry picked from commit 8fca1457399ac7892427fac1b9ab74a4ac653f05)
(cherry picked from commit 47ac40a7f943d0ce6e58f43330a73ac144c23aa1)
Output verbose info from efibootmgr 2021-11-18T14:50:01+00:00 Derek Higgins derekh@redhat.com 2021-08-03T11:10:44+00:00 0bf8769091b2855e8f1eb90977a7fabdce02f7f2 When debugging boot manager problems it can be advantageous to see all the full entries rather then just their labels. Depends-On: https://review.opendev.org/c/openstack/ironic/+/818394 Change-Id: I6a1bb78acaf5a4284727bdf533d4be6db2099f50 (cherry picked from commit caf695f70ab366498b46cb6f07f6751369c67e30) (cherry picked from commit 9753b61a51008da4f0b948b0c89665b762eb95c4) (cherry picked from commit 750934a20794efdcfe8340eec75996466c4c7665) 
When debugging boot manager problems it can be advantageous to
see all the full entries rather then just their labels.

Depends-On: https://review.opendev.org/c/openstack/ironic/+/818394
Change-Id: I6a1bb78acaf5a4284727bdf533d4be6db2099f50
(cherry picked from commit caf695f70ab366498b46cb6f07f6751369c67e30)
(cherry picked from commit 9753b61a51008da4f0b948b0c89665b762eb95c4)
(cherry picked from commit 750934a20794efdcfe8340eec75996466c4c7665)
Fix getting memory size in some lshw output 2021-06-25T22:27:16+00:00 Zane Bitter zbitter@redhat.com 2021-04-28T16:26:43+00:00 746f65bd8481aac32abb12768590e809697449e4 Due to a regression in lshw introduced by https://github.com/lyonel/lshw/pull/60, there are some versions in the wild that do not return sizes for memory banks <32GiB. In those cases, work around the problem by looking at the top-level size (if available) to find the total size. Previously we assumed that we only needed the top-level size when there was no list of memory banks. The issue is fixed upstream by https://github.com/lyonel/lshw/pull/65, but the erroneous patch is still present in the lshw-B.02.19.2-5.el8 package in CentOS 8.4 and 8.5. Note from Backport to Victoria branch: The stable test data was moved sometime during the Wallaby development cycle to a separate file, where as in Victoria and earlier, it is based in the test file itself. The required content was moved into the test file in line with where it was in the later versions. Change-Id: I6eb5981d28b9ae368239af0c1d0ec32ff79d95b3 Story: #2008865 Task: 42395 (cherry picked from commit ed791d97786f4ed37bf7b9f18eac8e2af46c3766) (cherry picked from commit bae3aec172df5b6ca9d609f29ea9feb9f6d11f13) (cherry picked from commit 97ce08d039099f569c48a2348c784e71e3693829) 
Due to a regression in lshw introduced by
https://github.com/lyonel/lshw/pull/60, there are some versions in the
wild that do not return sizes for memory banks <32GiB. In those cases,
work around the problem by looking at the top-level size (if available)
to find the total size. Previously we assumed that we only needed the
top-level size when there was no list of memory banks.

The issue is fixed upstream by https://github.com/lyonel/lshw/pull/65,
but the erroneous patch is still present in the lshw-B.02.19.2-5.el8
package in CentOS 8.4 and 8.5.

Note from Backport to Victoria branch:
The stable test data was moved sometime during the Wallaby development
cycle to a separate file, where as in Victoria and earlier, it is based
in the test file itself. The required content was moved into the test
file in line with where it was in the later versions.

Change-Id: I6eb5981d28b9ae368239af0c1d0ec32ff79d95b3
Story: #2008865
Task: 42395
(cherry picked from commit ed791d97786f4ed37bf7b9f18eac8e2af46c3766)
(cherry picked from commit bae3aec172df5b6ca9d609f29ea9feb9f6d11f13)
(cherry picked from commit 97ce08d039099f569c48a2348c784e71e3693829)
Add function to calculate memory 2021-06-25T22:27:10+00:00 Riccardo Pittau elfosardo@gmail.com 2020-04-23T14:32:23+00:00 e03d80d572c173aed66dcc10909b4df684e1c76b Move logic to calculate memory to its own function. Change-Id: I5ab98b6450ff45dff35ddae093a83140f37047a8 (cherry picked from commit 2738e57f2a6cacd44cf78331d6d77f3ae9f6a254) 
Move logic to calculate memory to its own function.

Change-Id: I5ab98b6450ff45dff35ddae093a83140f37047a8
(cherry picked from commit 2738e57f2a6cacd44cf78331d6d77f3ae9f6a254)
Reduce logging verbosity when collecting logs 2021-06-14T13:34:59+00:00 Dmitry Tantsur dtantsur@protonmail.com 2021-06-11T14:04:59+00:00 76d061ea87e633874f19569512d11f3d54d8193d It's not uncommon that some commands fail when collecting logs. We already log all failures in utils.execute, no need to duplicate them with a non-fatal ERROR logging. Change-Id: If151b3a3be979bd2b3ce01030e5d6242ad74eaa3 (cherry picked from commit 2fcf35e56d578d98d77ffc97b6441f83548d7704) 
It's not uncommon that some commands fail when collecting logs.
We already log all failures in utils.execute, no need to duplicate
them with a non-fatal ERROR logging.

Change-Id: If151b3a3be979bd2b3ce01030e5d6242ad74eaa3
(cherry picked from commit 2fcf35e56d578d98d77ffc97b6441f83548d7704)
Utilize CSV file for EFI loader selection 2021-06-11T14:08:03+00:00 Julia Kreger juliaashleykreger@gmail.com 2021-06-10T18:23:14+00:00 8287b96628afc38a8667b7a38e16d2a715011a36 Adds support to identify and utilize a CSV file to signal which bootloader to utilize, and set it when the OS is running as opposed to when EFI is running. This works around EFI loader potentially crashing some vendors hardware types when entry stored in the image does not match the EFI loader record which was utilzied to boot. Grub2+shim specifically specifically needs the CSV file name and entry label to match what the system was booted with in order to prevent the machine from potentially crashing. See https://storyboard.openstack.org/#!/story/2008962 and https://bugzilla.redhat.com/show_bug.cgi?id=1966129#c37 for more information. Change-Id: Ibf1ef4fe0764c0a6f1a39cb7eebc23ecc0ee177d Story: 2008962 Task: 42598 Co-Authored-By: Bob Fournier <bfournie@redhat.com> (cherry picked from commit 2fab70c36ba40a345a9dd01aeb5019681e567aa5) (cherry picked from commit bfa97cbbc2040d56bd0db14d1e1b83bb14f1a74c) (cherry picked from commit 474d2a91b06942a72c59d0f25ddd65fc255a3d16) 
Adds support to identify and utilize a CSV file to signal which
bootloader to utilize, and set it when the OS is running as opposed
to when EFI is running. This works around EFI loader potentially
crashing some vendors hardware types when entry stored in the
image does not match the EFI loader record which was utilzied to
boot.

Grub2+shim specifically specifically needs the CSV file name
and entry label to match what the system was booted with in order
to prevent the machine from potentially crashing.

See https://storyboard.openstack.org/#!/story/2008962
and https://bugzilla.redhat.com/show_bug.cgi?id=1966129#c37
for more information.

Change-Id: Ibf1ef4fe0764c0a6f1a39cb7eebc23ecc0ee177d
Story: 2008962
Task: 42598
Co-Authored-By: Bob Fournier <bfournie@redhat.com>
(cherry picked from commit 2fab70c36ba40a345a9dd01aeb5019681e567aa5)
(cherry picked from commit bfa97cbbc2040d56bd0db14d1e1b83bb14f1a74c)
(cherry picked from commit 474d2a91b06942a72c59d0f25ddd65fc255a3d16)