AMT connectivity lost after bios update

[rgord1]

I have experienced loss of AMT connectivity after updating an Intel bios. The only way it could be restored was through resetting the CMOS and re-provisioning from scratch.

Is it advisable to disable Intel MEBx before doing a bios update? If so, what steps need to be taken to accomplish this, so that after a bios update, MEBx can again be fully provisioned without resetting the CMOS?

[MIGUEL_C_Intel]

Hello, rgord1,

The BIOS update process is different for every system manufacturer including us. In your case, the BIOS is losing the previous configuration.  The CMOS clearing is wiping the pre-configured settings and putting the system in default.

I think the suggestion is accessing MEBx and keep a record of the options selected.  Pay attention to Network Access status, PKI DNS suffix, User Consent, and KVM status.

Regards,

Miguel C.

Intel Customer Support Technician

[rgord1]

Thanks for the response.

As I recall, the problem I encountered was that MEBx appeared to be configured properly in the updated bios, but MeshCommander was unable to locate the PC either through a scan or Add Computer. After clearing CMOS and reconfiguring MEBx, MeshCommander was able to locate the PC and a new TLS certificate was requested.

It seems that TLS authentication was the issue. If a vPro PC issues a certificate, would updating the bios require a new certificate to be issued, and if so, should the original certificate somehow be revoked before the update?

[Mr_vPro]

Hi rgord1,

if you have any issues related to Intel AMT please always provide exact ME FW version (major.minor.hotfix.build) and bit more system HW/OEM details (model of vPro PC)  it will make it easier for Intel Customer support team (Miguel C.) or other Intel folks (like me) to narrow down potential root cause area.

if your system is 12th Gen Core vPro with ME FW 16.0.x BIOS FW update Package (depending on OEM) may also include ME FW update inside same update package (Dell, HP, Intel NUC) and your ME FW may be updated to  16.1.y in which non TLS AMT support was deprecated so non encrypted AMT ports 16992/16994 are no longer listening. Instead of it AMT is accessible over TLS ports 16993/16995 with TLS encryption using self signed AMT certificate (you can also set it up with your own Root CA signed certs, or Intel EMA will do it).

https://software.intel.com/sites/manageability/AMT_Implementation_and_Reference_Guide/WordDocuments/Importantannouncements.htm

In Mesh Commander  you have to change from Digest /None to Digest /TLS:

until it is done you may perceive AMT as not responding anymore.

By design updating BIOS FW itself shall not require to disable AMT or unconfigure it or reconfigure it again via MEBx unless OEM BIOS update is reflashing complete Flash chip content which is against Intel recommendations (and has some implication on platform security).

rgds

Dariusz Wittek

Biz Client Technical Sales Specialist  |  Intel EMEA CCG Technical Sales

[rgord1]

That is all useful information, thank you. Particularly that it should not be necessary to disable AMT or unconfigure it or reconfigure it again via MEBx before reflashing the bios.

This is a 13th generation NUC, with AMT already requiring TLS authentication in the original bios. There was no non TLS AMT support to be deprecated.

As I indicated, after flashing the bios update MeshCommander could not find the PC either through a scan or Add Computer. If you, or the Intel vPro Customer support team could throw any light on why that happened, and better still, advise on steps to overcome it should it happen again if and when I update with the latest bios posted 8 March by Asus, I would be very grateful.

TIA!

[Mr_vPro]

Hi rgord1,

it is hard to tell now what went wrong with your NUC BIOS update - AMT shall not got unconfigured with just a BIOS update.

We/you do not have enough details about previous BIOS version and current one (versions not dates) - please also note NUC product were transitioned to ASUS so you shall raise support request to ASUS.

By design OEM BIOS update shall not require unconfigure  neither do unconfigure of AMT unless it is done in wrong way as complete flash chip reimage (which also is not correct design as per Intel recommendations).

If you disconnected CMOS RTC battery for BIOS update process for any reason it raises RTC battery reset flag which Intel MEBx FW receives on next boot at trigger to  perform full factory reset of Intel ME FW (including Intel AMT).

Once Intel AMT is reset to factory defaults it is no longer activated and does not respond to direct connections, although it still shall respond to RMCP Ping over Wired LAN interface when OS is up and running (this is what MeshCommander scan is based on).

rgds

Dariusz Wittek

Biz Client Technical Sales Specialist  |  Intel EMEA CCG Technical Sales

[rgord1]

Thanks for the response.

I have a support ticket in with Asus asking if the loss of AMT connectivity was an issue with the particular bios update I applied. So far, after elevating it to their "product support team" I have had no response.

Another bios update was released shortly after the one that resulted in the loss of connectivity. I would only attempt to apply that update after hearing back from Asus.

In my experience a MeshCommander scan will not find a vPro NUC unless and until it is provisioned for AMT. So I can say that I have not seen any instance of "...it still shall respond to RMCP Ping over Wired LAN interface when OS is up and running" prior to the device being provisioned for AMT.

[MIGUEL_C_Intel]

Hello, rgord1,

Mesh Commander was a great open-source tool; however, it arrived at its end of life and started presenting compatibility issues. Intel collaborated with the development team in the past.

I suggest start using Intel® Endpoint Management Assistant (Intel® EMA).

https://www.intel.com/content/www/us/en/download/19449/intel-endpoint-management-assistant-intel-ema.html

The zip file contains the manual installation guides.

If you have Azure Cloud, I suggest the version called Intel® Endpoint Management Assistant (Intel® EMA) Cloud Start Tool for Azure.  The configuration is easier, use the Simplified Template.  Less licensing cost with SQL Express.

Intel® Endpoint Management Assistant (Intel® EMA) Cloud Start Tool for Azure*

https://www.intel.com/content/www/us/en/download/19738/intel-endpoint-management-assistant-intel-ema-cloud-start-tool-for-azure.html

I will gladly help you with the configuration of Intel® EMA if necessary.  

Regards,

Miguel C.

Intel Customer Support Technician

[rgord1]

I have installed Intel EMA, and reached the stage of endpoint management of one of my vPro devices, a 12th generation Intel NUC that does not require TLS authenticatio for AMT connections.

The GUI advises that "The endpoint is provisioned by other tool and Intel EMA does not have the record. Please use that tool or MEBx to unprovision this endpoint first."

If I unprovision it:

1. Will I be able to reprovision it in Admin Control Mode through the Intel EMA interface?

2. Will I then be unable to access it through MeshCommander and/or Intel Manageability Commander?

TIA

[rgord1]

Thanks again.

I looked at Intel EMA a couple of years ago and it seemed to me overly complex compared with the more user friendly AMT connection tools MeshCommander and Intel Manageability Commander.

I will however attempt an installation as I have MySQL Server 5.7 installed already for another purpose. Can that be used for Intel EMA?

If I run into any difficulties with installation I'll call on your offer for assistance.

In any case, I suspect that if MeshCommander is unable to locate my vPro device after a bios update, then neither would Intel EMA, unless you think otherwise.

Regards

[Mr_vPro]

Hi,

Intel EMA can adopt Intel AMT already configured by external tool (for example Intel MEBx interface manually) via one of its API - see 

Intel EMA Rest API manual at https://www.intel.com/content/www/us/en/support/articles/000055621/software/manageability-products.html  and samples (Swagger)  Latestswagger.html 

Intel EMA deploys by default reversed AMT connectivity mode - CIRA (Client Initiated Remote Access) https://software.intel.com/sites/manageability/AMT_Implementation_and_Reference_Guide/default.htm?turl=WordDocuments%2Fdetaileddescription12.htm

Intel EMA SWARM server acts as Intel AMT MPS.
CIRA means AMT FW /HW connects out from the managed endpoint to EMA SWARM server with MTLS secured tunnel so it allows to manage endpoints with Intel AMT in corporate intranet as well as over Internet.

Alternatively you may set EMA AMT profile to TLS relay - when AMT will be configured to TLS listen mode -AMT console connects to Intel AMT in intranet network but EMA does not connect to AMT in TLS relay mode directly - it will do it via another PC running EMA Agent in same network segment - called Nearby Endpoints.

Intel EMA always will set Intel AMT into TLS encrypted mode with EMA signed/issued AMT TLS certs -also for MTLS CIRA tunnel mode.

rgds

darek

[rgord1]

I'm not so much concerned by Intel EMA's ability to access my endpoints (I know it can) as by the ability of Intel EMA to provision endpoints in Admin Control Mode. Currently I am without a valid certificate to enable Certificate Based Provisioning, so I'm limited to Host Based Provisioning.

I'm looking for a Powershell script that will generate an acceptable self-signed certificate. If you could point me to one I would be grateful!

TIA!

[MIGUEL_C_Intel]

Hello, rgord1,

Intel® EMA works with TLS 1.2 and higher. Even after doing the BIOS update, EMA will continue recognizing the endpoint.

The endpoint was provisioned with MeshCommander; perform the unprovisioning from the MEBx BIOS. 

MeshCommander and Manageability Commander might provision the endpoint after testing EMA if you keep the BIOS with the current BIOS (no updates).

Intel® EMA has some Server and endpoint prerequisites:

The Server needs to be running Windows Server 2019 or 2022.

The database is supported by SQL Enterprise or Express

Endpoints need to have Intel® AMT version 11.8.79 or higher.

Windows 10 or Windows 11

Provisioning in Admin Control Mode

This method requires a third-party Certificate chain (extra charge). The Certificate hash of the pre-authorized Vendors are included in the BIOS of the endpoints with Intel® vPro. This feature allows remote provisioning and allows the OOB connection.

1.3.6 Intel® AMT PKI Certificate of the Intel® EMA Server Installation Guide

https://www.intel.com/content/dam/support/us/en/documents/software/manageability-products/intel-ema-server-installation-and-maintenance-guide.pdf#page=11 

Vendor Certificates to Support Intel® AMT (bottom of the page)

https://www.intel.com/content/www/us/en/architecture-and-technology/vpro/active-management-technology/implementation.html

Regards,

Miguel C.

Intel Customer Support Technician

[rgord1]

I appreciate the assistance, but I think we may be talking at cross purposes.

1. The managed endpoints are not configured by MeshCommander, they are all provisioned in Admin Control Mode at the MEBx level independently of MeshCommander in each endpoint. MeshCommander is only used as a utility for connecting to and managing the endpoints.

2. My installation of Intel EMA is on a Windows 11 PC, not Windows Server.

3. I understand that provisioning in Admin Control Mode through Intel EMA requires a third-party Certificate. For cost reasons I am not in a position to purchase such a certificate, and so far my attempts to generate a self-signed certificate recognised by EMA have been unsuccessful. As noted elsewhere in this forum there is no Certificate Template Snap-In for Microsoft Management Console (MMC) in Windows 10 or Windows 11, so generation of a self-signed certificate is not possible by that method. I have attempted generation of a certificate with Powershell, but from what I have read what I generated was a SHA1 certificate no longer recognised as valid for AMT TLS configuration.

I'm still looking for a Powershell script that will generate an acceptable self-signed certificate. If you could point me to one I would be grateful.

Thanks again!

[MIGUEL_C_Intel]

Hello, rgord1,

I will try to answer your questions and give you a manage option for the endpoints.

1- Intel® EMA gives the warning message “endpoint handled by the third-party tool” if the endpoint was provisioned by a Third-party tool or manually from the MEBx.  In addition, if the endpoint is managed by Mesh Commander; the installation of the EMA agent file will fail; it is possible to install the software but, it will not provision the endpoint.

2- EMA needs to be installed in Windows Server 2019 or higher.  In addition, it is necessary to include the IIS role.  Before installing Intel® EMA software, we need to install SQL Enterprise 2017 or Express.

3- The Certificate needs to be done using Microsoft Management Console (MMC). 

The OS of the Server and the database are requirements of the software.  In the case of the Certificate, there is a workaround, but it requires manual access to the MEBx BIOS of each endpoint the first time.

Go to the EMA server and stop managing all the desired endpoints.

Uninstall the EMA agent file in the endpoint. Access the EMA agent installation file with Admin rights and select uninstall.

Then, 

1.  To prevent possible issues, perform a Full Unprovision of the endpoint from MEBEX BIOS.

2.  Go into MEBX and under Intel® AMT Configuration Network Access State choose Enable.

3.  Change the User Consent in MEBX to NONE.  This will allow us to do OOB KVM without any user interaction.

Note:

After Enabling the Network Access in MEBEX.  EMA WebUI and Intel® EMA configuration tool show the endpoint as provisioned in Admin mode by another instance, despite the EMA agent file being created in Client Control Mode.

4.  Return to the EMA server, and make sure CIRA mode is selected.

5.  Enable AMT Auto-setup and choose the right AMT profile.

6.  Choose HBP (Host Base Provisioning), and create your password, disable the randomized password.  This will make the adoption easier.

7.   Install the EMA agent files into the endpoint.  

8.   Now, download the Intel® Endpoint Management Assistant (Intel® EMA) API Sample Scripts in the EMA Server.  We will use the adopted scripts to pull the endpoint into EMA.

9.  The easiest way to adopt a single system is using the Adopt-AMTSetupBySearch PS script.

10.  Run the CLI request or you can do it by using the full command: 

PS> ./Adopt-AMTSetupBySearch.ps1 -emaServerURL EMAServer.demo.com -searchMethod hostnameStart -searchString laptop.

emaServerURL: FQDN

searchMethod: hostnameStart or hostnamematch

searchString: endpoint Name

11.  The Command will request the credentials of the Intel® EMA Tenant Administrator.

12.  The PowerShell script has examples in it, for custom scripting purposes.

13.  Bring up the Platform Manager and verify if everything went smoothly.  Also, verify in the EMA WebUI.  The endpoint should have now been adopted in Admin Control Mode (ACM) since that was set up in MEBX, despite the auto-setup specifying HBP. 

Regards,

Miguel C.

Intel Customer Support Technician

[rgord1]

The workaround is not working for me.

At step 7, Install the EMA agent files into the endpoint. This adds the endpoint to the group with AMT CIRA Profile, and  with the EMA configuration tool showing the endpoint as provisioned in Admin mode by another instance, and CIRA Not Connected.

At step 10 the CLI request runs without error, but at step 11 Command has not requested the credentials of the Intel EMA Tenant Administrator. Adding the parameter -noConfirm makes no difference to the outcome.

The script has not pulled the endpoint into EMA.

Running the script again after removing the endpoint from the group with "Stop managing endpoint" makes no difference to the outcome.

The exanple script

./Adopt-AMTSetupBySearch.ps1 -emaServerURL EMAServer.demo.com -searchMethod hostnameStart -searchString laptop -adoptedEndpointsFilePath C:\temp\adoptedEndpointList.txt -Verbose

does not create or write to a file C:\temp\adoptedEndpointList.txt.

[rgord1]

I think the root of the problem that I have with Intel EMA is that I have installed it on Windows 11. While it all appears to be functioning as expected, that may be the reason that the adopted scripts are not pulling the endpoints into EMA.

All I am looking for is a simple, user-friendly Windows-based application that can be used for OOB management of vPro devices on my home LAN. I don't need an enterprise solution, or Windows Server. It should be installable, configurable and operational under Windows 10 or Windows 11. The only application that currently fits those criteria is MeshCommander, now unsupported.

It is beyond comprehension to me that Intel is not providing such an application. The Intel Manageability Commander can still be used to manage endpoints with simple digest authentication, but is incapable of connecting to vPro devices that require TLS authentication.

When are we going to see a modern, fully-functional, up-to-date and supported, Windows-based OOB management tool from Intel?

[Mr_vPro]

Hi, just found that MeshCommander 0.9.6 can still be downloaded from https://meshcentral.com/meshcommander/MeshCommander-0.9.6.msi
not sure how long Ylian will keep it there.

[Mr_vPro]

Hi,

Intel Manageability Commander is able to manage Intel AMT systems also when configured with TLS encryption but it has to be able to verify AMT TLS cert trust chain vs Issuing CA CRL. Intel AMT/ME FW 16.1 and newer with Self signed  AMT TLS certificates and/or Intel EMA TLS relay mode configured TLS are not supported as IMC can't verify those certificate Issuing CA & its CRL by their nature.

Disabling AMT TLS certification  Issuing CA & CRL verification in IMC will cause security design issue (allowing untrusted certs to be used) and it is not going to be allowed by Intel Product Security team and SDL rules.

Let me take your feedback to Intel BU but I can't grant any changes in IMC to be done.

If you do not have Enterprise CA -which I think you do not have for home LAN - use of TLS with IMC is not possible due to reasons described above.

you may try to explore AMD SDK vPro modules for MS PowerShell https://www.intel.com/content/www/us/en/download/704388/intel-active-management-technology-intel-amt-software-development-kit-sdk.html if they allow to ignore TLS cert?

rgds

darek

[rgord1]

Understand my frustration that Intel Manageability Commander is unable to connect with 12th and 13th generation vPro NUcs that require TLS authentication, whereas MeshCommander is able to make those connections and trigger the issuing of a certificate.

You may say that support for 12th and 13th generation NUcs has been passed to Asus, but this is vPro, an Intel product, in NUCs that were manufactured by Intel, and Intel cannot provide a Windows-based OOB management application for these products?

Do you imagine that Asus will? I don't think so.

[Mr_vPro]

Hi,

you have mixed two things:

1. NUC support for BIOS update issue that unconfigured your AMT settings - it is now supported by ASUS not Intel.
   
   
2. Intel vPro/AMT management consoles which is broader topic not related to specific OEM or device.
   Intel vPro platforms with Intel AMT are built for business usage, for such use Intel offers Intel EMA SW.
   
   Intel Manageability Commander was provided as a companion app to  Intel SCS SW that was AMT configuration application while IMC was management console application. Intel SCS is EOL'ed since Dec 2022. 
   
   Current IMC version does not support ME FW 16.1 Self signed AMT TLS certs. 
   For home usage you may keep using AMT SDK tools and /or open source project MeshCommander 0.9.6 tool -but it is no longer supported by Intel.
   
   rgds
   darek