[alert variation=”alert-info”]We had previously identified what happened to those two boxes but that explanation is out of scope. This post is about how you can quickly recover form that situation.[/alert]
A few days ago we had a weird issue on a couple of servers after upgrading Acronis agent. Console was unable to start any backup plan, either automatically or manually, complaining that the agent was not registered anymore. That looked weird since we had updated our agents many times in the past but this time, somehow, agents unregistered or were unable to re-register.
Actually, this round of updates had caused issues on a couple of Linux servers too but those had been quickly fixed by rebooting those machines. We quickly noticed that servers that were failing both had Windows Server Core as their base OS and SQL Server was installed on them too and in the end we decided to access those machines and manually re-register those agents as detailed inside Acronis documentation. It was surprising to find that agent was unable to re-register, complaining that the endpoint certificate was invalid or, more specifically, the Certification Authority that signed the certificate was unknown.
Thing is that both those server were working fine before the update plus we seemed to be unable to re-register those agents even when using Acronis own endpoint. The same error was reported, which was so weird seems it could well be (though unlikely) that our own customized certificate had something wrong – even if it was working fine when accessed through browsers or on other machines – but Acronis own certificate could not have had any issue.
Updating root certificates
The very single fact that agent was reporting an unknown certificate authority was pretty indicative but we also had troubles connecting to the endpoint on our own, for example when using Powershell
Invoke-WebRequest -UseBasicParsing -Uri https://cloud.acronis.comDefinitely there was a problem connecting to SSL endpoints. While PowerShell was only complaining that the underlying connection was closed, Acronis agent reported a much more specific issue so we decided to investigate and check if we had any problems with our root certificates. We decided to update them altogether, just in case:
md C:\Temp
cd C:\Temp
# Download ALL root certificates from Windows Update
certutil.exe -generateSSTFromWU roots.sst
# Import ALL root certificates back inside root authorities
$sstContainer = (Get-ChildItem -Path C:\temp\roots.sst)
$sstContainer | Import-Certificate -CertStoreLocation Cert:\LocalMachine\RootRight after you do this, you need to reboot the affected machine to let it reload all root certificates.
Encryption issue
That didn’t fix the problem per se. Once the first machine was back, any attempt to connect to our endpoint or Acronis own one using Powershell was still failing. So we decided to investigate to check if machine was not using the proper cypher and we decided to force TLS1.2 in Powershell by using:
# Force PowerShell to use TLS1.2
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12Then you have to remember not to close that session or your will need to issue that command again. So now we attempted to connect to our endpoint or Acronis endpoint again and… bingo! It was working fine!
PS C:\temp> Invoke-WebRequest -UseBasicParsing -Uri https://backup.vaisulweb.cloud
StatusCode : 200
StatusDescription : OK
Content : <!DOCTYPE html><html lang=en><head><meta charset=utf-8><meta http-equiv=X-UA-Compatible
content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1"><meta
name=referrer content=o...
RawContent : HTTP/1.1 200 OK
Connection: keep-alive
Vary: Accept-Encoding
X-Frame-Options: DENY,SAMEORIGIN
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: ETag,Content-Length,Content-Typ...
Forms :
Headers : {[Connection, keep-alive], [Vary, Accept-Encoding], [X-Frame-Options, DENY,SAMEORIGIN],
[Access-Control-Allow-Credentials, true]...}
Images : {}
InputFields : {}
Links : {}
ParsedHtml :
RawContentLength : 570So we had two separate issues. We attempted to re-register agent again:
register_agent.exe -o register -t cloud -a https://cloud.acronis.com -u <account> -p <password>And now it worked fine! Geez! Backup plans were not working fine and executed without any issue. Hope it helps anyone!
