The Log4j Vulnerability (CVE-2021-44228) that was announced on the 10th December is currently being monitored by our teams. You can find any status updates below.
Summary
- A new critical remote code execution vulnerability in Apache Log4j2, a Java-based logging tool, is being tracked as CVE-2021-44228
- Exploit proof-of-concept code is widely available, and internet-wide scanning suggests active exploitations
- Major services and applications globally are impacted by the vulnerability due to the prevalence of Log4j2s use in many web apps
Status Update 18/01/22 17.00
Further to our updates on the Log4J vulnerability, which Circle has been tracking since December 2021, we are pleased to confirm that we have now completed the external and server scanning to check for the vulnerability and any breaches and applied mitigation where necessary. Where issues have been found, we have communicated with impacted customers and taken the appropriate action as required.
We continue to strongly recommend that you contact your software vendors to verify whether there are any available patches for impacted software applications not covered under your support contract with Circle. If you require our assistance to do this, please let your SDM or Account Manager know.
We continue to monitor security channels and our vendor platforms for advice relating to this vulnerability and will be in touch again if we need to take any further action relating to your systems.
Status Update 22/12/21 13.15
Our engineers continue to run further scans and review the outputs. To date, no further issues of concern have been detected. We will continue to scan and assess the results and feed back to any customers affected.
Status Update 21/12/21 13:30
Scans are currently ongoing and the remediation for vCenter has been done.
Status Update 20/12/21 10:45
We are now working through scanning all managed customer’s public-facing IP ranges. This will further help identify any immediate and vulnerable Log4J instances that need to be immediately dealt with as a matter of high priority.
VMware have also released additional mitigation steps for vCenter. We are planning to roll this out today to customers who have vCenter. We will be in touch with all affected customers again to let them know our plan.
Status Update 17/12/21 11:00
We have run the Datto component scans of all Windows servers within RMM for our customer-base. Where the scans have completed successfully, we have also mitigated any vulnerabilities found.
Where the scans have not completed successfully, we are reviewing the logs to determine next steps and will re-run once we have determined the cause of the failure.
Your Account Manager/SDM will be in touch with you if there are any issues which we need to discuss with you relating to the Windows server scans or any risks about which you need to be informed. If you do not hear from us, you should assume that your Windows server scans and any associated mitigations have been completed successfully.
We still strongly encourage you to engage with any third-party vendors or partners whose services you use to understand whether any patching may be required. Whilst our component will have mitigated the vulnerability, the application or software vendor may still need to apply a patch. If you require any assistance with this or would like to discuss this with us, please get in touch.
A further update will be posted here by midday Monday 20th December.
Status Update 16/12/21 14.40
We are continuing to run scans of Windows servers across our supported customer base using the Datto component. These scans are also mitigating any instances of vulnerability that are found.
We have a small number of instances where the scan has not run, and we are reviewing and actioning these to ensure that the scans are able to complete successfully.
We are receiving reports of the detected and mitigated vulnerabilities, which we are able to review. We are also receiving reports of any exploitations. In the event of detecting an exploitation, we will be in touch with you directly to discuss and agree next steps.
We strongly encourage you to engage with any third-party vendors or partners whose services you use to understand whether any patching may be required. Whilst our component will have mitigated the vulnerability, the application or software vendor may still need to apply a patch. If you require any assistance with this or would like to discuss this with us, please get in touch.
Whilst the Datto component addresses the Windows server estate, we have the following updates relating to network equipment for our customer base:
Dell:
Switches, routers and WiFi devices not vulnerable
SonicWALL
All Firewall devices not vulnerable
Cisco
https://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-apache-log4j-qRuKNEbd
Switches, routers and WiFi devices not vulnerable
Aruba
https://www.arubanetworks.com/assets/alert/ARUBA-PSA-2021-019.txt
Switches, routers and WiFi devices not vulnerable
Netgear
Yet to release full statement, but initial reports are that no products are vulnerable
Fortinet
https://www.fortiguard.com/psirt/FG-IR-21-245
Switches, routers and WiFi devices not vulnerable
A further update will be posted here by midday tomorrow.
Status Update 16/12/21 10:44
We have run further scans and mitigations using the Datto component and have seen positive results. Our plan now is therefore to:
- Install C++ on customer servers where this does not currently exist as this is required for the scans to run. There is no risk to service or stability from this being installed;
- Run scans and associated mitigations for all vulnerabilities detected. There is no risk to service or stability during the scan and the mitigations activities are very low-risk as they relate only to internal logging and not to the functionality of the services that are running;
- Review the mitigations and notify our customer base of any actions that are required for services or systems which are not supported by Circle;
- In the event of the scan failing to complete or any other issues, we will review, take action and contact you, if necessary, to update you on any further steps which may be required.
If you have any concerns or questions or do not wish us to install C++, run the scans or carry out the mitigations, please let us know so that we can take the appropriate action.
In parallel, the VMWare scripts which mitigate the vulnerability in vCenter have now been run for the vast majority of affected customers.
We continue to work with our Security Partners and vendors to ensure we stay informed of the latest developments and any new mitigations/patches and will inform you if these impact your services in any way.
If you have any further queries, please get in touch.
Status Update 15/12/21 14:00
We are currently undertaking more scanning in order to get further results to enable conversations with customers. We will be in touch with individual customers soon. In the meantime, you can find some frequently asked questions, here.
Status Update 14/12/21 15.30
We have tested the Datto component on our own userbase and are beginning to test on a limited customer set. So far, we can determine that the component appears to run successfully with no performance impact visible.
We plan to run this component in a staggered manner across our customer base starting from 10:00 tomorrow. Please contact us before then if you do not wish the component to be run on your service. The component will allow us to identify potential vulnerabilities, but no remediation will be made automatically and we will be in touch with you directly if we need to take any action as a result of the scan. The scan will be Windows Server OS based initially, highlighting any services/applications which are susceptible to this vulnerability. This will allow us to address the highest risk services, then we can review and aim to deploy to Windows endpoint OS (Windows 11/10 etc), to remediate these too.
In addition, VMWare has released scripts to automate their mitigation steps for vCenter. We have tested the vCenter mitigation which was released overnight, and we are planning to roll this out to customers who have vCenter. We will be in touch with all affected customers to let them know our plan and how they can opt-out if they do not wish for us to do this.
A further update will be posted here by midday tomorrow.
Status Update 14/10/21 10.30
Overnight, Datto has released a component which is able to identify and, where appropriate, mitigate instances of the vulnerability. Details of this are available on the Datto blog:
We are in the process of testing this script across our own userbase and will also test this on a limited customer set. Our aim is to be in a position to run this component across all customers in a phased manner from later today. We are currently determining whether there is any performance impact during the running of the component and will communicate with you prior to running this on your estate.
The component will allow us to identify potential vulnerabilities and give us options to investigate mitigation with you.
Further vendor statements and workarounds/mitigations are becoming available at the same time and we are continuing to assess and prioritise those. Our focus remains on securing public-facing services to mitigate the greatest areas of risk.
Further updates to follow by 1400.
Status Update 13/12/21 17:59
Our technical and security teams have reviewed further advice from our vendors and security partners. We are aware that there are likely to be other services used by our customers but outside Circle’s scope of support, which may also be impacted by this vulnerability. An example of this might be a piece of software or services that customers or employees access outside a VPN service. If you believe you may have such services or software running, please seek guidance and advice from your software or support providers and consider removing access if there are no mitigations/patches available.
Circle’s technical teams have been working to test the remediation on a number of key supported systems throughout the day. Further specific communication will be issued via your Account / Service Delivery Managers in due course.
Further updates to follow before 1030 tomorrow.
Status update 13/12/21 15:00
Our technical and security teams have compiled the list of the common products and vendors deployed across our customer base and are working through a review of the current vendor statements and known mitigations where they currently exist. This is a continually evolving situation meaning that more information is available every hour.
As this is a large-scale situation affecting many vendors and applications, we are liaising with vendors, monitoring the situation, and are now looking to remediate (where possible) the items identified for those vendors who have stated there are fixes/mitigations in place.
Patching forms a large part of the remediation with vendors working around the clock to provide patches to mitigate/resolve the vulnerabilities reported. Because of this, all vulnerabilities will not be addressed overnight. This will take time to address with so many vendors and applications affected.
Our Account/Service Delivery Managers will be in touch (as appropriate) to update you on steps we are taking to protect your services with individual plans for remediation.
Circle's own management toolsets, e.g., Datto and LogicMonitor, are not impacted by this vulnerability. Other common management toolsets, such as VMWare vCentre are affected.
A full list of current known applications and vendors can be found here - log4shell/software at main · NCSC-NL/log4shell · GitHub)
Further updates to follow before 18:00.
