Securing AEM Customer Code: Automated fixes for vulnerabilities in dependencies

AEM customer projects depend on many third-party libraries, and any of them can have security vulnerabilities. New ones are found all the time, and no team can keep track of them all by hand.

We will showcase what Adobe has built on automation to keep your customer code secure, across thousands of AEM environments, with an AI agents putting the fix straight into the customer's repository.
And also, we will show what customers can do themselves to stay ahead: an automated pipeline using Renovate that keeps dependencies up to date before they become a problem.

Konrad Windszus

How to mark a detected vulnerability in Security Health (https://experienceleague.adobe.com/en/docs/experience-manager-cloud-service/content/security/security-health) as false positive?

Ramon

Thanks - Security Health doesn't offer this yet today. There is some "ignore" feature available in ASO For Security Health, the UI shows a contact address for the related team (some issues are e.g. automatically marked as "ignored" for all customers on the backend side - with Adobe configuration) But yes, having an easier inline way might be interesting - will raise a feature request for it

Konrad Windszus

Do you expose detected vulnerabilities also via API?

Ramon

It's exposed via some API used by the frontend (this technically could be "reused" but may change) - but so far no publicly-defined API I will add it to our internal discussion / raise a feature request for it

Lisa Brown

How often is the security app updated against emerging cyber trends and threats and what data bases does it update itself against and/or what tools does it use to assess an update as relevant?

Ramon

For 3rd party vulnerabilities, the data is coming from public databases (NVD, github security advisories, ...). The Adobe scan does update information from the databases every few hours - the modules used by the customer is scanned once per day

Andres Bott

I understand the question related to the Vulnerability database, this is currently updated ~daily, we use a modified grype db and are looking to integrate other dbs as well to keep accuracy high

wolf

Will the tool decloak the uber-jar dependencies for the purpose of the scan? Dependencies in customer code on vulnerable versions of the bundles that AEM provides are hidden from view, and that is not a healthy state, as they may be exercising code paths that Adobe did not test.

Michael

In short: our code is always secure. There is no need to scan it. Famous last words :)

Ramon

yes, the scan is done based on libraries referenced from the customer project only. (if a library from the uber-jar is shaded / included in a different version in your project, it will be reported)

Shashi

Do we have control over this checklist, or is it possible to bypass this check?

Ramon

The security health information today is exposed as additional information only - there is no enforcement of changes in pipeline/other places. i.e. customers can decide to consider the findings or ignore them.

Konrad Windszus

Lots of vulnerabilities are usually detected in ACS AEM Commons. Does Adobe plan to contribute fixes for that library?

Andres Bott

as mentioned on stage, contributions are on our radar, and having scans at scale allows us also prioritize by usage

RolandGruber

What is the benefit compared to existing solutions like Dependabot or Renovate?

Thomas

Better customer visibility?

Michael

Scanning in production is already too late. Existing tools work on the PR level preventing vulnerabilities to begin with, which is where it should take place. The customer visibility argument does not stand.

Ramon

Few points on this one: * Using renovate is great (as seen in the 2nd part) => if every customer would use it, we may not even need this scan. (from experience, customers are not doing regular dependency updates) * The scan operates on a AEM/OSGi level (e.g. having a translation connector => includes a content package => includes a JAR file) - this is recognized by the Adobe scan So best indeed is a combination of both worlds

Andres Bott

another difference is AEM knowledge, we had situations where flagging a library was not helpful a) it was only pulled into the project transitively, so code scanners would not detect it (specially if it comes within a content package. b) once identified we also can capture where it was pulled in, make it easy to fix concrete example we had a shaded library withing the AWS SDK. without the tree information it was not possible to locate it.

Andres Bott

I think we should consider it in addition, we had instances where specific items where not picked up by code scanning tools

Lisa Brown

Could it prioritize security fixes by the severity of not fixing them? That might also help prioritize human review of code or the necessity of doing an off-cycle production deployment too.

Andres Bott

currently if more than one component is detected it is sorted by severity of the individual CVEs we keep looking into ways to avoid operational fatigue and only expose relevant information, might be a bit noisy for now

Konrad Windszus

How does the quote from John F. Kennedy about fixing the roof relate to the Security Health limitation: "Scans run only against production environments. Development, stage, RDE, and sandbox environments aren’t scanned." (https://experienceleague.adobe.com/en/docs/experience-manager-cloud-service/content/security/security-health#limitations) Are there plans to lift that limitation soon?

Michael

Scanning AEM instance with a vulnerable bundle is too late. You want to prevent them to end up in production to begin with (i.e. "shift left"). Existing tools like Sonar already integrate on a PR, which is where it all should take place.

Ramon

yes - scans on other environment types are planned to be added shortly. As indeed DEV is what is worked in - customers may want validate fixes etc. there / are adding new 3rd party libraries and should see the impact before reaching prod

Tad Reeves

What strategies have you used to successfully determine the blast radius of potential RenovateBot changes? A "successful build" doesn't necessarily tell you under which conditions the dependency update could kill the site. Assuming folks have come up with some AI to support such?

(see answer in talk video)