Solving Edge Delivery and AEM Performance in China: Comparing Approaches & Results
Delivering performant digital experiences in mainland China remains one of the most technically and operationally-complex challenges for global web platforms, with all too many site owners resigning themselves to poor performance in China.
This talk presents hard-earned lessons and architectural tradeoffs from efforts to improve the real-world performance of AEM Edge Delivery Services (EDS) and AEM Cloud Service sites specifically targeting audiences in mainland China. Our talk will detail the challenges we all face for any AEM site in China, and then detail the results & trade-offs we experienced in launching the first customer Document Authoring (i.e. Dark Alley)-powered Edge Delivery sites with userbases in China.
The aim of the presentation will be to enable AEM engineers to know what can be done to effectively and dramatically improve performance for the over one BILLION internet users who presently (probably) time out when trying to reach your site.
Jesse Pinkman
You say datadog, new relics all is blocked in China. Do you need to use chinese variants then?
Michal
The easiest solution is to use alternatives from Alicloud or Tencent.
Helge
Why is it that page load slower under load when in china?
Tad
We've noticed that at some times of day, specifically EU/US business hours, that the performance is significantly worse. Page performance is worse, deploying container images goes from a few minutes to several hours. Working theory is that the China firewall edge locations themselves are under load. But the main thing is that it's highly variable, but slightly better in the middle of the night.
Helge
How is image serving done when using eds with image modifier params?
Helge
Can we use dynamic media in china?
Michal
You need to pre-fetch all the media during the publications and push together with the HTML. All the content has to be inside of the mainland China. I mean If you want sub-seconds response times.
Ive
Did you have any services that are not completely working in China - like for example Google Maps - how did you solve that?
Michal
You need to replace the services with the China alternatives (Baidu, Alicloud, Tencent). This has to be replaced in conetent in CMS or during publications (in pipelines).
Ive
So basically you developed different set of blocks specially for China?
Tad
That's right. In the site we developed here, we've started to create blocks that have logic to switch Youtube embeds for the global site or an alternative (Youku or Vidyard China) if the `clientCountry` is CN.
Henry Kuijpers
So content still has to travel from an EU cluster to CN Edge? So content publishing can also have delays of more than 1 minute?
Michal
Exactly! But it's the DC to DC connection, so it's a bit more reliable. But the key is to use queuing system that will keep pushing until it's done.
Tomasz Sobczyk
Also this is ONLY publishing. And if it fails you can automatically retry it. There is no such option when consumer tries to load the page - after a minute or two the consumer is gone, and not coming back
Markus Haack - Adobe
With your approach can you also support content authors in China for EDS?
Michal
They would probably use corporate VPN to connect to Google services (for example), as Google are blocked in China.
Tomasz Sobczyk
I would consider potentially bringing your own cms inside China if you need large scale content operations there. Other option is to use authoring teams in Hong Kong etc where you have native speakers but sitting outside of great firewall
Robert
What about users interacting with BEs, e.g. like form submits. How to get this reliable?
Michal
Queuing system with retries, not a part of the demo, but supported. It's a subject of China regulatory restrictions (people data protection law)
Tomasz Sobczyk
You need to push these into China infra. Chinese government does not like Chinese consumers data leaving China soil (you have a lot of extra stuff to do when you want to move any data outside of China)
Jaybee
How do you manage transactional (egress) requests like add to cart or payment - to have the full experience?
Tomasz Sobczyk
You need to do deploy the store inside China
Michal
It's also subject to different regulatory laws. You should store the data in China, if it contains personal data.
GRANT
If this is an ecommerce site (Adobe Commerce/Magento) , how do you ensure that the communication back to the central database is not interupted? e.g presumably there is a central stock database, customer database, orders etc needed. or is this archtiecture just intended for brochure sites?
Michal
Queuing systems. You cannot process orders outside of China, because of the legal restrictions. The solutions works with more services, like full text search, facets.
Bertrand
I'm not sure you'll be willing to answer this question, but I'm curious about which mysterious protocol you are using to go through the Great Firewall unnoticed ;-)
Jörg
FTP :-)
Michal
We tried 5, but you are right!
Michal
I would suggest MQTT or Webhooks for that use case though.
Bertrand
Thanks! I thought you would need something more unexpected but if it's working, more power to you!
Michal
MQTT is actually designed for unreliable edge locations. Webhooks are simple, stateless and reliable. You just need to track proper way of ACK/NACK. Another thing is how to ensure the network and transport security, service discovery and fault tolerance including retries.
Robert
What is the way you use to trigger static page generation to warmup cache based on AEM page activations (including referenced pages, assets, possible custom resources from servlets and so on)?
Michal
We use Sling eventing to subscribe and consume various events for AEM. It's actually open sourced: https://github.com/streamx-dev/streamx-connector-aem