Your CX Quarterback: The Customer Data Platform (CDP)
Oct 28, 2021 • 7 Minute Read • Elizabeth Spranzani, Chief Technology Officer
See All Our Insights
Hitchhiker's Guide to Sitecore Architecture in 2022
Nov 29, 2021 • 17 Minute Read • Elizabeth Spranzani, Chief Technology Officer
A turning tide is ushering in major changes in the digital and Sitecore space. This could be compared to the mid-2000s when Classic ASP gave way to .NET and 2010s when MVC took over from Web Forms. Sitecore has ramped up their discussions, documentation, and marketing material around important architecture and development topics, including XM Cloud, SaaS, Headless, Experience Edge, JSS, Jamstack, Static Site Generation, Next.js, Vercel, and Deployment as a Service, among others.
As a platinum partner of Sitecore with 8 MVPs on staff, Verndale is fortunate to have participated in behind-the-scenes conversations, access to key decision-makers at Sitecore, and experts on staff who care deeply about where this is going. We're able to cut through the noise and read between the lines. We'd like to simplify the conversation in this post as much as possible, summarize these really important architectural features so you can understand what they are, and break down what we see as the next-best and long-term actions to be considering as you move forward with your favorite DXP platform.
Before we dive in, here are some key definitions to get you through the rest of the conversation:
Headless
In software development, a headless application has an architecture that completely separates the backend administration of content and data from the client-side rendering of the information. In most cases, this is done by exposing the information via APIs that a variety of front-end applications and frameworks can consume. They exist as two separate applications, potentially hosted in different locations.
SaaS
Software as a Service (SaaS) is a software licensing and delivery model in which software is licensed on a subscription basis and is centrally hosted in the Cloud. Instead of installing and maintaining software, you simply access it via the Internet, freeing yourself from complex software and hardware management. Upgrades are expected to be part of the service.
Jamstack
JavaScript + API + Markup. The entire front-end of a website is prebuilt into highly optimized static pages and assets and deployed to a CDN. This process of pre-rendering pages results in sites' increasing scalability and page render speeds. Jamstack reduces the cost, complexity, and risk of dynamic servers as critical infrastructure. However, it makes personalization, experimentation, and analytics tracking more complex.
MACH
This is an acronym that stands for Microservices, API-first, Cloud Native SaaS, Headless.
- Microservices are small applications that do one thing really well.
- APIs are the interfaces that talk to each other, exchange data, and surface functionality.
- Cloud, in this case, is specifically a SaaS (see above) platform.
- Headless: see above.
Sitecore has a very real end goal of being MACH-compliant. Why? You can read more about the benefits of MACH here DX ARCHITECTURE AT MACH SPEED, but to sum it up: MACH is agile and nimble, always up to date. Each component in the composable architecture is focused on what it's great at, and it's pluggable, scalable, replaceable, and continuously improved and evolved. The pieces can be switched out to always ensure best-of-breed solutions are in place. The dream is to be future-proof, where there are no more complex platform instantiations and upgrades.
There is a lot to do to get to MACH. Sitecore's acquisition activity has jump-started this goal, since all are headless SaaS systems and now are loosely coupled into Sitecore's composable DXP. The largest remaining hurdle is their core product for CMS: XM and XP. These are not SaaS and not traditionally headless (although APIs exist to enable headless). Sitecore's current managed hosting offering, Managed Cloud, is PaaS, not SaaS, but this is their first area of focus to try to get there.
Some of these concepts have been sitting on the fringes of Sitecore development practices for a few years now, but had yet to pick up steam outside of a minority of implementations and proof of concepts. This is partly due to a lack of knowledge, but also a lack of all the pieces being in place to fully support it. Being too far out ahead of any of these trends was risky, potentially resulting in an implementation that didn't have a future roadmap. The practices also introduced difficulty with finding partners and developers who could maintain it.
Many of our clients are validly asking what is the best approach for their needs. Should they be upgrading now? Should they switch to JSS? Where should they be hosting? Do they need to ditch their in-house .NET developers and find JavaScript gurus? The once clear Sitecore 5-year plan is now murky. Let's first discuss SaaS and Composable DXP.
Sitecore's Composable DXP Architecture
The popularity of MACH architecture, SaaS, and composable Digital Experience Platform (DXP) ecosystems has taken the digital world by storm. The origins of most premiere DXP players are not rooted in SaaS architecture, but they're trying to get there organically and through acquisition. Sitecore is leading the charge and has proven their committed to it through a series of 2021 acquisitions.
How Sitecore Could Get to SaaS
Sitecore has two products to choose from when purchasing the core CMS product: XM and XP. XM is web content management, a classic CMS. XP is XM plus a lightweight CDP, personalization/experimentation, analytics, and marketing automation features.
First, they need to get customers off of the XP product. It would be far too difficult to make XP SaaS, and it isn't necessary or recommended anymore with the more robust CDP/Experimentation SaaS platform, formerly known as Boxever, in the Sitecore suite.
Second, they needs to get XM customers into the cloud. SaaS requires vendor management of the infrastructure AND the software. That translates to Sitecore getting their customers into their Managed Cloud offering and then figuring out how they will maintain the software and roll out updates and upgrades 'automatically'. The problem is that the Sitecore CMS platform is really open for customization and most implementations have some kind of customization that would go against a SaaS-controlled offering. Those updates and upgrades can't be allowed to 'break' a production deployment. There needs to be an in-between step.
Sitecore's advice is get upgraded to 10.2, released November 4, 2021, and get into their Cloud hosting. Any potential features that will better enable SaaS will require this foundation.
As a Sitecore customer, let's assume you have made it to this point: XP out, upgraded to 10.2, and living happily on XM Cloud. To get closer to SaaS, you also have to remove the front-end application from the same solution (and become headless), at least in the production environment. XM needs to get out of delivery. The current traditional Sitecore install has the client website application baked into the same code base as the Sitecore administration tools. For SaaS, you want to relegate that deployment of the application to the sole purpose of content author page orchestration within the back-office tool: it can't be directly served to end users. When the front-end is being delivered completely "headlessly" by SaaS products, the SaaS claim is closer, which is where the next important thought comes in.
The Headless Conundrum:
A true headless front-end is not directly connected to a content management system (CMS). It can be written in any front-end framework that is desired, content is pulled from a RESTful service API, and the developer composes it into each page of the application. This is a problem for Sitecore and disregards one of the best selling points of a premium CMS platform: in-page editing, page orchestration and visual layout management. A lack of support for this is a downfall for most headless-first CMS platforms in the market today. Sitecore arguably has the BEST page orchestration of any CMS out there, but you're not able to use Sitecore's Experience Editor in a headless fashion because it requires a Sitecore instance. It's very much a Sitecore-bound application. You need to output the HTML in a particular way so that Sitecore can add a layer on top of the front-end code, making it interactive for a content author. This allows for updates directly to the page and ability to move elements around on the page (among many other actions). You also don't want to have two different versions of your code (one to enable Experience Editor and one that is headless) because that makes no sense from a maintainability perspective and would double the cost of implementation. Now what?
Enter JSS
Sitecore's first push to solve this problem is JSS: Sitecore JavaScript SDK. Until now, JSS was the only way to accomplish headless (sort of) while supporting Experience Editor page orchestration. JSS isn't really true headless. The Sitecore SDK is in there, and it's still a fairly tight coupling with Sitecore. But it's far closer to headless, and the Sitecore administration application is now separate from the 'head' delivery application. Another goal of JSS is to entice more front-end developers into the Sitecore world without having to learn the back-end stack.
JSS front-end is written in such a way to support various 'modes' that can be easily configured for different environments to support the different needs of those environments.
For example, when set to the mode to support Experience Editor (Integrated Mode), the JS will be rendered Server-Side. The components on the page will become editable when surfaced in an authoring interface and wrapped in extra HTML and JS. Layout data can be updated and then saved to the server. Sitecore authors can't tell the difference between a JSS site and MVC site because Experience Editor is supported in this Integrated Mode.
The same exact JSS app and code will work differently when rendered in a different mode for an end user in a production environment (Eg: Headless SSR mode which also has full Sitecore marketing/personalization engine support, or API-Only mode). It will be hosted on a Node server and will pull content and component rendering data from Sitecore's Headless Service (layout rendering service, previously known as JavaScript Server Components). This service is hosted either on a CD server or in Experience Edge. Finally, the Node server would render the page and deliver to visitors.
This achieves the goal of a single codebase that simultaneously enables Experience Editor authoring while delivering to end-users headlessly.
There are additional modes for development purposes as well, but those listed here are the modes pertinent to this discussion.
Another interesting aspect of JSS is it allows for code-first development. This means once it's connected to a Sitecore environment with a different mode, it can theoretically auto-generate templates and other elements in Sitecore based on definitions in the code. That generally isn't considered the best architectural practice though. This feature caters to the JavaScript developers who might not want the whole rigmarole of setting up a local Sitecore administrative environment.
Is JSS What We Should be Doing Now?
Not so fast. JSS has been out for a few years now. Making a decision to move from .NET development to all JavaScript on the front-end is not something most Sitecore partners and clients were tooled to do. Since there are limited developers with JSS experience, there is an inherent risk with a JSS implementation. A developer who leaves an organization would take all of their JSS knowledge with them. Also to consider: what if Sitecore decides not to support or maintain JSS in the future? This has happened in the past.
Additionally, JSS is still tied tightly to the Sitecore SDK. It doesn't truly deliver on the agile/nimble promise of headless on its own.
There is also performance to consider. Theoretically going JavaScript with Single Page Applications or Progressive Web Applications might be faster. Unfortunately, the interaction between JSS and the Headless Services delivered by the Content Delivery (CD) servers was not actually proving out to be performant. An implementation with clean JavaScript/React in the right places, well-built with .NET MVC and taking advantage of Sitecore's OOB caching, can have pretty great performance.
Until now, the tradeoff of these potential downsides might not be worth it just to attain headless + Experience Editor support. The promise and aspirations of what JSS could achieve were directionally sound, but the reality wasn't quite there. This segues to the next topic.
Agile, Nimble and Always-Up-To-Date…But What About Fast?
To speed up the bottleneck between where the Sitecore content would be delivered from and the front-end, Sitecore released a new product, Experience Edge.
Experience Edge is a globally distributed delivery platform and a SaaS version of Sitecore headless and layout services API in a CDN, which were hosted traditionally on CD servers. It acts as a Sitecore publishing target and provides globally replicated and scalable access to Sitecore items, layout, and media accessible via a GraphQL API. Experience Edge services both Sitecore XM and Sitecore Content Hub.
The idea is that both JSS applications as well as any application that wants to serve up Sitecore content headlessly now can do so at scale (but Edge is not required for JSS if you choose not to use it).
You can call the Edge GraphQL endpoints from a headless .NET Core application as well.
Another benefit of XM Experience Edge is that it has opened the door to Jamstack and Static Site Generation with JSS and now MVC. For Jamstack architectures in particular, Experience Edge provides an API that can be used for initial static markup generation and runtime data needs, including client-side fetch (AJAX) calls and incremental static regeneration.
The Need for Speed
This brings me to our next point on performance. It's great that Edge has now enabled faster and scalable content delivery to the front-end application. But what if the application itself is slow? We all know the statistics around how easy it is to lose visitors on your application if it's perceived to be slow. And we must also consider the shift to visitor traffic from mobile devices where guaranteed fast data is not always a given. How do we make it even faster?
The current hot trend answer to that question is Jamstack and Static Site Generation.
The premise is that your web application can be run through a static site generation process that will turn your content and markup into static HTML pages that can be hosted in a CDN and globally distributed at scale. This enables blazing fast site performance.
Static Site Generation (SSG) means there is no dynamic content (think personalization) and it doesn't talk to any servers. This is as opposed to Server Side Rendering (SSR), which is what a traditional Sitecore MVC application does on every request as it goes back to either the CD server or Experience Edge for more content. With SSG, content is not refreshed unless you build the application again. You can configure Experience Edge to call back to your infrastructure via webhook and notify when a publish operation has been completed. This enables SSG rebuilds when content changes.
You can also leverage Client Side Rendering (CSR), which is a mix of SSR and SSG. The fact that the markup is static does NOT mean the app itself is static. A static page is returned to the browser, but elements are going to be rendered on the client and certain parts of the page will trigger client-side function (CSR). Client-side JavaScript can still make asynchronous calls to APIs to fetch dynamic data for user interactivity (rehydration). This enables personalization, A/B testing, and analytics tracking. As an example: if you integrate with the Sitecore CDP via the JavaScript snippet, it's enabling personalization even on static pages.
Additionally there is a way to code your application for Incremental Static Generation (ISG), which allows for periodic and on-demand content updates. ISG is not completely Jamstack since it needs to ping a CD server (or Edge) to check for changes. This can result in a little latency and you will need to decide where to use ISG instead of SSG.
We glossed over an important part of the SSG story, though. How can you statically generate your site?
Next.js Handles the Static Site Generation
Next.js is an open-source development framework built on top of Node.js. It enables React-based web applications inclusive of static website generation. It provides ability to use both server-side rendering, static generation, and incremental static regeneration. By default, the Next.js starter app uses SSG for all pages, but components can be configured to make use of SSR and fetch personalized content. It can call back to get the content from Experience Edge.
Next.js can be hosted on Vercel, which is a SaaS hosting platform that provides CDN, auto-scaling, DDOS protection, and more. Cool side note: When Vercel is connected to a git repo and a new branch is created and committed, it will spin up a whole new environment to view the change.
With Sitecore 10.2 (Headless Rendering 19.0), HTML for individual components can be generated from either JSS or .NET MVC. It's fed into Experience Edge and then 'assembled' by Next.js.
Sitecore's support of MVC with SSG is new and a really important point. If this new offering works well, JSS is not a requirement anymore for headless with Experience Editor. Choosing between MVC and JSS could simply be a choice based on developer availability and preference.
Circling Back to SaaS
Vercel and Experience Edge are SaaS products. Sitecore XM's content can now publish to Experience Edge, which will be detected by Vercel and a rebuild of any statically rendered pages will be triggered. Therefore, when a site visitor requests a page, it will 100% be served by a SaaS layer and XM is no longer responsible or playing a direct role in the delivery of the request.
And with that logic, and if Sitecore can find a way to release CMS updates more automatically to their customers in XM Cloud, Sitecore will have found a path to SaaS.
Next Steps in Tightening Up Jamstack
There are still some potential bottleneck points when rehydrating statics pages through ISG and CSR. This could be made better if personalization logic could be created literally on the edge in the CDN or client-side, removing the need to call back to the origin. Containers could publish logic closer to the user and improve latency. The ability to do this is likely at least a year away, but it's on Sitecore's radar.
Circling Back to What's Best for Our Clients
We will need to help our Sitecore clients navigate this architectural road and make some important decisions. For all the promises of MACH, the reality of it isn't going to be easy. But easy or not it could be the right choice, because that may be where the industry is trending and at some point you need to go with the flow or be left behind. At Verndale, we have formed some opinions on what we have discussed here.
Our current opinions:
- Composable DXP is here to stay and we will recommend choices to go with as many of the loosely coupled SaaS platforms as possible instead of the monolithic options.
- We definitely will be recommending that you move away from Sitecore XP, upgrade to 10.2, and leverage Sitecore CDP and Personalize for anything that would have been handled by XP in the past.
- We recommend going to XM Cloud, managed by Sitecore, assuming they can price it reasonably.
- We probably recommend using XM Edge, because it comes with XM Cloud and you might as well leverage a CDN where you can.
As for the rest of the decisions, the jury is still out from our point of view but we are seriously considering all of it. It will require input from our clients for their preferences and their tolerance for risk and cost for something that really hasn't been thoroughly tested yet, and also some prescience and well-placed bets.
So What Will We Consider When Helping Our Clients?
At this point, we have to analyze if your application should be headless or not, Jamstack or not, and if you need to support Experience Editor or not. If you opt for headless, it likely makes sense to also opt for Jamstack. If not headless, then Jamstack doesn't make sense. We will consider the following architectural options:
- Classic MVC integrated with Experience Editor support (what we've been doing for years)
- Pure headless without Experience Editor support, run whatever front-end framework you want, Jamstack or not
- JSS headless with Experience Editor support
- MVC headless with Experience Editor support
- JSS headless + Jamstack with Experience Editor support
- MVC headless + Jamstack with Experience Editor support
Regarding the Decision Between JSS and MVC
If the MVC SSG does its job (we need to see how true this is) then that might be the easiest option due to the prevalence of .NET developers in the Sitecore world. But we need to look at the digital landscape outside of Sitecore and determine if supporting JavaScript developers and enabling our traditional developers to grow into more full stack (or specialized) engineers might make the most sense long term. For now, it could be a preference that we work through in coordination and collaboration with our clients.
Regarding the Decision for Headless
It's looking more and more like Sitecore is preparing for this to become the norm while still supporting Experience Editor. Headless is an important consideration in moving away from the monolith and freeing the front-end and back-end from each other so they can be updated and deployed separately with far less dependencies.
Regarding the Decision for Experience Editor Support
Sitecore is investing in a new user interface called Symphony (Front-End as a Service and page orchestration), which is the evolution of Experience Editor. It will allow marketers to manage content and digital assets from XM and Content Hub, and implement personalization, decisioning, and experimentation via Sitecore CDP and Personalize directly within a visual editor. Considering Sitecore is now a composable DXP where their other platform product offerings are available to other DXPs, then the reason for having Sitecore's XM CMS product could really come down to Symphony. Therefore, you will probably want to have support for the Experience Editor because that's why you bought the CMS product from Sitecore.
Regarding the Decision for Jamstack
Publishing and how quickly end users see changes to content needs to be a consideration, as well as how many steps exist and gates it must pass through to get there. It remains to be seen how fast this process will actually be, and if content does not appear, how complicated it will be to debug what happened. There will also need to be other development and coding considerations because everything with Jamstack will have to be handled with JavaScript requests instead of server-side call backs.
There are plenty of good coding and Sitecore practices to optimize performance and speed with the current infrastructure strategy of CM/CD servers. Your Sitecore application can be really fast if you are:
- Taking advantage of Sitecore caching
- Optimizing images for the web
- Serving images/videos and other media assets from a CDN
- Lazy-loading JavaScript and other files only when you need them
- Being careful with font usage, etc.
Blazing fast? Maybe not, but fast enough to keep your visitors on your site (for now). Therefore, you need to carefully consider why you might want to move to this Jamstack approach: do you need blazing fast and better scaling options? Is hosting and infrastructure too expensive right now for you and want to find some cost savings by moving away from CD servers to CDN delivery instead? If the answer to those questions is yes, then Jamstack is something to consider. If the answer is no, and a more direct publishing approach where there are less waypoints along the way for content to get derailed is more important, then Jamstack with Sitecore is probably not the best choice right now.
I know the choices may seem overwhelming, but the goal here is to make a choice that will be about 75% foundationally consistent and will support making it easy to change the other 25% if you decide to try a different way later. By no longer being in a monolithic system and perhaps headless, it means only changing out some aspects of the front-end application and switching between CD servers and CDNs. It's an exciting new frontier, and Verndale is here to help you on the ride.
Contact Us for more information on Sitecore architecture or for help developing your technology roadmap.