Machine learning model serving architectures
Xebia
APRIL 2, 2024
In this blog we will discuss the most common serving architectures 1 ; batch predicting, on-demand synchronous serving and streaming serving. Those are more advanced serving architecture warranting a blog post of their own. It is the time it takes for the “Reads predictions” (see figure 1) interaction to complete.
Let's personalize your content