The milliservice architecture

The milliservice architecture is the modern way of developing scalable, redundant software solutions. It combines the scalability and availability of microservices with the ease of implementation, debugging and deployment of traditional monolithic applications.

Milliservices vs microservices

Milliservices are larger than microservices. Microservices as advocated in cloud solutions like AWS Lambda or Azure Functions only cover one aspect of a business object. Like the “add a product” to a shopping cart, or the “check out” or “choose shipment” aspect. A milliservice, on the other hand, covers an entire real-world business object (like the entire “shopping cart” and/or the “user account”). It would implement all of the action methods for adding products, checking out, and choosing shipment for a shopping cart, all in one application.

In microservice architecture, multiple microservice instances together are responsible for one business object (like a specific shopping cart for one user). Each runs in its own process, and together they have to manage concurrent access to shared resources. In milliservice architecture, one single milliservice instance has the sole responsibility for one specific business object. The runtime guarantees that there is no more than one milliservice active at any given moment for a certain business object.

A milliservice can invoke other milliservices when necessary (like the credit card validation milliservice that is used during checkout to verify the payment), but still, there is only one milliservice per business object.

Because all aspects of a business object (like a shopping cart) are handled by the same milliservice instance, it is trivial to manage concurrency. For example, to allow only one write-operation (add product, checkout) at a time, or to forbid add-product when checkout is already in progress.

A microservice instance typically only performs one task at a time. It can perform a checkout for user #1, but not at the same time for user #2. Or it can add a product for user #1, but not at the same time handle a query for a list of current items in the shopping basket. For that, multiple instances would need to be deployed, and/or requests would have to be queued.

For milliservices, on the other hand, there is no such 1:1 relationship. Many milliservice instances (even of different types, like shopping cart, user account and product milliservices) live together in one process. Depending on the implementation, it is even possible that one single milliservice instance performs multiple actions in parallel on the same specific business object (like adding a product and doing a checkout for the same shopping cart), although the use cases for parallel processing of actions on the same business object are not as numerous as it might seem at first sight. So, where every microservice instance requires its own process, hundreds or thousands of milliservice instances can live within the same process and be served in parallel.

Milliservices vs monoliths

Milliservices are quite similar to traditional monolithic computer programs that combine many functionalities in one program. A milliservice application can implement the logic of all individual milliservices that make up the applications logic (like a monolith), but a milliservice application can also be as small as only one specific type of milliservice.

Note: Even in the latter case, when a milliservice application only contains one milliservice, a milliservice application is still larger than a microservice application, because a microservice application only covers one aspect of a business object, whereas a milliservice application covers all aspects of that business object. In addition to that, a typical microservice application can only perform one action for one business object at a time; a milliservice application can perform many actions for many different business objects in parallel.

The major difference between milliservices and monoliths is how business objects are coupled. In traditional monolithic applications that use object-oriented programming, business objects are implemented as objects that have (pointer) references to each other. A shopping cart object has a reference to the credit card validation object (that lives within the same process), and it can use the object reference to invoke methods (like `verify`).

Milliservice applications are also object-oriented, but there the references to other business objects are proxies (stubs). A shopping cart does not have a direct reference to the object that performs credit card validation, but it has a reference to a proxy object that looks exactly like a credit card validator (that is, it also has a ‘verify’ method), but under water, it first locates the application in the cluster that provides the intended credit card validator, then remotely invokes it (over the network), waits for the result, and then passes the result back to the shopping cart object that invoked the ‘verify’ method. Proxies provide spatial decoupling between milliservices.

Decoupling and scalability

The use of proxies in the milliservice architecture effectively decouples business objects. For the developer of a business object, it looks as if the referenced objects are within the same process. Methods can directly be invoked (typically via the async-await pattern), and errors are nicely thrown, just like with regular structured programming. That makes programming very straight forward. Under water, it is the use of proxies that provides scalability to the system. When an application grows (in terms of the number of users or traffic), one or more copies of the milliservice application can be deployed, and the load is automatically balanced over all of the instances without the actors noticing that the actor they want to invoke used to be in their own process, but now suddenly runs somewhere else.

Persistence and availability

One major difference between milliservices and both monoliths and microservices is that a milliservice framework provides integrated persistence. With ‘integrated’ we do not intend that it would not be possible to use external services (like a SQL database or a cloud database) to store state. No, we mean that unlike the monolith and microservice architectures, the concept of persistence is a key and integrated feature the milliservice architecture. A milliservice runtime must provide milliservices with an easy to use interface to persistence, so that milliservices can easily load and store their internal state, without having to bother with the details like which underlying storage provider to use (SQL, cloud, file system), how to authenticate, et cetera.

Why is persistence so important? Once milliservices are able to persist their internal state, it becomes possible to move milliservices from one application to the other. This is called reincarnation. When one application goes down, when the memory of one application is nearly full, or for any other reason, the milliservice can come to live in another application by stopping it from handling incoming calls, having it persist its state, creating a new instance in another process, and having that new instance reload its state from persistence.

Because proxies can have an automatic retry mechanism built in when milliservices are reincarnating, this all can happen without the application code noticing that the milliservice they are trying to invoke just moved to another application.

Besides the scalability that we already have seen to be provided by proxies, proxies thus also provide availability: when an application has to go down, the milliservices that lived on it instantly pop up on elsewhere – when someone needs them. As long as no requests are made to a certain milliservice, it will just stay asleep in the persistence store. Only when it is required, it will revive.

Actor-oriented programming

A convenient way of implementing milliservices is by means of the actor-oriented programming paradigm. In actor-oriented programming, business objects are implemented as actors. Actors are simply objects, but instead of having direct reference to other actor objects, they either receive a reference to a portal, or to a proxy object that the dependency-injection logic already obtained from a portal.

A portal in actor-oriented programming is similar to what an object factory is in object-oriented programming. Actors can use a portal to obtain a proxy to another actor. The proxy can be used as if it were a regular object, but under water, the proxy performs all the discovery logic and networking involves to find (or even instantiate) the requested actor, invoke the requested action on it, and return the result.

Darlean

Darlean facilitates the creation and running of milliservices by means of actor-oriented programming. It provides libraries that developers can use to create actor-oriented milliservice applications, and a runtime that can be used to run milliservice applications.

The runtime provides features like a distributed message bus (that applications use to interact with each other and with the runtime); actor discovery (which actor runs on which applications) and persistence (to store actor state on disk, in a SQL database, in the cloud, et cetera).

As such, it provides application developers with a novel way of programming and deployment that puts developers back in their strength. Developers are good at structured programming, and that is exactly what Darlean offers. To develop and deploy scalable, high-availability applications as easy as it always was to develop and deploy traditional monolithic computer programs, thus solving the microservice premium.