The field of backend engineering has witnessed remarkable growth and transformation over the years. As technology advances rapidly, the architecture and methodologies employed have also evolved significantly. We will explore the evolution of architecture in backend engineering, comparing the old ways with the new ones and highlighting the benefits and challenges of each approach.

Stage 1: Monolithic Architecture

The monolithic architecture was the prevailing approach in the early days of software development. Here the user interface, business logic and database were all bundled together in the same process. There was no client-server. In fact all code could even reside in a single file! This model built the entire application as a single unit. The code base encompassed all the functionalities and components required to run the application.

While monolithic architectures were relatively simple to develop and deploy, they suffered from several drawbacks. One significant challenge was that as the application grew, maintaining and scaling the monolithic system became increasingly difficult. Any new feature or an error in one component could affect the entire application, making maintenance and debugging time-consuming and error prone. Further, for scaling the strategy was vertical scaling, requiring more and more RAM, CPU and storage for the single monolithic process - which often would hit a practical limit.

Stage 2: Three Tier Architecture

The next evolution of architecture incorporated three main components of a software viz. presentation, application and data tier. Each tier run as a different process. The data tier stored information, the application tier handled logic, and the presentation tier was the graphical user interface (GUI). The three tiers were logical, not physical, and may or may not run on the same server. This separation allowed greater maintainability, scalability and re-usability of the components.

It was a linear architecture where the presentation tier spoke to the application tier which in turn spoke to data tier. Unlike the more modern counterpart - the MVC architecture which is triangular in nature, where all the three tiers can speak directly to each other, the Three Tier Architecture always followed a linear approach.

Presentation <--> Controller <--> Data

Stage 3: N-tier Architecture

N-tier architecture had multiple (2, 3, 4, 5 or more) tiers (processes) spanning across presentation, business logic, and data storage. The three components run on different logically and physically separate tiers. Being physically separated also means that each of these functions executes on different physical machines. A 3-tier architecture is also an N-tier architecture.

It still followed the linear narrative like the 3-tier architecture

The benefits of N-tier were that it enabled modular development, easier maintenance, independent scalability, replace-ability and re-usability of each tier. However, it introduced complexities in communication and data synchronisation between tiers.

Stage 4: Service Oriented Architecture

Service-Oriented Architecture (SOA) was a further evolution after N-tier architecture. Here a service is supposed to serve a business function as a standalone and a complete unit in itself. It provides a dedicated, cohesive business functionality as a one stop shop for a particular set of behaviours. A service is a single isolated unit which could be reused in making multiple enterprise solutions.

Communication: The SOA architecture uses a centralized enterprise service bus (ESB) to connect diverse services with multiple messaging protocols. Some of these protocols include SOAP, Advanced Messaging Queuing Protocol (AMQP), and Microsoft Message Queuing (MSMQ). If the ESB fails, all SOA services will be impacted.

Data Storage: Services may or may not share the same datastores.

The four main tenets of SOA

• Explicit boundaries

• Autonomy

• Shared API schema and contract (not code)

• Policy based compatibility

SOA offers reusability, scalability, and flexibility, but managing service discovery and dependencies can be challenging.

Stage 5: Microservices Architecture

The rise of microservices architecture marked a significant shift in backend engineering. Instead of building bulky services with shared database access and complex communication mechanisms under SOA, developers started breaking down the system into smaller, independent, decoupled components with dedicated datastores, enhancing autonomy, modularity, scalability, flexibility, reusability while supporting multiple communication protocols. Each microservice focuses on a specific functionality and can be developed, tested, and deployed independently. Multiple microservices together form a business level functionality.

Microservices architecture brought numerous advantages to backend engineering. Firstly, it enabled horizontal scalability. As the application grew, additional instances of specific microservices could be deployed to handle the increased load, ensuring efficient resource utilisation. For example, suppose an e-commerce application experiences a surge in traffic during a sale. Secondly, the modular nature of microservices approach allows smaller, micro teams to work in parallel, with each team responsible for a specific microservice. This improved development speed and facilitated easier debugging and maintenance, as issues were isolated to specific services. Thirdly with decent solution design in place, an issue in one part of the system doesn’t bring down the entire system

However, adopting a microservices architecture introduced new challenges. Managing distributed workflows and transactions spanning multiple microservices required special design patterns to handle them. Additionally, the complexity of deployment and observability increased with the distributed nature of microservices. For example, for debugging one needs to setup correlated traces and logs along to trace API requests as they flow across multiple microservices. For deployment of new releases, one needs to follow complex approaches like blue green or canary deployments.

Recent Trends: Server less Architecture

As technology continued to evolve, new architectural paradigms emerged in backend engineering. Two notable trends include server less architecture and event-driven systems.

Server less architecture abstracts away the need for deployment and infrastructure management for microservices. Developers can focus solely on writing microservices as sets of functions that respond to sync or async events without worrying about deployment or scaling. Cloud providers handle the infrastructure, automatically scaling the functions based on demand. The server less architecture enables developers to build highly scalable and cost-effective applications when usage traffic is not very high and they can enjoy pay-per-use pricing models. For instance, a server less function that handles image resizing can scale seamlessly as more images are uploaded to the application, ensuring efficient resource utilization without manual intervention.

Recent Trends: Async Event Driven Systems

Talking of event-driven systems, there are both sync and async kind of events. In synchronous events based communication the client knows the location of the API service it wants to call. This is a kind of coupling and inhibits scalability of functionality because it makes it difficult to add a new functionality in the system upon a particular event like change data capture. For ex. sending an email when a new user joins, or syncing user's data to the secondary datastore. Another problem of this approach is that it lacks resiliency. For ex. if a service goes down, all the services calling this service's APIs will also go down because their requests will time out.

For building highly interconnected loosely coupled systems, many applications rely on asynchronous event mechanisms where the producer of event does not know the consumer and vice versa. A microservice simply decides what events to produce and what to consume. It does not know of any other service's existence and location. This allows for near real-time processing of events, without depending on other services. Further, this loose coupling allows to add new workflows or features without changing rest of the system, by simply starting to attach consumers to pre-existing events. For resiliency and fault tolerance: In case consumers go down, events can also be persisted and replayed at ease by the message bus when the consumer is up.

Conclusion

Microservices and server less architectures facilitate parallel development, enabling faster delivery of features and easier maintenance. Event-driven systems promote near real-time processing and responsiveness. For example, a team can develop the user authentication microservice while another team focuses on the payment processing microservice.

Talking of fault tolerance, a single bug can impact the entire application in monolithic architectures. With microservices, server less, and event-driven systems, issues are isolated to specific services or functions, making debugging and maintenance easier. For example, using replication, where multiple instances of a service are deployed across different servers or data centers, ensures that if one instance fails, another can take over without impacting the overall system’s availability.

It is important to consider the context and scale when choosing between microservices and server less architectures. Microservices are generally preferred at high scales, where the complexity and size of the system require a more granular and distributed approach. Microservices allow for independent scaling, easier maintenance, and better fault isolation. On the other hand, server less architectures are beneficial for smaller-scale applications or when rapid development and deployment are essential. Server less provides automatic scaling and eliminates the need for infrastructure management, making it ideal for event-driven or sporadically used functionalities. It’s worth noting that in modern application backends, a combination of microservices, server less components, and traditional monolithic approaches is often employed to create robust and flexible solutions that best fit the specific requirements of the application.

The evolution of architecture in backend engineering has paved the way for more scalable, maintainable, and efficient systems. Each approach has its own benefits and challenges, from the old monolithic architecture to the new microservices paradigm and recent trends like serverless and event-driven systems. As technology continues to advance, it’s essential for backend engineers to stay abreast of these architectural transformations, adopting the best approach that aligns with their application requirements and goals.

If you haven't checkout the world's first meta-framework for microservices and serverless development, then check out Godspeed and see how Godspeed's meta-framework allows for higher level abstractions for developing modern microservices or service oriented applications. Give it a try for free!