Understanding ResNet Architecture

Very deep networks often result in gradients that vanishes as the gradient is back-propagated to earlier layers, repeated multiplication may make the gradient infinitely small. ResNet uses the concept of residual blocks that include shortcut skip connections to jump over some layers.

The authors of ResNet paper provide evidence showing that residual networks are easier to optimize, and can gain accuracy from considerably increased depth by reformulating the layers as learning residual functions with reference to the layer inputs.

Let’s understand it in more detail!!

#pytorch #cifar-10 #neural-networks #deep-learning #resnet #machine-learning

What is GEEK

Buddha Community

Serverless Vs Microservices Architecture - A Deep Dive

Companies need to be thinking long-term before even starting a software development project. These needs are solved at the level of architecture: business owners want to assure agility, scalability, and performance.

The top contenders for scalable solutions are serverless and microservices. Both architectures prioritize security but approach it in their own ways. Let’s take a look at how businesses can benefit from the adoption of serverless architecture vs microservices, examine their differences, advantages, and use cases.

#serverless #microservices #architecture #software-architecture #serverless-architecture #microservice-architecture #serverless-vs-microservices #hackernoon-top-story

Understanding ResNet Architecture

Very deep networks often result in gradients that vanishes as the gradient is back-propagated to earlier layers, repeated multiplication may make the gradient infinitely small. ResNet uses the concept of residual blocks that include shortcut skip connections to jump over some layers.

The authors of ResNet paper provide evidence showing that residual networks are easier to optimize, and can gain accuracy from considerably increased depth by reformulating the layers as learning residual functions with reference to the layer inputs.

Let’s understand it in more detail!!

#pytorch #cifar-10 #neural-networks #deep-learning #resnet #machine-learning

Road to Simplicity: Hexagonal Architecture [Part One]

Software writing taught me that: a well written software is a simple software.

So I started to think how to achieve simplicity in a methodological

way. This is the first story of a series about this methodology.

Naturally it’s a snapshot because it’s in constant evolution.

Simplicity

A definition of simplicity is:

The quality or condition of being easy to understand or do.

Oxford dictionary (https://www.lexico.com/en/definition/simplicity)

So, a simple software is a software that is easy to understand.

After all software are written by humans for humans. This implies

that they should be understandable. Simplicity guarantees that its

understandability isn’t an intellectual pain.

A software solves a problem. So to build the former you should understand the latter.

But to build a simple software you should understand - clearly - a problem.

First step: architecture

On the Martin Fowler blog there is a deep definition of architecture and its explanation:

“Architecture is about the important stuff. Whatever that is.”

On first blush, that sounds trite, but I find it carries a lot of richness.

It means that the heart of thinking architecturally about software is to decide what is important, (i.e. what is architectural), and then expend energy on keeping those architectural elements in good condition.

Ultimately the important stuffs are about the solved problem. In other words about the software domain.

So we need an architecture that allows us to express - clearly - the software domain.

I think that the hexagonal architecture (a.k.a. ports and adapter architecture) is an ideal candidate.

It’s based on layered architecture, so the outer layer depends on the inner layer. Each layer is represented as a hexagon.

Here a UML-like diagram to express the below concepts:

In this architecture the innermost hexagon is dedicated to the

software domain. Here we define domain objects and we express clearly:

• what the domain does as input port or use case (I prefer the latter because expressiveness).
• what the domain need, to fulfill use cases, as output port.

Conceptually on the sides of the domain layer there are use case and output port interfaces.

The communication between the outer layers and the domain layer happens through these interfaces.

The outer layer provides output port implementations and they use the use case interfaces.

The implementations and use case clients are are called adapter. Because they adapt our interface to a specific technology.

This relation is an instance of the dependency inversion principle. Simply put: high level concept, the domain, doesn’t rely on a specific

technology. Instead low level concept depends upon high level concept.

In other words our code is technology agnostic.

As you can see the concepts expressed in the outer layers are just details.

The real important stuff, the domain, is isolated and expressed clearly.

Code

A little project accompanies this series to show this methodology. It’s written in Java with the reactive paradigm from the beginning. For this reason the ReactiveX library is also used in the domain layer.

The software analyzes the capabilities (e.g. the java version, the

network speed and so on) of the machine and it exposes them through REST API.

It’s inspired by a real world software that I wrote because of work.

The first step is to define the innermost hexagon.

We can already identify:

• the main use case, expressed as GetCapabilitiesUseCase
• the object that describe the machine capabilities, expressed as Capabilities

The use case is an interface:

(if you never used ReactiveX: a Single means that the method will return asynchronously an object or an error)

public interface GetCapabilitiesUseCase {
  Single<Capabilities> getCapabilities();
}

The Capabilities objects are immutable (precisely they’re value objects). And there is an associated builder (I’m using lombok annotations to generate the code):

@RequiredArgsConstructor
@Value
@Builder
public class Capabilities {
  private final String javaVersion;
  private final Long networkSpeed;
}

#architecture #software-architecture #programming #java #hexagonal-architecture #reactive-programming #software-development #software-engineering

Rookout CTO Discusses Understandability, Architecture Styles

In a recent InfoQ podcast, Liran Haimovitch, CTO at Rookout, discussed the concept of “understandability” and how this relates to building modern software systems. Building on the concepts introduced in his recent InfoQ article, “Understandability: The Most Important Metric You’re Not Tracking”, he also discussed how complexity impacts a system’s understandability, and the benefits of live debugging tooling.

Haimovitch began by outlining the ideas behind understandability within software engineering, the foundations of which were borrowed from finance. At the core, understandability is the concept that a system should be presented so that an engineer can easily comprehend it. The more understandable a system is, the easier it will be for engineers to change it in a predictable and safe manner.

A system is understandable if it is complete, concise, clear, and organized.

When you speak of all the criteria, but especially completeness, then you must provide your engineers with visibility and information about all of those components. I mean, it doesn’t help if you have the source code, but you don’t know how the system is configured, or what records exist within the database.

In addition to providing access to the complete system, the information must be presented in a concise way: it is all too easy for engineers to experience cognitive overload when attempting to build a mental model of a system with hundreds of thousands of lines of code. Clarity and organization of such information further helps an engineer in understanding a system; adding appropriate metadata and providing effective methods to search and categorize information enables engineers to chunk ideas and attempt to understand components of the system in isolation.

There is a static and a dynamic component to understandability. Design time understandability is focused on architecture diagrams and code:

The static nature of the system [is] basically of how it was designed to be operated: what’s the source code, what’s the architecture diagram, what are the tables within the databases?

The runtime understandability is concentrated on data, data flows, and side effects. Haimovitch noted that the scale of the data within a system can impact understandability, and this can only be gathered dynamically:

If I have a [database] table, [does this] contain a hundred records, or a hundred million records? That’s going to be very different, from a system perspective. And some of that information can only be gathered at runtime.

#microservices #understandability #observability #development #architecture & design #devops #news

Event-Driven Architecture as a Strategy

Event-driven architecture, or EDA, is an integration pattern where applications are oriented around publishing events and responding to events. It provides five key benefits to modern application architecture: scalability, resilience, agility, data sharing, and cloud-enabling.

This article explores how EDA fits into enterprise integration, its three styles, how it enables business strategy, its benefits and trade-offs, and the next steps to start an EDA implementation.

Although there are many brokers you can use to publish event messages, the open-source software Apache Kafka has emerged as the market leader in this space. This article is focused on a Kafka-based EDA, but much of the principles here apply to any EDA implementation.

Spectrum of Integration

If asked to describe integration a year ago, I would have said there are two modes: application integration and data integration. Today I’d say that integration is on a spectrum, with data on one end, application on the other end, and event integration in the middle.

The spectrum of integration.

Application integration is REST, SOAP, ESB, etc. These are patterns for making functionality in one application run in response to a request from another app. It’s especially strong for B2B partnership and exposing value in one application to another. It’s less strong for many data use cases, like BI reporting and ML pipelines, since most application integrations wait passively to be invoked by a client, rather than actively pushing data where it needs to go.Data integration is patterns for getting data from point A to point B, including ETL, managed file transfer, etc. They’re strong for BI reporting, ML pipelines, and other data movement tasks, but weaker than application integration for many B2B partnerships and applications sharing functionality.

Event integration has one foot in data and the other in application integration, and it largely gets the benefits of both. When one application subscribes to another app’s events, it can trigger application code in response to those events, which feels a bit like an API from application integration. The events triggering this functionality also carry with them a significant amount of data, which feels a bit like data integration.

EDA strikes a balance between the two classic integration modes. Refactoring traditional application integrations into an event integration pattern opens more doors for analytics, machine learning, BI, and data synchronization between applications. It gets the best of application and data integration patterns. This is especially relevant for companies moving towards an operating model of leveraging data to drive new channels and partnerships. If your integration strategy does not unlock your data, then that strategy will fail. But if your integration strategy unlocks data at the expense of application architecture that’s scalable and agile, then again it will fail. Event integration strikes a balance between both those needs.

Strategy vs. Tactic

EDA often begins with isolated teams as a tactic for delivering projects. Ideally, such a project would have a deliberative approach to EDA and a common event message broker, usually cloud-native brokers on AWS, Azure, etc. Different teams select different brokers to meet their immediate needs. They do not consider integration beyond their project scope. Eventually, they may face the need for enterprise integration at a later date.

A major transition in EDA maturity happens when the investment in EDA shifts from a project tactic to enterprise strategy via a common event bus, usually Apache Kafka. Events can take a role in the organization’s business and technical innovation across the enterprise. Data becomes more rapidly shareable across the enterprise and also between you and your external strategic partners.

EDA Styles

Before discussing the benefits of EDA, let’s cover the three common styles of EDA: event notification, event-carried state transfer, and event sourcing.

Event Notification

This pattern publishes events with minimal information: the event type, timestamps, and a key-value like an account number or some other key of the entity that raised the event. This informs subscribers that an event occurred, but if subscribers need any information about how that event changed things (like which fields changed, etc.), it must invoke a data retrieval service from the system of record. This is the simplest form of EDA, but it provides the least benefit.

Event-Carried State Transfer

In this pattern, the events carry all information about the state change, typically a before and after image. Subscribing systems can then store their cache of data without the need to retrieve it from the system of record.

This builds resilience since the subscribing systems can function if the source becomes unavailable. It helps performance, as there’s no remote call required to access source information. For example, if an inventory system publishes the full state of all inventory changes, a sales service subscribing to it can know the current inventory without retrieving from the inventory system — it can simply use the cache it built from the inventory events, even during an inventory service outage.

It also helps performance because the subscriber’s data storage can be custom-tuned just for that subscriber’s unique performance needs. Using the previous example, perhaps the inventory service is best suited using a relational database, but the sales service could get better performance from a no-SQL database like MongoDB. Since the sales services no longer need to retrieve from the inventory service, it’s at liberty to use a different DBMS than the inventory service. Additionally, if the inventory service is having an outage, the sales service would be unaffected since it pulls inventory data from its local cache.

The cons are that lots of data is copied around and there is more complexity on the receivers since they have to sort out maintaining all the state they are receiving.

#integration #microservices #data #kafka #enterprise architecture #event driven architecture #application architecture