What is Ostara

Ostara is a modern tool for managing and monitoring actuator-enabled microservices that aims to make the process more user-friendly and straightforward.

One of our main goals with Ostara was to create a tool that just works out of the box, without needing anything besides a functioning Actuator API on the other end.

Ostara allows you to gain insights into the performance and health of your microservices by providing real-time monitoring of critical metrics such as CPU and memory usage.

One of the key benefits of Ostara is its simplicity. The tool is designed to be easy to use and requires minimal setup. All you need is a running instance of your microservice with Actuator enabled, and you can start monitoring and managing it with Ostara.

Why?

The first question to ask is why even use Ostara, when excellent tools such as Spring Boot Admin already exist?

As Spring Boot Admin users, we had to ask ourselves this question before embarking on the development journey of Ostara.

For all its greatness, one thing Spring Boot Admin that cannot be said about Spring Boot Admin is that it's plug and play or 'off-the-shelf'.

To setup Spring Boot Admin, we start by having to create Spring project for the server itself (In our case, which is true for many, also meant we have to setup a full CI/CD flow, complete with Helm charts and Docker images in order to create a full production flow)

If we needed additional features such as authentication, notifications and so on, we would need to add them directly to the project.

All of this, despite the infinite versatility it offers (because it is essentially a blank slate), raises the startup costs of Spring Boot Admin considerably and even more so when you remember that you have to continually maintain this project as with any other project within your code ecosystem, either due to security updates to unrelated dependencies you may have needed to add, or in order to update Spring to match your main project(s).

But aren't there ready-made images of Spring Boot Admin Server? A quick Google search will show a few (often outdated) pre-built ones that people in the community have created for different scenarios, including this official codecentric/spring-boot-admin Docker image which states:

This repository contains pre-build Docker Images containing basic and hence not production-ready builds of Spring Boot Admin.

Now onto the client, in order to register instances on Spring Boot Admin, you have to add the spring-boot-admin-starter-client dependency to your app and point it at the admin server. From personal production experience in mature projects, this isn't always a pleasant, straightforward experience.

If your project guards Actuator's endpoints with Spring Security, you also have to configure the client to send the server the credentials it will need when registering itself.

Alternatively, if you use Spring Cloud Discovery within your ecosystem you will be able to use that in order to discover instances, dependency free, but not hassle free, as you will again have to add this to your server's project and configure on both ends.

The second part is performance. Spring Boot Admin lacks in a lot of areas when it comes to performance. The simplest example is the Thread Dump page, where if you stick around long enough without refreshing, it will eventually crash your tab.

With Ostara, we aimed for a solution that just works performantly with your existing setups, no matter their size or scale and without requiring any prior modifications in your code or environment with the only prerequisite being the presence of Spring Actuator.

Feature Overview

• A robust and user-friendly UI for an ever-growing list of Actuator endpoints (Full list)

• Health monitoring of your applications and their individual instances

• Neatly organize your applications and instances into different folders, with color coding and unique icons

• Modify the loggers of your applications and individual instances on the fly

• Evict caches from applications and individual instances on the fly

• View cache statistics

• View HTTP request statistics

• Bean dependency visualization

• Spring Integration Graph visualization

• Application Security built in, interact with your secured Actuator instances with no extra code

The Future

Some of our planned features for future releases:

• Service discovery support

• Notifications

• Custom themes

• A plugin ecosystem for custom Actuator endpoints

• Support for non-official actuator endpoints, such as Togglz

• Native support for non-Spring Actuator facades such as gohealth, express-actuator and pyctuator

Setting up Ostara

Setting up Ostara is super simple, simply head over and download the latest release for your OS and follow the Quick Start to create your first instance and begin monitoring with Ostara

Contributions

Ostara is in pre release so we would greatly appreciate any feedback, recommendations and/or requests as well as any contributions. We would also love to hear from other Spring Boot developers. Head to the repository to participate!

If you already know the strategy pattern just jump to "The Library" section.

What is it?

According to Wikipedia, the strategy pattern is a behavioral software design pattern that enables selecting an algorithm at runtime. Instead of implementing a single algorithm directly, code receives run-time instructions as to which in a family of algorithms to use.

In simple words, the strategy pattern is used to specify how something should be done by providing a specific algorithm. For example, a strategy for payment could be Credit Card, PayPal, or any other payment provider.

Why should we use it?

The Strategy Pattern defines a family of algorithms, encapsulates each one, and makes them interchangeable. Strategy lets the algorithm vary independently from clients that use it (source).

In short, it makes our code cleaner, easier to maintain, and reusable.

The Strategy Pattern in Code

Implementations of the pattern consist of an interface defining the strategy, multiple concrete implementations of the strategy and a context using the strategy.

By Vanderjoe - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=60733582

Code Example

Let's look at the following simplified scenario:

• We have an application with users and we need to send them verification codes for 2FA.

• The application supports multiple communication options for receiving the verification code: Email, SMS, Notifications...

• Each user selects its preferred communication method.

First we will define our strategy interface:

interface VerificationCodeCommunicationStrategy {
    fun sendVerificationCode(user: User, code: String)
}

Now let's add our implementations:

class EmailCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override fun sendVerificationCode(user: User, code: String) {
        emailClient.sendEmail(user.email, "Verification code", "Your verification is $code.")
    }
}

class SmsCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override fun sendVerificationCode(user: User, code: String) {
        smsClient.sendSMS(user.phone, "Your verification is $code.")
    }
}

// ...

All that's left is to use the strategy.

enum class CommunicationMethod {
    Email, SMS, Notification
}

class VerificationCodeService {
    val verificationCodeStrategyMap: Map<CommunicationMethod, VerificationCodeCommunicationStrategy> = mapOf(
        CommunicationMethod.Email to EmailCommunicationStrategy(),
        CommunicationMethod.SMS to SmsCommunicationStrategy(),
        // ...
    )

    fun sendVerificationCode(user: User) {
        val code = generateVerificationCode()

        val verificationCodeStrategy = verificationCodeStrategyMap[user.preferredCommunicationMethod]
        verificationCodeStrategy?.sendVerificationCode(user, code)
    }
}

We can see how the code is clean and easy to maintain. Adding a new communication method would be easy and will not need to change existing code other than adding it to the map of strategies.

The Library

We use the strategy pattern a-looot in our projects. Moreover, like all developers we are lazy and want to make our life as easy as possible. This is why we created our library.

Code Example WITHOUT the library

Let's revise the previous example for a regular Spring Boot implementation.

When using the strategy pattern in a Spring Boot application the context and strategy implementations are usually Spring components.

We will start with the strategies:

interface VerificationCodeCommunicationStrategy {
    fun sendVerificationCode(user: User, code: String)
}

@Component
class EmailCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override fun sendVerificationCode(user: User, code: String) {
        emailClient.sendEmail(user.email, "Verification code", "Your verification is $code.")
    }
}

@Component
class SmsCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override fun sendVerificationCode(user: User, code: String) {
        smsClient.sendSMS(user.phone, "Your verification is $code.")
    }
}

// ...

The interface remains the same and the implementations receive the @Component annotation.

Now let's move on to the verification service.

@Service
class VerificationCodeServiceImpl(
    private val emailCommunicationStrategy: EmailCommunicationStrategy,
    private val smsCommunicationStrategy: SmsCommunicationStrategy
) {
    private val verificationCodeStrategyMap: Map<CommunicationMethod, VerificationCodeCommunicationStrategy> = mapOf(
        CommunicationMethod.Email to emailCommunicationStrategy,
        CommunicationMethod.SMS to smsCommunicationStrategy
    )

    fun sendVerificationCode(user: User) {
        val code = generateVerificationCode()

        val verificationCodeStrategy = verificationCodeStrategyMap[user.preferredCommunicationMethod]
        verificationCodeStrategy?.sendVerificationCode(user, code)
    }
}

We need to add the strategy implementations to the constructor and build the map manually. That's no fun!

Code Example WITH the library

The entire objective of the library is to make our life easier and the code cleaner and less bug prone. It does it by simply building the strategy map automatically.

We will start by modifying the strategy interface.

interface VerificationCodeCommunicationStrategy {
    @get:ComponentMapKey
    val method: CommunicationMethod

    fun sendVerificationCode(user: User, code: String)
}

As you can see we have added a new field method with the annotation @ComponentMapKey. This informs the library which field is the key for the strategy map.

Let's update the implementations according to the modified interface.

@Component
class EmailCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override val method = CommunicationMethod.Email

    override fun sendVerificationCode(user: User, code: String) {
        emailClient.sendEmail(user.email, "Verification code", "Your verification is $code.")
    }
}

@Component
class SmsCommunicationStrategy : VerificationCodeCommunicationStrategy {
    override val method = CommunicationMethod.SMS

    override fun sendVerificationCode(user: User, code: String) {
        smsClient.sendSMS(user.phone, "Your verification is $code.")
    }
}

// ...

Now let's modify the service to use the library.

@Service
class VerificationCodeServiceImpl {
    @ComponentMap
    private lateinit var verificationCodeStrategyMap: Map<CommunicationMethod, VerificationCodeCommunicationStrategy>

    fun sendVerificationCode(user: User) {
        val code = generateVerificationCode()

        val verificationCodeStrategy = verificationCodeStrategyMap[user.preferredCommunicationMethod]
        verificationCodeStrategy?.sendVerificationCode(user, code)
    }
}

That's it! All we had to do is to add the @ComponentMap annotation to the strategy map and voila! The map is built automatically from the components. No boiler-plate code. To add a new strategy all you need to do is to create a new implementation and it will be "magically" added to all relevant strategy maps.

Final Words

This library is the most used library in our projects. Although it's not life changing, it's just useful, helps us keep our code clean and prevents lots of recurring bugs from forgetting to add new strategies to their appropriate maps.

If you would like to use it you can start with the quick start guide.

We develop open source because we love it and want to contribute to the cause. If you would like to support us please consider starring the repository.

The short answer

We needed a simple to use, minimal boiler plate mapping engine, that is also flexible and supports the most advanced use cases.

The long answer

A few years ago we got tired of developing the same boiler plate code for CRUD and decided to develop a generic CRUD framework.

As a part of this framework we needed an object mapping library and could not find one that answered all of our requirements.

The first iteration of the library was part of the entire CRUD framework. However, some time ago we decided we want to start contributing to the open source community and we thought "What is the best way to contribute if not to release our CRUD framework?". The issue with this thought is that the framework is huge and contains lots of code, polishing it and getting it ready to be published as open source would take a lot of time (documentation anyone?).

Our solution was to separate the framework into multiple smaller libraries we can polish and publish in smaller time frames. The first of these libraries is ShapeShift.

WHAT

The short answer

ShapeShift will help you transform object of type A to object of type B easily, fast and without boiler plate code.

ShapeShift main features:

• Auto mapping
• Custom field transformers
• Default transformers
• Deep mapping
• Multiple mapping targets
• Conditional mapping
• Mapping decorators
• Seamless spring integration
• Native Kotlin DSL API
• Java builder API
• Annotations API
• Native Android support

The longer answer

ShapeShift's job seems simple, convert object of type A to object of type B. But to do its job in the best way possible, the library offers lots of features, to name a few:

• Advanced auto mapping - Automatically map fields with the same name from the source object to the target object. Control wether to map only fields of the same type or use the advanced default transformers feature to auto map fields of different types!
• Seamless Spring integration - ShapeShift's Spring Boot library allows to integrate it with Spring easily. Use Spring's power to create custom transformers beans with access to DB or other resources.

More about the advanced features with code examples is available in Kotlin version and Java version.

HOW

The short answer

ShapeShift supports multiple APIs to configure your mappings. Annotations, Kotlin DSL and Java Builder.

The long answer

ShapeShift's documentation is available here. You can check our Quick Start guide to get it up and running in 5 minutes.

Built with Kotlin in mind, ShapeShift was designed around its ecosystem and best practices. The library APIs for mapping with Kotlin:

• Annotations - Fully featured annotation based mapping, just add annotations to your objects and ShapeShift handles the rest. Including using custom field transformers, conditional mapping, advanced object decoration and much more.
• Kotlin DSL - Allowing you to define the relations between objects. This allows you to map objects you can't change (or don't want to), like objects from 3rd party libraries. Additionally you can define inline transformations, conditions and decorations, enabling deep customization and very advanced mapping.

ShapeShift also has full Java support available with multiple APIs for mapping:

• Annotations - Same API as Kotlin with full Java support.
• Java Builder - The equivalent Java API for the Kotlin DSL. Allowing you to define the relations between objects without modifying their code, and map objects you can't change (or don't want to).

Final words

We poured our love, time, knowledge and what not into ShapeShift. I'm proud of the result and our greatest achievement is to be able to help others with their work.

If you would like to contribute, please feel free to open issues, join as a contributor or star the repository. Thanks in advance!

The Experience — 11 Years

The Job — Senior Developer at Startup

—

Teaching some new recruits

It was my 3rd year at the startup, things were going well, the company was growing. We needed more working hands, so like any company, we hired a new group of devs, straight off the assembly line (because why spread it out, or get someone with experience). I don't exactly remember how it came to be, but I was in charge of getting them up to speed. This was my first true taste of teaching and leading.

I was always good at winging explanations, and presentations, so it was no different here. I showed them the ropes, the way things were put together, and what talked to what and how.

The problem came when I needed to explain the way the code was written, and why it was written that way. We were using GWT, and for those who don't know, it allowed us to write the client code as JAVA and use the same entities as we do with the DB.

To say that it was useful was an understatement, but like everything in life, this had its drawbacks. Namely everything you save in the DB is sent to the client. This could and did cause many performance issues, and looking back I'm pretty sure there were security issues stemming from this behavior. This code was not layered correctly, but I didn't quite know it yet.

The code that forced me to look and understand layers

It was around this time that my friend started working on another startup, there he worked with this outsourced dev. She had written the main structure of the server code, and had very distinct layers and separation.

I try not to go too deep in technical terms in these posts, but I kind of need to here. There are a few different actors in our story, and each has some specific needs and talents.

The DB

The job of the DB is simple, save the objects you need to, and allow you to query them when needed. So naturally, this is where you need to think of how to save things, how they are connected to the other entities in the system, and how you are going to query them.

The Server

Its main job is to process things, take requests from the client, do some logic, and save it to the DB. This is the man in the middle, between the client and the DB.

The Client

This is where things are displayed, not everything you need to do the job needs to be displayed. Sometimes you want to display things as a mix of other things.

Now I want to focus on the server, this is where my expertise is, and this is where the layers are most prevalent. It's the server's job to get tasks from the client and process them, update the DB, and send the results back for display. Let's break this down into meaningful layers.

[API] ← RO (Representation Object)→ [Service] ← Entity/RO → [Handler] ← Entity → [Dao/DB]

The client does requests against the server, this is deserialized to a RO.

This RO is then used by the service to do logic, this is where processes are done.

The service in turn works with the handlers, where short and transactional logic can be performed.

They in turn can use the DAO (Data Access Object) to persist and fetch data from the DB.

I'll dig deeper into the different layers, and what they are in charge of, and why they use the Objects they do to talk to the other layers.

The API

This is a facade, facing the client. Not all data needs to be transferred in order to do the requested action. For example, there could be fields on an object that are immutable, and there is no reason for the client to send them over when updating an object. Or there could be fields in the server that are only required for server logic, no reason to send them back via the API. This is why this layer works with ROs aka Representation Objects. They can have less fields, or aggregated fields, that can be used by the client.

This layer should be kept as “stupid” as you can, allowing the system to have multiple different types of APIs that call the same logic. The logic is based on the Service layer, and the API calls upon that layer to achieve its goals.

Service Layer

This is where most of the business logic lives, the different processes that constitute a system. These processes can be long, and can require multiple entities within the system. It's the service layer's responsibility to map the ROs into something more useful for handling the logic.

The service layer calls different Handlers, each with its own domain of responsibility to run the logic. These processes can be long, and should not be transactional (Create locks in the DB).

Handlers

Handlers have very well defined responsibilities, they usually work with only 1 entity, and they are the ones that can do transactional actions. They are also the only ones that talk to the DAO/DB.

As you know, DB transactions can cause locks. One of the things we strive for in good system design, is to minimize lock time and scope. This is why the handlers are separated from the services. In the handlers is where you will find the code most optimized, and single goal oriented.

DAO/DB

This is where persistence occurs. This layer is similar to the API layer in that it should be as simple and “stupid” as possible. Correct implementation of this allows us to simply (relatively) change DB by just re-implementing the DAO layer.

An example:

Lets take for example a process of publishing a new version of a post. This process is triggered by a request from the client, all that is needed is the post_id and version_number. So we have an RO with those fields only.

The process starts with some validation

Does that post_id exist?

Can the user requesting the publishing perform that action?

Is there a version_number like that?

Is it already published?

You can think of some more….

Now of course at the end of this, we want to transactionally update the is_published flag at the same time on the old version as the new. But during the time we are doing validations there is no reason to open a write transaction in the DB, and lock the entities. It is much better to do the validation process, call the handler just to do the swap, then continue after the transaction to create the result for the client.

This way we gained a few key points.

Shortest transaction time possible.

No connection to DB used if the request fails in the validation phase.

Code reuse - the method that publishes in the handler can later be used by different services (for example an automated time publisher).

Clear single responsibilities for each layer.

Practice what I preach

Over the years I have honed this layered design (I will get into it in later posts), but the basics remain. This code forced me to look at my code differently, and implement the layers, and I have to tell you, this has made my code much more readable and maintainable as well. It has also increased my development speed, because I can write each layer's interface, and then start filling in the methods.

I won't lie, it took me a while to get used to working this way, and even longer to get it right (I would put things in the handler that could have been in a service and split among multiple handlers), but it's a process. I urge you to try and split some code you are working on using these principals. Everyone has that 1 method that is so big that nobody wants to touch it. Go ahead, give it a go, I'm confident that after some time (and banging your head against the wall) you will be able to split it up and have much better code at the end.

Please check out my open source library and maybe even star it. I would appreciate it greatly!

When working with multi-layered applications, external libraries, a legacy code base or external APIs, we are often required to map between different objects or data structures.

In this tutorial, we will check out some Object Mapping libraries advanced features to simplify this task while saving development and maintenance time.

In our examples we will use the library ShapeShift. It's a light-weight object mapping library for Java/Kotlin with lots of cool features.

Auto Mapping

We will start with a bang. Auto mapping can and will save you lots of time and boiler-plate code. Some applications require manual mapping between objects, but most applications will save tons of time working on boring boiler-plate by just using this one feature. And it gets even better, with ShapeShift's default transformers we can even use automatic mapping between different data types.

Simple Mapping

Let's start with a simple example for auto mapping. We have our two objects, imagine they could also have tens, hundreds, or even thousands (for the crazy people here) of fields.

public class User {
    private String id;
    private String name;
    private String email;
    private String phone;
    // Getters, Setters, Constructors, Equals...
}

public class UserDTO {
    private String id;
    private String name;
    private String email;
    private String phone;
    // Getters, Setters, Constructors, Equals...
}

We want to map all the field from User to UserDTO. Using auto mapping we don't need to write any boiler-plate code. The mapper will be defined as follow:

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(User.class, UserDTO.class)
        .autoMap(AutoMappingStrategy.BY_NAME_AND_TYPE)
        .build();

Voila! All the fields will be mapped automatically without any manual boiler-plate code.

Advanced Mapping

In this example we will use the power of default transformers to take auto mapping even further.

public class User {
    private String id;
    private String name;
    private Date birthDate;
    // Getters, Setters, Constructors, Equals...
}

public class UserDTO {
    private String id;
    private String fullName;
    private Long birthDate;
    // Getters, Setters, Constructors, Equals...
}

Note that the types of the birthDate field are different in the source and destination classes. But using the power of default transformers we can still use auto mapping here.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(User.class, UserDTO.class)
        .autoMap(AutoMappingStrategy.BY_NAME)
        .build();

We changed the auto mapping strategy to BY_NAME so it will map fields also with different types. Now we need to register a default transformer to the ShapeShift instance in order for it to know how to transform Date to Long.

ShapeShift shapeShift = new ShapeShiftBuilder()
        .withTransformer(
                Date.class, 
                Long.class, 
                new DateToLongMappingTransformer(), 
                true // Default Transformer
        )
        .build();
    }

We can also add manual mapping on top of the auto mapping in order to add/change behavior. The source and destination classes have different names for the name field so we will add manual mapping for it.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(User.class, UserDTO.class)
        .autoMap(AutoMappingStrategy.BY_NAME)
        .mapField("name", "fullName")
        .build();

Auto mapping is great for use cases that does not require specific mapping. It helps reduce the amount manual boiler-plate code needed to configure mapping and also helps you keep your sanity.

Transformers

Transformers are very useful feature that allows you to transform the type/value of a field to a different type/value when mapping a field.

Some use cases we have been using widely:

• Transform date to long and vice versa between server and client objects.
• Transform JSON string to it's actual type and vice versa between server and client objects.
• Transform comma separated string to list of enums.
• Transform another object id to its object or one of its fields from the DB using Spring transformers.

Basic Transformers

We will start with a simple transformer example. Date-to-Long and Long-to-Date transformers:

public class DateToLongMappingTransformer implements MappingTransformer<Date, Long> {
    @Nullable
    @Override
    public Long transform(@NonNull MappingTransformerContext<? extends Date> context) {
        return context.getOriginalValue() != null ? context.getOriginalValue().getTime() : null;
    }
}

public class LongToDateMappingTransformer implements MappingTransformer<Long, Date> {
    @Nullable
    @Override
    public Date transform(@NonNull MappingTransformerContext<? extends Long> context) {
        return context.getOriginalValue() != null ? new Date(context.getOriginalValue()) : null;
    }
}

All we need to do now is to register them.

ShapeShift shapeShift = new ShapeShiftBuilder()
        .withTransformer(Date.class, Long.class, new DateToLongMappingTransformer(), true) // "true" is optional, we are registering the transformers as default transformers, more on that later.
        .withTransformer(Long.class, Date.class, new LongToDateMappingTransformer(), true)
        .build();

That's it! We can now use the transformers when mapping objects.

public class User {
    private String id;
    private String name;
    private Date birthDate;
    // Getters, Setters, Constructors, Equals...
}

public class UserDTO {
    private String id;
    private String name;
    private Long birthDate;
    // Getters, Setters, Constructors, Equals...
}

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(User.class, UserDTO.class)
        .mapField("id", "id")
        .mapField("name", "name")
        .mapField("birthDate", "birthDate").withTransformer(DateToLongMappingTransformer.class) // We don't have to state the transformer here because it is a default transformer
        .build();

Inline Transformers

In some use cases we want to transform the value but we don't need a reusable transformer and we don't want to create a class just for a one time use.

Inline transformers for the rescue! Inline transformers allow to transform the value without the need to create and register and transformer.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(Source.class, Target.class)
        .mapField("birthDate", "birthYear").withTransformer(context -> context.getOriginalValue() != null ? ((Date) context.getOriginalValue()).getYear() : null)
        .build();

Advanced Transformers

Transformers also allow us to do transformations with the DB or other data sources.

In this example we will use the power of the Spring Boot integration to create transformers with DB access.

We have three models:

• Job - DB entity.
• User - DB entity.
• UserDTO - Client model.

public class Job {
    private String id;
    private String name;
    // Getters, Setters, Constructors, Equals...
}

public class User {
    private String id;
    private String jobId;
    // Getters, Setters, Constructors, Equals...
}

public class UserDTO {
    private String id;
    private String jobName;
    // Getters, Setters, Constructors, Equals...
}

We want to convert the jobId on User to jobName on UserDTO by querying the job from the DB and setting it on the DTO.

In Spring's case, you generally avoid interaction with the application context from static functions or functions on domain objects.

We will use a ShapeShift's Spring integration to create a component as a transformer to access our DAO bean.

@Component
public class JobIdToNameTransformer implements MappingTransformer<String, String> {

    @Autowired
    private JobDao jobDao;

    @Nullable
    @Override
    public String transform(@NonNull MappingTransformerContext<? extends String> context) {
        if (context.getOriginalValue() == null) {
            return null;
        }
        Job job = jobDao.findJobById(context.getOriginalValue());
        return job.getName();
    }
}

All that's left to do is to use this transformer in our mapping.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(User.class, UserDTO.class)
        .mapField("id", "id")
        .mapField("jobId", "jobName").withTransformer(JobIdToNameTransformer.class)
        .build();

Another bonus of using transformers is their reusability. In some use cases, We could create more generic transformers that will have application-wide usage.

Default Transformers

When registering transformers you can indicate wether a transformer is a default transformer. A default transformer of types \ is used when you map a field of type \ to field of type \ without specifying a transformer to be used.

As we already seen, default transformers are useful for recurring transformations and especially for automatic mapping.

Deep Mapping

What if we want to map from/to fields that available inside a field which is an object? We can even do that, easily.

In order to access child classes we can use the full path of a field. Let's look at the following example.

public class From {

    private Child child = new Child();
    // Getters, Setters, Constructors, Equals...

    class Child {
        private String value;
        // Getters, Setters, Constructors, Equals...
    }
}

public class To {
    private String childValue;
    // Getters, Setters, Constructors, Equals...
}

We want to map the value field in Child class inside the From class to the childValue field in the To class. We will use the full path of value which is child.value.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(From.class, To.class)
        .mapField("child.value", "childValue")
        .build();

The full path is supported in both source and destination fields, it also supports multi level depth (e.g. x.y.z).

Conditional Mapping

Conditions allow us to add a predicate to a specific field mapping to determine whether this field should be mapped.

Using this feature is as easy as creating a condition.

public class NotBlankStringCondition implements MappingCondition<String> {
    @Override
    public boolean isValid(@NonNull MappingConditionContext<String> context) {
        return context.getOriginalValue() != null && !context.getOriginalValue().trim().isEmpty();
    }
}

And adding the condition to the desired field mapping.

public class SimpleEntity {
    private String name;
    // Getters, Setters, Constructors, Equals...
}

public class SimpleEntityDisplay {
    private String name = "";
    // Getters, Setters, Constructors, Equals...
}

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(SimpleEntity.class, SimpleEntityDisplay.class)
        .mapField("name", "name")
        .withCondition(NotBlankStringCondition.class)
        .build();

Inline Conditions

Like transformers, conditions can also be added inline using a function.

MappingDefinition mappingDefinition = new MappingDefinitionBuilder(SimpleEntity.class, SimpleEntityDisplay.class)
        .mapField("name", "name")
        .withCondition(context -> context.getOriginalValue() != null && !((String) context.getOriginalValue()).trim().isEmpty())
        .build();

Annotations Mapping

This specific feature receives lots of hate because it breaks the separation of concerns principle. Agreed, this could be an issue in some applications, but in some use cases where all objects are part of the same application it can also be very useful to configure the mapping logic on top of the object. Check out the documentation and decide for yourself.

Conclusion

Object mapping libraries are not the solution for every application. For small, simple applications using boiler-plate mapping functions are more than enough. But, when developing larger, more complex applications, object mapping libraries can take your code to the next level, saving you development and maintenance time. All of these while reducing the amount of boiler-plate code and overall improving the development experience.

On a personal note, I used to work with manual mapping functions and was ok with it. It was "just" some simple lines of code. After upgrading our applications to use object mapping as part of our "boiler-plate free" framework (We will discuss that framework at a later time), I can't go back. Now we spend more time on what's important and interesting and almost no time on boring boiler-plate code.

The Experience — 9 Years

The Job — Junior Developer at Startup

—

My first job interview

In my last year of uni Aviv (my now best friend and business partner) told me he is working at a startup that is now growing, and I should join him working there. I was nervous for my first job interview, they were working with a different stack than I had experience with, and let's just say I'm a little weird. A little background, I live and had grown up in a rural farming area, I wear shorts, and sandals (even in the dead of winter), not exactly “professional” looking. Anyway I vividly remember taking long pants and shoes in the car with me to the interview, thinking I would change right before I went in, to look the part. At the last moment I changed my mind, and told myself, “You should not try to pretend you are something you are not, they will either take you with your quirks or it's not the right place for you”. So I walked into the interview with the CEO of the company, in shorts and sandals. Lucky for me, the CEO was wearing torn jeans, and dirty worn sneakers, I felt right at home.

All this was to say something simple, but not easy. Be true to yourself, if you go work somewhere you won’t give your 100% if you feel out of place. I'm not saying you need to reject any job that isn't perfect, it's more like, find a job that doesn't compromise on your core, and be flexible about the “less important” stuff. In the long run this will be much better for you and the company.

Java you say

My first day on the job was an interesting one, I got to the new office, they gave me a laptop, and told me to install my environment. Easier said than done, till now it was a visual studio, now I needed this weird eclipse thing. Then I got the code and nothing runs :/ weird. After some back and forth, I learned that for java and eclipse I need to download the JDK (Java development toolkit), fine. Still nothing… Just a reminder it was my first day on the job, in a language/stack I didn't know, and I didnt want to be the new “idiot” they hired. Safe to say it took me the whole day or maybe even 2 just to get the code up and running.

Later that week I got my first task, I needed to copy some functionality that existed in the client area to the back office. This was actually a perfect first task, for many reasons, most of all it was 70% copy paste. This was great because I had barely knew how to read the java code I was looking at. Additionally the adaptation needed in the back office project was simple and existed in one form or another in different areas of the code. So basically the task was like a big jigsaw puzzle, I got 1 big piece, now I need to go hunt down some smaller ones and put it all together.

A small tangent

Over the years this “jigsaw puzzle” approach has really sunk in, and today I can tell you with confidence, if you structure the tasks given to developers (especially juniors) in this way, a few good things happen:

1. They get into the code much faster.
2. They write less buggy code.
3. They build confidence in their abilities. Finally as a business owner I can tell you from my experience, it takes a lot less time to see a positive ROI for their time.

This has led me to a core belief of mine (in regards to development). A good programmer needs to know 3 things:

1. How to read and understand code.
2. Be proficient in google searches for solutions to the problem at hand.
3. Know how to adapt other code to meet their current needs.

A great programmer is one that knows how/why things are done in this way, and understands optimization and deep diving into frameworks they use. While a good programmer just needs to know how to use the framework, and not how it works.

What is this Spring magic - using frameworks

The project we were writing was built upon Spring (a hugely popular dependency injection framework in Java). Now frameworks are powerful tools, but they hide many things under the hood. Over the years I have learned in depth how/why Spring works, but truth be told, as a junior developer back then it seemed like magic to me.

I want to take you through a few different stages of understanding that I use today when I am faced with a new framework.

Step 1 - Basics (The what)

In this step it is most important NOT to try to understand how it works, just to understand the tools and capabilities it provides. I have seen it many times that a junior gets “stuck” on trying to understand the depths of the framework, and the truth is they are just not ready to. They don’t fully understand WHAT the framework can do, so they have no chance to understand how.

Step 2 - Advanced Usage (The how)

Almost every framework I have used has many different features that compliment each other (makes sense). This stage is focused on that, using the framework to its fullest. Mixing and matching different features, making the whole greater than the sum of its parts. This is where the developer truly learns HOW to use the framework correctly.

Step 3 - Down the rabbit hole (The why)

Here is where things get truly interesting. There comes a time when you reach the limits of the framework and try to do something you think should be possible, but turns out it isn't. This is when you download the source code and start the deep dive to understand the WHY. Sometimes it's something at the core of how it's built, but other times it's just on the fringes of what the framework can do, and that means, no examples, no documentation. Then the only answer is to understand why the framework works the way it does.

Step 4 - Building upon the framework

This is the final stage, where you can start using the tools the framework gives you to build your own mini framework upon. Or even build your own libraries from scratch. This is where you start contributing to the framework, or start releasing some open source of your own.

Generalization is a powerful tool

After a year or 2 in the company, I got a new task, implementing a new PSP (Payment service provider), basically adding something like paypal to the clients payment options. This was a new area in the code for me, something I had no experience with. So off to the races, I went and looked at the implementation of a few different PSPs, and to my shock, I saw a ton of very similar code, makes sense since it doesn't really matter how the money comes in, we still need to update billing, create the relevant logs, and credit the account. All this was done time and time again, in the different PSPs, this was mainly because the objects we were getting back from the PSP api were so different.

Interfaces to the rescue, this is one of my first epiphanies, this has led me to proper code layers, and a few “new” design patterns (really just classes that give me a mix of design patterns with no boilerplate code). So I told my boss I needed more time to rewrite it correctly, and set to work. I created a handler that was in charge of the process of communication with the PSPs. It had methods for the different actions, like charge or refund and so forth. It managed the different steps of the process and called upon the different classes of the PSP for interpretation of the response received via API.

I don't want to get into the code (this is not the point of the post), but basically there was a PspResponse that all PSPs needed to inherit from, and then methods like: getIsSuccess() getErrorText() getAmount()

This took some time to refactor the old code, but in the long run reduced implementation time for new PSPs drastically, all the while reducing the amount of bugs (mainly because someone would always forget to update the new logic in some old PSP).

The experience I gained designing and implementing this led me down the path to becoming an architect. I felt so proud from my new creation, and caused me to look at code in a different way. This is a pivotal point in my career, and has caused me to think about how to write code correctly in the long run, and not just this feature.

Please check out my open source library and maybe even star it. I would appreciate it greatly!

The Experience — 7 Years

The Job — Uni Student + Freelancer

—

My first paying job

Let me set the mood, poor uni student, first year physics major, tooo many courses at the same time, not enough sleep; I think you get the idea (and possibly what it feels like). Now I know we would all love to have some money, to go out with, and enjoy our time. We can all find some minimum wage 9-5, but I was looking for something better, and thought I could use my skills as a developer to land some work.

Luckily for me, a family friend was looking for someone to develop something for them. The job was a good mix of my knowledge and skills. This friend owned a factory that builds custom water treatment and purification equipment, and he had a bottleneck in his quoting division. Every quote the sales team needed required an engineer to write up plans, then someone needed to calculate not only the costs of the different parts, but things like length of welds, amount of paint needed, amount of bolts needed, and so forth. Till this point it was done in excels with lookups and required a lot of know-how.

My job (should I choose to accept it) was to create a visual tool, that the sales agent can drag and drop to the parts needed, including piping and such, and the output would be a detailed quote, including everything mentioned above and more.

Working with/for a client as a freelancer

Now for all you developers out there, I'm sure you are thinking, “OK, he got some requirement documents on how/what to build”. That couldn't be farther from the truth (and surprisingly more common than I had expected). I had to basically interview all the people involved in the company, to try and get a feel of what was needed. After that I would sit down and write my own basic requirement documents, that I would then go over with the client.

If there is one thing you take away from this: “Always go over the requirements with the client in writing”

This has saved my @$$ many times. It took some time learning how to deal with clients, and I learned a ton. This first client helped me learn, the development skills aren't the only thing important, it is important to understand the clients needs, pain points, and where to focus my efforts, this has rang true throughout my career.

I was lucky (in a sadistic kind of way) that my first client had no idea on how to write requirements, and how to handle a development cycle. In the years to come, I would receive many “requirements” of the type “put a button there that does that”; and 9/10 times there is a better way to do things that answers the clients problems in a better way. I urge you to look at anything you are developing, and try to understand the business use case behind it, and look at the bigger picture. You will be surprised to find that this will help you tremendously write better code.

Just for an example, I was told to make a button for export of the quote, sounds reasonable enough, but then it would be the sales agent's job to send copies to the client, the management, and the project manager. I wouldn't have known this if I just would have developed the button. Instead I suggested that the system have projects in it, with the project managers info, and have a “send to email” button. This may seem minor, but after talking to the CEO I found out it saved him tons of heartache, because more than once, the wrong quote was sent to the wrong client, just because the agent attached the wrong file.

What you can’t learn in Uni

During my time at uni, I learned a ton of things, in many different subjects, including programming. A few caveats to this is the fact that most of the time, the languages that were taught were out of date, and the professors didn't really know how to program a product, more like write blocks of code. Sadly there is more truth than I had hoped for in the old saying “Those who can't, teach”.

I think I can confidently say I learned more in my first freelance job, then everything uni could teach. There is nothing like getting your hands “dirty” and just doing some work. Here is a small list of the top things I “learned on the job”:

1. Understand the business - Like I stated before, understand “why” your client needs the feature, not just how they see the feature implemented
2. 80/20 principle - Try and spend your time wisely covering most of the needs, and leaving “known issues” for the really rare edge cases.
3. KISS - “Keep it, simple stupid” (or somtimes I think it’s “Keep it simple, stupid”), try to simplify your code, break it into smaller bite sized pieces, that will pay dividends in the long run.
4. Performance isn't everything (or even important most of the time) - Today's computers are fast (like really really fast) and that fancy Binary search algorithm you learned, doesn't really matter in real life (most of the time), when you filter a list of even a few 10s of thousands, it’s not really going to change the runtime.
5. Use open source wherever you can - There are so many amazing open source projects out there, and they can cut your programming time by a huge factor. Additionally they can (usually) do it better then if you tried to write it yourself.

Please check out my open source library and maybe even star it. I would appreciate it greatly!

The Experience — 4 Years

The Job — The intelligence corps

It's my time to serve

This may be doxing myself a little, but where I live there is a mandatory military service, for 3 f%$#ing years. Luckily for me, I had some skills, computer skills. It works here like this, you go do a lot of tests, intelligence tests, aptitude tests, physical tests, and at the end you get stationed somewhere (usually NOT where you want to be). I was in the running for a few different positions, and then one day out of the blue, I got a phone call. Come now to interview for a new position, nothing that I ever heard of, well of course I went. They were looking for someone with programming experience that they didn't need to send to the military academy to learn, and I fit the bill…

One thing led to another, and within the week I found myself in basic training on my way to the intelligence corps, as a developer in a small unit. After a few weeks of basic training (even us developers go through boot camp), I arrived at my unit. This is where I found out that just me and 1 other guy are the only people in the unit that can “speak computer”, and no I don't just mean program. We were in charge of the whole system there, from the network, to the servers, and of course the software.

How to learn C# in 3 easy steps

1. Get enlisted to the military, to a plush desk job as a programmer.
2. Be 1 of 2 people in charge of the whole system.
3. Never forget that if you don't get on top of things within a few months, they will ship you out to somewhere much worse.

Easy right?

But in all seriousness, I had to teach myself C# on the job. At least I had the running code, and the coveted PC on the internet. You see in the intelligence corps, and probably most military installations, no one has internet access on their PC. Well technically neither did I, but what I did have was access to the only PC (that was air gapped from the network) which had internet access.

By this time of my life (luckily for me), google was a thing already, and there were more and more resources online about programming. Even more for things like C#, because it was Microsoft's. But (I hate it that there is always a “but”)… The computer in question is air gapped, so no copy paste.

Into the deep end

This was the first time I truly worked with a DB (I don't really count MS Access). It was my first time, and they threw me into the deep end (without floats). Our systems used Oracle, till this day I think it's an amazing DB, but it's a beast. Our system was complex, and today, I would have never built it this way (I didn't build it back then either, the system was operational when I got there).

I know I'm going to spark some debate about this, but I dont think business logic is supposed to be in stored procedures (there I said it). This architecture caused some very weird side effects, for example, releasing versions requiring simultaneous release of new stored procedures, or updates to old ones. Triggers that can cause chain reactions of events, and you can't quite see them in any IDE.

If that wasn't bad enough, we didn't really have a server side. The C# was a desktop app, installed via group policy to all PCs on the network. The server was basically the stored procedures in Oracle. So things like creating thread pools to handle large tasks, or locks on editing were all out of the question.

This is where I started thinking about how code should be built, and it was the start of the path to architecture. Even if it still took a few years to find out what I should do, at least now, I know what I shouldn't. It was an important realization, one that I have since seen on some of the juniors I mentored. Writing code without bugs is important, but writing code in a maintainable way, is just as important.

My partner in crime

About half way through my service, the other guy I was with finished his tour. It was up to me to find a replacement. I had access to candidates (and it's not like they had CVs, they are all 18 year old fresh out of high school), I gave them tests to do, asked them questions. Remember back then I couldn't send them anything online, all by phone, or in person.

After a few weeks, and at least 50 candidates, I had found him, my new partner. Again we couldn't send him to learn in any course, so I had to find someone with the knowhow from home. Additionally even after I selected him, it took months for him to get to me. He had to get security clearance, and go through boot camp. So months later he was finally here, I would like to introduce you to a new character (that will have a recurring role in the articles to come), my business partner (and best friend).

Like any relationship it was rocky at first. I won't go into the details, but what I can say is that we went through a lot together. Things that built our relationship strong. Little did I know (spoilers) that we would be working together till this day (over 16 years).

Please check out our open source library and maybe even star it ;-) We would appreciate it greatly.

Multi-layered applications often have similar but different object models, where the data in two models may be similar but the structure and concerns of the models are different. Writing the mapping code is a tedious and error-prone task. Object mapping makes it easy to convert one model to another.

Simple Example

Let's say we have a REST API application with two models. One for the server and one client facing:

User (Server):

data class User(
    val id: String,
    val name: String,
    val email: String,
    // isBlocked is server side only
    val isBlocked: Boolean
)

UserDTO (Client):

data class UserDTO(
    val id: String,
    val name: String,
    val email: String
)

Conversion

We have a couple of options:

Extension Methods

We can use simple extension methods.

fun User.toUserDTO(): UserDTO {
    return UserDTO(id, name, email)
}

Simple and easy, but requires additional boiler-plate code we need to maintain every time we change the model.

Object Mapping

We will use the open source object mapping library ShapeShift for the examples.

Annotations

The first step is to add the annotations to the fields we want to map.

@DefaultMappingTarget(UserDTO::class)
data class User(
    @MappedField
    var id: String = "",
    @MappedField
    var name: String = "",
    @MappedField
    var email: String = "",
    // isBlocked is server side only and not mapped to client DTO
    var isBlocked: Boolean = false
)

All that's left is to convert.

val shapeShift = ShapeShiftBuilder().build()
val user = User("xyz", "john doe", "john@email.com", false)
val userDTO = shapeShift.map<UserDTO>(user)

DSL

In some cases we can't change the data classes code (or don't want to). For these use cases we create a standalone mapper between the two classes using Kotlin DSL.

val mapper = mapper<User, UserDTO> {
    User::id mappedTo UserDTO::id
    User::name mappedTo UserDTO::name
    User::email mappedTo UserDTO::email
}

And now the conversion. Note that we are registering the mapper to the ShapeShift instance.

val shapeShift = ShapeShiftBuilder().withMapping(mapper).build()
val user = User("xyz", "john doe", "john@email.com", false)
val userDTO = shapeShift.map<UserDTO>(user)

Seems like a lot of work for a simple conversion, right? Maybe... Let's continue to the next example.

Auto Mapping

What if I told you there is a way to remove almost if not all the mapping boiler-plate code? In many use cases we have similar classes we need to map between. This is where automatic mapping comes to the rescue.

Let's continue with the same classes.

data class User(
    val id: String,
    val name: String,
    val email: String,
    val isBlocked: Boolean
)

data class UserDTO(
    val id: String,
    val name: String,
    val email: String
)

Using automatic mapping our mapper will be the following.

val mapper = mapper<User, UserDTO> {
    autoMap(AutoMappingStrategy.BY_NAME_AND_TYPE)
}

Voila! No boiler-plate code to configure fields manually. Auto mapping also comes with advanced features for more complex use cases.

Generic Example

When our projects start to scale, we want to add generic functionality to improve productivity and decrease maintenance. For example:

• Generic CRUD controller for entities.
• Generic export to excel/pdf/...

In order for us to be able to implement these generic functionality we need to be able to convert our objects back and forth between classes.

We can use the factory method pattern to do it, simple and straightforward, but agin requires more boiler-plate code we need to maintain when we add new models.

Another option is (surprise surprise) to use object mapping.

inline fun <reified ExportModel : Any> export(model: Any) {
    val shapeShift = ShapeShiftBuilder().build()
    val exportModel = shapeShift.map<ExportModel>(model)
    // export logic...
}

val user = User(/*...*/)
export<UserExport>(user)

That's it. Zero boiler-plate code and we got ourselves the generic functionality we know and love.

Type Safety

Let's take it further and add type safety to the export method. For that we will add two interfaces:

interface ExportModel {
}

interface BaseModel<EM: ExportModel> {
}

Now we will update our classes to implement the interfaces.

class UserExport: ExportModel {
    // ...
}

class User: BaseModel<UserExport> {
    // ...
}

And change our generic method respectively.

inline fun <reified EM : ExportModel> export(model: BaseModel<EM>) {
    val shapeShift = ShapeShiftBuilder().build()
    val exportModel = shapeShift.map<EM>(model)
    // export logic...
}

That's it. We got ourselves type safe generic export method.

val user = User(/*...*/)
export(user)

Spring Example

Taking it up a notch to the next level. We have three models.

• Job - DB entity.
• User - DB entity.
• UserDTO - Client model.

@Entity
@Table(name = "jobs")
class Job {
    var id: String = ""
    var name: String = ""
}

@Entity
@Table(name = "users")
class User {
    var id: String = ""
    var jobId: String? = null
}

class UserDTO {
    var id: String = ""
    var jobName: String? = null
}

We need to convert the jobId on User to jobName on UserDTO by querying the job from the DB and setting it on the DTO.

In Spring's case, you generally avoid interaction with the application context from static functions or functions on domain objects and because of that they cannot be invoked from extensions.

Object mapping to the rescue (surprise!). Let's create a custom transformer to do the heavy lifting for us.

@Component
class JobIdToNameTransformer(
    private val jobDao: JobDao
) : MappingTransformer<String, String>() {
    override fun transform(context: MappingTransformerContext<out String>): String? {
        context.originalValue ?: return null
        val job = jobDao.findJobById(context.originalValue!!)
        return job.name 
    }
}

The complicated part is over. Now we just need to add the annotations to the User model.

@Entity
@Table(name = "users")
@DefaultMappingTarget(UserDTO::class)
class User {
    @MappedField
    var id: String = ""
    @MappedField(transformer = JobIdToNameTransformer::class, mapTo = "jobName")
    var jobId: String? = null
}

That's all. The great part is that we can use the same transformer every time we need to convert a job id to job name on any model.

val shapeShift = ShapeShiftBuilder().build()
val user = User(/*...*/)
val userDTO = shapeShift.map<UserDTO>(user)

A full spring example code is available here.

Pros & Cons

Each conversion method has its own pros & cons.

Extension Methods

Pros

• Simple implementation.
• Clear code - no annotations "magic".

Cons

• Lots of boiler plate code.
• Limited functionality.
• Does not scale well.

Object Mapper

Pros

• Little to none boiler-plate code.
• Auto mapping.
• Generic code.
• Advanced features.
• Code reuse.

Cons

• Requires to learn new library.
• Annotations "magic".

Summary

Like most things in development, there is no right answer here. The answer is that it depends on the project. For small simple projects extension methods are more than enough to convert classes, but for large enterprise projects using an object mapper can give the freedom to achieve better architecture and more generic code that can keep moving forward with large codebase and lots of models.

The Experience — 2 Years

The Job — High school

Isn’t learning fun?

To set the scene, it was early 2000s in a small farming village. Yes we had internet, but it was dial up, and no stack overflow or anything close yet, the best you could hope for is a forum post.

So how did we learn? TL;DR — Books and lots of them…

I remember going to the library looking for books, swapping with friends. If you got the expensive ones you even got a CD (you know the round thing with a hole in it) with them with code examples and lesson projects, not that the CD ever worked (or was even there) from the library. No searching the book for your problem or an example, no copy paste, nothing. Just read and hope you remember later where you saw something about that problem you are having.

Yes the internet existed, but there wasn’t much in the way or programming guides, tutorials, or even examples. One of the most useful tools was IRC channels (for you kids out there, a really old chat). Tell you the truth, I don’t even remember how I found what channel to write to, but I remember talking to people from around the world about the most silly things, things that today you will find the solution to as the top search result on google.

I know this is weird, but you can learn at school

When I reached the last year of high school, I took a tech class. In this class we learned things from electrical engineering basics, mechanical design, and of course programming. The main problem was, I had taught myself more than what the teacher reached in class. Nonetheless I gained a great resource, my teacher could answer questions, point me in the right direction.

First of all, I would like to say, not all teachers are bad (yes many are, but not all). Luckily for me our high school tech teacher saw my potential (especially after I broke the school’s network). I used to get bored in class, everyone was learning C and VB6 at a level I passed a long time ago on my own. That is when my teacher came to my rescue.

My first real project

Computer vision, you know when computers look at an image and try to understand what is there. Well for that time it was no AI models or anything, it was purely algorithmic. My mission should I choose to accept it (over being thrown out of class) was to take this “new” library that allowed me to capture an image from a webcam in VB6 and process it (pixel by pixel), and try and make out a shape in an image, simple geometric shapes (circle, square, triangle).

So my best friend and I set out to this task, it was soooo interesting we would have all nighters trying to solve it. Let me take you through the basics.

Go over the image one pixel at a time to try and find a place with a big change in color (black shape on white paper). When I reach it, go in a pattern to find its neighbor that is also a border pixel. Do this in a loop till I can’t find any more border pixels. This worked (in theory). There is always a “but”… It all went to hell when we would hit these edge cases of a few pixels out of place.

That’s when it hit me, the solution may be obvious to you, but to me it wasn’t. Well, it turns out there is a use for the data structure “stack” we’ve been taught. We put all pixels in the stack, and each time we would hit a brick wall, we would pop a pixel and try again. This would continue till we popped them all out and got back to our starting pixel.

This turned out to be very effective, and solved our biggest problem.

We actually found a few solutions for understanding what shape it was.

From the trivial, see how many “straight” lines made up the shape, which worked pretty well for simple shapes. To the most advanced solution, that I am still proud of till this day, calculate the area of the shape, and the perimeter of the shape, and see what constant best defines the relation.

The moral of the story

I told you this whole story to come to a small point, one that shaped the next few years of my development life. Data structures & design patterns are there for a reason, and even though it might not be obvious, you ask yourself “why would I ever need that”. At some point you will, and it’s good to keep them handy.

If you would like to check out our open source library and leave feedback we would appreciate it greatly.

In this tutorial, we will check out some Object Mapping libraries advanced features to simplify this task while saving development and maintenance time.

In our examples we will use the library ShapeShift. It's a light-weight object mapping library for Kotlin/Java with lots of cool features.

Auto Mapping

We will start with a bang. Auto mapping can and will save you lots of time and boiler-plate code. Some applications require manual mapping between objects, but most applications will save tons of time working on boring boiler-plate by just using this one feature. And it gets even better, with ShapeShift's default transformers we can even use automatic mapping between different data types.

Simple Mapping

Let's start with a simple example for auto mapping. We have our two objects, imagine they could also have tens, hundreds, or even thousands (for the crazy people here) of fields.

class User {
    var id: String = ""
    var name: String? = null
    var email: String? = null
    var phone: String? = null
}

class UserDTO {
    var id: String = ""
    var name: String? = null
    var email: String? = null
    var phone: String? = null
}

We want to map all the field from User to UserDTO. Using auto mapping we don't need to write any boiler-plate code. The mapper will be defined as follow:

val mapper = mapper<User, UserDTO> {
    autoMap(AutoMappingStrategy.BY_NAME_AND_TYPE)
}

Voila! All the fields will be mapped automatically without any manual boiler-plate code.

Advanced Mapping

In this example we will use the power of default transformers to take auto mapping even further.

class User {
    var id: String = ""
    var name: String? = null
    var birthDate: Date? = null
}

class UserDTO {
    var id: String = ""
    var fullName: String? = null
    var birthDate: Long? = null
}

Note that the types of the birthDate field are different in the source and destination classes. But using the power of default transformers we can still use auto mapping here.

val mapper = mapper<User, UserDTO> {
    autoMap(AutoMappingStrategy.BY_NAME)
}

We changed the auto mapping strategy to BY_NAME so it will map fields also with different types. Now we need to register a default transformer to the ShapeShift instance in order for it to know how to transform Date to Long.

val shapeShift = ShapeShiftBuilder()
    .withTransformer(DateToLongMappingTransformer(), true)
    .build()

We can also add manual mapping on top of the auto mapping in order to add/change behavior. The source and destination classes have different names for the name field so we will add manual mapping for it.

val mapper = mapper<User, UserDTO> {
    autoMap(AutoMappingStrategy.BY_NAME)
    User::name mappedTo UserDTO::fullName
}

Auto mapping is great for use cases that does not require specific mapping. It helps reduce the amount manual boiler-plate code needed to configure mapping and also helps you keep your sanity.

Transformers

Transformers are very useful feature that allows you to transform the type/value of a field to a different type/value when mapping a field.

Some use cases we have been using widely:

• Transform date to long and vice versa between server and client objects.
• Transform JSON string to it's actual type and vice versa between server and client objects.
• Transform comma separated string to list of enums.
• Transform another object id to its object or one of its fields from the DB using Spring transformers.

Basic Transformers

We will start with a simple transformer example. Date-to-Long and Long-to-Date transformers:

class DateToLongMappingTransformer : MappingTransformer<Date, Long> {
    override fun transform(context: MappingTransformerContext<out Date>): Long? {
        return context.originalValue?.time
    }
}

class LongToDateMappingTransformer : MappingTransformer<Long, Date> {
    override fun transform(context: MappingTransformerContext<out Long>): Date? {
        context.originalValue ?: return null
        return Date(context.originalValue)
    }
}

All we need to do now is to register them.

val shapeShift = ShapeShiftBuilder()
    .withTransformer(DateToLongMappingTransformer(), true) // "true" is optional, we are registering the transformers as default transformers, more on that later.
    .withTransformer(LongToDateMappingTransformer(), true)
    .build()

That's it! We can now use the transformers when mapping objects.

class User {
    var id: String = ""
    var name: String? = null
    var birthDate: Date? = null
}

class UserDTO {
    var id: String = ""
    var name: String? = null
    var birthDate: Long? = null
}

val mapper = mapper<User, UserDTO> {
    User::id mappedTo UserDTO::id
    User::name mappedTo UserDTO::name
    User::birthDate mappedTo UserDTO::birthDate withTransformer DateToLongMappingTransformer::class // We don't have to state the transformer here because it is a default transformer
}

Inline Transformers

In some use cases we want to transform the value but we don't need a reusable transformer and we don't want to create a class just for a one time use.

Inline transformers for the rescue! Inline transformers allow to transform the value without the need to create and register and transformer.

val shapeShift = ShapeShiftBuilder()
        .withMapping<Source, Target> {
            // Map birthDate to birthYear with a transformation function
            Source::birthDate mappedTo Target::birthYear withTransformer { (originalValue) ->
                originalValue?.year
            }
        }
        .build()

Advanced Transformers

Transformers also allow us to do transformations with the DB or other data sources.

In this example we will use the power of the Spring Boot integration to create transformers with DB access.

We have three models:

• Job - DB entity.
• User - DB entity.
• UserDTO - Client model.

class Job {
    var id: String = ""
    var name: String = ""
}

class User {
    var id: String = ""
    var jobId: String? = null
}

class UserDTO {
    var id: String = ""
    var jobName: String? = null
}

We want to convert the jobId on User to jobName on UserDTO by querying the job from the DB and setting it on the DTO.

In Spring's case, you generally avoid interaction with the application context from static functions or functions on domain objects.

We will use a ShapeShift's Spring integration to create a component as a transformer to access our DAO bean.

@Component
class JobIdToNameTransformer(
    private val jobDao: JobDao
) : MappingTransformer<String, String>() {
    override fun transform(context: MappingTransformerContext<out String>): String? {
        context.originalValue ?: return null
        val job = jobDao.findJobById(context.originalValue!!)
        return job.name 
    }
}

All that's left to do is to use this transformer in our mapping.

val mapper = mapper<User, UserDTO> {
    User::id mappedTo UserDTO::id
    User::jobId mappedTo UserDTO::jobName withTransformer JobIdToNameTransformer::class
}

Another bonus of using transformers is their reusability. In some use cases, We could create more generic transformers that will have application-wide usage.

Default Transformers

When registering transformers you can indicate wether a transformer is a default transformer. A default transformer of types \ is used when you map a field of type \ to field of type \ without specifying a transformer to be used.

As we already seen, default transformers are useful for recurring transformations and especially for automatic mapping.

Deep Mapping

What if we want to map from/to fields that available inside a field which is an object? We can even do that, easily.

In order to access child classes we can use the .. operator. Let's look at the following example.

class From {
    var child: Child = Child()

    class Child {
        var value: String?
    }
}

class To {
    var childValue: String?
}

We want to map the value field in Child class inside the From class to the childValue field in the To class. We will create a mapper with the .. operator.

val mapper = mapper<From, To> {
    From::child..From.Child::value mappedTo To::childValue
}

Let's take it one step further with multi level depth.

class From {
    var grandChildValue: String?
}

class To {
    var child: Child = Child()

    class Child {
        var grandChild: GrandChild = GrandChild()
    }

    class GrandChild {
        var value: String?
    }
}

To access the grand child field we just use the .. operator twice.

val mapper = mapper<From, To> {
    From::grandChildValue mappedTo To::child..To.Child::grandChild..To.GrandChild::value
}

Conditional Mapping

Conditions allow us to add a predicate to a specific field mapping to determine whether this field should be mapped.

Using this feature is as easy as creating a condition.

class NotBlankStringCondition : MappingCondition<String> {
    override fun isValid(context: MappingConditionContext<String>): Boolean {
        return !context.originalValue.isNullOrBlank()
    }
}

And adding the condition to the desired field mapping.

data class SimpleEntity(
    val name: String
)

data class SimpleEntityDisplay(
    val name: String = ""
)

val mapper = mapper<SimpleEntity, SimpleEntityDisplay> {
    SimpleEntity::name mappedTo SimpleEntityDisplay::name withCondition NotBlankStringCondition::class
}

Inline Conditions

Like transformers, conditions can also be added inline using a function.

val mapper = mapper<SimpleEntity, SimpleEntityDisplay> {
    SimpleEntity::name mappedTo SimpleEntityDisplay::name withCondition {
        !it.originalValue.isNullOrBlank()
    }
}

Annotations Mapping

This specific feature receives lots of hate because it breaks the separation of concerns principle. Agreed, this could be an issue in some applications, but in some use cases where all objects are part of the same application it can also be very useful to configure the mapping logic on top of the object. Check out the documentation and decide for yourself.

Conclusion

Object mapping libraries are not the solution for every application. For small, simple applications using boiler-plate mapping functions are more than enough. But, when developing larger, more complex applications, object mapping libraries can take your code to the next level, saving you development and maintenance time. All of these while reducing the amount of boiler-plate code and overall improving the development experience.

On a personal note, I used to work with manual mapping functions and was ok with it. It was "just" some simple lines of code. After upgrading our applications to use object mapping as part of our "boiler-plate free" framework (We will discuss that framework at a later time), I can't go back. Now we spend more time on what's important and interesting and almost no time on boring boiler-plate code.

Oh how naive we were... Of course some truly appreciated our contribution, yet others didn't understand what we are doing, and the crème de la crème just wanted to hate.

I think that in today's world it's hard to put your work out there. When you post your code you feel very vulnerable, a bit naked. It's like your soul is out there for everyone to see and criticize. And don't get me wrong, I think criticism is very important and it's one of the best ways to learn and grow.

On the other hand, it's very easy to completely dismiss someone else's work. It takes almost no time and costs nothing. Also I'm pretty sure some people enjoy it very much.

So when you meet someone like that, how should you react? The first instinct is to ignore or "yell" back, dismiss all criticism completely and move forward. I decided to do otherwise, breathe and count from 1 to 10, slowly, like my mom used to say, read it again, and think where it is coming from.

For example, in one of these scenarios a comment ended with "I would stay away from it" because of a disagreement with the project's coding style. Spoiler: after a few back and forths the conversation ended with "Good luck though!". How about that?

What is crazy and beautiful about programming. We think it is objective, that it is exact science and there is right and wrong. But the truth is, it is also art, and there is a huge subjective component to it.

What I realized is just that my coding philosophy is different. Even though we are still in a disagreement, it was fascinating to learn that there are completely different points of view for doing the exact same thing, and I learned a thing or two in the process.

What is the point of all of this? I think open source is amazing. For some reason we have crazy drive to develop stuff for free just to share it with the world and help others. Nonetheless, I think there is a lot of room for improvement in our discussions and critique. Maybe I'm naive again, however I still believe the community, our community can do better and hope for better communication between us all (and hopefully over a beer).

If you would like to check out our open source library and leave feedback we would appreciate it greatly.

The code was written by an Ukrainian developer I have never met, but loved in my heart. It wasn't the first time I saw a layered application, but it was the best of times. She put everything there, clean controllers layer for the API, service layer to handle process logic, handlers layer to tackle fine grained actions, and of course DAO layer to handle all the DB stuff. From then on, all my future projects followed her beautiful layering and everything was better.

Fast forward a couple of years, my partner and I are now managing the R&D team somewhere (not interesting & not important). Like most teams, we are being swamped with feature requests, and other bullshit the management finds "CRITICAL". Our code-base is still quite organized with the layering we know and love but we start noticing some issues (Add dramatic sound...).

Every time we need to add a new CRUD entity to the system, even the most basic entity for some kind of configuration, we need to add A LOT of boiler plate code. We start by copying an existing entity code, then:

• Update the controller with the new entity. Clean not relevant code.
• Update the service with the new entity, again. Clean not relevant code.
• Update the handler with the new entity, again... Clean not relevant code.
• Update the DAO with the new entity, again....... Clean not relevant code.

The next image is not for the faint of heart. It contains the boiler-plate code required for one the SIMPLEST use cases. The horror!

Just writing it and remembering the old times made me tired. And more important, adding the new entity is the easy part, now we have more code we need to maintain. Our code base is growing and doing system wide changes becomes impossible.

Then one day, we are on a flight to some conference, disconnected from the world and connected to our inner thoughts (no wifi...). My partner, comes to me and say, "What if we develop a generic CRUD framework without boiler-plate, just add an entity and everything else is available in seconds". At first I was skeptical, but he is the smart one between us so I give it another thought and decides to roll with it.

Fast forward again, my partner and his trusted sidekick have kept their promise and delivered their boiler-plate free CRUD framework, after countless of fights and iterations (to this day). I don't want to bore you with too much code, but I just have to show a simple example to honor everything we lost along the way.

@Entity
@Table(name = "example")
@DefaultMappingTarget(ExampleDisplay::class)
class Example(
    @MappedField
    @Column
    var exampleString: String?,

    @MappedField
    @Column
    var exampleBoolean: Boolean?,

    // ...

) : AbstractJpaEntity()

@DefaultMappingTarget(Example::class)
data class ExampleDisplay(
    @MappedField
    var exampleString: String? = null,

    @MappedField
    var exampleBoolean: Boolean? = null,

    // ...

) : AbstractJpaDisplay()

@RestController
@RequestMapping("example")
class ExampleController : BaseCRUDController<Example, ExampleDisplay>() {
}

That's it! Now I know what you are going to tell me. Where are your layers that you couldn't shut up about?! The beautiful thing is, they are all here. Behind the scenes the framework has all the layers and it allows you to use hooks and other mechanisms to add additional logic/frameworks/... Since then, the smart guys have added permissions, security and more on top of the CRUD framework, according to the requirements of each project.

So your next thought is probably "Where do I sign?". Hold your horses for a moment, I know your honest current thought is that I'm stupid and I know nothing, fair. Also I would be happy to give you the CRUD framework but it's not truly open source ready. If you want to take a look anyway it's available here.

Because it's a large framework and it will take time to bring it to the level that we want for an open source release, we decided to start small and separate our object mapping capabilities into a separate library. Of course the hardest part was to find a name for this library but in the end we settled for ShapeShift.

If you are not sure what is object mapping or why you need it you can read more about it here.

If you have any thoughts/comments/disses you would like to throw at me I would be more than happy to connect.

The Experience — 0 Years

The Job — Start of High school

Where do I even begin?

Like I stated in the prologue, I was fascinated with tech from a very young age. What I failed to mention before is that thanks to my loving and understanding parents, my curiosity was not killed in school. That’s mostly thanks to the fact that in my house I could always take any “junk” home to take apart and learn how it ticks.

My first steps

At the age of 13, I was already pretty good on a PC, but haven’t really dived into programming yet. My first real time was when I went to a few VB6 classes and convinced my parents to buy me a book.

For me it was like magic, I could tell the computer what to do and it would do it, as long as I didn’t fuck it up somehow (which happened a lot at that time). I wrote small dumb things, what most would call a “Hello World”.

I’ve been a bad boy

It was around the my second year of High school when I noticed how bad the computer system at school was, how weak the security was, and of course I had to find out if I could beat it. But being the goodie two shoes I was, I was scared to do it, and if it wasn’t for the fact that the guy in charge of said system told me I was wrong (he thought it was secure), I wouldn’t have even tried.

So I did what had to be done, I got hold of *.pwl files (win 98 password files, and yes I am a dinosaur), and found out how to crack them. I wrote my first brute force script and made all the PCs in class run it at night. Luckily for me the password was short and within a few weeks it was cracked.

You have got to start somewhere

Yes I know what you are going to say, what a geek, nerd, … Yes, I spent my free time teaching myself programming. No, I wouldn’t change that. I’m not much of a reader (till this day I can’t read for fun), but somehow going through a programming book was not a chore and I learned a ton. I think the most important part was that it interested me and I was passionate about it.

Today is different, but I highly recommend to anyone to just give it a try, go take a course online, or use YouTube, or even just go though GitHub. Today it’s sooo much easier and more accessible. Who knows you may just find that programming is your dream job.

If you would like to check out our open source library and leave feedback we would appreciate it greatly.

It’s rude not to introduce myself, so I’ll start with a little background, just to get us started. I can call myself a developer for over 15 years now, and a tinkerer for a few more. I started playing with technology from the very young age of 7, when I entered and won a race event where I built a toy solar car. Till today, as a system architect and having some successes (and failures) at tech startups, and running my own dev & consulting business.

What’s the point of all this

As of late, my team and I have released the first of our libraries as open source, this has made me look back at the path that led me here. My main goal is to share that story, the ups and downs (and sometimes sideways) and hopefully it will help someone on their path. I can’t promise it will be interesting all the time (or even pretty), but I hope you will join me for the journey.

I’ll try and keep it brief

I will be writing this as a few different posts, each one not much longer than this one, as I want it to be a quick and fun read. So this is the plan:

1. The story of a Noob — My somewhat humble beginnings
2. The journey begins — Teaching myself in the age before google
3. They don’t call it intelligence for nothing — My military career
4. School can’t teach you everything — My time at Uni and my first freelance work
5. It’s time to settle down — My first real job
6. Good code is like an onion, it has layers — The right way to write code
7. My eureka moment — After writing the same code over and over, something had to change
8. It’s ALIVE — My solution to CRUD boilerplate, and the evolution of the layers
9. Shit just got real — Using what I learned to land a big contract
10. Time to give back — The birth of my first open source

In my research I discovered that it is possible and quite easy to use a git repository for this purpose. Great! However, I feel that did not go far enough for my use. For once, I did not want to have to manually package and index the repository on every chart change and so I decided to expand on it.

The end result

At the end, we should have a Helm repository that updates itself on every change to the git repository structured as follows:

├── master
│   ├── charts/
│   │   └── example/
│   └── sync_repo.sh
└── repo
    ├── example-0.1.0.tgz
    └── index.yaml

The full example is available at my GitHub

The setup

For this, create a new git repository, clone it locally and create a new chart in the charts directory;

mkdir charts
cd charts
helm create example

Once that's done and committed, we will create an orphan branch to store the actual Helm repository. Why an orphan branch? On every commit to the repository, CircleCI will do its own commit to update the Helm repository.

While this step is optional, it prevents developers from having to pull-rebase every time they make a change to the repo, and has the added benefit of keeping the history of the master branch clean. Create an orphan branch called repo as follows:

git checkout --orphan repo
git rm -rf . 
touch index.yaml
git add index.yaml
git commit -m 'Initial Commit'
git push -u origin repo

Now let's return to the master branch. Our next step is to create a script which our CI will use on every commit. The script will package all charts, and re-generate the index.yaml file.

Use your text editor of choice to create sync_repo.sh and add the following to it:

#!/bin/sh
mkdir -p repo
cd repo
helm package ../charts/*
helm repo index .

At last, we add the last piece of this puzzle and integrate GitHub Actions into this process.

Our action will pick off where our shell script left us off; It will clone the separate repo branch and copy the generated files to it, finally committing and pushing back the changes to origin.

For this, we'll create the following action at .github/workflows/sync_repo.yml.

name: Sync Helm Repo
on:
  push:
    branches: [ master ]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          ref: 'master'
          path: 'master'
      - uses: actions/checkout@v2
        with:
          ref: 'repo'
          path: 'repo'   
      - uses: azure/setup-helm@v1
      - name: Sync Helm Repo
        run: cd master && sh sync_repo.sh
      - name: Move repo folder
        run: |
          mv master/repo/* repo/
          cd repo
          git config user.email "idan@elhalwani.com"
          git config user.name "Idan Elhalwani"
          git add -A
          git diff --quiet && git diff --staged --quiet || git commit -m "Update repo [SKIP CI]" -m "${{ github.event.head_commit.message }}"
          git push

Let's break down the action;

# Move our generated files from the master branch folder to the repo branch folder
mv master/repo/* repo/
cd repo
# Configure git for when we're ready to commit
git config user.email "idan@elhalwani.com"
git config user.name "Idan Elhalwani"
# Add all new files and attempt to commit. If there is nothing new, the commit won't go through and nothing will be pushed
git add -A
git diff --quiet && git diff --staged --quiet || git commit -m "Update repo [SKIP CI]" -m "${{ github.event.head_commit.message }}"
git push

Conclusion

At this point, we have a fully functional and automatically updating Helm repository. To use it, get the raw URL for the repo branch.

In our case, we will add the repository as follows:

helm repo add example https://raw.githubusercontent.com/idane/helm-repo-example/repo