How to use higher-order function programming to improve the simplicity of your code

Functions are first-class citizens of Go language. This article adopts a high-order function approach to abstract the query conditions for querying DB using gorm, abstracting various complex combinations of queries for multiple tables into a unified method and a configuration class, improving the simplicity and elegance of the code, and at the same time can improve the efficiency of developers.

Background

There is a DB table, and the business needs to do filtering queries by different fields in this table, which is a very common requirement, and I believe this requirement is inseparable for everyone who does business development. For example, we have a table that stores user information, and the simplified table structure is as follows.

CREATE TABLE `user_info` (
  `id` bigint unsigned NOT NULL AUTO_INCREMENT COMMENT '自增主键',
  `user_id` bigint NOT NULL COMMENT '用户id',
  `user_name` varchar NOT NULL COMMENT '用户姓名',
  `role` int NOT NULL DEFAULT '0' COMMENT '角色',
  `status` int NOT NULL DEFAULT '0' COMMENT '状态',
  PRIMARY KEY (`id`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='用户信息表';

This table has several key fields, user_id, user_name, role, status. if we want to do filtering by user_id, then we are generally in the dao layer to write a method like this (for the sake of concise model code, all sample code here omitted the error handling part).

func GetUserInfoByUid(ctx context.Context, userID int64) ([]*resource.UserInfo) {
   db := GetDB(ctx)
   db = db.Table(resource.UserInfo{}.TableName())
   var infos []*resource.UserInfo
   db = db.Where("user_id = ?", userID)
   db.Find(&infos)
   return infos
}

If the business needs to query by user_name, then we need to write a similar method to query by user_name.

func GetUserInfoByName(ctx context.Context, name string) ([]*resource.UserInfo) {
   db := GetDB(ctx)
   db = db.Table(resource.UserInfo{}.TableName())
   var infos []*resource.UserInfo
   db = db.Where("user_name = ?", name)
   db.Find(&infos)
   return infos
}

As you can see, the code of the two methods are extremely similar, and if we need to query by role or status, then we have to come up with several more methods, resulting in a lot of similar methods. Of course, it is easy to think that we can solve this problem by using one method with a few more inputs, so we merged the above two methods into the following method, which can support filtering queries by multiple fields .

func GetUserInfo(ctx context.Context, userID int64, name string, role int, status int) ([]*resource.UserInfo) {
   db := GetDB(ctx)
   db = db.Table(resource.UserInfo{}.TableName())
   var infos []*resource.UserInfo
   if userID > 0 {
      db = db.Where("user_id = ?", userID)
   }
   if name != "" {
      db = db.Where("user_name = ?", name)
   }
   if role > 0 {
      db = db.Where("role = ?", role)
   }
   if status > 0 {
      db = db.Where("status = ?", status)
   }
   db.Find(&infos)
   return infos
}

Accordingly, the code that calls this method needs to be changed as well.

//只根据UserID查询
infos := GetUserInfo(ctx, userID, "", 0, 0)
//只根据UserName查询
infos := GetUserInfo(ctx, 0, name, 0, 0)
//只根据Role查询
infos := GetUserInfo(ctx, 0, "", role, 0)
//只根据Status查询
infos := GetUserInfo(ctx, 0, "", 0, status)

This kind of code can be very difficult for both those who write it and those who read it. We’ve only listed four parameters here, so imagine what this code would look like if there were a dozen to 20 fields in the table that needed to be filtered. First, the GetUserInfo method itself has a lot of parameters and is full of ! = 0 and ! = "" judgments, and it is important to note that 0 must not be a valid value for the field, otherwise ! = 0 would be a problem. Secondly, as the caller, it is clear that only based on a field filter query, but had to fill a 0 or "" for other parameters to occupy, and the caller should be particularly careful, because if you are not careful, you may fill the role to the status position, because they are the same type, the compiler will not check for any errors, it is easy to make business bugs.

Solution

If there are segments to solve this kind of problem, then the above writeup can only be considered bronze, next we look at silver, gold and king.

Beginner

A more common solution to this problem is to create a new structure, put the various query fields in this structure, and then pass this structure as an input to the query method in the dao layer. And where the dao method is called, the structure containing the different fields is constructed according to the respective needs. In this example, we can build a UserInfo structure as follows.

type UserInfo struct {
   UserID int64
   Name string
   Role int32
   Status int32
}

Passing UserInfo as an input to the GetUserInfo method makes the GetUserInfo method look like this.

func GetUserInfo(ctx context.Context, info *UserInfo) ([]*resource.UserInfo) {
   db := GetDB(ctx)
   db = db.Table(resource.UserInfo{}.TableName())
   var infos []*resource.UserInfo
   if info.UserID > 0 {
      db = db.Where("user_id = ?", info.UserID)
   }
   if info.Name != "" {
      db = db.Where("user_name = ?", info.Name)
   }
   if info.Role > 0 {
      db = db.Where("role = ?", info.Role)
   }
   if info.Status > 0 {
      db = db.Where("status = ?", info.Status)
   }
   db.Find(&infos)
   return infos
}

Accordingly, the code that calls this method needs to be changed.

//只根据userD查询
info := &UserInfo{
   UserID: userID,
}
infos := GetUserInfo(ctx, info)
//只根据name查询
info := &UserInfo{
   Name: name,
}
infos := GetUserInfo(ctx, info)

This code is actually much better than the initial approach, at least the dao layer method has been changed from many inputs to one and the caller’s code can build parameters according to their needs without the need for many null placeholders. But the problem is also obvious: there are still a lot of null judgments, and a redundant structure is introduced. It would be a shame if we ended there.

In addition, if we extend the business scenario, we use not equal-value queries, but multiple-value queries or interval queries, such as the query status in (a, b), then how to extend the above code? Is it necessary to introduce a method, the method is tedious aside, what is the name of the method will let us tangle for a long time; perhaps we can try to expand each parameter from a single value into an array, and then assign the value of the place from = to in () way, all the parameters of the query use in obviously not so friendly to performance.

Advanced

Next we look at the golden solution. In the above method, we introduced a redundant structure and couldn’t avoid doing a lot of null assignment in the dao layer methods. So can we not introduce the redundant structure UserInfo and avoid these ugly null assignments as well? The answer is yes, functional programming can solve this problem very well, first we need to define a function type.

`1`	`type Option func(*gorm.DB)`

Define Option to be a function whose input type is *gorm.DB and whose return value is null.

Then define a function for each field in the DB table that needs to be filtered for a query, and assign a value to that field, like the following.

func UserID(userID int64) Option {
   return func(db *gorm.DB) {
      db.Where("`user_id` = ?", userID)
   }
}
func UserName(name string) Option {
   return func(db *gorm.DB) {
      db.Where("`user_name` = ?", name)
   }
}
func Role(role int32) Option {
   return func(db *gorm.DB) {
      db.Where("`role` = ?", role)
   }
}
func Status(status int32) Option {
   return func(db *gorm.DB) {
      db.Where("`status` = ?", status)
   }
}

In the above set of code, the input is a filter value of a field that returns an Option function, and the function’s function is to assign the input to the current [db *gorm.DB] object. This is what we mentioned at the beginning of the article higher-order functions, not unlike our ordinary functions, which return a simple type of value or a wrapped type of structure, this higher-order function returns a function with a certain function. Here more than one sentence, although the go language well supported functional programming, but due to its currently lack of support for generic , resulting in the use of higher-order function programming does not bring more convenience to developers, so in the usual business code to write higher-order functions or slightly rare. And developers familiar with JAVA know that JAVA Map, Reduce, Filter and other higher-order functions are very comfortable to use.

Well, with this set of functions, let’s look at the dao layer of the query method how to write.

func GetUserInfo(ctx context.Context, options ...func(option *gorm.DB)) ([]*resource.UserInfo) {
   db := GetDB(ctx)
   db = db.Table(resource.UserInfo{}.TableName())
   for _, option := range options {
      option(db)
   }
   var infos []*resource.UserInfo
   db.Find(&infos)
   return infos
}

By comparing the method with the one at the beginning, you can see that the method’s input has been changed from a number of different types of parameters to a set of functions of the same type, so when dealing with these parameters, there is no need to judge the null one by one, but directly use a for loop to get it done, which is much more concise than before.

So how to write the code to call the method, here it is given directly.

//只使用userID查询
infos := GetUserInfo(ctx, UserID(userID))
//只使用userName查询
infos := GetUserInfo(ctx, UserName(name))
//使用role和status同时查询
infos := GetUserInfo(ctx, Role(role), Status(status))

Whether we use any single parameter or a combination of multiple parameters to query, we can write it as we like, without paying attention to the order of the parameters, which is concise and clear, and the readability is also very good.

Consider the extension scenario mentioned above. If we need a multi-value query, such as querying multiple status, then we just need to add a small function to Option.

func StatusIn(status []int32) Option {
   return func(db *gorm.DB) {
      db.Where("`status` in ?", status)
   }
}

The same is true for other fields or equivalent queries, and the simplicity of the code speaks for itself.

The ultimate

If you can optimize to the above advanced stage, it’s already very clean, and it’s perfectly fine if you stop there. But if you want to go further and pursue the ultimate, then please read on!

In the above method, we have already solved the problem of cumbersome code for a table with multiple field combinations through higher-order functions, but for different table queries, we still have to write a query method for each table, so is there room for further optimization? We find that this set of higher-order functions defined in Option has nothing to do with a table at all, but simply assigns values to gorm.DB. So if we have multiple tables, each with common fields like user_id, is_deleted, create_time, and update_time, then we don’t need to define them all over again, just one in Option, and we can reuse them for each table query. Thinking further, we find that Option maintains some silly code that we don’t need to write manually each time, we can use script generation to scan through the DB tables and generate an Equal method, In method, Greater method, Less method for each non-repeating field, which can solve all the tables to do equal-value queries, multi-value queries, interval queries by different fields This can solve the problem of equal-value queries, multi-value queries, and interval queries in all tables by different fields.

After solving the problem of Option, for each table of various combinations of queries, you only need to write a very simple Get method, for the convenience of seeing, we post here again.

func GetUserInfo(ctx context.Context, options ...func(option *gorm.DB)) ([]*resource.UserInfo) {
   db := GetDB(ctx)
   db = db.Table(resource.UserInfo{}.TableName())
   for _, option := range options {
      option(db)
   }
   var infos []*resource.UserInfo
   db.Find(&infos)
   return infos
}

The above query method is written for the user_info table, if there are other tables, we need to write a Get method similar to this one for each table. If we take a closer look at the Get method for each table, we will see that there are two differences between these methods.

The return value type is different.
TableName is not the same.

If we can solve these two problems, then we can solve all table queries using one method. First of all, for the first point of inconsistent return values, we can refer to json.unmarshal and pass in the return type as a parameter, because the pointer type is passed in, so we don’t need to give the return value again; and for the problem of inconsistent tableName, we can actually add an Option method in the same way as above for different parameters to to solve the problem.

func TableName(tableName string) Option {
   return func(db *gorm.DB) {
      db.Table(tableName)
   }
}

After this transformation, our dao-level query method looks like this.

func GetRecord(ctx context.Context, in interface{}, options ...func(option *gorm.DB)) {
   db := GetDB(ctx)
   for _, option := range options {
      option(db)
   }
   db.Find(in)
   return
}

Note that we have changed the method name from GetUserInfo to GetRecord because this method supports queries not only for the user_info table, but for all tables in a library. This means that instead of having one class for each table and many query methods under each class, we now have One method for all queries for all tables.

Then let’s see how the code to call this method is written.

1
2
3

//根据userID和userName查询
var infos []*resource.UserInfo
GetRecord(ctx, &infos, TableName(resource.UserInfo{}.TableName()), UserID(userID), UserName(name))

The example of querying the user_info table is still given here, specifying the tableName and return type in the call.

After this transformation, we end up with a simple method [GetRecord] + an auto-generatable configuration class [Option] for multiple combinations of queries for all tables in a library. The simplicity and elegance of the code has been improved a bit more. The downside is that the query method is called with two extra parameters, one for the return value variable and one for the tableName, which is a bit less aesthetically pleasing.

Summary

The abstraction of the grom query conditions here greatly simplifies the writing of the DB combination query and improves the simplicity of the code. For other update, insert, delete operations, you can also borrow this idea to do a certain degree of simplification, because of space relations we do not repeat here.

Table of Contents