Duff’s Device is probably one of the most puzzling pieces of C code to date, and Duff shows us the incredible properties of the switch statement. Understanding Duff’s Device helps us implement a plain coroutine. Before we start introducing the Duff device, we need to understand the concept called loop unrolling. loop unrolling is an optimization method to speed up the execution of a program at the expense of its
The principle of RxJava thread switching
RxJava is designed on the basis of chained calls, and by setting different schedulers, you can flexibly switch between threads and execute the corresponding Task. Let’s take a look at how this switching pattern is implemented. Scheduler Scheduler is the abstract parent class of all RxJava schedulers, and subclasses need to override its createWorker() to return a Worker instance that accepts and executes a Task; it can also override its
The principle of RxJava chain calls
RxJava uses a chain call design similar to the Stream API , providing filter, map, observeOn and other common operators . Unlike the Builder pattern, which does not require any order of method calls, RxJava’s operator calls need to maintain sequential relationships. A typical chain call scenario is as follows.
1 2 3 4 5 6 Observable .create(...) .filter(...) .map(...) .observeOn(...) .subscribe(...) The chain call is triggered from subscribe() and we look at the corresponding source code.
Use BPF to increase the throughput of Go network programs by 8 times
The classic bpf (classical Berkeley Packet Filter) is a technique that works very well to improve performance in some special Go underlying network programming situations.
Background I have previously developed a Go UDP application where the client and server communicate via a raw socket via a UDP program. The purpose of the program is rather specific, so I’ll present a simple program as an example here.
Actually, I’m not being strict when I say I’m using the rawsocket approach.
go-querystring: a tool for converting structs to URL query strings
Requirement Scenario When a backend service calls a third-party API, a common requirement is to build a URL query string. net/url is available in the go standard package to solve this problem, url.Values is essentially a map[string][] string, and provides a series of methods (Add, Del, Set) to manipulate the parameters, and eventually converts the map to a URL query string via the Encode() method.
However, there is some repetitive work involved, such as.
Using worktree to maintain multiple branches
The git worktree is not technically a new feature, it has been available for several years and was introduced in the 2.5 release in July 2015. A Worktree is multiple workspaces linked to a unified repository. A git repository can support multiple worktrees, corresponding to different branches.
I saw a tweet from Guido van Rossum on Twitter about why he didn’t know about the git worktree usage before. This piqued my curiosity, as I don’t usually encounter scenarios where I need to use this command in my work.
Calling C code in Go
In addition to its compact syntax and good concurrency support, Go has the advantage of being able to call C code. You can write C code directly in Go source code, or you can refer to external libraries in C. This allows you to rewrite the language where there are performance bottlenecks, or where certain features are missing in Go and third parties but are readily available in C. This
The difference between the Git commands reset and revert
In team development, using Git as a versioning tool makes it easy to collaborate with multiple people to manage parallel development. However, if you or someone else’s commit contaminates a remote branch, you need to restore the remote code. Git provides two commands to do this: reset and revert. These two commands have very different effects, so if you’re not sure how they work, you’ll need to understand them to
Using pprof to analyze and optimize Go programs
In a production environment, Go programs may occasionally experience CPU spikes, and the monitoring panel can’t see what’s causing the spike, except in cases of high concurrency at certain times. The Go language natively provides the tool pprof, which is explained by Google as a tool for visualizing and analyzing data.
By using Go pprof, you can profile the CPU performance, memory usage, Goroutine wait share resource, and mutex lock of your program.
How to get the first n rows of data for each key from a single table
Recently, the project needs to implement version control of single data, so there is a table (foo) dedicated to storing key data and another table (bar) dedicated to storing Data data, how to get the latest version of all the key data in the bar table? Let’s take a look at the schema example.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 -- foo table DROP TABLE IF EXISTS "foo"; CREATE TABLE `foo` ( `id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `name` TEXT NULL, `key` TEXT NULL, `created_at` DATETIME NULL, `updated_at` DATETIME NULL ); -- bar table CREATE TABLE `bar` ( `id` INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, `foo_id` INTEGER NULL, `is_deleted` INTEGER NULL, `timestamp` INTEGER NULL, `created_at` DATETIME NULL, `data` TEXT NULL, `memo` TEXT NULL ) The name + key in the foo table is a unique value, so it will be a one-to-many state, and a key will correspond to multiple sets of data in the bar.
Timing of using Go Channel and Goroutine
Channel Basics Channels are divided into read and write. If you use a lot of channels in the real world, please note that reading and writing channels anywhere in the code may cause different situations. Therefore, to avoid abusing channels in the team, I usually limit the situations in which I can only write and in which I can only read. If you mix them up, it will be very difficult to debug them, and also very difficult for the reviewer to read and understand.
Go Modules How to Use Private GIT Repository?
Suppose you have a private Git Repository in your project, how do you solve it? Right now go mod defaults to proxy.golang.org to grab the latest data, but if you want to grab the private one, you need to do it some other way:
1 go env -w GOPRIVATE=github.com/appleboy The above represents the go command that tells you to read github.com/appleboy as soon as you encounter it, no need to go through the Proxy process.
Want to learn k8s but don't have an environment? Build one easily with minikube
Kubernetes (often referred to as K8s) is an open source system for automating the deployment, scaling, and management of “containerized applications”. The system was designed by Google and donated to the Cloud Native Computing Foundation (now part of the Linux Foundation) for use.
It is designed to provide “a platform for automating the deployment, scaling, and running of application containers across host clusters”. It supports a range of container tools, including Docker and others.
A Loki built on top of ClickHouse - cLoki
cLoki is a more flexible Loki-compatible LogQL API built on top of ClickHouse.
Built-in Explore UI and LogQL CLI for querying and extracting data. Native Grafana and LogQL API for querying, processing, ingesting, tracking, and alerting. Powerful pipeline to dynamically search, filter, and extract data from logs, events, and more. Ingestion and push APIs compatible with LogQL, PromQL, InfluxDB, Elastic, etc. Works with agents such as Promtail, Grafana-Agent, Vector, Logstash, Telegraf, etc.
Finish the Job in the Worker before stopping the Go service
When dealing with Http Handler or closing a database connection, etc., suppose there are multiple Jobs in a service with a live worker, how do I wait until all the Jobs are finished before shutting down and deleting the service (using Docker)? Here is the whole operation process:
Problems encountered When the service is shut down or forced to use ctrl + c to stop, the service should wait until all workers have completed all jobs.
Accelerating Image Compilation with Docker BuildKit
Now the deployment of the compilation process will certainly use Docker, regardless of testing and deployment are implemented in Docker as far as possible, to achieve environmental isolation. But how to shorten Docker in the compilation of Image time, this is a problem, this article to introduce you to an experimental feature is BuildKit.
Prepare beforehand Since BuildKit is an experimental feature, the default installation of Docker will not enable this feature.
Context Usage Scenarios and Introduction in Golang
Context was officially included in the official standard library in Go language version 1.7. Why do we introduce the use of context today? The reason is simple. When learning Go for the first time and writing APIs, you will often see that the first parameter in the http handler is ctx context.Context, and what exactly is the purpose and meaning of this context used here? This article is to bring you to understand what context is and how it is used.
Go Modcache
I believe that all developers writing Go language projects must be using Go module now, and Go module files are written in /go/pkg/mod directory by default, so when you want to string CI/CD process, because it is not under the project path, each container cannot share the /go/pkg/mod path, resulting in repeated downloads of third-party packages. In fact, the cross-container solution can be solved by Drone’s Temporary Volumes, but eventually, when you want to finish the compilation process, you can pack the final mod directory and save it for the next CI/CD deployment process, and if you can change the /go/pkg/mod path, you can dynamically adjust the directory structure.
Byte Order Exploration: Big End vs. Small End
Today we talk about an important computer concept that you may have heard of, but rarely delve into, and that is byte order (Endianness).
1. Concepts Byte order refers to the order in which multiple bytes of data are arranged in memory . This is rather abstract, but it is well understood using a graphical explanation.
Memory is like a row of rooms, and each byte is a room. Each room has a door number (memory address), starting from 0, then 1, 2 ……
Advantages of GraphQL over RESTful API
As a web engineer, you can’t afford not to know what GraphQL is. It is an API Query syntax for front-end and back-end communication, which significantly improves the cooperation model between front-end and back-end, and this article will introduce to you why you should learn GraphQL, and organize the three major advantages of GraphQL, so that you can understand what is different from traditional RESTful API. Of course, we are not telling developers to abandon RESTful APIs, but to decide on different technology stacks depending on the project, for example, service to service calls are not suitable to use GraphQL, but to use the lighter RESTful API or gRPC.