Introduction to gonew

Recently, Go official blog introduced a new tool called gonew. The tool supports based on go project template clone and create a Go project belongs to you. gonew tool introduced to greatly simplify the creation of Go projects, at the same time, due to the support of custom project templates, but also to improve the standardisation of Go projects. gonew tool has just been put into the Go tools project code repository, is still in the experimental stage, the follow-up may add new features, but the current core features (core functionality) will continue to be retained.

Data replication and partitioning of databases

In distributed databases allowing distributed nodes of the database to have copy backups through replication is mainly for the following purposes: Scalability, because the amount of data carried by a single machine is limited. Fault tolerance and high availability, in a distributed system, single-machine failure is the norm, more redundancy in the event of single-machine failure other machines can take over in a timely manner. Performance, if the user access to the situation of cross-region, can be deployed through the multi-location near the access to reduce the time delay.

How to hide a Linux process

Overview I recently came across a project, libprocesshider, which can be used to hide a Linux process, and out of curiosity, tried it out, and found that it involves a few interesting points, so I’ll document them. Principle We usually check the process by ps command, or when the machine indicator is abnormal, we usually use top command to check the process situation of the system, so if we make these two programs “can’t see” the process that we want to hide, won’t we complete the function of process hiding?

Deeper Understanding of Observation - Principles, back porting and performance

SwiftUI follows the Single Source of Truth principle, where only modifying the state to which a View is subscribed can change the view tree and trigger a re-value of the body, which in turn refreshes the UI. When first released, SwiftUI provided property wrappers such as @State, @ObservedObject, and @EnvironmentObject for state management. In iOS 14, Apple added @StateObject, which completes the state management of SwiftUI by complementing the case where the View holds an instance of a reference type.

Creating a SWAP Partition in Linux

gcc compiles with the error cc: Internal error: Killed (program cc1). This is basically due to insufficient memory. This can be solved by temporarily modifying the swap partition. 1. swap What swap partitions do When a Linux system runs out of physical memory, a portion of the physical memory can be freed for use by currently running programs. The freed space may come from programs that have not operated for a long time, and the freed space is temporarily saved in SWAP space, and then restored from SWAP to memory when those programs want to run.

K8sPrinciples of the Kubernetes Scheduler

kube-scheduler is one of the core components of kubernetes, which is mainly responsible for the scheduling function of the entire cluster resources, according to specific scheduling algorithms and strategies, the pod will be scheduled to the optimal node above the working nodes, so as to make more reasonable and full use of the cluster resources, which is also a very important reason why we choose to use kubernetes. This is a very important reason why we chose to use kubernetes.

The Go Developer's Guide to Using Apache Arrow: Reading and Writing Parquet Files

Apache Arrow is an open, language-independent columnar memory format, and in the previous articles in this series we have focused on memory representation and memory manipulation. But for a database system or big data analytics platform, the data can’t and can’t be kept in memory all the time, and although memory is currently big and cheap enough, its volatility also dictates that we still have to serialise the data and store it on disk or some low-cost storage service (e.

Installing and cleaning the 32-bit runtime environment on 64-bit Linux systems

I previously bought a used Asus RT-AC1900P wireless router with Merlin installed to use as a home gateway. The previous owner used a 2.6.36.4brcmarm kernel. Since it is more than 10 years old, I don’t need to think about the WireGuard module, but I can still compile tunneling modules such as IPIP/GRE, I will write about the compilation process later. Broadcom only has a 32-bit toolchain, and my server is 64-bit Ubuntu, so I can’t run it directly.

std::sort in C++

The current implementation of std::sort in C++ uses Introsort and Insertion Sort together. When the total number of elements is less than a threshold (usually 16), then sort will only use insertion sort. In Introsort, when a certain depth is reached (the logarithmic value of the number of elements), the sort is converted to a heap sort. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 /// This is a helper function.

Real World Go Design Patterns - Factory Patterns

Factory pattern is a creation pattern, is used to create new objects of a design pattern, it can be a complex object construction process abstracted out (abstract class), so that the abstract process of different implementation methods can be constructed to different representations of the object (attribute). The 23 design patterns include the Abstract Factory pattern, the Factory Method pattern, and others have been summarised as the Simple Factory pattern. This factory relies heavily on interfaces, abstract classes and concrete class implementations.

New Network Performance Optimisation Technique - BIG TCP

I was looking through the Cilium Release Blog and found a kernel technology called Big TCP, which was just merged into the kernel around February this year. The introduction said that it is specially designed for high-speed networks with more than 100Gbit, and it can significantly reduce the latency while increasing the throughput by 50%, so I went to learn more about this technology. High-speed network performance bottlenecks and misconceptions When it comes to network performance, many people will naturally associate it with NIC performance, but those who have done network performance optimisation will know that the bottleneck is more on the CPU.

Performance issues with uniform_int_distribution in libc++

Context Some time ago, I found that the performance difference between a piece of C++ code compiled under macOS with the included Clang and GCC with Homebrew was nearly an order of magnitude, and here are the runtimes: GCC-13 Homebrew: 300 Apple Clang: 2170 The code is as follows: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 #include <algorithm> #include <array> #include <chrono> #include <iostream> #include <memory> #include <random> constexpr size_t FILE_N = 1e8; constexpr size_t DATA_R = (1 << 23); int main() { for (size_t i = 0; i < 4; ++i) { auto start = std::chrono::high_resolution_clock::now(); std::random_device rd; std::mt19937_64 gen(rd()); std::uniform_int_distribution<> dis(0, DATA_R); std::shared_ptr<std::array<size_t, FILE_N>> data( new std::array<size_t, FILE_N>()); std::generate(data->begin(), data->end(), [&dis, &gen]() { return dis(gen); }); auto end = std::chrono::high_resolution_clock::now(); std::cout << std::chrono::duration_cast<std::chrono::milliseconds>(end - start) .

Mock in Go Unit Testing

I’ve been developing in Go since 2018, and it’s been five years since then. I didn’t understand the Go ecosystem and lacked tools, so I chose a very simple but usable solution to do unit testing Mock, and then along with the development of the business, the R&D team was split into different groups. Recently, some teams are working on new projects, but the unit tests are still using the same temporary solution from five years ago.

Replace the map implementation in the golang standard library, SwissTable is faster?

In the latest instalment of the Go dev team’s weekly meeting, they discuss Google3 re-evaluating the performance of SwissTable, and that it might be worth following up with a replacement for the standard library map. SwissTable [MichaelP]: Google3 may want to do some benchmarking on this. Maps are used heavily in google3. This may be of value to us. So this may be worth investing in. Some people may not understand SwissTable (or called Swiss Tables) is what Google3 and what?

Using testify to assist with Go testing

Although I can not be regarded as a Go standard library “purist”, but in the test is still mostly based on the standard library testing package and go test framework, in addition to the need to mock time, basically did not use a third-party Go testing framework. Recently looked at the Apache arrow code, found that arrow’s Go implementation of the use of testify project organization and auxiliary testing.

Protobuf Guide

Let’s start with a brief introduction to Protocol Buffers (protobuf), which is a data serialization protocol developed by Google (similar to XML, JSON). It has many advantages, but also has some disadvantages that need to be noted: Advantages: High efficiency: Protobuf stores data in binary format, which is more compact and faster than text formats such as XML and JSON. Serialization and deserialization are also fast. Cross-language support: Protobuf supports a variety of programming languages, including C++, Java, Python and so on.

Compare Postgres and MySQL

According to 2023 Stack Overflow research (https://survey.stackoverflow.co/2023/), Postgres has replaced MySQL as the most admired, desired database. As Postgres gains momentum, the choice between Postgres and MySQL becomes more difficult. If you look at the number of installations, MySQL is probably still the largest open source database in the world. Postgres prides itself on being the world’s most advanced open source relational database. Bytebase works closely with a variety of databases and their derivatives because of the need to integrate with them, and Google Cloud SQL (https://cloud.

Scheme language

I’ve been reading SICP (Structure and Interpretation of Computer Programs) recently, and I feel that I’ve benefited a lot from it, so I’m going to summarize and share some of what I’ve learned.SICP is very comprehensive, and the content includes functional programming, hierarchical and abstraction of data structures, object-oriented, infinite streams, meta-cyclic interpreters, inert value, non-deterministic programming, logic programming, assembly language and machines, compilation principles, and more. The theme of this series of articles is continuation, and the main topics may include:

The Go Developer's Guide to Using Apache Arrow: Advanced Data Structures

After studying the previous two articles, we know about the various Arrow array types and their layouts in memory, and we understand some of the mechanisms and usage principles of the Go arrow implementation for memory management. Arrow’s array type is just a fixed-length sequence of values of the same type. In practice, the array type more often than not serves as a base type, and we need more advanced data structures that have the ability to combine base types.

The right way to destroy Linux threads

In the Linux system, threads are lightweight execution units, the correct destruction of threads can avoid memory leaks and other problems, Linux threads have joinable and detached two kinds of properties. In my previous article, I covered some historical background of Linux threads and the NPTL thread model. In this article, the correct way to destroy Linux threads will be explained with the above knowledge in mind. In Linux, threads are lightweight execution units, and destroying them properly can avoid memory leaks and other problems.