First, concurrency safety Golang actually provides a shared memory-based approach in addition to the CSP programming model. For example, there are. sync.Mutex: Mutual exclusion lock sync.WaitGroup: Wait group, wait for all the Goroutines in the group to finish before exiting Atomic operations. For example atomic.AddUint64 (sync/atomic package), thread-safe, no locking required Single instance objects (sync.Once package), which are only initialized once when accessed concurrently by multiple Goroutines The following example
Understand some of the terms used in ICP programming
This article introduces a number of terms that must be understood to program on IC, including Node, Replica, WebAssembly, Canister, Actor, Ledger, Wallet, Principal, and Controller.
This article should be the best article to introduce these terms.
Node Node is the physical or virtual machine node that hosts the IC, which is well understood and needs no introduction.
Replica In IC, a replica is an IC protocol process running on a node.
Kubernetes Containers and image GC Principles Explained
Container GC Exiting containers continue to use system resources, such as storing a lot of data on the filesystem and the CPU and memory that Docker applications use to maintain these containers. Docker itself does not automatically delete exiting containers, so kubelet takes over this responsibility. kubelet container recycling is used to delete exiting containers to save space on nodes and improve performance. While container GC is good for space
memory ballast and gc tuner are history
memory ballast and auto gc tuner are optimizations to solve the problem of Go triggering frequent GCs when memory is not fully utilized, resulting in high CPU usage of GCs.
memory ballast tricks GOGC by allocating a huge object (typically several GB) on the heap, allowing Go to use as much heap space as possible to reduce the frequency of GC triggers.
The auto gc tuner shared by uber later is a bit smarter, setting the program’s memory usage threshold and dynamically setting the GOGC every time a GC is triggered by a callback to the user’s finalizer function during a GC, so that the memory used by the application gradually converges to the target.
Go 1.19 Beta 1 released
Go 1.19 is expected to be released in August 2022, and the beta version of Go 1.19 is now available. Here are the main changes in GO 1.19.
Changes to the language There are only minor changes to the language, with minor corrections to the scope of type parameters in method declarations. Existing programs are not affected.
Memory Model The Go memory model has been modified to be consistent with the memory model used by C, C++, Java, JavaScript, Rust, and Swift.
Template functions in C++
1 2 3 4 5 6 7 8 template<typename T> void Swap(T &a ,T &b) { T temp; temp = a; a = b; b = temp; } When using template functions, the compiler generates the appropriate function definition based on the actual type.
Overloading templates Not all types use the same algorithm, and it is possible to overload template function definitions just like regular functions.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 template<typename T> void Swap(T &a ,T &b); //#1 template<typename T> void Swap(T *a ,T *b,int n);//#2 The last parameter is the specific type int main() { int i =10,j=20; Swap(i,j);//Use #1 const int Lim = 8; int d1[Lim]={0,1,2,3,4,5,6,7}; int d2[Lim]={7,6,5,4,3,2,1,0}; Swap(d1,d2,Lim);//Use #2 } template<typename T> void Swap(T &a ,T &b) { T temp; temp = a; a = b; b = temp; } template<typename T> void Swap(T *a ,T *b,int n) { T temp; for(int i=0;i<n;i++) { temp =a[i]; a[i]=b[i]; b[i]=temp; } } Template Limitations In some cases, the corresponding operation of type T only applies to arrays, and the template function does not work if T is a structure.
The basics of Rust
I. What is Rust Rust is a general-purpose, compiled programming language developed under the leadership of Mozilla. The design guidelines are “safe, concurrent, and practical”, and support functional, concurrent, procedural, and object-oriented programming styles. – Wikipedia
II. Language characteristics Language features
Rust is much closer in deep syntax to languages in the meta-language family like Haskell. Basically, every part of a function body is an expression, even a control flow operator.
Go 1.19 New Features Preview
On May 7, 2022, the Go 1.19 development branch entered the new feature freeze phase, which means that only bugs can be fixed and no new features can be added to Go 1.19. As the previous version of Go 1.18 was delayed by one month due to the introduction of generic changes, this directly led to a shortened development cycle for Go 1.19.
Although the development cycle is nearly a month shorter, Go 1.
Graceful exit mechanism for Pods in Kubernetes
Kubernetes provides a Pod graceful exit mechanism that allows Pods to complete some cleanup before exiting. But if something goes wrong while performing cleanup, will the Pod exit properly? How long does it take to exit? Can the exit time be specified? Does the system have default parameters? There are several details that we should pay attention to, and this article will start from these details to sort out the behavior of Kubernetes components and their parameters in each case.
Multi-platform compilation for Golang
Golang supports cross-compilation for generating executable programs on one platform for another platform.
1. Mac Compile 64-bit executables for Linux, Windows platforms under Mac.
1 2 CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build test.go CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build test.go 2. Linux Compiling 64-bit executable programs for Mac, Windows platforms under Linux.
1 2 CGO_ENABLED=0 GOOS=darwin GOARCH=amd64 go build test.go CGO_ENABLED=0 GOOS=windows GOARCH=amd64 go build test.go 3. Windows Compiling 64-bit executable programs for Mac, Linux platforms under Windows.
Controlling Kubernetes with client-go
Earlier I have built a local k3s cluster of two machines and deployed Crawlab (a distributed crawler management platform developed using Golang) to achieve a local distributed cluster service environment. After the service is up, we need Kubernetes to control the container nodes dynamically, monitor the container running status, and achieve even the expansion. Fortunately, Kubernetes directly provides python and golang clients, which can easily implement Kubernetes API operations. So
Use sync.Pool in Golang to reduce GC pressure
1 Preface In a nutshell: save and reuse temporary objects, reduce memory allocation, and reduce GC pressure.
sync.Pool works roughly as follows.
A local object pool poolLocal is created for each P to minimize concurrency conflicts, taking advantage of GMP features. Each poolLocal has a private object, and access to private objects is given priority to avoid complex logic. Use pin to lock the current P during Get and Put to prevent the goroutine from being preempted and causing program chaos.
Offline deployment of K8s in an intranet ubuntu environment
Recently working in an intranet environment, I needed to build a K8s cluster from scratch and perform a microservice migration. The following table shows the pre-researched scenarios for offline deployment of K8S. Item Language Star Fork Off-line deployment support kops Golang 13.2k 4.1k Not supported kubespray Ansible 11.1k 4.7k Support, you need to build your own installation package kubeasz Ansible 7.2k 2.7k Support, you need to build your own installation
Descheduler Secondary scheduling for a more balanced Kubernetes load
1. Why do you need secondary scheduling The role of the Kubernetes scheduler is to bind Pods to a particular best-of-breed node. In order to do this, the scheduler needs to perform a series of filters and scoring.
Kubernetes scheduling is based on Request, but the actual usage values of each Pod are dynamically changing. After a period of time, the load on the nodes is uneven. Some nodes are overloaded, while others are underused.
How to Renew Kubernetes Certificates
By default, Kubernetes certificates need to be renewed every other year, and the following is a documented certificate renewal process.
1. View Certificate Check the certificate expiration time on the Master node:
1 2 3 4 5 6 7 8 9 10 11 12 13 kubeadm certs check-expiration CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED admin.conf Apr 02, 2023 09:53 UTC 296d no apiserver Apr 02, 2023 09:53 UTC 296d ca no apiserver-kubelet-client Apr 02, 2023 09:53 UTC 296d ca no controller-manager.
4 Ways to View Log File Write Speed in Linux
Sometimes we need to check the growth rate of a file, such as a log file, to get a feel for the load on the system, because in general, the faster the logs are written, the heavier the load on the system.
In this article, we will introduce several ways to view the growth rate of logs in Linux, as follows.
Using dd The first thing to introduce is dd, as the dd command comes with almost all major distributions and requires no additional installation, as follows.
Cross-origin resource sharing(CORS)
Cross-origin resource sharing (CORS) is a mechanism for making restricted resources of a web page accessible to pages from other domains based on HTTP headers. Through this mechanism, pages can freely use images, styles, scripts, iframes, and videos from different sources (cross-origin).
In general, some cross-origin requests (especially ajax) are prohibited by the same-origin policy. CORS defines a way to allow web application servers to perform cross-origin access control, thus enabling secure cross-origin data transfer.
Batch Modify Git Repository Commit Mailboxes
After reinstalling the system a few days ago, I accidentally committed a lot of commits with author "root"<root@localhost> and looked for a way to modify them.
Create a new shell script called fix.sh in the repository root directory and type the following.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 #!/bin/bash git filter-branch --env-filter ' an="$GIT_AUTHOR_NAME" am="$GIT_AUTHOR_EMAIL" cn="$GIT_COMMITTER_NAME" cm="$GIT_COMMITTER_EMAIL" if [ "$GIT_COMMITTER_EMAIL" = "Wrong email" ] then cn="Correct name" cm="Correct email" fi if [ "$GIT_AUTHOR_EMAIL" = "Wrong email" ] then an="Correct name" am="Correct email" fi export GIT_AUTHOR_NAME="$an" export GIT_AUTHOR_EMAIL="$am" export GIT_COMMITTER_NAME="$cn" export GIT_COMMITTER_EMAIL="$cm" ' When finished, run .
Linux Systemd Getting Started
Systemd is a centralized system and setup manager for the Linux computer operating system, including daemons, libraries and applications, developed by Lennart Poettering. The goal was to provide a better framework to represent the dependencies between system services, to enable parallel startup of services during system initialization, and to reduce the system overhead of the shell, eventually replacing the System V and BSD style init programs that are now commonly used.
Microsoft GitHub Announces "Phasing Out" of Atom Code Editor! The focus will shift to VSCode
Are you still using GitHub’s Atom text editor? Well, here’s the bad news - Atom will soon be “retired”!
GitHub Officially Announces the “Retirement” of the Atom Code Editor On June 8, Pacific Time, the official GitHub blog officially posted, “Today, we’re announcing that we’ll be archiving Atom on December 15, 2022, and archiving all projects under the organization.”
GitHub, for its part, says it has decided to “…discontinue Atom to further our commitment to bringing fast and reliable software development to the cloud through Microsoft Visual Studio code and GitHub code spaces.