I love Rust Procedural Macros

Requirement Today I came across a requirement to generate a random instance of an enumeration type. Unlike Python, this is not as convenient and requires a specific Trait implementation with Rust. The simplest idea is to number the different members of the enum type, generate a random number, instantiate the corresponding member, and if the member has data, recursively generate that data at random. 1 2 3 4 5 6

TUN Mode

TUN is a Layer 3 virtual network device provided by the kernel and implemented by software to replace the real hardware. It is equivalent to opening a port at the Layer 3 (network layer) location of the system network stack and handing over the eligible (route matching) Layer 3 packets to the corresponding user space software for processing, and the user space software can also inject packets into the system network stack through the TUN device.

Manually implement zero-overhead async trait in Rust with GAT

In this article, we will explain how to implement a zero-overhead async trait in Rust using GAT, using a series of RocksDB-like iterators as an example. The code in this article requires nightly Rust to compile. We will implement two iterators. TestIterator: Produces an ordered sequence of KVs. ConcatIterator: stitch together sequences of multiple iterators. Example: TestIterator::new(0, 5) will produce the following sequence. 1 2 3 4 5 key_00000 -> value_00000 key_00001 -> value_00001 key_00002 -> value_00002 key_00003 -> value_00003 key_00004 -> value_00004 ConcatIterator::new(vec!

Go container package

Java has a rich set of built-in container classes, and different containers are used to handle various business scenarios. Although Go has many similarities with Java in language design, it doesn’t support many container data structures natively, only map and slice. The standard library’s container package extends the container data structure to support three containers: Heap, LinkedList and Circular List. Containers Familiarity with C++ and Java should give you a

Optimization of Rust Enum Layout

I learned a little bit about Rust Enum today, and before you start reading, you can guess what the output of the following Rust code is on a common 64 bit machine. 1 2 3 4 5 6 7 8 struct A (i64, i8); struct B (i64, i8, bool); fn main() { dbg!(std::mem::size_of::<A>()); dbg!(std::mem::size_of::<Option<A>>()); dbg!(std::mem::size_of::<B>()); dbg!(std::mem::size_of::<Option<B>>()); } In this Rust Playground you can see the result. Rust enum is essentially

Memory-related fields in the Linux top command

The well-known top command in Linux lists the utilization of system resources by each process. There are several fields VIRT , RES , SHR , CODE , DATA and so on, which describe the memory usage of the processes. (The last two are not displayed by default, you need to press the F key to bring up the relevant fields). But what do they actually mean? Documentation for the top command Execute man top to view the top command’s manual.

Remove duplicates from Hive SQL query results

The problem encountered in this article is to de-duplicate the data from Hive SQL SELECT with certain columns as key. The following is a step-by-step discussion. DISTINCT When it comes to de-duplication, DISTINCT naturally comes to mind. But in Hive SQL, it has two problems. DISTINCT will use all the columns from SELECT as keys for de-duplication. That is, as long as one column has different data, DISTINCT will consider it different and keep it.

Rust Announces 2024 Roadmap, Will Lower Learning Barriers

As the Rust language continues to evolve, it’s natural to need plans for where the language will go next. So the Rust language design team (Rust Lang Team) recently published a roadmap for updates to the Rust language in 2024 on the official blog, providing A look at the future of Rust. It is important to note that it is still early in 2024, so the Rust roadmap may change over time on this basis.

Does Go limit the Committer community? Every change needs to be reviewed by 2 Google employees

Google engineer Russ Cox announced in a mailing list to golang-dev on Monday that the company has decided that every change to the Go programming language will need to be reviewed by two Google employees before it can be released to users (up from one). The company has decided that every change to the Go programming language will have to be reviewed by two Google employees (up from one) before it can be released to users.

Mounting/unmounting a disk on Linux

View the disks on your machine fdisk is a tool used on Linux to manipulate disk partition tables. Using the fdisk -l command, you can list all the disks and their partitions recognized by the system. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Disk /dev/sda: 599.1 GB, 599051206656 bytes, 1170021888 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 262144 bytes Disk label type: # masked Disk identifier: # masked Device Boot Start End Blocks Id System /dev/sda1 2048 204802047 102400000 83 Linux /dev/sda2 204802048 213190655 4194304 82 Linux swap / Solaris /dev/sda3 * 213190656 1170020351 478414848 83 Linux Disk /dev/sdb: 6118.

Data Constraint Language CUE

CUE is an open source data constraint language designed to handle configurations, structures, data and execute them. Most people start using CUE because they want to do data validation and generate configurations. Then they go on to use CUE for data templates, runtime input validation, code generation, scripting, pipline, and more. First you need to understand that CUE is not a programming language in the usual sense, it is a Turing non-complete programming language.

DeepL Deception Tactics in Api Design

This blog post should have been published last year when I finished reversing the DeepL client, but I delayed starting it because I was afraid that once the relevant details were made public, they would be widely adopted by others and blocked by the DeepL official. Some time ago I released DeepL Docker image of Free Api, and also made the binary public on GitHub. I believe DeepL will be in action soon, so I think it’s time to make the details public.

Unix domain socket

Introduction Stream Socket and Datagram Socket are usually based on tcp and udp protocols for data transfer respectively. Both sockets have one common feature, that is, they require an IP address and a port to establish a connection between the client and the server. This article explains a special socket that does not require a traditional IP address and port, but uses the file system for data interaction between programs, and can only be used on unix systems.

How to shutdown containers gracefully

I recently read through the official Docker Docker Reference documentation again and found that There are still a lot of details to dig into. For writing Dockerfile, most of the time there is no big problem as long as you follow it. But it would be more interesting to understand more deeply. To talk about how to gracefully close the container, we have to mention the idea of Signal, and

Constructing HTTP request URLs in the Go language more standardly and securely

In reviewing the code that my colleagues have written to initiate external HTTP requests, I have rarely seen a more standard (or correct and safe) way to construct the URL of an HTTP request. What is standard practice, if you ask me? I probably can’t tell you exactly. However, I have a few simple criteria of my own. Protocol: Does the request work without http://? Path: Does it stitch out

Getting started with Dagger

Recently, Docker founder Solomon Hykes announced the launch of a new product Dagger, a new DevOps platform designed to solve some of the problems in the DevOps process for developers. Dagger has raised $20 million in Series A funding led by Redpoint Ventures, with participation from Nat Fireman, former CEO of GitHub, Brian Stevens, former CTO of Red Hat, and Ellan Pao, former CEO of Reddit. Dagger wants to help

How does Go respond to supply chain attacks?

Go’s official blog introduction has their mitigation measures for supply chain attacks. Go’s toolchain and design are said to include attack risk mitigation considerations at all stages. All builds are “locked” External changes (such as releasing a new version of a dependency) do not affect Go builds. Unlike the configuration files used by most other package managers, Go modules do not have a separate list of constraints and lock files for locking specific versions.

SSL Performance Testing for Nginx

Although the title says SSL performance testing, it doesn’t really matter if it’s SSL or not, it’s just that the configuration of the test is different. We should focus more on the general principles of performance testing than on the testing methodology of a specific metric. The specific test metrics may vary with the test business and test environment, but the core test methodology is the same. Since I am

Enabling Pod Security Policies for Kubernetes Clustering

The demos and examples in this article are validated in the v1.18.17 cluster. Pod Security Policies Pod Security Policies (hereafter referred to as psp or pod security policies) is a cluster-level global resource that provides fine-grained authorization control over pod creation and updates. Specifically, a psp object defines a set of security conditions that must be met by a pod’s spec field, along with the default values of the applicable

Setting up the shared memory of a kubernetes Pod

Users can use shared memory to do some for communication (vs golang’s “shared memory by communication”). On a kvm or physical machine, the size of shared memory available to the user is about half of the total memory. Here’s how shared memory looks like on my pve machine, /dev/shm is the size of the shared memory. 1 2 3 4 5 6 7 8 root@pve:~# free -h total used free shared buff/cache available Mem: 47Gi 33Gi 3.