I previously bought a used Asus RT-AC1900P wireless router with Merlin installed to use as a home gateway. The previous owner used a 2.6.36.4brcmarm kernel. Since it is more than 10 years old, I don’t need to think about the WireGuard module, but I can still compile tunneling modules such as IPIP/GRE, I will write about the compilation process later. Broadcom only has a 32-bit toolchain, and my server is 64-bit Ubuntu, so I can’t run it directly. Today I’m going to share how to install and clean up a 32-bit runtime environment on a 64-bit Linux system.

Running 32-bit programs in a pure 64-bit Linux environment may encounter errors.

1
2
$ ./arm-none-eabi-gcc --version
-bash: ./arm-none-eabi-gcc: No such file or directory

The executable is there, but bash says it doesn’t exist. Checking the file information confirms that it is a 32-bit application:

1
2
3
$ file ./arm-none-eabi-gcc
./arm-none-eabi-gcc: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-l
inux.so.2, for GNU/Linux 2.6.8, stripped

So why is it “possible” to get an error? You can try it out. If it’s a 32-bit programme compiled in Go, it’s likely to work fine. The difference is that Go programs are statically linked by default, and do not rely on dynamic link libraries installed on the system. And 64-bit CPUs inherently support running 32-bit instructions, so most Go programs will run directly.

View Go program information.

1
2
$ file led
led: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, Go BuildID=lyyBrECmPyAxisV7K8CW/4BUObJmofsE9O5rEljuA/KDrmeLORek8eFaiXW8ov/s4oxB_6x564bJCZyGtO6, with debug_info, not stripped

Here statically linked means statically linked. Static links do not depend on so files on the system.

So the so-called 32-bit runtime environment is a set of related dynamic link libraries that ensure that 32-bit applications load the corresponding so-files when they run. All we need to do is to install the libraries.

First turn on the 32-bit architecture, which is the i386 architecture.

1
$ sudo dpkg --add-architecture i386

Then install the i386 architecture core dynamic libraries.

1
$ sudo apt-get install gcc-multilib g++-multilib

The only thing left to do is to install the libraries that depend on the specific application. There are two types of libraries, those with the word 32 in their name, such as lib32stdc++6, and those with the architecture specified in the library name, such as libelf1:i386.

Then it works fine. But my VPS hard drive is too small, only 20G, not enough, so I had to switch to a friend’s machine. But I’m also an obsessive-compulsive person, I have to clean up the packages I don’t use.

If we remove the i386 architecture directly, the system will indicate an error.

1
2
dpkg --remove-architecture i386
dpkg: error: cannot remove architecture 'i386' currently in use by the database

Obviously, this is because 32-bit packages are already installed on the system. We can find them by using dpkg with grep.

1
dpkg -l | grep :i386

Of course, you can also use dpkg’s built-in command dpkg -l "*:i386", but I think it’s more flexible and versatile to use it with grep.

Then you can use awk {print $2} to extract the package name, which is the second column, and finally use xargs with aptitude to batch remove it.

1
dpkg -l | grep :i386 | awk {print $2} | xargs aptitude purge -y

The -y option at the end here is to tell aptitude to just delete it and not ask the user for confirmation. However, given the thrilling experience I had last time, I took a more conservative approach. I started by making the package to be deleted a single line and splitting it with a space.

1
dpkg -l | grep :i386 | awk {print $2} | xargs echo

The xargs echo at the end does the trick. Then run aptitude manually and see what happens.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
sudo aptitude purge gcc-12-base:i386 libc6:i386 libc6-i386 libcom-err2:i386 libcrypt1:i386 libgcc-
s1:i386 libgmp-dev:i386 libgmp10:i386 libgmpxx4ldbl:i386 libgpm2:i386 libgssapi-krb5-2:i386 libidn2-0:i386 libk5crypto3:
i386 libkeyutils1:i386 libkrb5-3:i386 libkrb5support0:i386 libmpc-dev:i386 libmpc3:i386 libmpfr-dev:i386 libmpfr6:i386 l
ibncurses5:i386 libnsl2:i386 libnss-nis:i386 libnss-nisplus:i386 libssl3:i386 libstdc++6:i386 libtinfo5:i386 libtirpc3:i
386 libunistring2:i386
The following packages will be REMOVED:
  gcc-12-base:i386{p} libc6:i386{p} libc6-i386{p} libcom-err2:i386{p} libcrypt1:i386{p} libgcc-s1:i386{p}
  libgmp-dev:i386{p} libgmp10:i386{p} libgmpxx4ldbl:i386{pu} libgpm2:i386{p} libgssapi-krb5-2:i386{p}
  libidn2-0:i386{p} libk5crypto3:i386{p} libkeyutils1:i386{p} libkrb5-3:i386{p} libkrb5support0:i386{p}
  libmpc-dev:i386{p} libmpc3:i386{p} libmpfr-dev:i386{pu} libmpfr6:i386{pu} libncurses5:i386{p} libnsl2:i386{p}
  libnss-nis:i386{pu} libnss-nisplus:i386{pu} libssl3:i386{p} libstdc++6:i386{p} libtinfo5:i386{pu}
  libtirpc3:i386{p} libunistring2:i386{p}
0 packages upgraded, 0 newly installed, 29 to remove and 0 not upgraded.
Need to get 0 B of archives. After unpacking 35.5 MB will be freed.
Do you want to continue? [Y/n/?]

The system will find other packages that have dependencies and delete them together, this step requires user confirmation. I looked at the packages to be deleted and felt that there was no problem, so I pressed Y to continue. I didn’t expect the system to display a new prompt.

1
2
3
4
5
6
The following ESSENTIAL packages will be REMOVED!
  libcrypt1:i386

WARNING: Performing this action will probably cause your system to break!
         Do NOT continue unless you know EXACTLY what you are doing!
To continue, type the phrase "I am aware that this is a very bad idea":

It said that libcrypt1:i386 was a core package and that removing it might break the system. I panicked a bit. Don’t mess up the system like last time. But to be reasonable, libcrypt1:i386 should only be used by 32-bit programmes, and the system won’t use it, and programmes like SSH are also 64-bit, and shouldn’t depend on it. It might be worth checking which version SSH depends on.

1
2
3
4
5
$ ldd /usr/sbin/sshd|grep crypt
        libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007effdb000000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007effdb532000)
        libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007effdaac2000)
        libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007effda9fc000)

The path /lib/x86_64-linux-gnu shows that the dependency is on the 64-bit version. We use dpkg -L libcrypt1:i386 to view the package files.

1
2
3
4
/lib/i386-linux-gnu/libcrypt.so.1
/lib/i386-linux-gnu/libcrypt.so.1.1.0
/usr/share/doc/libcrypt1/changelog.Debian.gz
/usr/share/doc/libcrypt1/copyright

The path is /lib/i386-linux-gnu, so it doesn’t affect 64-bit programs like SSH and can be removed!

To remove it, you have to use the --force-remove-protected parameter of dpkg.

1
$ sudo dpkg --purge --force-remove-protected libcrypt1:i386 ...

After deletion you can clean up the i386 architecture.

1
$ sudo dpkg --add-architecture i386

Finally rebooted the system. Still able to log in normally, hooray. Above is the whole content of this article.