Pipeable

Pipeable is perhaps a rather controversial way of programming C++.

There is pipeable in Boost: pipeable - Boost.HigherOrderFunctions 0.6 documentation - master. It is a part of the hof library

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

#include <boost/hof.hpp>
#include <cassert>
using namespace boost::hof;

struct sum
{
    template<class T, class U>
    T operator()(T x, U y) const
    {
        return x+y;
    }
};

int main() {
    assert(3 == (1 | pipable(sum())(2)));
    assert(3 == pipable(sum())(1, 2));
}

The HOF library stands for Higher-order functions for C++, and its author Paul Fultz II is also a celebrity. There are two Posts in his blog related to the topic of this article.

  1. Pipable functions in C++14
  2. Extension methods in C++

Similarly, there are Extension methods in Boost. methods).

Fundamentals

We note that the origin of Pipeable programming comes from pipe operations in the OS Shell, e.g.

1

cat 1.txt | grep 'best of' | uniq -c | head -10

In particular, when the C++ operator ’ ’ (logical or) can also be overloaded, everything seems to fall into place.

The key C++ idea is to do ’ ’ operator overloading as well as ‘()’ operator overloading.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

template<class F>
struct pipe_closure : F
{
    template<class... Xs>
    pipe_closure(Xs&&... xs) : F(std::forward<Xs>(xs)...)
    { }
};

template<class T, class F>
decltype(auto) operator|(T&& x, const pipe_closure<F>& p)
{
    return p(std::forward<T>(x));
}

The user class F can then have the ability to operate with ‘|’ under the pipe_closure decoration (provided that F also implements the ‘()’ operator overload.

1
2
3
4
5
6
7
8

struct add_one_f
{
    template<class T>
    auto operator()(T x) const // `T const &x` is better
    {
        return x + 1;
    }
};

Then the pipeable tandem operation looks like this.

1
2
3

const pipe_closure<add_one_f> add_one = { };
int number_3 = 1 | add_one | add_one;
std::cout << number_3 << std::endl;

Note that the ‘()’ operator accepts the generic class T as an input, so that it receives the lhs element forwarded by pip_closure<F>::operator |, the left-hand operand of the " " operator. In this way, the result of the lhs expression is piped to the rhs operand in operator ()<T>(), thus completing the pipeline operation.

Slightly improved

Obviously this idea can be further embellished.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

namespace hicc::pipeable {

    template<class F>
    struct pipeable {
    private:
        F f;

    public:
        pipeable(F &&f)
            : f(std::forward<F>(f)) { }

        template<class... Xs>
        auto operator()(Xs &&...xs) -> decltype(std::bind(f, std::placeholders::_1, std::forward<Xs>(xs)...)) const {
            return std::bind(f, std::placeholders::_1, std::forward<Xs>(xs)...);
        }
    };

    template<class F>
    pipeable<F> piped(F &&f) { return pipeable<F>{std::forward<F>(f)}; }

    template<class T, class F>
    auto operator|(T &&x, const F &f) -> decltype(f(std::forward<T>(x))) {
        return f(std::forward<T>(x));
    }

} // namespace hicc::pipeable

Then, to be able to have a more elegant presentation of.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

void test_piped() {
    using namespace hicc::pipeable;

    {
        auto add = piped([](int x, int y) { return x + y; });
        auto mul = piped([](int x, int y) { return x * y; });
        int y = 5 | add(2) | mul(5) | add(1);
        hicc_print("    y = %d", y);

        int y2 = 5 | add(2) | piped([](int x, int y) { return x * y; })(5) | piped([](int x) { return x + 1; })();
        // Output: 36
        hicc_print("    y2 = %d", y2);
    }
}

The above pipeable class and piped(…) are from Functional pipeline in C++11 - Victor Laskin’s Blog, his implementation is more meaningful, so we like it a bit more.

ACK: the following image is from his blog

Paul Fultz II’s motivation is more of a priority, I just never really use boost, it’s too large and complete to be used in engineering at all. I prefer something relatively lightweight, yet relatively complete.

If you’re interested in Paul Fultz II’s implementation, in addition to boost::hof::pipeable, he also made the open source Linq: pfultz2/Linq: Linq for list comprehension in C++. This is a C++ implementation of the C# Linq technology, which is amazing and crazy.

successor

Here is the extended reading time.

ranges

Of course, it is also important to mention the std::ranges range library, which is one of the changes brought by C++20, but the C++20 specification has not been released for long enough for engineering applications to be realistic.

ranges also has an incubation project of the same name ericniebler/range-v3: Range library for C++14/17/20, basis for C++20’s std::ranges, which can be used if you don’t want to enable c++20 right away, requires clang 3.6+ or gcc 4.9.1, so older projects say no pressure.

In general, ranges provide a filtering tool that allows you to filter, sort, mapreduce, etc. on top of a pipeline filter.

But range v3 is also a mixed bag.

How to understand the scope library

In fact, this guy is just as inexplicable as other terms like protocol, concept, etc. Do you know what a concept is?

Do you know what a concept is? iykyk, I’m a real Chinese person who doesn’t have any idea about concepts.

However, C++20’s std::concept is a weird thing.

Their actual meanings have to go into the etymology of the English roots, so they are hard for Chinese people to understand intuitively.

Here, let’s ignore the ranges etymology and use std::ranges::view to understand it. In other words, a ranges library is a cross-section of a set, a view that can be observed, and then a view that can be multiplied or added, or the mean can be calculated, or other aggregation operations can be done, or sorting can be done, or map reduce can be done.

The question of what you can do is not a question of ranges, but how you are going to do it on the view provided by ranges. So to make good use of the ranges library, you should have a good understanding of functional programming. For those who have an understanding of C# Linq techniques, RxJava can easily understand the core of std::ranges.

Use

As for using ranges, refer to the examples they provide.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

#include <iostream>
#include <string>
#include <vector>

#include <range/v3/view/filter.hpp>
#include <range/v3/view/transform.hpp>
using std::cout;

int main()
{
    std::vector<int> const vi{1, 2, 3, 4, 5, 6, 7, 8, 9, 10};
    using namespace ranges;
    auto rng = vi | views::filter([](int i) { return i % 2 == 0; }) |
               views::transform([](int i) { return std::to_string(i); });
    // prints: [2,4,6,8,10]
    cout << rng << '\n';
}

It’s hard to say if this is progress.

Other Languages

Kotlin

1
2
3
4
5
6
7
8

val animals = listOf("raccoon", "reindeer", "cow", "camel", "giraffe", "goat")

// grouping by first char and collect only max of contains vowels
val compareByVowelCount = compareBy { s: String -> s.count { it in "aeiou" } }

val maxVowels = animals.groupingBy { it.first() }.reduce { _, a, b -> maxOf(a, b, compareByVowelCount) }

println(maxVowels) // {r=reindeer, c=camel, g=giraffe}

Kotlin would look better if it were RxJava

1
2
3
4
5
6

Observable.just("Apple", "Orange", "Banana")
.subscribe(
  { value -> println("Received: $value") }, // onNext
  { error -> println("Error: $error") },    // onError
  { println("Completed!") }                 // onComplete
)

If Kotlin didn’t always come with a JVM, I would be completely and utterly bored.

Of course, Rx also has a c++ version, RxCpp, which tastes like this.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

#include "rxcpp/rx.hpp"
namespace Rx {
using namespace rxcpp;
using namespace rxcpp::sources;
using namespace rxcpp::operators;
using namespace rxcpp::util;
}
using namespace Rx;

#include <regex>
#include <random>
using namespace std;
using namespace std::chrono;

int main()
{
    random_device rd;   // non-deterministic generator
    mt19937 gen(rd());
    uniform_int_distribution<> dist(4, 18);

    // for testing purposes, produce byte stream that from lines of text
    auto bytes = range(0, 10) |
        flat_map([&](int i){
            auto body = from((uint8_t)('A' + i)) |
                repeat(dist(gen)) |
                as_dynamic();
            auto delim = from((uint8_t)'\r');
            return from(body, delim) | concat();
        }) |
        window(17) |
        flat_map([](observable<uint8_t> w){
            return w |
                reduce(
                    vector<uint8_t>(),
                    [](vector<uint8_t> v, uint8_t b){
                        v.push_back(b);
                        return v;
                    }) |
                as_dynamic();
        }) |
        tap([](vector<uint8_t>& v){
            // print input packet of bytes
            copy(v.begin(), v.end(), ostream_iterator<long>(cout, " "));
            cout << endl;
        });

    // ...
		// full codes at: https://github.com/ReactiveX/RxCpp
    return 0;
}

What can only be said, bad is bad in […] (…) { … } is an outrageous and rather unattractive lambda function body. When compared to kotlin’s lambda, it’s a retreat.

In fact, this has been the case with all the changes since C++11, creating a lot of weird and unintuitive stuff. I’m afraid it would be dead if not for some new goodies and history.

About the controversy I mentioned

First of all on the one hand, C++ anonymous functions are in terrible shape. See C++0x lambdas suck, C++ lambda ugly, etc.

On the other hand, we have to see that this is also historical, not to mention that C++’s functional style originally had its own style, and ranges of things like standards came too late, too many different implementations caused confusion, and standards, well, always late.

With C# closures, Java 8 closures, and Kotlin closures in the foreground, C++ can’t win this battle under the preconceptions.

About Extension Method

This concept, is part of Protocol-oriented Progamming (PPP).

Swift

In Swift language, you can extend any class by means of Protocol (Extensions). It behaves somewhat like this.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

let pythons = ["Eric", "Graham", "John", "Michael", "Terry", "Terry"]
let beatles = Set(["John", "Paul", "George", "Ringo"])

extension Collection {
    func summarize() {
        print("There are \(count) of us:")

        for name in self {
            print(name)
        }
    }
}

// Both Array and Set will now have that method, so we can try it out
pythons.summarize()
beatles.summarize()

Another example, adding a reverse() method to the String class (note: String has a native method reversed()).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

import Foundation

extension String {
    func reverse() -> String {
        var word = [Character]()
        for char in self {
            word.insert(char, at: 0)
        }
        return String(word)
    }
}

var bobobo = "reversing"
print(bobobo.reverse())
// gnisrever

The examples above do not make use of protocol constraints, so they only show the ability to extend existing classes. You can extend a generic class that meets a specific protocol constraint, thus giving that group of generic classes an additional method. This is too large to expand on here.

Kotlin

Kotlin provides the means for Extension Function, which can be considered a disruptive and innovative design for Java.

This time the example is slightly different.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

fun String.reverseCaseOfString(): String {
    val inputCharArr = toCharArray()
    var output = ""
    for (i in 0 until inputCharArr.size) {
        output += if (inputCharArr[i].isUpperCase()) {
            inputCharArr[i].toLowerCase()
        } else {
            inputCharArr[i].toUpperCase()
        }
    }
    return output
}

This adds the reverseCaseOfString() method to the String class, which is no different from String.length().

1

print("CaseSensitive".reverseCaseOfString())

Those of you who are mobile developers should be familiar with this capability of Swift and Kotlin.

In C++

I’ve been studying and dreaming for a long time (several years, spanning a decade) to be able to do the same thing in C++. But even at c++23 this is still not possible.

As for creating an equivalent C++ class library, the conclusion, after many failed attempts, is that it is still not possible.

A less-than-perfect way to extend the functionality of existing classes through C++ is through the Decorator Pattern, which is the most standardized solution, but is badly shaped and cumbersome:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

#include <iostream>
#include <string>

struct Shape {
  virtual ~Shape() = default;

  virtual std::string GetName() const = 0;
};

struct Circle : Shape {
  void Resize(float factor) { radius *= factor; }

  std::string GetName() const override {
    return std::string("A circle of radius ") + std::to_string(radius);
  }

  float radius = 10.0f;
};

struct ColoredShape : Shape {
  ColoredShape(const std::string& color, Shape* shape)
      : color(color), shape(shape) {}

  std::string GetName() const override {
    return shape->GetName() + " which is colored " + color;
  }

  std::string color;
  Shape* shape;
};

int main() {
  Circle circle;
  ColoredShape colored_shape("red", &circle);
  std::cout << colored_shape.GetName() << std::endl;
}

Using the decorate pattern, you can either decorate the implementation of existing methods or add new ones (CIrcle.Resize()), but you need to display the construction to the decorator class to do so (ColoredShape colored_shape("red", &circle)). An improvement would be to use the template class for mixin, but that would be a limited improvement.