How to play Dapr without Kubernetes?

Dapr is designed as an enterprise-class microservices programming platform for developers that is independent of specific technology platforms and can run “anywhere”. Dapr itself does not provide an “infrastructure”, but rather uses its own extensions to adapt to specific deployment environments. In its current state, a true native Dapr application can only be deployed in a K8S environment if you wish to take it into production. While Dapr also provides support for Hashicorp Consul, there does not appear to be a stable version available.

Kubernetes is not “standard” for many companies, and for some reason they may have a homegrown microservices or elastic cloud platform that may be more valuable for Dapr to adapt to. We’ve done some feasibility studies on this over the past two weeks and found that it’s really not that hard, so we’ll go over the general solution with a very simple example.

1. NameResolution Component

Although Dapr provides a series of programming models, such as service invocation, publish-subscribe and actor models, the one that is widely used should be service invocation. We know that service invocation in microservice environment needs to solve the problems of service registration and discovery, load balancing, elastic scaling, etc. In fact, Dapr does not do anything in this area, as mentioned above, Dapr itself does not provide the infrastructure, it leaves these functions to specific deployment platforms (such as K8S) to solve. The only component in Dapr related to this is a simple NameResolution component.

From a deployment point of view, all the functionality of Dapr is embodied in the Sidecar paired with the application. All we need to do when making a service call is to specify the ID (AppID) of the target application where the service is located. Service requests (HTTP or gRPC) are routed from the application to the sidecar, which “routes” the request to the appropriate node. If deployed on a Kubernetes cluster, the addressing of service requests is no longer an issue if the target service’s identity and other relevant metadata (namespace, cluster domain, etc.) are specified. In fact, the NameResolution component embodies the “resolution” for “Name” that solves the problem of converting, for example, the Dapr application-specific identifier AppID into a deployment environment-based application identifier. environment-based application identity. From the code provided by dapr, it currently registers 3 types of NameResolution components as follows.

mdns: uses mDNS (Multicast DNS) for service registration and discovery, which is the type used by default if not explicitly configured. Since mDNS is only a type of DNS implemented using broadcast communication in small-scale networks, it is not suitable for formal generation environments at all.
kubernetes: Adapted to Kubernetes name resolution, currently offering a stable version.
consul: Name resolution for HashiCorp Consul, currently the latest version is Alpha.

2. Resolver

A registered NameResolution component is intended to provide a Resolver object, which is represented by the following interface. As shown in the following code snippet, the Resolver interface provides two methods. The Init method is called when the application is started and carries as parameters Metadata related to the current application instance (including the application identity and port, as well as Sidecar’s HTTP and gRPC ports, etc.) and the configuration for the current NameResolution component configuration for the current NameResolution component. For each service call, information about the target application identity and namespace is encapsulated by Sidecar into a ResolveRequest interface, and the ReolveID method of the Resolver object is invoked with the most parameters, resulting in a representation that matches the current deployment environment and is used to leverage the full target service call with the infrastructure This identifier is used to leverage the full target service invocation with the infrastructure.

package nameresolution

type Resolver interface {
    Init(metadata Metadata) error
    ResolveID(req ResolveRequest) (string, error)
}

type Metadata struct {
    Properties    map[string]string `json:"properties"`
    Configuration interface{}
}

type ResolveRequest struct {
    ID        string
    Namespace string
    Port      int
    Data     map[string]string
}

3. Simulating Service Registration and Load Balancing

Assuming we have a private microservices platform that implements basic service registration, load balancing, and even elastic scaling, if we want to use Dapr on this platform, we just need to provide a corresponding Resolver object using a custom NameResolution component. We use an ASP.NET Core MVC application to simulate the microservices platform we wish to adapt to, as follows: The HomeController maintains a list of applications and endpoints (IP+port) using the static field _applications. For a service call against an application, we implement simple load balancing by polling the corresponding endpoint. For ease of description, we will refer to the application as “ServiceRegistry”.

public class HomeController: Controller
{
    private static readonly ConcurrentDictionary<string, EndpointCollection> _applications = new();

    [HttpPost("/register")]
    public IActionResult Register([FromBody] RegisterRequest request)
    {
        var appId = request.Id;
        var endpoints = _applications.TryGetValue(appId, out var value) ? value : _applications[appId] = new();
        endpoints.TryAdd(request.HostAddress, request.Port);
        Console.WriteLine($"Register {request.Id} =>{request.HostAddress}:{request.Port}");
        return Ok();
    }

    [HttpPost("/resolve")]
    public IActionResult Resolve([FromBody] ResolveRequest request)
    {
        if (_applications.TryGetValue(request.ID, out var endpoints) && endpoints.TryGet(out var endpoint))
        {
            Console.WriteLine($"Resolve app {request.ID} =>{endpoint}");
            return Content(endpoint!);
        }
        return NotFound();
    }
}

public class EndpointCollection
{
    private readonly List<string> _endpoints = new();
    private int _index = 0;
    private readonly object _lock = new();

    public bool TryAdd(string ipAddress, int port)
    {
        lock (_lock)
        {
            var endpoint = $"{ipAddress}:{port}";
            if (_endpoints.Contains(endpoint))
            {
                return false;
            }
            _endpoints.Add(endpoint);
            return true;
        }
    }

    public bool TryGet(out string? endpoint)
    {
        lock (_lock)
        {
            if (_endpoints.Count == 0)
            {
                endpoint = null;
                return false;
            }
            _index++;
            if (_index >= _endpoints.Count)
            {
                _index = 0;
            }
            endpoint = _endpoints[_index];
            return true;
        }
    }
}

The HomeController provides two Action methods, the Register method is used to register the application and is called by the Init method of the custom Resolver. The other method, Resolve, is used to complete the process of getting a specific endpoint based on the requested application representation, and is called by the ResolveID method of the custom Resolver. The parameter types of these two methods, RegisterRequest and ResolveRequest, are defined as follows, with the latter having the same definition as the interface of the same name given earlier. Both Actions output the appropriate text in the console showing the registered application information and the resolved endpoint.

public class RegisterRequest
{
    public string Id { get; set; } = default!;
    public string HostAddress { get; set; } = default!;
    public int Port { get; set; }
}

public class ResolveRequest
{
    public string ID { get; set; } = default!;
    public string? Namespace { get; set; }
    public int Port { get; }
    public Dictionary<string, string> Data { get; } = new();
}

4. Customizing the NameResolution component

Since Dapr does not support dynamic registration of components, we have to pull down its source code, modify it, and recompile it. There are two git operations involved here, dapr and components-contrib, the former for the core runtime and the latter for the community-driven contributed components. We put the cloned source code in the same directory.

dapr and components-contrib

We named our custom NameResolution component “svcreg” (meaning service registration), so we created a directory of the same name in the components-contrib/nameresolution directory (In this directory we will see the definition of several NameResolution components mentioned above) and defined the component code in the svcreg.go file in that directory. The complete definition of the NameResolution component is shown below.

package svcreg

import (
 "bytes"
 "encoding/json"
 "errors"
 "fmt"
 "io/ioutil"
 "net/http"
 "strconv"

 "github.com/dapr/components-contrib/nameresolution"
 "github.com/dapr/kit/logger"
)

type Resolver struct {
 logger           logger.Logger
 registerEndpoint string
 resolveEndpoint  string
}

type RegisterRequest struct {
 Id, HostAddress string
 Port            int64
}

func (resolver *Resolver) Init(metadata nameresolution.Metadata) error {

 var endpoint, appId, hostAddress string
 var ok bool

 // Extracts register & resolve endpoint
 if dic, ok := metadata.Configuration.(map[interface{}]interface{}); ok {
  endpoint = fmt.Sprintf("%s", dic["endpointAddress"])
  resolver.registerEndpoint = fmt.Sprintf("%s/register", endpoint)
  resolver.resolveEndpoint = fmt.Sprintf("%s/resolve", endpoint)
 }
 if endpoint == "" {
  return errors.New("service registry endpoint is not configured")
 }

 // Extracts AppID, HostAddress and Port
 props := metadata.Properties
 if appId, ok = props[nameresolution.AppID]; !ok {
  return errors.New("AppId does not exist in the name resolution metadata")
 }
 if hostAddress, ok = props[nameresolution.HostAddress]; !ok {
  return errors.New("HostAddress does not exist in the name resolution metadata")
 }
 p, ok := props[nameresolution.DaprPort]
 if !ok {
  return errors.New("DaprPort does not exist in the name resolution metadata")
 }
 port, err := strconv.ParseInt(p, 10, 32)
 if err != nil {
  return errors.New("DaprPort is invalid")
 }

 // Register service (application)
 var request = RegisterRequest{appId, hostAddress, port}
 payload, err := json.Marshal(request)
 if err != nil {
  return errors.New("fail to marshal register request")
 }
 _, err = http.Post(resolver.registerEndpoint, "application/json", bytes.NewBuffer(payload))

 if err == nil {
  resolver.logger.Infof("App '%s (%s:%d)' is successfully registered.", request.Id, request.HostAddress, request.Port)
 }
 return err
}

func (resolver *Resolver) ResolveID(req nameresolution.ResolveRequest) (string, error) {

 // Invoke resolve service and get resolved target app's endpoint ("{ip}:{port}")
 payload, err := json.Marshal(req)
 if err != nil {
  return "", err
 }
 response, err := http.Post(resolver.resolveEndpoint, "application/json", bytes.NewBuffer(payload))
 if err != nil {
  return "", err
 }
 defer response.Body.Close()
 result, err := ioutil.ReadAll(response.Body)
 if err != nil {
  return "", err
 }
 return string(result), nil
}

func NewResolver(logger logger.Logger) *Resolver {
 return &Resolver{
  logger: logger,
 }
}

As shown in the code snippet above, we define the core Resolver structure, which has a logger field for logging and two additional fields registerEndpoint and resolveEndpoint, representing the URLs of the two APIs provided by ServiceRegistry, respectively. In the Init method implemented for the Resolver structure, we extract the configuration from the metadata as parameters, and further extract the address of the ServiceRegistry from the configuration and add the routing paths “/register” and " /resolve" to initialize the registerEndpoint and resolveEndpoint fields of the Resolver structure. Next, we extract the AppID, IP address and internal gRPC port number (through which the external application calls the Sidecar of the current application) from the metadata, which are encapsulated into the RegisterRequest structure and serialized into JSON strings, and used as input to call the corresponding Web API to complete the corresponding service registration.

In the ResolveID implementation, we directly serialize the ResolveRequest structure as a parameter into JSON and call the Resolve API. the string carried in the body of the response is the endpoint (IP+Port) parsed for the target application, which we directly use as the return value of ResolveID.

5. Registering a Custom NameResolution Component

The custom NameResolution component needs to be explicitly registered to the executable daprd representing Sidecar, and the source file where the entry program is located is dapr/cmd/daprd/main.go. We first import the package where svcreg is located as follows.

“github.com/dapr/components-contrib/nameresolution/svcreg”.

// Name resolutions.
nr "github.com/dapr/components-contrib/nameresolution"
nr_consul "github.com/dapr/components-contrib/nameresolution/consul"
nr_kubernetes "github.com/dapr/components-contrib/nameresolution/kubernetes"
nr_mdns "github.com/dapr/components-contrib/nameresolution/mdns"
nr_svcreg "github.com/dapr/components-contrib/nameresolution/svcreg"

In the main function, we find the part of the code used to register the NameResolution component and just follow the instructions for registering the svcreg as we did for the other NameResolution components. The NewResolver function used to provide the Resolver in the registration code is defined in the svcreg.go file above.

runtime.WithNameResolutions(
 nr_loader.New("svcreg", func() nr.Resolver {
  return nr_svcreg.NewResolver(logContrib)
 }),
 nr_loader.New("mdns", func() nr.Resolver {
  return nr_mdns.NewResolver(logContrib)
 }),
 nr_loader.New("kubernetes", func() nr.Resolver {
  return nr_kubernetes.NewResolver(logContrib)
 }),
 nr_loader.New("consul", func() nr.Resolver {
  return nr_consul.NewResolver(logContrib)
 }),
),

6. Compile and deploy daprd.exe

So far, all the programming work has been done, next we need to recompile daprd.exe which represents Sidecar.

As you can see from the code snippet above, the package paths of dapr are prefixed with “github.com/dapr”, so we need to modify the go.mod file (dapr/go.mod) to redirect the dependency paths to the local directory, so we added the paths for so we add a replacement rule for “github.com/dapr/components-contrib” as follows.

replace (
 go.opentelemetry.io/otel => go.opentelemetry.io/otel v0.20.0
 gopkg.in/couchbaselabs/gocbconnstr.v1 => github.com/couchbaselabs/gocbconnstr v1.0.5
 k8s.io/client => github.com/kubernetes-client/go v0.0.0-20190928040339-c757968c4c36
 github.com/dapr/components-contrib => ../components-contrib
)

After switching the current directory to “dapr/cmd/daprd/”, executing “go build” from the command line will generate a daprd.exe executable file in the current directory. Now we need to use this new daprd.exe to replace the current one, which is located in the %userprofile%.dapr\bin directory.

cmd

7. Configuring svcreg

As we have already mentioned, Dapr uses the mDNS-based NameResolution component by default (for which the registered name is “mdns”). To enable our custom component “svcreg”, we need to modify Dapr’s configuration file (%userprofile%.dapr\config.yaml). As shown in the following code snippet, we not only set the name of the used component to “svcreg” (the name provided when registering the NameResolution component in dapr/cmd/daprd/main.go), but also set the URL of the service registration API (http://127.0.0.1:3721) in the configuration (the URL extracted by the Init method of Resolver is from here).

apiVersion: dapr.io/v1alpha1
kind: Configuration
metadata:
  name: daprConfig
spec:
  nameResolution:
    component: "svcreg"
    configuration:
      endpointAddress: http://127.0.0.1:3721
  tracing:
    samplingRate: "1"
    zipkin:
      endpointAddress: http://localhost:9411/api/v2/spans

8. Testing the Effects

We now write a Dapr application to verify that the custom NameResolution component works. We use the example of a service call provided in ASP.NET Core 6 Framework Revealed Example Demo [03]: Dapr First Experience. App2, defined as follows, is an ASP.NET Core application that uses routing to provide APIs for addition, subtraction, multiplication, and division operations.

 using Microsoft.AspNetCore.Mvc;
 using Shared;

 var app = WebApplication.Create(args);
 app.MapPost("{method}", Calculate);
 app.Run("http://localhost:9999");

 static IResult Calculate(string method, [FromBody] Input input)
 {
     var result = method.ToLower() switch
     {
         "add" => input.X + input.Y,
         "sub" => input.X - input.Y,
         "mul" => input.X * input.Y,
         "div" => input.X / input.Y,
         _ => throw new InvalidOperationException($"Invalid method {method}")
     };
     return Results.Json(new Output { Result = result });
 }
public class Input
{
    public int X { get; set; }
    public int Y { get; set; }
}

public class Output
{
    public int   Result { get; set; }
    public DateTimeOffset  Timestamp { get; set; } = DateTimeOffset.Now;
}

App1 with the following definition is a console application that calls the four APIs of the appeal using the Dapr client SDK.

 using Dapr.Client;
 using Shared;

 HttpClient client = DaprClient.CreateInvokeHttpClient(appId: "app2");
 var input = new Input(2, 1);

 await InvokeAsync("add", "+");
 await InvokeAsync("sub", "-");
 await InvokeAsync("mul", "*");
 await InvokeAsync("div", "/");

 async Task InvokeAsync(string method, string @operator)
 {
     var response = await client.PostAsync(method, JsonContent.Create(input));
     var output = await response.Content.ReadFromJsonAsync<Output>();
     Console.WriteLine( $"{input.X} {@operator} {input.Y} = {output.Result} ({output.Timestamp})");
 }

After starting the ServiceRegistry, we start App2 and the console will elaborate the following output. As you can see from the name of the NameResolution component in the output, our custom svcreg is being used.

NameResolution component in the output

Since the Resolver’s Init method is called when the application is started to register, this is also reflected in the output of the ServiceRegistry as shown below. You can see that the AppID of the registered instance is “app2” and the corresponding endpoint is “10.181.22.4:60840”.

corresponding endpoint

Then we start App1 again, and the output shown below indicates that all four service calls completed successfully.

four service calls completed successfully

The application instance of the launched App1 is also registered in the ServiceRegistry. Four service calls result in four calls to the Resolver’s ResolveID method, which is also reflected in the output of the ServiceRegistry.

output of the ServiceRegistry

Reference

https://www.cnblogs.com/artech/p/dapr-custom-name-resolution.html

Table of Contents