Prerequisites

In microservices architecture, if the downstream dependencies do not do request degradation processing, the downstream abnormal dependencies are not isolated, and it is likely that one or two services or as small as one or two interface abnormalities will lead to the unavailability of all upstream services and even affect the whole business line.

Request degradation processing is still relatively mainstream is Netfilx produced Hystrix.

Hystrix works on the following principles.

  • Isolating requests based on thread pools or semaphores, once the downstream service fails to respond within the specified configured timeout will enter a pre-defined or default degraded implementation.
  • The status of each request is recorded, and the rate of processing failure within a sliding window exceeds a set threshold to trigger the fuse (Circle Breaker) to open, and all requests go directly to the preset or default degradation logic after the fuse is opened.
  • After the fuse is opened and the time since the fuse was opened or the last trial request release exceeds the set value, the fuse device enters the half-open state and allows the release of a trial request.
  • After the request success rate increases, the fuse is determined to be closed based on statistical data and all requests are released normally.

Instead of going into the details of Hystrix, we will move on to how to use Hystrix in Spring Cloud Gateway, mainly including the built-in Hystrix filters and custom filters combined with Hystrix to achieve the functionality we want. In addition to introducing the spring-cloud-starter-gateway dependency, you also need to introduce spring-cloud-starter-netflix-hystrix.

Instead of going into the details of Hystrix, we will move on to how to use Hystrix in Spring Cloud Gateway, mainly including the built-in Hystrix filters and custom filters combined with Hystrix to achieve the functionality we want. In addition to introducing the spring-cloud-starter-gateway dependency, you also need to introduce spring-cloud-starter-netflix-hystrix.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

<dependencies>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-gateway</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
    </dependency>
</dependencies>

Use the built-in Hystrix filter

The built-in Hystrix filter is the HystrixGatewayFilterFactory, which supports the following configurations.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

public static class Config {
    // If the following Setter is configured as null, name will be used as the HystrixCommandKey of the Hystrix
    private String name;
    // Setter property of Hystrix, mainly used to configure the KEY and other properties of the command
    private Setter setter;
    // The target URI for degradation must start with forward, and the URI will match the controller method applied to the gateway
    private URI fallbackUri;

	public String getName() {
		return name;
	}

	public Config setName(String name) {
		this.name = name;
		return this;
	}

    public Config setFallbackUri(String fallbackUri) {
	    if (fallbackUri != null) {
			setFallbackUri(URI.create(fallbackUri));
		}
		return this;
	}

    public URI getFallbackUri() {
	    return fallbackUri;
    }
    
    // Note that for this method, the configured fallbackUri should start with forward as the schema, otherwise it will throw an exception
    public void setFallbackUri(URI fallbackUri) {
        if (fallbackUri != null && !"forward".equals(fallbackUri.getScheme())) {
			throw new IllegalArgumentException("Hystrix Filter currently only supports 'forward' URIs, found " + fallbackUri);
		}
		this.fallbackUri = fallbackUri;
	}

	public Config setSetter(Setter setter) {
		this.setter = setter;
		return this;
	}
}

In addition,

(1) the global Hystrix configuration will also take effect for HystrixGatewayFilterFactory;

(2) HystrixGatewayFilterFactory can be used as default-filters for all routing configurations as under-the-hood filters and function as such.

For point (1), if we configure the following in application.yaml.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

// The execution timeout is 1 second and will take effect for the HystrixGatewayFilterFactory bound to the following route order_route
hystrix.command.fallbackcmd.execution.isolation.thread.timeoutInMilliseconds: 1000

spring:
  cloud:
    gateway:
      routes:
      - id: order_route
        uri: http://localhost:9091
        predicates:
        - Path=/order/**
        filters:
        - name: Hystrix
          args:
            name: HystrixCommand
            fallbackUri: forward:/fallback

The configured hystrix.command.fallbackcmd.execution.isolation.thread.timeoutInMilliseconds will take effect on the HystrixGatewayFilterFactory bound to the route order_route.

For point (2), we can configure HystrixGatewayFilterFactory as the default filter, so that all routes will be associated with this filter, but it is recommended not to do so unless necessary: HystrixGatewayFilterFactory is the default filter.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

spring:
  cloud:
    gateway:
      routes:
        - id: order_route
          uri: http://localhost:9091
          predicates:
            - Path=/order/**
      default-filters:
        - name: Hystrix
          args:
            name: HystrixCommand
            fallbackUri: forward:/fallback

When I was testing, I found that the Setter mentioned above could not be configured, presumably because the Setter object of Hystrix is multi-packaged and there is no way to set the property for the time being. Next we have to add a controller method to the gateway service for handling redirected /fallback requests.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

@RestController
public class FallbackController {

    @RequestMapping(value = "/fallback")
    @ResponseStatus
    public Mono<Map<String, Object>> fallback(ServerWebExchange exchange, Throwable throwable) {
        Map<String, Object> result = new HashMap<>(8);
        ServerHttpRequest request = exchange.getRequest();
        result.put("path", request.getPath().pathWithinApplication().value());
        result.put("method", request.getMethodValue());
        if (null != throwable.getCause()) {
            result.put("message", throwable.getCause().getMessage());
        } else {
            result.put("message", throwable.getMessage());
        }
        return Mono.just(result);
    }
}

Controller method entries are handled by the internal components of Spring Cloud Gateway and can call back some useful types such as ServerWebExchange instances, specific exception instances and so on.

Custom Filters with Hystrix

HystrixGatewayFilterFactory should meet business needs in most cases, but here also do a customization of a filter that integrates Hystrix and implements the following functionality.

  • Creates a new instance of the Hystrix command to be invoked based on each request URL.
  • Each URL can specify a unique thread pool configuration, or use the default if not specified.
  • A separate Hystrix timeout can be configured for each URL.

This means that each different external request URL is isolated by Hystrix using a thread pool. Of course, such a filter only makes sense if the number of different URLs for external requests is limited, otherwise there is a risk of creating too many thread pools causing system performance degradation, which is counterproductive. The modification is as follows.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174

@Component
public class CustomHystrixFilter extends AbstractGatewayFilterFactory<CustomHystrixFilter.Config> {

    private static final String FORWARD_KEY = "forward";
    private static final String NAME = "CustomHystrix";
    private static final int TIMEOUT_MS = 1000;
    private final ObjectProvider<DispatcherHandler> dispatcherHandlerProvider;
    private volatile DispatcherHandler dispatcherHandler;
    private boolean processConfig = false;

    public CustomHystrixFilter(ObjectProvider<DispatcherHandler> dispatcherHandlerProvider) {
        super(Config.class);
        this.dispatcherHandlerProvider = dispatcherHandlerProvider;
    }

    private DispatcherHandler getDispatcherHandler() {
        if (dispatcherHandler == null) {
            dispatcherHandler = dispatcherHandlerProvider.getIfAvailable();
        }

        return dispatcherHandler;
    }

    @Override
    public List<String> shortcutFieldOrder() {
        return Collections.singletonList(NAME_KEY);
    }


    @Override
    public GatewayFilter apply(Config config) {
        processConfig(config);
        return (exchange, chain) -> {
            ServerHttpRequest request = exchange.getRequest();
            String path = request.getPath().pathWithinApplication().value();
            int timeout = config.getTimeout().getOrDefault(path, TIMEOUT_MS);
            CustomHystrixCommand command = new CustomHystrixCommand(config.getFallbackUri(), exchange, chain, timeout, path);
            return Mono.create(s -> {
                Subscription sub = command.toObservable().subscribe(s::success, s::error, s::success);
                s.onCancel(sub::unsubscribe);
            }).onErrorResume((Function<Throwable, Mono<Void>>) throwable -> {
                if (throwable instanceof HystrixRuntimeException) {
                    HystrixRuntimeException e = (HystrixRuntimeException) throwable;
                    HystrixRuntimeException.FailureType failureType = e.getFailureType();
                    switch (failureType) {
                        case TIMEOUT:
                            return Mono.error(new TimeoutException());
                        case COMMAND_EXCEPTION: {
                            Throwable cause = e.getCause();
                            if (cause instanceof ResponseStatusException || AnnotatedElementUtils
                                    .findMergedAnnotation(cause.getClass(), ResponseStatus.class) != null) {
                                return Mono.error(cause);
                            }
                        }
                        default:
                            break;
                    }
                }
                return Mono.error(throwable);
            }).then();
        };
    }

    /**
     * YAML parsing does not support '/' in MAP's KEY, so here you can only use '-' instead
     *
     * @param config config
     */
    private void processConfig(Config config) {
        if (!processConfig) {
            processConfig = true;
            if (null != config.getTimeout()) {
                Map<String, Integer> timeout = new HashMap<>(8);
                config.getTimeout().forEach((k, v) -> {
                    String key = k.replace("-", "/");
                    if (!key.startsWith("/")) {
                        key = "/" + key;
                    }
                    timeout.put(key, v);
                });
                config.setTimeout(timeout);
            }
        }
    }

    @Override
    public String name() {
        return NAME;
    }

    private class CustomHystrixCommand extends HystrixObservableCommand<Void> {

        private final URI fallbackUri;
        private final ServerWebExchange exchange;
        private final GatewayFilterChain chain;

        public CustomHystrixCommand(URI fallbackUri,
                                    ServerWebExchange exchange,
                                    GatewayFilterChain chain,
                                    int timeout,
                                    String key) {
            super(Setter.withGroupKey(HystrixCommandGroupKey.Factory.asKey(key))
                    .andCommandKey(HystrixCommandKey.Factory.asKey(key))
                    .andCommandPropertiesDefaults(HystrixCommandProperties.Setter().withExecutionTimeoutInMilliseconds(timeout)));
            this.fallbackUri = fallbackUri;
            this.exchange = exchange;
            this.chain = chain;
        }

        @Override
        protected Observable<Void> construct() {
            return RxReactiveStreams.toObservable(this.chain.filter(exchange));
        }

        @Override
        protected Observable<Void> resumeWithFallback() {
            if (null == fallbackUri) {
                return super.resumeWithFallback();
            }
            URI uri = exchange.getRequest().getURI();
            boolean encoded = containsEncodedParts(uri);
            URI requestUrl = UriComponentsBuilder.fromUri(uri)
                    .host(null)
                    .port(null)
                    .uri(this.fallbackUri)
                    .build(encoded)
                    .toUri();
            exchange.getAttributes().put(GATEWAY_REQUEST_URL_ATTR, requestUrl);
            ServerHttpRequest request = this.exchange.getRequest().mutate().uri(requestUrl).build();
            ServerWebExchange mutated = exchange.mutate().request(request).build();
            return RxReactiveStreams.toObservable(getDispatcherHandler().handle(mutated));
        }
    }

    public static class Config {

        private String id;
        private URI fallbackUri;
        /**
         * url -> timeout ms
         */
        private Map<String, Integer> timeout;

        public String getId() {
            return id;
        }

        public Config setId(String id) {
            this.id = id;
            return this;
        }

        public URI getFallbackUri() {
            return fallbackUri;
        }

        public Config setFallbackUri(URI fallbackUri) {
            if (fallbackUri != null && !FORWARD_KEY.equals(fallbackUri.getScheme())) {
                throw new IllegalArgumentException("Hystrix Filter currently only supports 'forward' URIs, found " + fallbackUri);
            }
            this.fallbackUri = fallbackUri;
            return this;
        }

        public Map<String, Integer> getTimeout() {
            return timeout;
        }

        public Config setTimeout(Map<String, Integer> timeout) {
            this.timeout = timeout;
            return this;
        }
    }
}

In fact, most of the code is similar to the built-in Hystrix filter, only the command transformation function part and the configuration loading processing part have been changed. The configuration file is as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

spring:
  cloud:
    gateway:
      routes:
        - id: hystrix_route
          uri: http://localhost:9091
          predicates:
            - Host=localhost:9090
          filters:
            - name: CustomHystrix
              args:
                id: CustomHystrix
                fallbackUri: forward:/fallback
                timeout:
                  # 这里暂时用-分隔URL，因为/不支持
                  order-remote: 2000
  application:
    name: route-server
server:
  port: 9090

The gateway adds a /fallback processing controller as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

@RestController
public class FallbackController {

    @RequestMapping(value = "/fallback")
    @ResponseStatus
    public Mono<Map<String, Object>> fallback(ServerWebExchange exchange, Throwable throwable) {
        Map<String, Object> result = new HashMap<>(8);
        ServerHttpRequest request = exchange.getRequest();
        result.put("path", request.getPath().pathWithinApplication().value());
        result.put("method", request.getMethodValue());
        if (null != throwable.getCause()) {
            result.put("message", throwable.getCause().getMessage());
        } else {
            result.put("message", throwable.getMessage());
        }
        return Mono.just(result);
    }
}

Intentional downstream service interruption points.

1
2
3
4
5
6
7
8

curl http://localhost:9090/order/remote

response ：
{
    "path": "/fallback",
    "method": "GET",
    "message": null   # <== Here the message is null because it is a timeout exception
}

Just in line with the expected results.

Summary

This article is just to Hystrix and filter application to provide a usable example and problem solving ideas, specific how to use or need for real scenarios.

Reference https://www.throwx.cn/2019/05/25/spring-cloud-gateway-hystrix/