Kotlin Coroutine and Retrofit

Retrofit 2.6.0 supports defining interfaces with Kotlin suspend functions. This article describes how to create the most comfortable coroutine experience by customizing the Retrofit Call Adapter and Converter.

Call Adapter: custom request execution logic, including thread switching, etc.
Converter: Customize the deserialization logic, how to convert the bytes obtained from the request into objects.

Spoiler for the final result.

// Retrofit 接口定义
// 简洁起见，后文省略外面这个 UserApi
interface UserApi {
  suspend fun getUser(id: Int): 
    ApiResponse<User>
}

data class User(val name: String)

// 调用示例 1：
lifecycleScope.launch {
  retrofit.create<UserApi>()
    .getUser(1)
    .getOrNull()
    // 主线程更新 UI
    ?.let { binding.nameLabel.text = it.name }
}

// 调用示例 2：
lifecycleScope.launch {
  val user: User = retrofit.create<UserApi>()
    .getUser(1)
    .guardOk { return@launch }
  // 拿到非 null 的 User 继续后面的业务逻辑
}

// 还没有结束，文章最后会介绍一个进一步简化的方案 ;-)

This solution was inspired by Jake Wharton’s Making Retrofit Work for You talk. Jake is also the maintainer of Retrofit. In his talk, he recommends taking advantage of the custom deserialization and request execution APIs that Retrofit provides to adapt adapt your own business logic and interfaces.

Background

Suppose our interface returns JSON data like this, with an errcode field returning 0 on a successful request, and a data field holding the data.

{
  "errcode": 0,
  "msg": "",
  "data": {
    "id": 1,
    "name": "Peter Parker"
  }
}

The exception errcode is not 0 and the msg field returns the error message displayed to the user.

{
  "errcode": 401,
  "msg": "无权访问"
}

Retrofit interface design

Let’s set aside the implementation for a moment and explore how to design Retrofit’s interface to make it more comfortable for callers to use Coroutine.

Get rid of the “envelope”

A comfortable envelope should make it as easy as possible for the caller, the simpler the better. You can see that the data that’s really useful to the business is inside data, with an “envelope” on the outside. In most cases we just need to take the normal data and continue with the subsequent business logic. It would be very redundant to have to manually check errcode == 0 every time we call it. One of the simplest designs is to simply return the envelope-removed data type.

suspend fun getUser(
  @Query("id") id: Int
): User

data class User(val id: Int, val name: String)

// 在主线程开启协程并更新 UI
// 🚨 危险：请求异常会让应用崩溃
lifecycleScope.launch {
  val user = retrofit.create<UserApi>().getUser(1)
  binding.userNameLabel.text = user.name
}

Exception handling

The design of returning the data type inside the envelope directly works in theory: it’s nice to call it normally, and if an exception occurs you can get specific exception information with try catch. However, by the design of the Kotlin Coroutine, we should call the wrapped suspend function directly in the main thread. If the function throws an exception, it will be thrown in the main thread, causing the application to crash. This is also evident from the function signature: once the User data type is not returned properly, the runtime can only throw an exception. The caller then has to do a try catch, which is very cumbersome to write. What’s worse is that people can forget about try catch altogether, and are likely to write the wrong

// - Kotlin 标准库的 runCatching，比 try catch 写起来舒服一点点
// - 🚨 错误的 try catch，无法捕获 launch 协程块的异常
runCatching {
  lifecycleScope.launch {
    val user = retrofit
      .create<UserService>()
      .getUser(1)
    binding.userNameLabel.text = user.name
  }
}

Watch out! The try catch in the above example was written incorrectly. If an exception is thrown inside the Coroutine block, it will still crash, because the wrong try catch is made on the Coroutine builder launch. The Coroutine builder returns immediately after opening the Coroutine block in the CoroutineScope, and the Coroutine inside the builder is executed concurrently with the code around launch. The exception logic inside the Coroutine block cannot be caught by the try catch outside of launch. The correct way to write try catch is inside the Coroutine block.

lifecycleScope.launch {
  val user = runCatching { retrofit.create<UserService>().getUser(1) }
    .onFailure { if (it is CancellationException) throw it }
    .getOrNull() ?: return@launch
  binding.userNameLabel.text = user.name
}

In addition, the try catch suspend function needs to be careful to re-throw CancellationException, otherwise the Coroutine block may not be cancelled in time. See JCIP Notes - Interruption and Cancellation for details. -kotlin-coroutines).

💡 A good wrapper design should make the right way to write the simplest , the simplest way to write by default is the right way to write.

To avoid the hassle and potential mistakes of try catch concurrent exceptions, I recommend catching all exceptions in the suspend function internal wrapper and reflecting the exceptions in the function signature.

One option is to return the type nullable. This would take advantage of Kotlin’s null-safe operator ?, ? : and !:.

suspend fun getUser(
  @Query("id") id: Int
): User?

lifecycleScope.launch {
  retrofit.create<UserApi>()
    .getUser(1)
    ?.let { binding.nameLabel.text = it.name }
}

This seems to be a rather authentic and elegant design, and is recommended. But with nullable we can’t tell the caller what type of exception has occurred. The only possibilities for the caller are success ! = null or failure == null. But this distinction is sufficient in many cases.

In addition, exceptions should be handled in a consistent place in the project, for example by showing the user a hint when errcode ! = 0, report network request exceptions, etc. Doing ad hoc (ad hoc) exception handling everywhere in the business call interface is not robust enough: people can forget to do exception handling at all, or handle it very roughly. Also, exception handling code can create a lot of redundancy and make it hard to see the normal code logic.

Retrofit’s Call Adapter can help us embed custom logic in Retrofit’s execution logic to achieve the goal of uniformly catching and handling all exceptions. A reference implementation will be given later.

As a rule of thumb, you should not be catching exceptions in general Kotlin code. That’s a code smell. Exceptions should be handled by some top-level framework code of your application to alert developers of the bugs in the code and to restart your application or its affected operation.

– Roman Elizarov, Project Lead for Kotlin

As a rule, do not catch exceptions in Kotlin business logic code. Exceptions should be handled consistently in the top-level infrastructure code of the application: for example, by escalating or retrying the steps that went wrong.

Designing ApiResponse types

In order for the caller to get the exception information, it is inevitable to stuff the return value inside a shell that reflects the success/failure result. But instead of deserializing the return format as is, we do some encapsulation. For example, in the normal case of a request, the msg field is of no use and can be omitted. The result of the request can be divided into roughly three cases.

Normal response: we can get the data needed by the subsequent business logic from the data field.
Business logic exception: the interface request is successful, but the backend returns data telling us that the business logic is abnormal and we need to display the exception information in the UI.
Other technical exceptions: network request errors, deserialization errors, etc., which we may need to escalate depending on the situation.

Implemented into the code, it can be represented by Kotlin Sealed Class.

sealed class ApiResponse {
  // 正常响应情况调用方不需要 errcode, msg
  data class Ok<T>(
    val data: T
  )
  
  data class BizError<T>(
    val errcode: Int,
    val msg: String
  ): ApiResponse<T>
  
  data class OtherError<T>(
    val throwable: Throwable
  ): ApiResponse<T>
}

suspend fun getUser(@Query("id") id: Int)
  : ApiResponse<User>

lifecycleScope.launch {
  val response = retrofit.create<UserApi>().getUser(1)
  
  // 可以使用 when 对 ApiResponse 的类型进行区分
  // 作为表达式使用的时候可以利用 when
  // 穷尽枚举的特性
  when (response) {
    is ApiResponse.Ok -> {/**/}
    is ApiResponse.BizError -> {/**/}
    is ApiResponse.OtherError -> {/**/}
  }
}

Add some null-safe syntactic sugar

It is much safer for us to reflect exceptions in the type system rather than throwing them. But the vast majority of scenarios callers don’t need, and shouldn’t have to do, such detailed exception handling. So we add a pair of extensions that allow the caller to use the syntactic sugar of Kotlin nullable.

fun <T> ApiResponse<T>.getOrNull(): T? = when(this) {
  is Ok -> data
  is BizError, is OtherError -> null
}
fun <T> ApiResponse<T>.getOrThrow(): T = when(this) {
  is Ok -> data
  is BizError -> throw BizException(errcode, msg)
  is OtherError -> throw throwable
}

class BizException(
  val errcode: Int
  override val msg: String
): RuntimeException()

// 调用方
lifecycleScope.launch {
  retrofit.create<UserApi>()
    .getUser(1)
    .getOrNull()
    ?.let { binding.nameLabel.text = it.name }
}

The naming of the functions refers to the Kotlin standard library conventions like get getOrNull , first firstOrNull : the first class throws an exception, the second returns the nullable type. Considering that client-side exception throwing is very dangerous, we name get getOrThrow to emphasize it in the method name. (Actually, you can also consider a version without the exception throwing, which nobody in the project probably uses.)

Borrow from Swift’s `guard` keyword

getOrNull is often used followed by a ? let only handles success cases: if the request is successful, use this data it to update the UI, otherwise nothing happens. If the failure case requires some action, you can use if / else or when to determine the type.

val response = retrofit.create<UserApi>().getUser(1)

if (response is ApiResponse.Ok) {
  val user: User = response.data
  // ...
} else {
  // 更新 UI 展示异常状态
  pageState.value = PageState.Error
}

If … else if too much nesting will make the code less readable, use the early exit style, we first deal with the failure case and exit the current block, so that the successful case all the way down, more simple and clear

val response = retrofit.create<UserApi>.getUser(1)

if (response !is ApiResponse.Ok) {
  pageState.value = PageState.Error
  return
}

val user: User = response.data
// ...
// 拿到非 null 的 User 继续后面的业务逻辑

But some people think that the early exit style is not robust enough, because there is a risk of forgetting to write an early exit return, causing logical errors.

Swift loves early exit so much that it added the keyword guard. guard is like if, but with an extra layer of assurance: the compiler makes sure that the else block returns or throws and exits the current block, making early exit just as robust as if … else.

guard let user = getUser(1) else {
  pageState.value = PageState.Error
  return
}

// ...
// 拿到非 null 的 User 继续后面的业务逻辑

In Kotlin, we can achieve a similar effect with the inline extensions. The key is that the block returns Nothing.

inline fun <T> ApiResponse<T>.guardOk(
  block: () -> Nothing
): T {
    if (this !is ApiResponse.Ok<T>) {
        block()
    }
    return this.data
}

val user: User = retrofit.create<UserApi>
  .getUser(1)
  .guardOk {
    pageState.value = PageState.Error
    return@launch
  }

// ...
// 拿到非 null 的 User 继续后面的业务逻辑

Implementation: Retrofit Call Adapter

In order for Retrofit to catch all exceptions, we write a CatchingCallAdapterFactory that inherits from Retrofit’s CallAdapter.Factory. This CatchingCallAdapterFactory exposes an ErrorHandler for configuring the global exception handling logic.

val retrofit = Retrofit.Builder()
  .baseUrl(/**/)
  .addCallAdapterFactory(CatchingCallAdapterFactory(
    object: CatchingCallAdapterFactory.ErrorHandler {
      // 如果是业务逻辑异常给用户展示错误信息
      override fun onBizError(errcode: Int, msg: String) {
        toast("$errcode - $msg")
      }
      // 如果是其他异常进行上报
      override fun onOtherError(throwable: Throwable) {
        report(throwable)
      }
    }  
  ))
  //...

CatchingCallAdapterFactory Reference implementation.

class CatchingCallAdapterFactory(
  val defaultErrorHandler: ErrorHandler? = null
) : CallAdapter.Factory() {

  // 用于配置全局的异常处理逻辑
  interface ErrorHandler {
    fun onBizError(errcode: Int, msg: String)
    fun onOtherError(throwable: Throwable)
  }

  override fun get(
    returnType: Type, 
    annotations: Array<out Annotation>, 
    retrofit: Retrofit
  ): CallAdapter<*, *>? {
    // suspend 函数在 Retrofit 中的返回值其实是 `Call`
    // 例如：Call<ApiResponse<User>>
    if (getRawType(returnType) != Call::class.java) return null
    check(returnType is ParameterizedType)

    // 取 Call 里边一层泛型参数
    val innerType: Type = getParameterUpperBound(0, returnType)
    // 如果不是 ApiResponse 则不由本 CallAdapter.Factory 处理
    if (getRawType(innerType) != ApiResponse::class.javava) return null
    
    // 获取后续代理
    val delegate: CallAdapter<*, *> = retrofit
      .nextCallAdapter(this, returnType, annotations)

    return CatchingCallAdapter(
      innerType, 
      retrofit, 
      delegate, 
      defaultErrorHandler
    )
  }

  class CatchingCallAdapter(
    val dataType: Type,
    val retrofit: Retrofit,
    val delegate: CallAdapter<*, *>,
    val errorHandler: ErrorHandler?
  ) : CallAdapter<Any, Call<Any>> {
    override fun responseType(): Type 
        = delegate.responseType()
    override fun adapt(call: Call<Any>): Call<Any> 
        = CatchingCall(call, dataType as ParameterizedType, errorHandler)
  }

  class CatchingCall(
    private val delegate: Call<Any>,
    private val wrapperType: ParameterizedType,
    private val errorHandler: ErrorHandler?
  ) : Call<Any> {
  
    override fun enqueue(
      // suspend 其实是 callback
      // suspend 的返回值通过这个 callback 传递
      callback: Callback<Any>
    ): Unit = delegate.enqueue(object : Callback<Any> {
      override fun onResponse(call: Call<Any>, response: Response<Any>) {
        // 无论请求响应成功还是失败都回调 Response.success
        if (response.isSuccessful) {
          val body = response.body()
          if (body is ApiResponse.BizError<*>) {
            errorHandler?.onBizError(body.errcode, body.msg)
          }
          callback.onResponse(this@CatchingCall, Response.success(body))
        } else {
          val throwable = HttpException(response.code(), response)
          errorHandler?.onOtherError(throwable)
          callback.onResponse(
            this@CatchingCall,
            Response.success(ApiResponse.OtherError(throwable))
          )
        }
      }

      override fun onFailure(call: Call<Any>, t: Throwable) {
        errorHandler?.onOtherError(t)
        callback.onResponse(
          this@CatchingCall,
          Response.success(ApiResponse.OtherError<Any>(t))
        )
      }
    })

    override fun clone(): Call<Any> = 
      CatchingCall(delegate, wrapperType, errorHandler)
    override fun execute(): Response<Any> = 
      throw UnsupportedOperationException()
    override fun isExecuted(): Boolean = delegate.isExecuted
    override fun cancel(): Unit = delegate.cancel()
    override fun isCanceled(): Boolean = delegate.isCanceled
    override fun request(): Request = delegate.request()
    override fun timeout(): Timeout = delegate.timeout()
  }
}

Implementation: Retrofit Converter

For different cases of ApiResponse, we need to configure custom JSON deserialization parsing logic. Retrofit can be adapted to different deserialization libraries by injecting custom type converters (not necessarily just JSON data formats, but also XML, Protocol Buffers, etc.) via addConverterFactory.

JSON deserialization library selection

Currently, the Kotlin project recommends using Moshi, which has much better support for Kotlin than Gson. For example, the following example.

data class User(
  val name: String
)

val user = gson.fromJson("{}", User::class.java)

println(user) // User(name=null)
user.name.length 💣// NullPointerException!

Gson creates a User type object through reflection, but Gson does not distinguish between Kotlin nullable/non-nullable types and directly returns an object with null properties, causing us to throw a null pointer exception when we subsequently use this “broken” object. Our CatchingCallAdapter is supposed to catch all exceptions, including deserialization, but Gson’s behavior escapes our exception catching logic and intrudes into the business logic code.

Moshi does not have this problem and throws a uniform JsonDataException when it gets data that it cannot parse. OtherErrorwhen caught byCatchingCallAdapter`.

The advantages of Moshi over Gson can be found at the following link.

Please don’t use Gson. 2 out of 3 maintainers agree: it’s deprecated. Use Moshi, Jackson, or kotlinx.serialization which all understand Kotlin’s nullability. Gson does not and will do dumb things, and it won’t be fixed. Please abandon it.

– Signed, a Gson maintainer.

The above quote is from Jake Wharton. Moshi is recommended as a priority for new projects, and migration is risky for projects already using Gson, so caution is advised.

With Moshi, there are currently several options.

use reflection like Gson, but with an indirect dependency on the 2.5 MiB size kotlin-reflect .
use annotation processors to generate JsonAdapter for all classes marked @JsonClass(generateAdapter = true).
same as 2 code generation, but instead of annotation processors, use Kotlin Symbol Processing.
similar to 1, but using kotlinx-metadata, which is more lightweight than kotlin-reflect.

3 and 4 are in the MoshiX project and seem to be slightly experimental; also note that code generation has the benefit of higher performance, but the generated code is not small and requires explicitly configuring the appropriate JsonAdapter for all the classes that need to be deserialized, which is a bit invasive for existing projects.

kotlinx.serialization is the official Kotlin serialization/deserialization scheme, and is also an annotated markup, code generation scheme. However, code generation is integrated into the compiler (similar to @Parcelize and KSP), and the development experience is probably better, with richer Kotlin feature support, and should be the preferred choice on Kotlin solution. However, streaming parsing is not supported at the moment, see this issue.

On balance, it seems that we can use Moshi for now, and search for replacement annotations for migration when kotlinx.serialization is mature.

Moshi Implementation

Here is the reference implementation of Moshi’s custom parsing ApiResponse, Gson is much the same

class MoshiApiResponseTypeAdapterFactory : JsonAdapter.Factory {

  override fun create(
    type: Type, 
    annotations: MutableSet<out Annotation>, 
    moshi: Moshi
  ): JsonAdapter<*>? {
    val rawType = type.rawType
    if (rawType != ApiResponse::class.java) return null

    // 获取 ApiResponse 的泛型参数，比如 User
    val dataType: Type = (type as? ParameterizedType)
      ?.actualTypeArguments?.firstOrNull() 
      ?: return null
        
    // 获取 User 的 JsonAdapter
    val dataTypeAdapter = moshi.nextAdapter<Any>(
      this, dataType, annotations
    )

    return ApiResponseTypeAdapter(rawType, dataTypeAdapter)
  }

  class ApiResponseTypeAdapter<T>(
    private val outerType: Type,
    private val dataTypeAdapter: JsonAdapter<T>
  ) : JsonAdapter<T>() {
    override fun fromJson(reader: JsonReader): T? {
      reader.beginObject()

      var errcode: Int? = null
      var msg: String? = null
      var data: Any? = null

      while (reader.hasNext()) {
        when (reader.nextName()) {
          "errcode" -> errcode = reader.nextString().toIntOrNull()
          "msg" -> msg = reader.nextString()
          "data" -> data = dataTypeAdapter.fromJson(reader)
          else -> reader.skipValue()
        }
      }

      reader.endObject()
      
      return if (errcode != 0)
        ApiResponse.BizError(
          errcode ?: -1, 
          msg ?: "N/A"
        ) as T
      else ApiResponse.Ok(
        errcode = errcode, 
        data = data
      ) as T?
    }

    // 不需要序列化的逻辑
    override fun toJson(writer: JsonWriter, value: T?): Unit 
      = TODO("Not yet implemented")
  }
}

Use.

private val moshi = Moshi.Builder()
  .add(MoshiApiResponseTypeAdapterFactory())
  .build()

val retrofit = Retrofit.Builder()
  .baseUrl(/**/)
  .addCallAdapterFactory(CatchingCallAdapterFactory(
    object: CatchingCallAdapterFactory.ErrorHandler {
      // 如果是业务逻辑异常给用户展示错误信息
      override fun onBizError(errcode: Int, msg: String) {
        toast("$errcode - $msg")
      }
      // 如果是其他异常进行上报
      override fun onOtherError(throwable: Throwable) {
        report(throwable)
      }
    }  
  .addConverterFactory(
    MoshiConverterFactory.create(moshi)
  )
  // 配置 OkHttp，API 鉴权等逻辑在这里配置
  .client(/**/)
  .build()

One More Thing: Using Result as a Return Value

The example at the beginning of the article uses the runCatching method provided by the Kotlin standard library for try catch. The return value of the runCatching method is Result, which provides a number of useful methods.

suspend fun getUser(id: Int): Result<User>

lifecycleScope.launch {
  val result = getUser(1)
    .onFailure {/**/}
    .onSuccess {/**/}

  result.isSuccess
  result.isFailure

  val exception: Throwable? = result.exceptionOrNull()
  val user1: User? = result.getOrNull()
  val user2: User = result.getOrThrow()
}

Previously, Kotlin did not allow Result to be the return value of a function. This restriction has been removed in Kotlin 1.5. This allows us to consider using Result as the return type of a Retrofit interface method.

1
2

// 需要 Kotlin 1.5
suspend fun getUser(id: Int): Result<User>

With Result, the caller gets the exception information, but cannot distinguish BizError from OtherError at the outermost level. In practice, however, few callers need to make this distinction, and it seems like a good tradeoff to make this rarely used case a bit of a pain.

Even more promising is Kotlin’s plan to make the null-safe operator also available to Result, so we can write it like this

// 需要 Kotlin 1.5，以及尚未发布的特性

// 调用示例 1：
lifecycleScope.launch {
  retrofit.create<UserApi>()
    .getUser(1)
    ?.let { binding.nameLabel.text = it.name }
}

// 调用示例 2：
lifecycleScope.launch {
  val user: User = retrofit.create<UserApi>()
    .getUser(1)
    ?.run { return@launch }
  // 拿到非 null 的 User 继续后面的业务逻辑
}

Thanks to the null-safe operator that works directly on the Result type, we don’t need to define extensions to convert to nullable types, and the calls are more streamlined.

The Call Adapter for the suspend function and the Result return value can be referenced or used directly from this library: yujinyan/retrofit-suspend-result-adapter.

If the interface in your project is wrapped in an “envelope” like the example in this article, you can write a Converter using your own deserialization library. this test case provides a reference implementation of Moshi.

Table of Contents