Error: rpc error: code = DeadlineExceeded desc = context deadline exceeded

Camunda 8 Topics Discussion & Questions
1

I request help to get command zbctl status to report information. Currently it times out. I’m using Linux 3.10.0-957.10.1.el7.x86_64, CentOS 7 running on a VM. I edited zeebe.cfg.toml, started elastisearch in one shell, then started bin/broker in a second shell. Is there a preferred distro of Linux I should use? Is there some config option I need to set for Linux? In a third shell, I can run the version command and get a response.

$ ~/zeebe/zeebe-broker-0.17.0/bin/zbctl version
zbctl 0.17.0 (commit: 856aca0a)
$ ~/zeebe/zeebe-broker-0.17.0/bin/zbctl status
Error: rpc error: code = DeadlineExceeded desc = context deadline exceeded
Usage:
  zbctl status [flags]

Flags:
  -h, --help   help for status

Global Flags:
      --address string   Specify the Zeebe addressFlag

from logs/zeebe.log:

21:55:29.844 [] [main] INFO  io.zeebe.gateway - Version: 0.17.0
21:55:29.847 [] [main] INFO  io.zeebe.gateway - Starting gateway with configuration {
  "enable": true,
  "network": {
    "host": "0.0.0.0",
    "port": 26500
  },
  "cluster": {
    "contactPoint": "0.0.0.0:26501",
    "transportBuffer": "128M",
    "requestTimeout": "15s"
  },
  "threads": {
    "managementThreads": 1
  }
}
21:55:32.346 [service-controller] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.broker.transport - Bound subscriptionApi.server to /0.0.0.0:26504
21:55:32.349 [service-controller] [0.0.0.0:26501-zb-actors-1] INFO  io.zeebe.transport.endpoint - Registering endpoint for node '0' with address '0.0.0.0:26502' on transport 'managementApi.client'
21:55:32.366 [service-controller] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.broker.transport - Bound managementApi.server to /0.0.0.0:26502
21:55:32.378 [] [main] INFO  io.zeebe.transport.endpoint - Registering endpoint for node '-1' with address '0.0.0.0:26501' on transport 'broker-client'
21:55:32.383 [] [main] INFO  io.zeebe.transport.endpoint - Registering endpoint for node '-1' with address '0.0.0.0:26501' on transport 'broker-client-internal'
21:55:32.387 [service-controller] [0.0.0.0:26501-zb-actors-1] INFO  io.zeebe.transport.endpoint - Registering endpoint for node '0' with address '0.0.0.0:26504' on transport 'subscriptionApi.client'
21:55:32.391 [service-controller] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.transport - Bound clientApi.server to /0.0.0.0:26501
21:55:32.397 [service-controller] [0.0.0.0:26501-zb-actors-1] INFO  io.zeebe.transport - Bound replicationApi.server to /0.0.0.0:26503
21:55:38.142 [service-controller] [0.0.0.0:26501-zb-actors-1] INFO  io.zeebe.raft - Created raft partition-0 with configuration RaftConfiguration{heartbeatInterval='250ms', electionInterval='1s', leaveTimeout='1s'}
21:55:38.211 [topology] [0.0.0.0:26501-zb-actors-1] INFO  io.zeebe.transport.endpoint - Registering endpoint for node '0' with address '0.0.0.0:26502' on transport 'managementApi.client'
21:55:38.211 [topology] [0.0.0.0:26501-zb-actors-1] INFO  io.zeebe.transport.endpoint - Registering endpoint for node '0' with address '0.0.0.0:26503' on transport 'replicationApi.client'
21:55:38.211 [topology] [0.0.0.0:26501-zb-actors-1] INFO  io.zeebe.transport.endpoint - Registering endpoint for node '0' with address '0.0.0.0:26504' on transport 'subscriptionApi.client'
21:55:39.830 [logstream.partition-0.blockIdx.writer] [0.0.0.0:26501-zb-fs-workers-1] WARN  io.zeebe.logstreams - Can't find address of snapshot position. Rebuilding block index.
21:55:40.023 [partition-0] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.raft - Joined raft in term 2
21:55:43.377 [exporter] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.logstreams - Recovering state of partition 0 from snapshot
21:55:43.674 [exporter] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.logstreams - Recovered state of partition 0 from snapshot at position 8589934592
21:55:44.023 [zb-stream-processor] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.logstreams - Recovering state of partition 0 from snapshot
21:55:44.911 [zb-stream-processor] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.logstreams - Recovered state of partition 0 from snapshot at position -1
21:55:45.255 [exporter] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.processor - Start scanning the log for error events.
21:55:45.256 [exporter] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.processor - Finished scanning the log for error events.
21:55:47.426 [exporter] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.broker.exporter.elasticsearch - Exporter opened
21:55:47.497 [zb-stream-processor] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.processor - Start scanning the log for error events.
21:55:47.499 [zb-stream-processor] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.processor - Finished scanning the log for error events.
22:28:11.199 [] [Broker close Thread] INFO  io.zeebe.broker.system - Closing...
22:28:11.572 [exporter] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.broker.exporter.elasticsearch - Exporter closed
22:28:11.695 [exporter] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.logstreams - Write snapshot for stream processor exporter at event position 12884901888.
22:28:11.751 [exporter] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.logstreams - Creation of snapshot exporter took 56 ms.
22:28:11.753 [zb-stream-processor] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.logstreams - Write snapshot for stream processor zb-stream-processor at event position -1.
22:28:11.872 [zb-stream-processor] [0.0.0.0:26501-zb-actors-0] INFO  io.zeebe.logstreams - Creation of snapshot zb-stream-processor took 119 ms.
22:28:11.989 [] [Broker close Thread] INFO  io.zeebe.broker.system - Broker closed
2

Hi @kwalker17, are you running all of these commands in the VM?

I just dowloaded the 0.17 distribution, unzipped it, then ran:

bin/broker

in one shell, and

bin/zbctl.darwin status --address 127.0.0.1

in another (I’m running this on a Mac).

I get a response from the broker.

I suggest trying this first: run it with no modifications to establish a baseline, before making any changes.

3

Hello @jwulf!

For sure I can get a non-timeout response on my iMac (not a VM). Unfortunately, the output for zbctl.darwin status was missing the line with “Partition 0 Leader”. I’ve had this problem before. … which is why I tried again on my Linux VM.

However, taking your advice to re-install zeebe-broker on my iMac has resolved the problem of zbctl.darwin status missing the line with “Partition 0 Leader”.

Any thoughts on why re-installing zeebe-broker fixed the problem on my iMac?

We will want our solution to run on Linux. How can I make it work there? Why does it time out?

Thank you for your help,
Kim

4

I downloaded and repeated the steps on an Ubuntu 19.04 machine (running natively), and it works for me.

I can start it in docker on Ubuntu (using https://github.com/jwulf/zeebe-operate-docker), which uses Debian as the base image, and that works fine for me as well.

If you do the clean install on the CentOS VM, does that fix it?

5

I retried on the CentOS 7 vm; still doesn’t work.

I downloaded and repeated the steps on my laptop (which is also CentOS 7) and… it works. Thank you!

1 Like
6