Uwe Koch: Hi,
I’m interested to try out Camunda Cloud for my private use / self hosted and looking forward to use it with Optimize (really can’t await it!).
Though I successfully got Camunda Cloud running as decribed on https://docs.camunda.io/docs/self-managed/overview/ I’m struggling to startup Optimize via
https://docs.camunda.io/docs/self-managed/optimize-deployment/setup/ as well 
Is there any step-by-step guide like for the base components ?
I suppose that I additionally need an elastic container, too…
Many thanks for any hints on guides, links, videos, etc. for that scenario: Sel-fmanaged: Cam Cloud + Optimize
Uwe
Uwe Koch: Hi,
I’m interested to try out Camunda Cloud for my private use / self hosted and looking forward to use it with Optimize (really can’t await it!).
Though I successfully got Camunda Cloud running as decribed on https://docs.camunda.io/docs/self-managed/overview/ I’m struggling to startup Optimize via
https://docs.camunda.io/docs/self-managed/optimize-deployment/setup/ as well
Is there any step-by-step guide like for the base components ?
I suppose that I additionally need an elastic container, too…
Many thanks for any hints on guides, links, videos, etc. for that scenario: Sel-fmanaged: Cam Cloud + Optimize
Uwe
zell: Hey Uwe 
I have also not much experience with hosting/deploying Optimize but I know that it has a dendency to IAM, which means you also need to set up this https://docs.camunda.io/docs/self-managed/iam/what-is-iam/|https://docs.camunda.io/docs/self-managed/iam/what-is-iam/
And yes it also needs a connection to elastic, which contains exported zeebe data.
So for your deployment I would expect the following:
• Zeebe with elastic exporter
• Elasticsearch for Zeebe and Optimize
• IAM (+ postgres)
• Optimize
Hope this helps.
Other than that, do you ran into any specific issues, where I maybe can help?
Greets
Chris
Uwe Koch: Thanks, Chris.
so there seems to be more needed like keycloak, postgres, etc…?
So it would become more and more helpful (not only for me) to find a getting started guide or sth. else for the real beneit for the community: optimize for free use !
If after succcessfully managed that and I would run into additional questions, I’ll contact you 
Uwe
zell: Afaik keycloak is only necessary in newer upcoming versions 
but I might be wrong.
I will work the next months to bring all of that into the camunda cloud helm charts, so I can share my experience and will post here if something is executable to try it out. 
P.S.: the docs will I think also updated regarding this
Maximilian Mack: @zell is there a way to deactivate IAM completely for demo purposes? A complete IAM setup incl. postgres is a bit off-track atm…
zell: I know this is possible with operate but with optimize i think not. Maybe @Felix Müller (Camunda) or @menski knows?
Felix Müller (Camunda): right now there is no way to deactivate IAM for Optimize, like the idea for simple getting started though.
also a getting started guide makes lots of sense - @Uwe Koch please reach out what is not working for Optimize then we can check with the Optimize team
Maximilian Mack: ok thx for the answer… boots postgres
Uwe Koch: @Felix Müller (Camunda) I would like to have one docker-compose file like https://github.com/camunda-cloud/camunda-cloud-get-started/blob/master/docker-compose.yaml that includes sections for IAM and Optimize. So with one docker-compose command I could fire everything up (after setting passwords, secrets, etc…).
Felix Müller (Camunda): Thanks Uwe!
cc<@U0306HUR99T>
Eric Lundberg (Camunda): Thanks <@U0304449YQ1>! I’ve reviewed this with my tech lead and it makes complete sense. I’ve forwarded on the request to the owners of the repo to make it easier to get started with Optimize
Eric Lundberg (Camunda): cc @Amara Graham 
Uwe Koch: As I said, I can’t await it 
So I copied together what I could find from the docs and made a first docker-compose file which includes Zeebe, IAM and Optimize. Had to adjust some ports (hope to made it the right way). Elasticsearch should be already included by the Zeebe docker-compose part, so I hoped to re-use it for optimize.
But there seem some problems for optimize to connect to that elastic instance because of missing nodes?
09:04:16.022 [main] INFO o.c.o.s.e.OptimizeElasticsearchClientFactory - No Elasticsearch nodes available, waiting [7595] ms to retry connecting
09:04:23.630 [main] ERROR o.c.o.s.e.OptimizeElasticsearchClientFactory - Failed getting number of cluster nodes.
java.net.ConnectException: Connection refused
Perhaps anyone @Felix Müller (Camunda) can help?
Perhaps there’s also a better place for the all-in-one docker-compose file. But because I dont know where it could be, I attach it her.
zell: Cool stuff @Uwe Koch ! I really like your secrets like: NurDoenerMachtSchoener 
Maybe you have to change - OPTIMIZE_ELASTICSEARCH_HOST=localhost to - OPTIMIZE_ELASTICSEARCH_HOST=elasticsearch
Uwe Koch: Thanks @zell for the hint. Now it seems to look a bit better. Optimize now complains about elastic 7.16.1 which is not supported - had to switch to 7.16.2 instead. Seems to work with zeebe.
After all Optimize starts, but calling localhost:8090 results in {"errors":["client.notFound"]} .
Any hint?
See my actual docker.yaml attached.
zell: Tbh I’m not sure. What is the default for the network driver if you are not setting it ? I think it is also bridge right? I can see in an internal docker compose that this is not set. Maybe @Sebastian Bathke has an idea
zell: Do you see any errors in the log? Is latest really latest? Maybe use a direct version.
Uwe Koch: Optimize logs say V3.7.1. So that must be ok.
The only error i could find is in iam. It seems for me that optimize can’t connect with it’s secret.
IAM log:
2022-02-03 11:03:35.844 INFO 1 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2022-02-03 11:03:35.846 INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2022-02-03 11:03:35.855 INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 9 ms
2022-02-03 11:03:36.652 ERROR 1 --- [nio-8080-exec-6] i.c.iam.security.token.TokenService : Failed to validate token (potential attack)
io.camunda.iam.security.token.exception.ValidationSecurityException: token.invalid
at io.camunda.iam.security.token.TokenService.validate(TokenService.kt:73)
at io.camunda.iam.security.token.TokenService.parse(TokenService.kt:102)
Anything else necessary for optimize & IAM? I thought IAM_CLIENTSECRET should be enough.
Sebastian Bathke: @Uwe Koch I suspect the issue originates from the fact that IAM doesn’t know the client yet, in a local docker compose I used for testing I had the following env vars set for the iam container.
CLIENTS_0_CLIENT_ID: optimize
CLIENTS_0_NAME: Optimize
CLIENTS_0_CLIENT_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
CLIENTS_0_LOGOUT_URL: <http://localhost:8090/api/authentication/logout-callback>
CLIENTS_0_BASE_URL: <http://localhost:8090>
IAM_CLIENT_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
IAM_CLIENT_BASE_URL: <http://localhost:8090>
IAM_CLIENT_LOGOUT_URL: <http://localhost:8090/logout>
Another challenge you will encounter with a local compose setup is that the value of CAMUNDA_OPTIMIZE_IAM_ISSUER_URL is actually used for both redirecting the user in the browser for logging in to iam and for the Optimize backend to call iam to validate a token. Which is something we consider decoupling in future. With a docker compose setup I had to make use of local domains that work outside and inside of the docker network.
I can dump you my sample file here
zell: Thanks @Sebastian Bathke 
I also used your example to try it locally. I tried to apply the configs to @Uwe Koch example but it seems the keys also need to match specific requirements. I got at the end:
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [io.camunda.iam.client.crypto.ClientSecretAttributeConverter]: Constructor threw exception; nested exception is com.nimbusds.jose.KeyLengthException: The Content Encryption Key length must be 128 bits (16 bytes), 192 bits (24 bytes), 256 bits (32 bytes), 384 bits (48 bytes) or 512 bites (64 bytes)
Sebastian Bathke: hmm maybe I didn’t had the latest IAM version, can you try the 1.3.0 tag?
zell: I used 1.3.0 seems that CAMUNDA_OPTIMIZE_IAM_CLIENTSECRET="UeberDenBergIstEsKuerzerAlsZuFuss" makes issues since it is 33 bytes long 
If I use your secret, I see not that exception anymore. But still I can’t connect. I copied your example now completely. Had to replace the optimize image, but even now it doesn’t work. It seems to redirect me at least to <http://iam.localtest.me:9090/api/authorize?client_id=optimize&redirect_uri=http%3A%2F%[…]Fapi%2Fauthentication%2Fcallback&response_type=code&scope=> but then nothing.
Uwe Koch: Same redirection URL with my approach. Additionally get a {"errors":["client.notFound"]}
Sebastian Bathke: and no login form from iam in this case? is direct access to <http://iam.localtest.me:9090> working for you?
zell: For me not. :man-shrugging:
Sebastian Bathke: weird, what do you get when trying that? let’s maybe check this together when you have time, I’m blocked till 17:00 today though. Would ping tomorrow if that works for you?
zell: Sounds good!
Eric Lundberg (Camunda): Hey all,
Once we hammer out the IAM issues and have something we can share with the broader public, let’s bring in @Amara Graham to get this published!
zell: Hey folks,
I had today a Session with @Sebastian Bathke and we got it working at the end! 
Thanks for that again 
I will document here our findings so maybe @Uwe Koch can also test this out.
One problem we encountered with the shared config by @Sebastian Bathke was that I was not able to resolve <http://localtest.me|localtest.me> Based on this page https://readme.localtest.me/ this should actually possible.
Nslookup showed the following:
$ nslookup <http://localtest.me|localtest.me>
Server: 192.168.178.1
Address: 192.168.178.1#53
Non-authoritative answer:
*** Can't find <http://localtest.me|localtest.me>: No answer
After editing the /etc/hosts file:
127.0.0.1 <http://iam.localtest.me|iam.localtest.me>
It worked. So this would be solution one
Alternative:
We tried also another way, with network_mode: host setting on all containers. This worked as well. This solution has some down sides since we use no longer the docker networks and it is no longer isolated. But it works.
Resources:
• https://docs.docker.com/compose/compose-file/compose-file-v3/#network_mode
• https://earthly.dev/blog/docker-networking/#the-host-driver
• https://docs.docker.com/network/host/
Below the working config:
version: '2.4'
services:
iam-backend:
image: <http://docker.io/camunda/iam:1.3.0|docker.io/camunda/iam:1.3.0>
container_name: iam
hostname: iam
depends_on:
- iam-database
environment:
DB_URL: jdbc:<postgresql://localhost:5432/iam>
DB_USER: camunda
DB_PASSWORD: mydatabasepassword
DATABASE_ENCRYPTION_KEY: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
FRONTEND_URL: <http://localhost:9090>
BACKEND_URL: <http://localhost:9090/api>
ENFORCE_HTTPS: "false"
CLIENTS_0_CLIENT_ID: optimize
CLIENTS_0_NAME: Optimize
CLIENTS_0_CLIENT_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
CLIENTS_0_LOGOUT_URL: <http://localhost:8090/api/authentication/logout-callback>
CLIENTS_0_BASE_URL: <http://localhost:8090>
IAM_CLIENT_SECRET: XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
IAM_CLIENT_BASE_URL: <http://localhost:8090>
IAM_CLIENT_LOGOUT_URL: <http://localhost:8090/logout>
FEATURE_USER_MANAGEMENT: "true"
FEATURE_SELF_SIGN_UP: "true"
SERVER_PORT: 9090
healthcheck:
test: wget <http://localhost:8081/actuator/health> -q -O - > /dev/null 2>&1
interval: 30s
timeout: 15s
retries: 5
network_mode: host
iam-database:
image: postgres:12.7-alpine
container_name: iam-database
environment:
POSTGRES_DB: iam
POSTGRES_USER: camunda
POSTGRES_PASSWORD: mydatabasepassword
healthcheck:
test: pg_isready -d iam -U camunda
interval: 30s
timeout: 15s
retries: 5
network_mode: host
optimize:
image: camunda/optimize:3.7.1
container_name: optimize
environment:
- JAVA_OPTS=-Xms1g -Xmx1g -XX:MaxMetaspaceSize=256m
- SPRING_PROFILES_ACTIVE=ccsm
- OPTIMIZE_ELASTICSEARCH_HOST=localhost
- OPTIMIZE_API_ACCESS_TOKEN=secret
- CAMUNDA_OPTIMIZE_IAM_ISSUER_URL=<http://localhost:9090>
- CAMUNDA_OPTIMIZE_IAM_CLIENTID=optimize
- CAMUNDA_OPTIMIZE_IAM_CLIENTSECRET=XALaRPl5qwTEItdwCMiPS62nVpKs7dL7
- CAMUNDA_OPTIMIZE_SECURITY_AUTH_COOKIE_SAME_SITE_ENABLED=false
- CAMUNDA_OPTIMIZE_UI_LOGOUT_HIDDEN=true
- CAMUNDA_OPTIMIZE_ZEEBE_ENABLED=true
- CAMUNDA_OPTIMIZE_ZEEBE_NAME=zeebe-record
- CAMUNDA_OPTIMIZE_ZEEBE_PARTITION_COUNT=1
restart: always
cpu_count: 4
mem_limit: 2g
network_mode: host
elasticsearch:
image: <http://docker.elastic.co/elasticsearch/elasticsearch:7.16.2|docker.elastic.co/elasticsearch/elasticsearch:7.16.2>
container_name: elasticsearch
environment:
- cluster.name=elasticsearch
- bootstrap.memory_lock=true
- discovery.type=single-node
- "ES_JAVA_OPTS=-Xms1g -Xmx1g"
ulimits:
memlock:
soft: -1
hard: -1
restart: always
cpu_count: 4
mem_limit: 2g
network_mode: host
No matter what solution you used above important is the following:
IAM Auth Setup:
• After the containers are running (make sure to wait enough, the apps take a while to start)
• Connect to http://localhost:9090 and login via user: demo pw: demo
• Go to the Permissions Tab
• Click on the Add Button
◦ Type: write:* into the Definition test field
◦ Application should be Optimize
◦ Click the Add Button
• Got to the Roles Tab
◦ Click on the FULL_ACCESS Role
◦ Click on edit on the Permissions section
◦ Enable the new permission for Optimize
• After set up the permission you can go to http://localhost:8090
◦ Ideally you should already be logged in
◦ If not, use user: demo pw: demo again
zell:
Uwe Koch: Awesome 
. Will try it out and add Zeebe, Operate, Tasklist again.
Uwe Koch: I had no success with solution 1. I added 127.0.0.1 <http://iam.localtest.me|iam.localtest.me> but after entering demo/demo in IAM form nothing happens. No error, still the same form.
So I upgraded V2. It’s a start, but after all I must say network_mode: host can’t be a solution because you don’t have the host port / container port isolation. In our case we have both operate and tasklist working internally on port 8080 and with this setting only the first wins 
So I only can use operate or tasklist at the same time actually.
See my docker-compose file attached. It should require only the manual IAM settings for the Optimize App from above.
Any idea to move either tasklist or operate to a different port as quick fix?
Sebastian Bathke: @Uwe Koch you can make use of the env var SERVER_PORT to assign different ports to tasklist/operate
Uwe Koch: Yes, that finally works :man-with-bunny-ears-partying: thank you very much!
Uwe Koch: So here’s my working compose file.
zell: Thanks for sharing @Uwe Koch happy that it works now 
Note: This post was generated by Slack Archivist from a conversation in the Zeebe Slack, a source of valuable discussions on Zeebe (get an invite). Someone in the Slack thought this was worth sharing!
If this post answered a question for you, hit the Like button - we use that to assess which posts to put into docs.