We´re running into severe issues with running the camunda-spring-boot-client in a containerised environment, where the container’s filesystem must be read-only.
This restriction may not be circumvented in our environments and causes problems, because the camunda client apparently needs to cache the oauth token it receives after performing a login.
For workers this only results in an ugly error message, but they still work.
But the @Deployment-annotation appears to just break: no Deployment happens AND the application shuts down.
Caused by: java.nio.file.FileSystemException: /home/nonroot/.camunda: Read-only file system
I have tried to work around our environments Read-only-filesystem-restriction, but nothing works.
Is there any workaround-configuration for amunda itself? I have found nothing in the documentation but do not believe, that we are the first and only people to run a camunda client in a read-only environment.
You need to configure the OAuth token cache to use a writable location in your read-only filesystem environment. I found the following relevant resources:
Are you allowed to use ephemeral write storage?
ie. map a ramdisk to /home/nonroot and allow Camunda to write to RAM?
When the container is torn down, the contents of the ramdisk are destroyed, so nothing is actually written.
I now used a workaround mounting an ephemeral volume under /tmp
Configuring the oauth-credential-storage-path to /tmp/camunda-tokens then works.
Just configuring that path to /tmp kept failing (silently).
It´s a counterintuitive interaction and one that I think should be addressed at some point in the future.
thank you for reaching out on this and sorry to hear about the trouble with the cache. I’m glad you could find a workaround and agree that this is counter-intuitive.
I would be interested in your perspective on my statement that I rather want to make in-memory the default, as I consider the usage of the file system cache to be rather rare, given that I expect the majority of users to run their workers in a stateless setup.
I find it very interesting to see the history of this design choice and the issues it has caused in various environments.
Personally I feel like I lack the overview of the multitude of runtime environments that may or may not benefit from a file-system cache to judge what the default should be. But I think that it would be beneficial, if Camunda were to offer a configuration parameter that allowed the use of in-memory caching instead of file-system caching.
If in-memory caching ends up proving to be preferable, maybe it could be the default in some future iteration.
Thank you for taking an interest in this issue I think it´s great when community issues result in considerations for improving Camunda!