Firehose Enhancement: support specifying API KEY + endpoint to retreive JWT #3012
Description
Do you want to request a feature or report a bug?
This is a feature.
What is the current behavior?
When connecting to firehose, you can specify a JWT (JSON Web Token) to connect to a service provider that is running firehose. (This is not necsesary if you run your own firehose.)
The current configuration only allows to specify the JWT directly. However, JWT should expire after a few hours and so then you need to update the JWT within the configuration so it is ready next time graph connects to firehose.
In the graph we're doing server-to-server communication having a long-running JWT is not really a problem, but firehose is also used by non-server (client-side) applications, so in those cases the JWT needs to expire. Firehose cannot distinguish these 2 different cases.
What is the expected behavior?
As a hack this can be solved with an external process that stops your graph node, updates the JWT and restarts it with the updated JWT.
What we need is configuration options that provides an API key and and endpoint URL to retrieve the JWT. Then the graph code knows to get a new JWT when establishing (re-establishing) a connection. The StreamingFast team have built this sort of infrastructure in the past.
Workaround: service providers could limit access to firehose by IP address and not require JWT.