investigate coder_script ordering methods

[stirby]

Problem

We want to tackle execution ordering for coder_script in H2 2025. As discussed with the mango team, we need an engineer to investigate two potential options before committing to an RFC and timeline.

The outcome of this investigation should tell us for each approach:

• How long it would take a single engineer to ship a working MVP (above all else)
• Any existing tools that may be used to achieve the MVP
• Potential (major) technical hurdles associated in implementation

Then, it is the responsibility of product to:

• Identify potential limitations on the resulting user experience
• Reach out to customers on whether the proposed solutions solve their trouble
• investigate usage in telemetry

Option 1 optimizes for delivery while satisfying most use cases, and option 2 focuses on elegance and permanence.

Option 1: Grouped execution ordering

In a coder_script, you may define an execution_group that determines the sequence in which scripts will be run. You may guarantee the s scripts in execution group n>m will run after others in group m. All scripts in a given group will run in parallel as they do today. The default group is 0(?) for backward compatibility. This allows most scripts to be run in parallel, but still provides rudimentary ordering.

resource "coder_script" "first_script" {
    ...
    exec_group = 1
}

resource "coder_script" "second_script" {
    ...
    exec_group = 2
}

resource "coder_script" "third_script" {
    ...
    exec_group = 5     # gaps allowed
}

Option 2: Signal based

We introduce new command line tools for sending or waiting on signals in bash like so:

coder script waitfor "mysignal"
coder script signal "mysignal"

Thus, the execution ordering is defined in the bash script, and signals can be hit/waited upon arbitrarily in the scripts. One drawback is that we can't parse the script ordering from Terraform easily. There is potential to use this outside of the agent context, but for the sake of this investigation we're only focused on coder_script utilization

[matifali]

Have we considered something like depends_on = coder_script.NAME that could allow us to have a clear graph of execution order?

resource "coder_script" "first_script" {
    ...
}

resource "coder_script" "second_script" {
    ...
    depends_on = ["coder_script.first_script"]
}

resource "coder_script" "third_script" {
    ...
    depends_on = ["coder_script.second_script"]
}

resource "coder_script" "third_script2" {
    ...
    depends_on = ["coder_script.second_script"]
}

flowchart TD
    A[coder_script.first_script] --> B[coder_script.second_script]
    B --> C[coder_script.third_script]
    B --> D[coder_script.third_script2]

Loading

[stirby]

How would this work with modules @matifali?

[matifali]

@stirby We can update the modules to expose the coder_script.id as an output and explicitly pass them to be used in other scripts/modules in depends_on.

This may not be very clean, though.

[phorcys420]

I agree that depends_on would be nice, but I feel like it would also cause confusion with Terraform's built-in depends_on . And yes, would not be very clean sadly due to how Terraform handles module namespaces :-(

I am into the signal-based system because this way, you don't need to depend on a script, you can depend on part of a script instead.

So if Script A does X, Y and Z :

• You could have Script B depend on Y.
• And you could have Script C's X² subtask depend on both Script B's completion and Script A's Z subtask.

My opinion on this is that scripts can do multiple things, and a signal-based approach makes it easier, otherwise to replicate this you'd need a million scripts in a very intertwined manner.

Though the issue with signals is that debugging will probably be hell.

[phorcys420]

Another thought though, if we want ordering, I think we also have to revisit how we currently handle logging. Once you have too many modules, it's impossible to tell where a module's logs starts and where it ends.