8.4.11: JIT issue with Generator

[AlexAT]

Description

php.ini

zend_extension=opcache.so
opcache.enable_cli=on
opcache.jit=tracing
opcache.jit_buffer_size=32M

Reproducer code:

<?php

function test()
{
  $iterations = 1000000;
  $counter = 0;

  yield true;
  while (true) {
    $counter++;
    if ($counter == $iterations) break;
if (mt_rand(0, 10000000) > 9999999) { echo("$counter\n"); return; } # take note iteration count is lower!
    yield true;
  }
  echo($counter."\n");
}

foreach (test() as $v) { }

Bug description:
Results in loop mostly (but not always!) terminating by injected mt_rand() based termination condition at fixed counter value of "65".

It seems counter variable somehow stops increasing at generator invocations. In Windows build it does stop at "65" as well.

If mt_rand() termination is commented out, script expectedly never completes at wrong ("65") runs, so most of the time, while it should always complete at 1000000 iterations. I do not know why it completes sometimes, maybe due to address space randomization, maybe due to something else.

Disabling JIT resolves the issue, the script always works correctly.

Confirmed in both 8.4.10 for Linux x86-64 NTS built from source and 8.4.11 NTS Windows ZIP build downloaded from PHP site.

Correct output for multiple consecutive runs (random values and "1000000" completions as expected):

963792
956411
1000000
209645
1000000
75357
1000000
842156
512774
1000000

...

Problem occurence output (take note how much "65" appears there):

822760
65
65
1000000
65
65
65
65
1000000
65

...

Code workaround found (should help to debug it):

<?php

function test()
{
  $iterations = 1000000;
  $counter = 0;

  yield true;
  while (true) {
    if ($counter == $iterations) break;
    $counter++;
if (mt_rand(0, 10000000) > 9999999) { echo("$counter\n"); return; } # take note iteration count is lower!
    yield true;
  }
  echo($counter."\n");
}

foreach (test() as $v) { }

(basically, swapping if() clause and counter increment)
makes the script always work properly.

Please tell if I can add anything else to that report.

PHP Version

PHP 8.4.10 (cli) (built: Jul 11 2025 10:12:19) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.4.10, Copyright (c) Zend Technologies
    with Zend OPcache v8.4.10, Copyright (c), by Zend Technologies

PHP 8.4.11 (cli) (built: Jul 29 2025 18:02:29) (NTS Visual C++ 2022 x64)
Copyright (c) The PHP Group                                                                                                                                                                             
Zend Engine v4.4.11, Copyright (c) Zend Technologies
    with Zend OPcache v8.4.11, Copyright (c), by Zend Technologies

Operating System

Oracle Linux 10 x86-64, Windows 11 x86-64

[AlexAT]

Apparently the count of "65" iterations before issue occurence is not random.
Changing opcache.jit_hot_loop setting from 64 to 128 made this number "129".
Setting it to 0 (avoiding hot loop compilation) resolves the issue.
So somehow related to hot loops compilation.

[nielsdos]

I made a slight modification to trigger it more easily:

function test()
{
  $counter = 0;

  while (true) {
    $counter++;
    if ($counter == 1000000) break;
if (\mt_rand(0, 0) > 9999999) { echo("$counter\n"); return; } # take note iteration count is lower!
    yield true;
  }
  echo($counter."\n");
}

On 8.3 we get the following JIT trace checks:

TRACE-2$test$8: ; (unknown)
	sub $0x10, %rsp
	mov $EG(jit_trace_num), %rax
	mov $0x2, (%rax)
	cmp $0x4, 0x58(%r14)
	jnz jit$$trace_exit_0
	add $0x1, 0x50(%r14) ###### 0x50(%r14) corresponds to $counter, it is increased here by 1
	jo jit$$trace_exit_1
etc

On 8.4 we get this:

TRACE-2$test$8:
	leaq EG(jit_trace_num)(%rip), %rax
	movl $2, (%rax)
	cmpb $4, 0x58(%r14)
	jne jit$$trace_exit_0
	movq 0x50(%r14), %rax ###### $counter value loaded
	movq %rax, %rbx
	addq $1, %rbx ###### %rbx = counter value, but not stored back
	jo jit$$trace_exit_1
etc

In the 8.4 assembly the value of %rbx is never stored back and so the increment is lost.

EDIT: forcing the variable to be stored by modifying zend_jit_store_var_if_necessary_ex works around the issue
EDIT 2: Dumping register allocation shows that the "store" flag is not set for $counter even though I think it should. Although on 8.3 it also doesn't have ZREG_STORE set...

[AlexAT]

Could it be it considers RBX to stay allocated inside the whole loop (register allocated for variable inside the function/loop context) and does not take it into account this is a generator loop and so allocated registers will all be lost on yield?

Not sure how it's done internally but it feels sane to terminate register allocation context by yield.
Basically issuing yield should store all register-allocated variables back to their respective locations and reset all register allocations.

[arnaud-lb]

Something like this should fix the issue:

diff --git a/ext/opcache/jit/zend_jit_vm_helpers.c b/ext/opcache/jit/zend_jit_vm_helpers.c
index f4d261cd753..1a584a2c6e8 100644
--- a/ext/opcache/jit/zend_jit_vm_helpers.c
+++ b/ext/opcache/jit/zend_jit_vm_helpers.c
@@ -937,11 +937,16 @@ zend_jit_trace_stop ZEND_FASTCALL zend_jit_trace_execute(zend_execute_data  *ex,
                        break;
                }
 
+               const zend_op *prev = opline;
                handler = (zend_vm_opcode_handler_t)ZEND_OP_TRACE_INFO(opline, offset)->call_handler;
 #ifdef HAVE_GCC_GLOBAL_REGS
                handler();
                if (UNEXPECTED(opline == zend_jit_halt_op)) {
-                       stop = ZEND_JIT_TRACE_STOP_RETURN;
+                       if (prev->opcode == ZEND_YIELD) {
+                               stop = ZEND_JIT_TRACE_STOP_INTERPRETER;
+                       } else {
+                               stop = ZEND_JIT_TRACE_STOP_RETURN;
+                       }
                        opline = NULL;
                        halt = ZEND_JIT_TRACE_HALT;
                        break;
@@ -951,7 +956,11 @@ zend_jit_trace_stop ZEND_FASTCALL zend_jit_trace_execute(zend_execute_data  *ex,
                rc = handler(ZEND_OPCODE_HANDLER_ARGS_PASSTHRU);
                if (rc != 0) {
                        if (rc < 0) {
-                               stop = ZEND_JIT_TRACE_STOP_RETURN;
+                               if (prev->opcode == ZEND_YIELD) {
+                                       stop = ZEND_JIT_TRACE_STOP_INTERPRETER;
+                               } else {
+                                       stop = ZEND_JIT_TRACE_STOP_RETURN;
+                               }
                                opline = NULL;
                                halt = ZEND_JIT_TRACE_HALT;
                                break;

Currently YIELD ends the trace with ZEND_JIT_TRACE_STOP_RETURN and is handled mostly like RETURN. Changing to ZEND_JIT_TRACE_STOP_INTERPRETER causes JIT to consider that variables are live after the trace returns E.g. in

php-src/ext/opcache/jit/zend_jit_trace.c

Lines 3299 to 3303 in 18276a8

	for (i = 0; i < op_array->last_var + op_array->T; i++) {
	int var = STACK_VAR(stack, i);
	if (var >= 0 && RA_HAS_REG(var)
	&& !(RA_REG_FLAGS(var) & (ZREG_LOAD|ZREG_STORE|ZREG_LAST_USE))) {
	RA_REG_FLAGS(var) |= ZREG_STORE;

[AlexAT]

Oh yeah, reverse cause, it considers local variables are not needed anymore then like with return. Seems like so.