NullReferenceException #2 in TestRunnerService

[danieljsinclair]

Bug Report: NullReferenceException in TestRunnerService

Summary

The TestRunnerService class throws NullReferenceException when Unity's Test Runner callbacks are triggered outside of the MCP server's execution context.

Environment

• Unity Version: 2021.3.45f1
• MCP Unity Package: com.gamelovers.mcp-unity@219eccc2b5
• Unity Test Framework: 1.1.33
• Platform: macOS (likely affects all platforms)

Reproduction Steps

1. Open Unity project with MCP Unity package installed
2. Create a TestRunnerService instance (via "Tools/MCP Unity/Debug call path" menu or MCP server)
3. Run tests manually via Unity's Test Runner window (not through MCP)
4. Observe repeated NullReferenceException errors in console

Expected Behavior

The service should handle Unity Test Runner callbacks gracefully without throwing exceptions.

Actual Behavior

Multiple NullReferenceException errors are thrown repeatedly:

NullReferenceException: Object reference not set to an instance of an object
McpUnity.Services.TestRunnerService.TestFinished (UnityEditor.TestTools.TestRunner.Api.ITestResultAdaptor result) (at ./Library/PackageCache/com.gamelovers.mcp-unity@219eccc2b5/Editor/Services/TestRunnerService.cs:164)

ArgumentNullException: Value cannot be null. Parameter name: source
McpUnity.Services.TestRunnerService.BuildResultJson (System.Collections.Generic.List`1[T] results, UnityEditor.TestTools.TestRunner.Api.ITestResultAdaptor result) (at ./Library/PackageCache/com.gamelovers.mcp-unity@219eccc2b5/Editor/Services/TestRunnerService.cs:197)

Root Cause

The TestRunnerService constructor registers itself as a global callback with Unity's TestRunnerApi:

public TestRunnerService()
{
    _testRunnerApi = ScriptableObject.CreateInstance<TestRunnerApi>();
    _testRunnerApi.RegisterCallbacks(this);  // Global registration
}

However, the _results field is only initialized in ExecuteTestsAsync():

public async Task<JObject> ExecuteTestsAsync(...)
{
    _results = new List<ITestResultAdaptor>();  // Only initialized here
}

Problem: When Unity runs tests outside the MCP context, it calls the registered callbacks, but _results is null because ExecuteTestsAsync() was never called.

Suggested Fix

Add defensive initialization in the constructor:

public TestRunnerService()
{
    _testRunnerApi = ScriptableObject.CreateInstance<TestRunnerApi>();
    _testRunnerApi.RegisterCallbacks(this);
    _results = new List<ITestResultAdaptor>(); // Initialize here
}

Alternative: Add null checks in callback methods:

public void TestFinished(ITestResultAdaptor result)
{
    _results ??= new List<ITestResultAdaptor>();
    _results.Add(result);
}

private JObject BuildResultJson(List<ITestResultAdaptor> results, ITestResultAdaptor result)
{
    results ??= new List<ITestResultAdaptor>();
    // ... rest of method
}

Impact

• Console spam with repeated exceptions
• Poor developer experience when mixing MCP and Unity Test Runner
• Potential confusion about actual test failures

Workaround

Avoid running Unity tests manually when MCP Unity service is active, or restart Unity to clear callback state when errors occur.

[CoderGamester]

@danieljsinclair thank you for the fix suggestion
I pushed the fix to main