Skip to content

Refactor LLVMCodeBuilder #642

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 79 commits into from
Aug 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
f850bcc
Remove llvmprocedure.h
adazem009 May 5, 2025
de624f1
LLVMCodeBuilder: Mark m_instruction as deprecated
adazem009 Jul 22, 2025
6330ed6
Add LLVMInstructionList class
adazem009 Jul 22, 2025
142e7e5
LLVMCodeBuilder: Rename m_instructions
adazem009 Jul 22, 2025
7af866e
LLVMCodeBuilder: Store instructions in LLVMInstructionList
adazem009 Jul 22, 2025
b07f1e1
LLVMInstructionList: Add missing const qualifiers
adazem009 Jul 22, 2025
2edff3b
LLVMInstructionList: Add containsInstruction() method
adazem009 Jul 22, 2025
7db5a2a
LLVMCodeBuilder: Use raw pointers for variable and list instructions
adazem009 Jul 22, 2025
b00c6da
LLVMCodeBuilder: Read instructions from m_instructions
adazem009 Jul 22, 2025
b0b8c7e
LLVMInstructionList: Add predicate-based contains method
adazem009 Jul 22, 2025
207b328
LLVMInstructionList: Add empty() method
adazem009 Jul 22, 2025
80032ee
Use raw pointers for loop var/list write instructions
adazem009 Jul 22, 2025
7d80b20
LLVMCodeBuilder: Drop m_instructionList
adazem009 Jul 22, 2025
0a81319
Ensure LLVM instructions are not deleted
adazem009 Jul 22, 2025
b419577
LLVMCodeBuilder: Store variable pointer in LLVMVariablePtr
adazem009 Jul 23, 2025
da38039
Implement variableTypeChanges() for basic cases
adazem009 Jul 23, 2025
5c9f495
LLVMLoopAnalyzer: Implement nested loops in variableTypeChanges()
adazem009 Jul 23, 2025
5b0c770
LLVMLoopAnalyzer: Handle if statements in variableTypeChanges()
adazem009 Jul 23, 2025
9c43261
LLVMLoopAnalyzer: Fix null variable test case for variableTypeChanges()
adazem009 Jul 24, 2025
33b37d6
LLVMLoopAnalyzer: Get write value type from a separate method
adazem009 Jul 24, 2025
2d6dc53
LLVMLoopAnalyzer -> LLVMTypeAnalyzer
adazem009 Jul 24, 2025
26b2f6d
LLVMTypeAnalyzer: Implement variableType() for basic cases
adazem009 Jul 24, 2025
2c2bf75
LLVMTypeAnalyzer: Ignore unknown pre-loop type
adazem009 Jul 24, 2025
ee90056
variableTypeChangesInLoop() -> variableTypeChangesInBranch()
adazem009 Jul 25, 2025
6cddc41
LLVMTypeAnalyzer: Handle all nested loop/if statement cases
adazem009 Jul 25, 2025
41b04d8
LLVMTypeAnalyzer: Drop variableTypeChangesInBranch()
adazem009 Jul 25, 2025
bcf26d7
LLVMTypeAnalyzer: Fix type mismatch in MultipleWritesBoolToNumber test
adazem009 Jul 25, 2025
8698a95
LLVMTypeAnalyzer: Support more edge cases
adazem009 Jul 25, 2025
54400de
LLVMTypeAnalyzer: Remove leftover dead code
adazem009 Jul 25, 2025
5b94db0
LLVMTypeAnalyzer: Fix some test case names
adazem009 Jul 26, 2025
ea7c9b7
LLVMTypeAnalyzer: Ignore writes after pos in if statements
adazem009 Jul 26, 2025
ad367dc
LLVMTypeAnalyzer: Use isVariableWrite() to check variable writes
adazem009 Jul 26, 2025
3f4d19f
LLVMTypeAnalyzer: Use Variable instead of LLVMVariablePtr
adazem009 Jul 26, 2025
e8b26f5
Revert "LLVMCodeBuilder: Store variable pointer in LLVMVariablePtr"
adazem009 Jul 26, 2025
e89f882
LLVMTypeAnalyzer: Handle cross-variable dependencies
adazem009 Jul 26, 2025
a76435b
LLVMTypeAnalyzer: Add test cases for reporters assigned to variables
adazem009 Jul 26, 2025
547c506
LLVMTypeAnalyzer: Move find branch end logic to branchEnd()
adazem009 Jul 27, 2025
005357a
LLVMTypeAnalyzer: skipBranch() -> branchStart()
adazem009 Jul 27, 2025
c9342a9
LLVMTypeAnalyzer: Implement listTypeAfterBranch() for basic cases
adazem009 Jul 27, 2025
f9afdfc
LLVMTypeAnalyzer: Handle list clear instruction
adazem009 Jul 30, 2025
ac4920e
LLVMTypeAnalyzer: Fix else branch handling in branchStart()
adazem009 Jul 30, 2025
9fc99eb
LLVMTypeAnalyzer: Implement listType() for basic cases
adazem009 Jul 31, 2025
528d3cb
LLVMTypeAnalyzer: Fix sequential if statements/loops
adazem009 Jul 31, 2025
5c77e11
LLVMTypeAnalyzer: Handle cross-list dependencies
adazem009 Aug 1, 2025
ae4690a
Add mixed test suite for LLVMTypeAnalyzer
adazem009 Aug 1, 2025
bd0d45e
LLVMTypeAnalyzer: Add pointer const qualifiers
adazem009 Aug 1, 2025
133814e
LLVMTypeAnalyzer: Do not process query point in variableType()
adazem009 Aug 1, 2025
0521d04
Use LLVMTypeAnalyzer for static type analysis
adazem009 Aug 1, 2025
846b5f8
Revert "Comment out stop all test"
adazem009 Aug 1, 2025
83f3aa5
Uncomment more engine test cases
adazem009 Aug 1, 2025
096036f
LLVMCodeBuilder: Create a global resume function in LLVMCompilerContext
adazem009 Aug 2, 2025
cdba367
Uncomment Clones test case
adazem009 Aug 2, 2025
8746ca8
Engine: Properly handle broadcasts with the same name but different case
adazem009 Aug 2, 2025
42bd6a8
Uncomment all engine bug tests
adazem009 Aug 2, 2025
9b7ae5a
Store raw data pointer in List
adazem009 Aug 3, 2025
fb7ad85
LLVMCodeBuilder: Use pointer to list data pointer
adazem009 Aug 3, 2025
26a5c63
Add LLVMTestUtils class
adazem009 Aug 3, 2025
e70709f
LLVMCodeBuilder: Move operator tests to separate test suites
adazem009 Aug 3, 2025
65e2ed8
LLVMTestUtils: Properly test NaN values
adazem009 Aug 3, 2025
ee32ec4
Do not pass IRBuilder to LLVMTypes
adazem009 Aug 3, 2025
ef69b9f
Store custom LLVM types in LLVMCompilerContext
adazem009 Aug 3, 2025
baa3435
Move external function resolve methods to a new class
adazem009 Aug 4, 2025
ba37c7f
Drop LLVMLoopScope
adazem009 Aug 4, 2025
b69af81
Move LLVM helpers to a new class
adazem009 Aug 4, 2025
4174776
LLVMCodeBuilder: Use instruction pointer instead of reference
adazem009 Aug 4, 2025
9850bed
LLVMCodeBuilder: Switch to state machine approach
adazem009 Aug 4, 2025
5695176
LLVMBuildUtils: Add builder getter
adazem009 Aug 4, 2025
d2256f8
Add a separate class for building LLVM instructions
adazem009 Aug 4, 2025
9776569
Add empty LLVM instruction groups
adazem009 Aug 4, 2025
30bfcc8
LLVMBuildUtils: Add getters for target variables and lists
adazem009 Aug 4, 2025
b1a141d
Store LLVM script function arguments in LLVMBuildUtils
adazem009 Aug 4, 2025
b53b497
LLVMBuildUtils: Add LLVM module getter
adazem009 Aug 4, 2025
4fb695e
LLVMBuildUtils: Add compiler context getter
adazem009 Aug 4, 2025
7635506
LLVMBuildUtils: Add type analyzer
adazem009 Aug 4, 2025
3198ff4
LLVMBuildUtils: Add script function getter
adazem009 Aug 4, 2025
c21ea7b
LLVMTypeAnalyzer: Add missing include guard
adazem009 Aug 5, 2025
0659c6a
Move LLVM instructions to separate classes
adazem009 Aug 5, 2025
c28d358
Rename code builder finalize method to build
adazem009 Aug 5, 2025
0c65342
Use LLVMTestUtils in LLVMCodeBuilder test
adazem009 Aug 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion include/scratchcpp/list.h
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,13 @@ class LIBSCRATCHCPP_EXPORT List : public Entity
void setMonitor(Monitor *monitor);

/*! Returns a pointer to the raw list data. */
inline ValueData *data() const { return m_dataPtr->data(); }
inline ValueData *data() const { return m_rawDataPtr; }

/*!
* Returns a pointer to pointer to the raw list data.
* \note This is used internally by compiled code for various optimizations.
*/
inline ValueData *const *dataPtr() const { return &m_rawDataPtr; }

/*!
* Returns a pointer to the list size.
Expand Down Expand Up @@ -247,6 +253,8 @@ class LIBSCRATCHCPP_EXPORT List : public Entity
value_free(&m_dataPtr->back());
m_dataPtr->erase(m_dataPtr->end());
}

m_rawDataPtr = m_dataPtr->data();
}

inline size_t getAllocSize(size_t x)
Expand All @@ -264,6 +272,7 @@ class LIBSCRATCHCPP_EXPORT List : public Entity

spimpl::unique_impl_ptr<ListPrivate> impl;
veque::veque<ValueData> *m_dataPtr = nullptr; // NOTE: accessing through pointer is faster! (from benchmarks)
ValueData *m_rawDataPtr = nullptr;
size_t m_size = 0;
};

Expand Down
4 changes: 2 additions & 2 deletions src/engine/compiler.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ std::shared_ptr<ExecutableCode> Compiler::compile(Block *startBlock, CodeType co
}

impl->block = nullptr;
return impl->builder->finalize();
return impl->builder->build();
}

while (impl->block) {
Expand Down Expand Up @@ -122,7 +122,7 @@ std::shared_ptr<ExecutableCode> Compiler::compile(Block *startBlock, CodeType co
impl->substackEnd();
}

return impl->builder->finalize();
return impl->builder->build();
}

/*!
Expand Down
43 changes: 20 additions & 23 deletions src/engine/internal/engine.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -528,8 +528,8 @@ void Engine::step()
m_frameActivity = !m_threads.empty();

// Resolve stopped broadcast scripts
std::vector<Broadcast *> resolved;
std::vector<Thread *> resolvedThreads;
std::unordered_map<Broadcast *, Thread *> stoppedBroadcasts; // sender thread
std::unordered_map<Thread *, bool> runningStatus; // sender thread

for (const auto &[broadcast, senderThread] : m_broadcastSenders) {
std::unordered_map<Broadcast *, std::vector<Script *>> *broadcastMap = nullptr;
Expand All @@ -544,44 +544,41 @@ void Engine::step()
broadcastMap = &m_broadcastMap;
}

bool found = false;
bool isRunning = false;

if (broadcastMap->find(broadcast) != broadcastMap->cend()) {
const auto &scripts = (*broadcastMap)[broadcast];

for (auto script : scripts) {
if (std::find_if(m_threads.begin(), m_threads.end(), [script](std::shared_ptr<Thread> thread) { return thread->script() == script; }) != m_threads.end()) {
found = true;
isRunning = true;
break;
}
}
}

if (found) {
// If a broadcast with the same name but different case
// was considered stopped before, restore the promise.
if (std::find(resolvedThreads.begin(), resolvedThreads.end(), senderThread) != resolvedThreads.end()) {
senderThread->promise();
resolvedThreads.erase(std::remove(resolvedThreads.begin(), resolvedThreads.end(), senderThread), resolvedThreads.end());
}
} else {
Thread *th = senderThread;
if (runningStatus.find(senderThread) == runningStatus.cend() || isRunning)
runningStatus[senderThread] = isRunning;

if (std::find_if(m_threads.begin(), m_threads.end(), [th](std::shared_ptr<Thread> thread) { return thread.get() == th; }) != m_threads.end()) {
auto promise = th->promise();
if (!isRunning)
stoppedBroadcasts[broadcast] = senderThread;
}

if (promise)
promise->resolve();
}
for (const auto &[broadcast, senderThread] : stoppedBroadcasts) {
m_broadcastSenders.erase(broadcast);

resolved.push_back(broadcast);
resolvedThreads.push_back(th);
// Resolve broadcast promise
Thread *th = senderThread;

if (std::find_if(m_threads.begin(), m_threads.end(), [th](std::shared_ptr<Thread> thread) { return thread.get() == th; }) != m_threads.end()) {
auto promise = th->promise().get();

// Resolve only if all broadcasts of the same name but different case are stopped
if (promise && !runningStatus[senderThread])
promise->resolve();
}
}

for (Broadcast *broadcast : resolved)
m_broadcastSenders.erase(broadcast);

m_redrawRequested = false;

// Step threads
Expand Down
2 changes: 1 addition & 1 deletion src/engine/internal/icodebuilder.h
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ class ICodeBuilder
public:
virtual ~ICodeBuilder() { }

virtual std::shared_ptr<ExecutableCode> finalize() = 0;
virtual std::shared_ptr<ExecutableCode> build() = 0;

virtual CompilerValue *addFunctionCall(const std::string &functionName, Compiler::StaticType returnType, const Compiler::ArgTypes &argTypes, const Compiler::Args &args) = 0;
virtual CompilerValue *addTargetFunctionCall(const std::string &functionName, Compiler::StaticType returnType, const Compiler::ArgTypes &argTypes, const Compiler::Args &args) = 0;
Expand Down
11 changes: 9 additions & 2 deletions src/engine/internal/llvm/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -1,25 +1,32 @@
target_sources(scratchcpp
PRIVATE
llvmbuildutils.cpp
llvmbuildutils.h
llvmcodebuilder.cpp
llvmcodebuilder.h
llvmregister.h
llvmconstantregister.h
llvminstruction.h
llvminstructionlist.cpp
llvminstructionlist.h
llvmifstatement.h
llvmloop.h
llvmcoroutine.cpp
llvmcoroutine.h
llvmvariableptr.h
llvmlistptr.h
llvmprocedure.h
llvmtypes.cpp
llvmtypes.h
llvmfunctions.cpp
llvmloopscope.h
llvmfunctions.h
llvmtypeanalyzer.cpp
llvmtypeanalyzer.h
llvmcompilercontext.cpp
llvmcompilercontext.h
llvmexecutablecode.cpp
llvmexecutablecode.h
llvmexecutioncontext.cpp
llvmexecutioncontext.h
)

add_subdirectory(instructions)
30 changes: 30 additions & 0 deletions src/engine/internal/llvm/instructions/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
target_sources(scratchcpp
PRIVATE
instructionbuilder.cpp
instructionbuilder.h
instructiongroup.cpp
instructiongroup.h
processresult.h
)

target_sources(scratchcpp
PRIVATE
functions.cpp
functions.h
math.cpp
math.h
comparison.cpp
comparison.h
string.cpp
string.h
logic.cpp
logic.h
control.cpp
control.h
variables.cpp
variables.h
lists.cpp
lists.h
procedures.cpp
procedures.h
)
91 changes: 91 additions & 0 deletions src/engine/internal/llvm/instructions/comparison.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
// SPDX-License-Identifier: Apache-2.0

#include "comparison.h"
#include "../llvminstruction.h"
#include "../llvmbuildutils.h"

using namespace libscratchcpp;
using namespace libscratchcpp::llvmins;

ProcessResult Comparison::process(LLVMInstruction *ins)
{
ProcessResult ret(true, ins);

switch (ins->type) {
case LLVMInstruction::Type::CmpEQ:
ret.next = buildCmpEQ(ins);
break;

case LLVMInstruction::Type::CmpGT:
ret.next = buildCmpGT(ins);
break;

case LLVMInstruction::Type::CmpLT:
ret.next = buildCmpLT(ins);
break;

case LLVMInstruction::Type::StrCmpEQCS:
ret.next = buildStrCmpEQCS(ins);
break;

case LLVMInstruction::Type::StrCmpEQCI:
ret.next = buildStrCmpEQCI(ins);
break;

default:
ret.match = false;
break;
}

return ret;
}

LLVMInstruction *Comparison::buildCmpEQ(LLVMInstruction *ins)
{
assert(ins->args.size() == 2);
const auto &arg1 = ins->args[0].second;
const auto &arg2 = ins->args[1].second;
ins->functionReturnReg->value = m_utils.createComparison(arg1, arg2, LLVMBuildUtils::Comparison::EQ);

return ins->next;
}

LLVMInstruction *Comparison::buildCmpGT(LLVMInstruction *ins)
{
assert(ins->args.size() == 2);
const auto &arg1 = ins->args[0].second;
const auto &arg2 = ins->args[1].second;
ins->functionReturnReg->value = m_utils.createComparison(arg1, arg2, LLVMBuildUtils::Comparison::GT);

return ins->next;
}

LLVMInstruction *Comparison::buildCmpLT(LLVMInstruction *ins)
{
assert(ins->args.size() == 2);
const auto &arg1 = ins->args[0].second;
const auto &arg2 = ins->args[1].second;
ins->functionReturnReg->value = m_utils.createComparison(arg1, arg2, LLVMBuildUtils::Comparison::LT);

return ins->next;
}

LLVMInstruction *Comparison::buildStrCmpEQCS(LLVMInstruction *ins)
{
assert(ins->args.size() == 2);
const auto &arg1 = ins->args[0].second;
const auto &arg2 = ins->args[1].second;
ins->functionReturnReg->value = m_utils.createStringComparison(arg1, arg2, true);

return ins->next;
}

LLVMInstruction *Comparison::buildStrCmpEQCI(LLVMInstruction *ins)
{
assert(ins->args.size() == 2);
const auto &arg1 = ins->args[0].second;
const auto &arg2 = ins->args[1].second;
ins->functionReturnReg->value = m_utils.createStringComparison(arg1, arg2, false);

return ins->next;
}
25 changes: 25 additions & 0 deletions src/engine/internal/llvm/instructions/comparison.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
// SPDX-License-Identifier: Apache-2.0

#pragma once

#include "instructiongroup.h"

namespace libscratchcpp::llvmins
{

class Comparison : public InstructionGroup
{
public:
using InstructionGroup::InstructionGroup;

ProcessResult process(LLVMInstruction *ins) override;

private:
LLVMInstruction *buildCmpEQ(LLVMInstruction *ins);
LLVMInstruction *buildCmpGT(LLVMInstruction *ins);
LLVMInstruction *buildCmpLT(LLVMInstruction *ins);
LLVMInstruction *buildStrCmpEQCS(LLVMInstruction *ins);
LLVMInstruction *buildStrCmpEQCI(LLVMInstruction *ins);
};

} // namespace libscratchcpp::llvmins
Loading