SpvGenTwo

SpvGenTwo is a SPIR-V building and parsing library written in C++17, no other dependencies! No STL or other 3rd-Party library needed. The library comes with its own set of SPIR-V definitions generated from the machine readable grammar and therefore does not require any vulkan or spirv-headers includes. The generator can be found here: rustspvgen.

I built this library as a 'slim' backend for runtime material/shader-editors (like Proto) to avoid introducing enormous libraries like DXC (including LLVM and Frontend) or SPIRV-Tools to the codebase.

SpvGenTwo is still under development, many parts are still missing, but all the building blocks are there. SpvGenTwo is designed to be extensible and customizable, it is fairly easy to implement new SPIR-V instructions and extensions, use custom allocators and define own type inference rules. Note that it is still possible to generate invalid SPIR-V modules because not all inputs are checked yet. Use the SPIR-V validator spirv-val from the SDK and have the specification at hand while using this library.

I mainly focused on Shader capabilities, so the Kernel and OpenCL side is a bit under-developed. Any community contributions in that regard are very welcome!

Overview:

• Examples
  • Test Project
• Building
• Tools
  • SPIR-V Disassembler
  • SPIR-V Reflector
  • SPIR-V Linker
• Documentation
• Contributing
• Copyright and Licensing
• Roadmap
• Gallery
• Coverage
  • Core
  • GLSL.std.450
  • OpenCl.std.100

Examples

ConsoleLogger log;
HeapAllocator alloc; // custom user allocator

// create a new spir-v module
Module module(&alloc, &log);

// configure capabilities and extensions
module.addCapability(spv::Capability::Shader);
module.addCapability(spv::Capability::VulkanMemoryModelKHR);
module.addExtension(spv::Extension::SPV_KHR_vulkan_memory_model);
Instruction* ext = module.getExtensionInstructionImport(u8"GLSL.std.450");
module.setMemoryModel(spv::AddressingModel::Logical, spv::MemoryModel::VulkanKHR);

// global variables
Instruction* uniformVar = module.uniform<vector_t<float, 3>>(u8"u_Position");

// float add(float x, float y)
Function& funcAdd = module.addFunction<float, float, float>(u8"add", spv::FunctionControlMask::Const);
{
    BasicBlock& bb = *funcAdd; // get entry block to this function

    Instruction* x = funcAdd.getParameter(0);
    Instruction* y = funcAdd.getParameter(1);

    Instruction* z = bb.Add(x, y);
    bb.returnValue(z);
}

// void entryPoint();
{
    EntryPoint& entry = module.addEntryPoint(spv::ExecutionModel::Fragment, u8"main");
    entry.addExecutionMode(spv::ExecutionMode::OriginUpperLeft);
    BasicBlock& bb = *entry; // get entry block to this function

    Instruction* uniVec = bb->opLoad(uniformVar);
    Instruction* cross = bb.ext<GLSL>()->opCross(uniVec, uniVec); // use GLSL.std.450 extension
    Instruction* s = bb->opDot(cross, uniVec);
    entry->call(&funcAdd, s, s); // call add(s, s)
    entry->opReturn();
}

// custom spir-v binary serializer:
BinaryFileWriter writer("test.spv");
module.finalizeAndWrite(writer);

The resulting SPIR-V binary when disassembled using spirv-dis:

; SPIR-V
; Version: 1.0
; Generator: SpvGenTwo SPIR-V IR Tools(30); 0
; Bound: 20
; Schema: 0
               OpCapability Shader
               OpCapability VulkanMemoryModelKHR
               OpExtension "SPV_KHR_vulkan_memory_model"
          %1 = OpExtInstImport "GLSL.std.450"
               OpMemoryModel Logical VulkanKHR
               OpEntryPoint Fragment %main "main" %u_Position
               OpExecutionMode %main OriginUpperLeft
               OpName %u_Position "u_Position"
               OpName %add "add"
               OpName %main "main"
      %float = OpTypeFloat 32
    %v3float = OpTypeVector %float 3
%_ptr_Uniform_v3float = OpTypePointer Uniform %v3float
          %9 = OpTypeFunction %float %float %float
       %void = OpTypeVoid
          %3 = OpTypeFunction %void
 %u_Position = OpVariable %_ptr_Uniform_v3float Uniform
        %add = OpFunction %float Const %9
         %11 = OpFunctionParameter %float
         %12 = OpFunctionParameter %float
         %13 = OpLabel
         %14 = OpFAdd %float %11 %12
               OpReturnValue %14
               OpFunctionEnd
       %main = OpFunction %void None %3
         %15 = OpLabel
         %16 = OpLoad %v3float %u_Position None
         %17 = OpExtInst %v3float %1 Cross %16 %16
         %18 = OpDot %float %17 %16
         %19 = OpFunctionCall %float %add %18 %18
               OpReturn
               OpFunctionEnd

Test Project

Set CMake option SPVGENTWO_BUILD_TESTS to TRUE to build included examples:

• Types
• Constants
• ControlFlow
• FunctionCall
• Extensions
• ExpressionGraph
• Linkage
• FragmentShader
• GeometryShader
• ComputeShader

Project Structure

SpvGenTwo is split into 5 folders:

• lib contains the foundation to generate SPIR-V code. SpvGenTwo makes excessive use of its allocator interface, no memory is allocated from the heap. SpvGenTwo comes with its on set of container classes: List, Vector, String and HashMap. Those are not built for performance, but they shouldn't be much worse than standard implementations (okay maybe my HashMap is not as fast as unordered_map, build times are quite nice though :). Everything within this folders is pure C++17, no other dependencies (given that SPVGENTWO_REPLACE_PLACEMENTNEW and SPVGENTWO_REPLACE_TRAITS are used).
• common contains some convenience implementations of abstract interfaces: HeapAllocator uses C malloc and free, BindaryFileWriter uses fopen, ConsoleLogger uses vprintf, ModulePrinter uses snprintf. It also has some additional classes like Callable (std::function replacement), Graph, ControlFlowGraph, Expression and ExprGraph, they follow the same design principles and might sooner or later be moved to lib if needed.
• test contains small, self-contained code snippets that each generate a SPIR-V module to show some of the fundamental mechanics and APIs of SpvGenTwo.
• dis is a SPIR-V disassembler tool like spirv-dis to print assembly language text.
• refl is a SPIR-V reflection tool like SPIRV-Reflect to extract descriptor bindings and other relevant info from SPIR-V binary modules.
• link is a SPIR-V linker like spirv-link for merging symbols of multiple modules into a new output module.
• test contains Catch2 unit tests. When using Visual Studio, Test Adapter for Catch2 can be used with the test/catch2.runsettings config file.

Building

Use the supplied CMakeLists.txt to generate project files for your build system. SpvGenTwo allows the user to use standard library headers (<type_traits>, <new> etc) instead of my hand-made replacements (see stdreplament.h).

• SPVGENTWO_BUILD_TESTS is set to FALSE by default. If TRUE, an executable with sources from the 'test' folder will be built.
  • Note that the SpvGenTwoTest executable project depends on SPIR-V Tools which requires python to be present for its tests (even when building without tests facepalm)
• SPVGENTWO_BUILD_DISASSEMBLER is set to FALSE by default. If TRUE, an executable with sources from the 'dis' folder will be built.
• SPVGENTWO_BUILD_REFLECT is set to FALSE by default. If TRUE, an executable with sources from the 'refl' folder will be built.
• SPVGENTWO_BUILD_LINKER is set to FALSE by default. If TRUE, an executable with sources from the 'link' folder will be built.
• SPVGENTWO_REPLACE_PLACEMENTNEW is set to TRUE by default. If FALSE, placement-new will be included from <new> header.
• SPVGENTWO_REPLACE_TRAITS is set to TRUE by default. If FALSE, <type_traits> and <utility> header will be included under spvgentwo::stdrep namespace.
• SPVGENTWO_LOGGING is set to TRUE by default, calls to module.log() will have not effect if FALSE.
• SPVGENTWO_ENABLE_WARNINGS is set to TRUE by default and will enable most pedantic warnings-as-errors for all targets except the tests.
• SPVGENTWO_ENABLE_OPERANDVALIDATION is set to TRUE by default and enables an additional validation step for every makeOp/opXZY call which is not necessary for non-development builds.

Note that I mainly develop on Windows using Clang and MSVC but I'll also try to support GCC/linux. I don't have any Apple hardware so I can't debug any issues there, but you are welcome to contribute fixes for this platform.

Tools

SpvGenTwo includes a couple of CLI tools to explore and test the libraries capabilities.

SPIR-V Disassembler

SpvGenTwoDisassembler source can be found at [dis/source/dis.cpp](dis/source/dis