Jackalope / jen (public) (License: GPLv3 or later version) (since 2018-10-24) (hash sha1)
----> ABOUT:

3D rendering and computing framework based on Vulkan API.

Libraries:
- simdcpp submodule (see my simdcpp repo)
- jmath submodule (see my jmath repo)
- mesh (constexpr generation of cubes, spheres, icosahedrons subdivisions)
- atlas (1D lines and 2D rectangles cutting)
- jlib submodule (see my jlib repo)
- jrf submodule (see my jrf repo)
- vkw (Vulkan API C++ wrapper)
Modules:
- compute (run compute shaders on gpu)
- graphics (draw models with clustered forward rendering and onscreen text)
- resource manager (load meshes, models, textures, scene data from
files and create related objects in graphics module)

----> INSTALLING:

To download all the parts of this framework it's enough to launch
git clone with recursive flag:

$ git clone —recursive ssh://rocketgit@ssh.rocketgit.com/user/Jackalope/jen

After this look at git tags:

$ git tag

It is recommended to use a tagged version instead of the latest commit,
because the first commit after the tagged one mostly includes incompatible
parts of future changes for the next version.

$ git checkout v0.1.0

----> DEPENDENCIES:

To use JEN as CMake subdirectory and successfully build programs with it
you need to make sure you have all of its dependencies:
- compiler: Clang or GCC, support for C++17. Clang 10+ or GCC 9+ is recommended,
compiling on Windows OS is tricky and requires something like MinGW with MSYS,
there are also some complications to go through to make dependencies work;
- GLFW3 library, supported version is 3.2.1;
- FreeType library, if graphics module will be used;
- Vulkan API headers, and optional validation layers to debug sneaky problems,
you also need Vulkan support in your graphics driver to run compiled programs;
- LibZip can be necessary, if JRF is used to read zip files;
- CMake, for obvious reasons;
- glslangValidator to compile shader for the graphics module.

CMake must be able to find GLFW3, Vulkan and FreeType (for graphics)
with find_package().

----> HOW TO USE IT:

To use JEN, you need to add it as a subdirectory:

add_subdirecroty(${PATH_TO_JEN})

There are several configuration options:
- JEN_MODULE_COMPUTE - turn compute module on for compiling and including;
- JEN_MODULE_GRAPHICS - turn graphics module on ...;
- JEN_MULTITHREADED_DRAW_FRAME - draw_frame function will use thread pool queue
instead of linear executing;
- JEN_MODULE_RESOURCE_MANAGER - resource manager module ON, if graphics is ON;
- JEN_VLK_VALIDATION - enable Vulkan Validation Layers to debug some errors
related to JEN. This will often produce false-positive,
as well as true-positive errors.

Look in CMakeLists.txt at JenExamples repo for details on how to use and
configure JEN automatically:

$ git clone ssh://rocketgit@ssh.rocketgit.com/user/Jackalope/JenExamples

Also I recommend to compile and run examples to make sure it works correctly.

----> SUPPORTED HARDWARE:

JEN has not been tested well, because it requires running it on large amount of
different hardware to do so. It must work with mesa driver and modern
Intel i965 GPUs as well as AMD GPUs.


----> DOCUMENTATION:

You can generate Doxygen documentation, to do so
turn on any of JEN_DOXYGEN_* options and run documentation target in cmake:

$ cmake -G %1 -DJEN_DOXYGEN_HTML=ON -DJEN_DOXYGEN_LATEX=ON
$ cmake —build —target documentation

Resource manager is not documented because it still requires large enhancements.
List of commits:
Subject Hash Author Date (UTC)
Compute descriptor set improvement, support for different descriptor types 5aad5ee0e4304b2035a652f4d4ca28e7a75bc93d Jackalope 2020-02-29 16:10:50
changed doxygen commentary style in DevMemUse description 9e9a548d6c50363a841af5d4897268aadba6ebed Jackalope 2020-02-29 16:08:33
make buffer functions const 24e343bc583629d08fad4d3dadb36a2fd72a4ca7 Jackalope 2020-02-29 16:08:04
added missing check in draw_drame cc6483e804d463fe4d1c41746692ec98bfbe53e8 Jackalope 2020-02-19 12:40:19
solved freee bug, draw data have pointer to user light data instead of applying every frame f60e7d3908651512a90488966141620306d54151 Jackalope 2020-02-19 10:50:22
solved lights flickefing issue c5cad408d3d9d908ac52760ebf1c42e7d578e61e Jackalope 2020-02-19 10:48:46
apply camera and frustum in single function ac6b09e32f0afa65eef918fb7bdb935cb99da2d8 Jackalope 2020-02-18 16:32:41
draw data wrong camera aspect fix 5d9270631a2ff3232c4928d811d55079ee30e6af Jackalope 2020-02-18 16:27:11
release macro fixed ebf6d8302ac06701d5fed4d4e153e0bc5c332990 Jackalope 2020-02-18 16:26:21
vkw memory barrier and buffer memory barrier 1df196563968fe2d0ab53f989b54b502500709e1 Jackalope 2020-02-17 18:31:15
removed forgotten redundand lines 24427796fde68c4d8a92b528ee876136da9ff365 Jackalope 2020-02-17 04:28:57
fix wrong cmd buffer begin flag f02a4272c22bfe918e4df881531699c723cf1156 Jackalope 2020-02-17 04:15:22
add temporary fix for some validation layers false-positives 9787cf6494ee83fe8c7313d0d3e7ce96ad0190e4 Jackalope 2020-02-17 06:34:31
updated descriptor pool size 8384cb8149bcba04addc908f0a5e67f291eff841 Jackalope 2020-02-17 02:43:08
new types headers for shorter integers and floating points 98da0d130bb5c78c16daaf1ffc56b046e524f756 Jackalope 2020-02-17 02:33:27
framework compute module 2441c321b81324b9873288a7c0ccb1f59fa982df Jackalope 2020-02-15 03:26:36
added conditional compiling JEN_PARALLEL_DRAW_FRAME macro 2db8f1d2583983542795ce43fe67a4505f697d39 Jackalope 2020-02-15 02:43:10
disable clusters transfer if memory is HOST_VISIBLE and DEVICE_LOCAL a85177626c71f008c4decf4a88c2ef61e6f3dda1 Jackalope 2020-02-15 02:20:00
one time submit conditional flag for cmdbuffer 1151a5f79a5c1a4e1b41ef286afe5f102a551111 Jackalope 2020-02-14 20:27:44
draw frame bug fixes ed1a9f52ad8356bd490add96ecae71d2113d2dbb Jackalope 2020-02-14 18:21:27
Commit 5aad5ee0e4304b2035a652f4d4ca28e7a75bc93d - Compute descriptor set improvement, support for different descriptor types
Author: Jackalope
Author date (UTC): 2020-02-29 16:10
Committer name: Jackalope
Committer date (UTC): 2020-02-29 16:10
Parent(s): 9e9a548d6c50363a841af5d4897268aadba6ebed
Signer:
Signing key:
Signing status: N
Tree: aaef1ce2ddb09e2bd54597828af814db867892f6
File Lines added Lines deleted
src/compute/binding_set.h 45 31
src/compute/bindings.h 82 0
src/compute/compute.h 39 26
src/compute/pipeline.h 14 8
File src/compute/binding_set.h changed (mode: 100644) (index 05fdbe3..9d03ca9)
1 1 #pragma once #pragma once
2 2
3 3 #include "../graphics/draw_stages/descriptors.h" #include "../graphics/draw_stages/descriptors.h"
4 #include "bindings.h"
4 5
5 namespace jen::vk {
6 using Buffer = DeviceBufferPart;
6 namespace jen::vk
7 {
8 struct Bindings {
9 jl::rarray<const BindingBufferView> uniform_texel_buffer;
10 jl::rarray<const BindingBufferView> storage_texel_buffer;
11 jl::rarray<const BindingBuffer> uniform_buffer;
12 jl::rarray<const BindingBuffer> storage_buffer;
13 };
7 14
8 struct ComputeData
15 struct BindingsSet
9 16 { {
10 17 [[nodiscard]] Result [[nodiscard]] Result
11 init(Device *p_dev, vkw::DescrLayout setLayout, uint32_t uniform_size,
12 const Buffer *p_buffers, uint32_t buffer_count)
18 init(Device *p_dev, vkw::DescrLayout setLayout, const Bindings &bi)
13 19 { {
20 uint32_t numPoolPart = 0;
21 uint32_t numSets = 0;
22 jl::array<vkw::DescrPoolPart,4> pool_parts;
23 auto put_part = [&numSets, &numPoolPart, &pool_parts]
24 (vkw::DescrType dt, auto part) {
25 if (part.count32() > 0) {
26 pool_parts[numPoolPart].type = dt;
27 numSets += pool_parts[numPoolPart].count = part.count32();
28 ++numPoolPart;
29 }
30 };
31 put_part(vkw::DescrType::UNIFORM_TEXEL_BUFFER, bi.uniform_texel_buffer);
32 put_part(vkw::DescrType::STORAGE_TEXEL_BUFFER, bi.storage_texel_buffer);
33 put_part(vkw::DescrType::UNIFORM_BUFFER, bi.uniform_buffer);
34 put_part(vkw::DescrType::STORAGE_BUFFER, bi.storage_buffer);
35
14 36 Result res; Result res;
15 res = uniform.init(p_dev, uniform_size);
37 res = pool.init(*p_dev, {}, {pool_parts.begin(), numPoolPart}, numSets);
16 38 if (res != VK_SUCCESS) if (res != VK_SUCCESS)
17 39 return res; return res;
18 jl::array<vkw::DescrPoolPart,2> pool_parts;
19 constexpr static const auto DESCR = vkw::DescrType::STORAGE_BUFFER;
20 pool_parts = {vkw::DescrPoolPart{vkw::DescrType::UNIFORM_BUFFER, 1},
21 vkw::DescrPoolPart{DESCR, buffer_count+1}};
22 res = pool.init(p_dev->device, {}, pool_parts, 2);
23 if (res != VK_SUCCESS)
24 goto CU;
25 40
26 41 res = pool.allocate_set(p_dev->device, setLayout, &set); res = pool.allocate_set(p_dev->device, setLayout, &set);
27 if (res != VK_SUCCESS)
28 goto CP;
29
30 set.set(p_dev->device, 0, vkw::DescrType::UNIFORM_BUFFER,
31 uniform.allocation.range());
32 for (uint32_t i = 0; i < buffer_count; ++i)
33 set.set(p_dev->device, i+1, DESCR, p_buffers[i].range());
42 if (res != VK_SUCCESS) {
43 pool.destroy(p_dev->device);
44 return res;
45 }
34 46
35 return res;
47 auto &set_ = set;
48 auto set_buffers = [&set_, p_dev] (vkw::DescrType dt, auto sets) {
49 for (auto &b : sets)
50 set_.set(*p_dev, b.binding, dt, b.part.range());
51 };
52 auto set_views = [&set_, p_dev] (vkw::DescrType dt, auto sets) {
53 for (auto &b : sets)
54 set_.set(*p_dev, b.binding, dt, b.view);
55 };
56 set_views(vkw::DescrType::UNIFORM_TEXEL_BUFFER, bi.uniform_texel_buffer);
57 set_views(vkw::DescrType::STORAGE_TEXEL_BUFFER, bi.storage_texel_buffer);
58 set_buffers(vkw::DescrType::UNIFORM_BUFFER, bi.uniform_buffer);
59 set_buffers(vkw::DescrType::STORAGE_BUFFER, bi.storage_buffer);
36 60
37 CP: pool.destroy(p_dev->device);
38 CU: uniform.destroy(p_dev);
39 61 return res; return res;
40 62 } }
41 void write_uniform_data(vkw::DeviceSize offset, vkw::DeviceSize size,
42 void *p_src) {
43 jassert(offset + size <= uniform.allocation.size(),
44 "region exceeds buffer");
45 memcpy(uniform.allocation.p_data() + offset, p_src, size);
46 }
47 63 void destroy(Device *p_dev) { void destroy(Device *p_dev) {
48 64 pool.destroy(*p_dev); pool.destroy(*p_dev);
49 uniform.destroy(p_dev);
50 65 } }
51 66
52 Descriptors::UniformBuffer uniform;
53 67 vkw::DescrSet set; vkw::DescrSet set;
54 68 vkw::DescrPool pool; vkw::DescrPool pool;
55 69 }; };
File src/compute/bindings.h changed (mode: 100644) (index e69de29..1bd6055)
1 #pragma once
2
3 #include "../device/device.h"
4
5 namespace jen::vk {
6 namespace BindingUseFlag { enum {
7 TRANSFER_SRC = vkw::BufferUsage::TRANSFER_SRC,
8 TRANSFER_DST = vkw::BufferUsage::TRANSFER_DST,
9 UNIFORM_TEXEL = vkw::BufferUsage::UNIFORM_TEXEL,
10 STORAGE_TEXEL = vkw::BufferUsage::STORAGE_TEXEL,
11 UNIFORM = vkw::BufferUsage::UNIFORM,
12 STORAGE = vkw::BufferUsage::STORAGE,
13 }; }
14 using BindingUseMask = uint32_t;
15
16 struct BindingCreateInfo {
17 vkw::DeviceSize size;
18 BindingUseMask use;
19 vkw::BindNo bindingNo;
20 };
21 using BindingCreateInfos = jl::rarray<const BindingCreateInfo>;
22
23 struct BindingBuffer {
24 [[nodiscard]] Result
25 init(Device *p_d, const BindingCreateInfo &info) {
26 DevMemUsage mem_use;
27 bool map = info.use & BindingUseFlag::TRANSFER_DST
28 or
29 info.use & BindingUseFlag::TRANSFER_SRC;
30
31 if (info.use & BindingUseFlag::UNIFORM_TEXEL
32 or
33 info.use & BindingUseFlag::UNIFORM)
34 mem_use = DevMemUsage::DYNAMIC_DST;
35 else
36 mem_use = DevMemUsage::STATIC;
37
38 binding = info.bindingNo;
39
40 return p_d->buffer_allocator
41 .allocate(info.size, 0, mem_use, info.use, map, &part);
42 }
43 void destroy(Device *p_d) {
44 p_d->buffer_allocator.deallocate(part);
45 }
46
47 void write_data(vkw::DeviceSize offset, vkw::DeviceSize size, void *p_src) {
48 jassert(offset + size <= part.size(), "region exceeds buffer");
49 memcpy(part.p_data() + offset, p_src, size);
50 }
51 [[nodiscard]] Result
52 read_data(vkw::DeviceSize offset, vkw::DeviceSize size, void *p_dst) {
53 jassert(offset + size <= part.size(), "region exceeds buffer");
54 jassert(part.is_mapped() and not part.is_flush_needed(),
55 "something is wrong");
56 memcpy(p_dst, part.p_data() + offset, size);
57 return VK_SUCCESS;
58 }
59
60 DeviceBufferPart part;
61 vkw::BindNo binding;
62 };
63
64 struct BindingBufferView : BindingBuffer {
65 [[nodiscard]] Result
66 init(Device *p_d, const BindingCreateInfo &info, VkFormat format) {
67 Result res;
68 res = BindingBuffer::init(p_d, info);
69 if (res != VK_SUCCESS)
70 return res;
71 res = view.init(*p_d, part.buffer, format, part.offset(), part.size());
72 if (res != VK_SUCCESS)
73 p_d->buffer_allocator.deallocate(part);
74 return res;
75 }
76 void destroy(Device *p_d) {
77 view.destroy(*p_d);
78 p_d->buffer_allocator.deallocate(part);
79 }
80 vkw::BufferView view;
81 };
82 }
File src/compute/compute.h changed (mode: 100644) (index 966301b..1e8af63)
... ... namespace jen::vk
26 26 syncs.destroy(*p_device); syncs.destroy(*p_device);
27 27 } }
28 28 [[nodiscard]] Result [[nodiscard]] Result
29 create_pipeline(uint32_t binding_count, const char *p_file_path,
29 create_pipeline(const Bindings &bi, const char *p_shader_file_path,
30 30 ComputePipeline *p_dst) { ComputePipeline *p_dst) {
31 return p_dst->init(p_device->device, binding_count, p_file_path);
31 return p_dst->init(p_device->device, bi, p_shader_file_path);
32 32 } }
33 33 [[nodiscard]] Result [[nodiscard]] Result
34 create_buffers(uint32_t count, const vkw::DeviceSize *p_sizes,
35 Buffer *p_dst) {
34 create_bindings(BindingCreateInfos infos, BindingBuffer *p_dst) {
36 35 Result res; Result res;
37 uint32_t buse = vkw::BufferUsage::STORAGE;
38 for (uint32_t i = 0; i < count; ++i) {
39 res = p_device->buffer_allocator
40 .allocate(p_sizes[i], 0, DevMemUsage::DYNAMIC_DST, buse, true,
41 &p_dst[i]);
36 for (uint32_t i = 0; i < infos.count32(); ++i) {
37 res = p_dst[i].init(p_device, infos[i]);
42 38 if (res != VK_SUCCESS) { if (res != VK_SUCCESS) {
43 39 while (i > 0) while (i > 0)
44 p_device->buffer_allocator.deallocate(p_dst[--i]);
40 p_dst->destroy(p_device);
45 41 return res; return res;
46 42 } }
47 43 } }
48 44 return VK_SUCCESS; return VK_SUCCESS;
49 45 } }
50 46 [[nodiscard]] Result [[nodiscard]] Result
51 create_computeData(const ComputePipeline &pipeline, uint32_t uniform_size,
52 const Buffer *p_buffers, uint32_t buffer_count,
53 ComputeData *p_dst) {
54 return p_dst->init(p_device, pipeline.setLayout,
55 uniform_size, p_buffers, buffer_count);
47 create_bindings(BindingCreateInfos infos, VkFormat *p_formats,
48 BindingBufferView *p_dst) {
49 Result res;
50 for (uint32_t i = 0; i < infos.count32(); ++i) {
51 res = p_dst->init(p_device, infos[i], p_formats[i]);
52 if (res != VK_SUCCESS) {
53 while (i > 0)
54 p_dst->destroy(p_device);
55 return res;
56 }
57 }
58 return VK_SUCCESS;
59 }
60 [[nodiscard]] Result
61 create_bindingSet(const ComputePipeline &pipeline, const Bindings &bindings,
62 BindingsSet *p_dst) {
63 return p_dst->init(p_device, pipeline.setLayout, bindings);
56 64 } }
57 65 [[nodiscard]] Result [[nodiscard]] Result
58 compute(const ComputePipeline &pl, const ComputeData &cd,
66 compute(const ComputePipeline &pl, const BindingsSet &set,
59 67 math::v3u32 group_count) math::v3u32 group_count)
60 68 { {
61 69 Result res; Result res;
 
... ... namespace jen::vk
69 77 return res; return res;
70 78 cmd.cmd_set_pipeline(pl.pipeline, vkw::BindPoint::COMPUTE); cmd.cmd_set_pipeline(pl.pipeline, vkw::BindPoint::COMPUTE);
71 79
72 cmd.cmd_set_descr_sets(vkw::BindPoint::COMPUTE, pl.layout, cd.set, 0);
80 cmd.cmd_set_descr_sets(vkw::BindPoint::COMPUTE, pl.layout, set.set, 0);
73 81 cmd.cmd_dispatch(*reinterpret_cast<vkw::Vector3D*>(&group_count)); cmd.cmd_dispatch(*reinterpret_cast<vkw::Vector3D*>(&group_count));
74 82
75 83 res = cmd.end(); res = cmd.end();
 
... ... namespace jen::vk
81 89 } }
82 90
83 91 [[nodiscard]] Result [[nodiscard]] Result
84 read_data(Buffer *p_buffer, vkw::DeviceSize offset, vkw::DeviceSize size,
92 read_data(const DeviceBufferPart &buffer,
93 vkw::DeviceSize offset, vkw::DeviceSize size,
85 94 void *p_dst) { void *p_dst) {
86 jassert(offset + size <= p_buffer->size(), "region exceeds buffer");
95 jassert(offset + size <= buffer.size(), "region exceeds buffer");
87 96 Result res; Result res;
88 97 res = syncs.fences[0].wait(*p_device, vkw::TIMEOUT_INFINITE); res = syncs.fences[0].wait(*p_device, vkw::TIMEOUT_INFINITE);
89 98 if (res != VK_SUCCESS) if (res != VK_SUCCESS)
90 99 return res; return res;
91 jassert(p_buffer->is_mapped() and not p_buffer->is_flush_needed(),
92 "something wrong");
93 memcpy(p_dst, p_buffer->p_data() + offset, size);
100 jassert(buffer.is_mapped() and not buffer.is_flush_needed(),
101 "something is wrong");
102 memcpy(p_dst, buffer.p_data() + offset, size);
94 103 return VK_SUCCESS; return VK_SUCCESS;
95 104 } }
96 void destroy_compute_data(ComputeData *p_cd) {
97 p_cd->destroy(p_device);
105 void destroy_bindingSet(BindingsSet *p_set) {
106 p_set->destroy(p_device);
107 }
108 void destroy_bindings(BindingBuffer *p_bs, uint32_t count = 1) {
109 for (uint32_t i = 0; i < count; ++i)
110 p_bs[i].destroy(p_device);
98 111 } }
99 void destroy_buffers(Buffer *p_bs, uint32_t count) {
112 void destroy_bindings(BindingBufferView *p_bs, uint32_t count = 1) {
100 113 for (uint32_t i = 0; i < count; ++i) for (uint32_t i = 0; i < count; ++i)
101 p_device->buffer_allocator.deallocate(p_bs[i]);
114 p_bs[i].destroy(p_device);
102 115 } }
103 116 void destroy_pipeline(ComputePipeline *p_pl) { void destroy_pipeline(ComputePipeline *p_pl) {
104 117 p_pl->destroy(p_device->device); p_pl->destroy(p_device->device);
File src/compute/pipeline.h changed (mode: 100644) (index 61efb0e..787b911)
... ... namespace jen::vk {
6 6 struct ComputePipeline struct ComputePipeline
7 7 { {
8 8 [[nodiscard]] Result [[nodiscard]] Result
9 init(vkw::Device device, uint32_t buffer_count, const char *p_filepath)
9 init(vkw::Device device, const Bindings &bi, const char *p_shader_filepath)
10 10 { {
11 11 Result res; Result res;
12 12
13 res = shader.init(device, p_filepath);
13 res = shader.init(device, p_shader_filepath);
14 14 if (res != VK_SUCCESS) if (res != VK_SUCCESS)
15 15 return res; return res;
16 16
17 jl::array<vkw::DescrBind,1024> dbinds;
18 dbinds[0] = vkw::DescrBind::compute(0, vkw::DescrType::UNIFORM_BUFFER, 1);
19 for (uint32_t i = 0; i < buffer_count; ++i)
20 dbinds[i+1] = vkw::DescrBind::compute(i+1, vkw::DescrType::STORAGE_BUFFER,1);
21
22 res = setLayout.init(device, {dbinds.begin(), buffer_count+1});
17 jl::array<vkw::DescrBind,256> dbinds;
18 uint32_t numBinds = 0;
19 auto put_part = [&numBinds, &dbinds] (vkw::DescrType dt, auto part) {
20 for (auto &b : part)
21 dbinds[numBinds++] = vkw::DescrBind::compute(b.binding, dt, 1);
22 };
23 put_part(vkw::DescrType::UNIFORM_TEXEL_BUFFER, bi.uniform_texel_buffer);
24 put_part(vkw::DescrType::STORAGE_TEXEL_BUFFER, bi.storage_texel_buffer);
25 put_part(vkw::DescrType::UNIFORM_BUFFER, bi.uniform_buffer);
26 put_part(vkw::DescrType::STORAGE_BUFFER, bi.storage_buffer);
27
28 res = setLayout.init(device, {dbinds.begin(), numBinds});
23 29 if (res != VK_SUCCESS) if (res != VK_SUCCESS)
24 30 goto CSH; goto CSH;
25 31
Hints:
Before first commit, do not forget to setup your git environment:
git config --global user.name "your_name_here"
git config --global user.email "your@email_here"

Clone this repository using HTTP(S):
git clone https://rocketgit.com/user/Jackalope/jen

Clone this repository using ssh (do not forget to upload a key first):
git clone ssh://rocketgit@ssh.rocketgit.com/user/Jackalope/jen

Clone this repository using git:
git clone git://git.rocketgit.com/user/Jackalope/jen

You are allowed to anonymously push to this repository.
This means that your pushed commits will automatically be transformed into a merge request:
... clone the repository ...
... make some changes and some commits ...
git push origin main