wgpu_core/command/
bundle.rs

1/*! Render Bundles
2
3A render bundle is a prerecorded sequence of commands that can be replayed on a
4command encoder with a single call. A single bundle can replayed any number of
5times, on different encoders. Constructing a render bundle lets `wgpu` validate
6and analyze its commands up front, so that replaying a bundle can be more
7efficient than simply re-recording its commands each time.
8
9Not all commands are available in bundles; for example, a render bundle may not
10contain a [`RenderCommand::SetViewport`] command.
11
12Most of `wgpu`'s backend graphics APIs have something like bundles. For example,
13Vulkan calls them "secondary command buffers", and Metal calls them "indirect
14command buffers". Although we plan to take advantage of these platform features
15at some point in the future, for now `wgpu`'s implementation of render bundles
16does not use them: at the hal level, `wgpu` render bundles just replay the
17commands.
18
19## Render Bundle Isolation
20
21One important property of render bundles is that the draw calls in a render
22bundle depend solely on the pipeline and state established within the render
23bundle itself. A draw call in a bundle will never use a vertex buffer, say, that
24was set in the `RenderPass` before executing the bundle. We call this property
25'isolation', in that a render bundle is somewhat isolated from the passes that
26use it.
27
28Render passes are also isolated from the effects of bundles. After executing a
29render bundle, a render pass's pipeline, bind groups, and vertex and index
30buffers are are unset, so the bundle cannot affect later draw calls in the pass.
31
32A render pass is not fully isolated from a bundle's effects on push constant
33values. Draw calls following a bundle's execution will see whatever values the
34bundle writes to push constant storage. Setting a pipeline initializes any push
35constant storage it could access to zero, and this initialization may also be
36visible after bundle execution.
37
38## Render Bundle Lifecycle
39
40To create a render bundle:
41
421) Create a [`RenderBundleEncoder`] by calling
43   [`Global::device_create_render_bundle_encoder`][Gdcrbe].
44
452) Record commands in the `RenderBundleEncoder` using functions from the
46   [`bundle_ffi`] module.
47
483) Call [`Global::render_bundle_encoder_finish`][Grbef], which analyzes and cleans up
49   the command stream and returns a `RenderBundleId`.
50
514) Then, any number of times, call [`render_pass_execute_bundles`][wrpeb] to
52   execute the bundle as part of some render pass.
53
54## Implementation
55
56The most complex part of render bundles is the "finish" step, mostly implemented
57in [`RenderBundleEncoder::finish`]. This consumes the commands stored in the
58encoder's [`BasePass`], while validating everything, tracking the state,
59dropping redundant or unnecessary commands, and presenting the results as a new
60[`RenderBundle`]. It doesn't actually execute any commands.
61
62This step also enforces the 'isolation' property mentioned above: every draw
63call is checked to ensure that the resources it uses on were established since
64the last time the pipeline was set. This means the bundle can be executed
65verbatim without any state tracking.
66
67### Execution
68
69When the bundle is used in an actual render pass, `RenderBundle::execute` is
70called. It goes through the commands and issues them into the native command
71buffer. Thanks to isolation, it doesn't track any bind group invalidations or
72index format changes.
73
74[Gdcrbe]: crate::global::Global::device_create_render_bundle_encoder
75[Grbef]: crate::global::Global::render_bundle_encoder_finish
76[wrpeb]: crate::global::Global::render_pass_execute_bundles
77!*/
78
79#![allow(clippy::reversed_empty_ranges)]
80
81use alloc::{
82    borrow::{Cow, ToOwned as _},
83    string::String,
84    sync::Arc,
85    vec::Vec,
86};
87use core::{
88    convert::Infallible,
89    num::{NonZeroU32, NonZeroU64},
90    ops::Range,
91};
92
93use arrayvec::ArrayVec;
94use thiserror::Error;
95
96use wgpu_hal::ShouldBeNonZeroExt;
97use wgt::error::{ErrorType, WebGpuError};
98
99#[cfg(feature = "trace")]
100use crate::command::ArcReferences;
101use crate::{
102    binding_model::{BindError, BindGroup, PipelineLayout},
103    command::{
104        BasePass, BindGroupStateChange, ColorAttachmentError, DrawError, IdReferences, MapPassErr,
105        PassErrorScope, RenderCommand, RenderCommandError, StateChange,
106    },
107    device::{
108        AttachmentData, Device, DeviceError, MissingDownlevelFlags, RenderPassContext,
109        SHADER_STAGE_COUNT,
110    },
111    hub::Hub,
112    id,
113    init_tracker::{BufferInitTrackerAction, MemoryInitKind, TextureInitTrackerAction},
114    pipeline::{PipelineFlags, RenderPipeline, VertexStep},
115    resource::{
116        Buffer, DestroyedResourceError, Fallible, InvalidResourceError, Labeled, ParentDevice,
117        RawResourceAccess, TrackingData,
118    },
119    resource_log,
120    snatch::SnatchGuard,
121    track::RenderBundleScope,
122    Label, LabelHelpers,
123};
124
125use super::{pass, render_command::ArcRenderCommand, DrawCommandFamily, DrawKind};
126
127/// Describes a [`RenderBundleEncoder`].
128#[derive(Clone, Debug, Default, PartialEq, Eq, Hash)]
129#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
130pub struct RenderBundleEncoderDescriptor<'a> {
131    /// Debug label of the render bundle encoder.
132    ///
133    /// This will show up in graphics debuggers for easy identification.
134    pub label: Label<'a>,
135    /// The formats of the color attachments that this render bundle is capable
136    /// to rendering to.
137    ///
138    /// This must match the formats of the color attachments in the
139    /// renderpass this render bundle is executed in.
140    pub color_formats: Cow<'a, [Option<wgt::TextureFormat>]>,
141    /// Information about the depth attachment that this render bundle is
142    /// capable to rendering to.
143    ///
144    /// The format must match the format of the depth attachments in the
145    /// renderpass this render bundle is executed in.
146    pub depth_stencil: Option<wgt::RenderBundleDepthStencil>,
147    /// Sample count this render bundle is capable of rendering to.
148    ///
149    /// This must match the pipelines and the renderpasses it is used in.
150    pub sample_count: u32,
151    /// If this render bundle will rendering to multiple array layers in the
152    /// attachments at the same time.
153    pub multiview: Option<NonZeroU32>,
154}
155
156#[derive(Debug)]
157#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
158pub struct RenderBundleEncoder {
159    base: BasePass<RenderCommand<IdReferences>, Infallible>,
160    parent_id: id::DeviceId,
161    pub(crate) context: RenderPassContext,
162    pub(crate) is_depth_read_only: bool,
163    pub(crate) is_stencil_read_only: bool,
164
165    // Resource binding dedupe state.
166    #[cfg_attr(feature = "serde", serde(skip))]
167    current_bind_groups: BindGroupStateChange,
168    #[cfg_attr(feature = "serde", serde(skip))]
169    current_pipeline: StateChange<id::RenderPipelineId>,
170}
171
172impl RenderBundleEncoder {
173    pub fn new(
174        desc: &RenderBundleEncoderDescriptor,
175        parent_id: id::DeviceId,
176    ) -> Result<Self, CreateRenderBundleError> {
177        let (is_depth_read_only, is_stencil_read_only) = match desc.depth_stencil {
178            Some(ds) => {
179                let aspects = hal::FormatAspects::from(ds.format);
180                (
181                    !aspects.contains(hal::FormatAspects::DEPTH) || ds.depth_read_only,
182                    !aspects.contains(hal::FormatAspects::STENCIL) || ds.stencil_read_only,
183                )
184            }
185            // There's no depth/stencil attachment, so these values just don't
186            // matter.  Choose the most accommodating value, to simplify
187            // validation.
188            None => (true, true),
189        };
190
191        // TODO: should be device.limits.max_color_attachments
192        let max_color_attachments = hal::MAX_COLOR_ATTACHMENTS;
193
194        //TODO: validate that attachment formats are renderable,
195        // have expected aspects, support multisampling.
196        Ok(Self {
197            base: BasePass::new(&desc.label),
198            parent_id,
199            context: RenderPassContext {
200                attachments: AttachmentData {
201                    colors: if desc.color_formats.len() > max_color_attachments {
202                        return Err(CreateRenderBundleError::ColorAttachment(
203                            ColorAttachmentError::TooMany {
204                                given: desc.color_formats.len(),
205                                limit: max_color_attachments,
206                            },
207                        ));
208                    } else {
209                        desc.color_formats.iter().cloned().collect()
210                    },
211                    resolves: ArrayVec::new(),
212                    depth_stencil: desc.depth_stencil.map(|ds| ds.format),
213                },
214                sample_count: {
215                    let sc = desc.sample_count;
216                    if sc == 0 || sc > 32 || !sc.is_power_of_two() {
217                        return Err(CreateRenderBundleError::InvalidSampleCount(sc));
218                    }
219                    sc
220                },
221                multiview_mask: desc.multiview,
222            },
223
224            is_depth_read_only,
225            is_stencil_read_only,
226            current_bind_groups: BindGroupStateChange::new(),
227            current_pipeline: StateChange::new(),
228        })
229    }
230
231    pub fn dummy(parent_id: id::DeviceId) -> Self {
232        Self {
233            base: BasePass::new(&None),
234            parent_id,
235            context: RenderPassContext {
236                attachments: AttachmentData {
237                    colors: ArrayVec::new(),
238                    resolves: ArrayVec::new(),
239                    depth_stencil: None,
240                },
241                sample_count: 0,
242                multiview_mask: None,
243            },
244            is_depth_read_only: false,
245            is_stencil_read_only: false,
246
247            current_bind_groups: BindGroupStateChange::new(),
248            current_pipeline: StateChange::new(),
249        }
250    }
251
252    pub fn parent(&self) -> id::DeviceId {
253        self.parent_id
254    }
255
256    /// Convert this encoder's commands into a [`RenderBundle`].
257    ///
258    /// We want executing a [`RenderBundle`] to be quick, so we take
259    /// this opportunity to clean up the [`RenderBundleEncoder`]'s
260    /// command stream and gather metadata about it that will help
261    /// keep [`ExecuteBundle`] simple and fast. We remove redundant
262    /// commands (along with their side data), note resource usage,
263    /// and accumulate buffer and texture initialization actions.
264    ///
265    /// [`ExecuteBundle`]: RenderCommand::ExecuteBundle
266    pub(crate) fn finish(
267        self,
268        desc: &RenderBundleDescriptor,
269        device: &Arc<Device>,
270        hub: &Hub,
271    ) -> Result<Arc<RenderBundle>, RenderBundleError> {
272        let scope = PassErrorScope::Bundle;
273
274        device.check_is_valid().map_pass_err(scope)?;
275
276        let bind_group_guard = hub.bind_groups.read();
277        let pipeline_guard = hub.render_pipelines.read();
278        let buffer_guard = hub.buffers.read();
279
280        let mut state = State {
281            trackers: RenderBundleScope::new(),
282            pipeline: None,
283            bind: (0..hal::MAX_BIND_GROUPS).map(|_| None).collect(),
284            vertex: Default::default(),
285            index: None,
286            flat_dynamic_offsets: Vec::new(),
287            device: device.clone(),
288            commands: Vec::new(),
289            buffer_memory_init_actions: Vec::new(),
290            texture_memory_init_actions: Vec::new(),
291            next_dynamic_offset: 0,
292        };
293
294        let indices = &state.device.tracker_indices;
295        state.trackers.buffers.set_size(indices.buffers.size());
296        state.trackers.textures.set_size(indices.textures.size());
297
298        let base = &self.base;
299
300        for command in &base.commands {
301            match command {
302                &RenderCommand::SetBindGroup {
303                    index,
304                    num_dynamic_offsets,
305                    bind_group,
306                } => {
307                    let scope = PassErrorScope::SetBindGroup;
308                    set_bind_group(
309                        &mut state,
310                        &bind_group_guard,
311                        &base.dynamic_offsets,
312                        index,
313                        num_dynamic_offsets,
314                        bind_group,
315                    )
316                    .map_pass_err(scope)?;
317                }
318                &RenderCommand::SetPipeline(pipeline) => {
319                    let scope = PassErrorScope::SetPipelineRender;
320                    set_pipeline(
321                        &mut state,
322                        &pipeline_guard,
323                        &self.context,
324                        self.is_depth_read_only,
325                        self.is_stencil_read_only,
326                        pipeline,
327                    )
328                    .map_pass_err(scope)?;
329                }
330                &RenderCommand::SetIndexBuffer {
331                    buffer,
332                    index_format,
333                    offset,
334                    size,
335                } => {
336                    let scope = PassErrorScope::SetIndexBuffer;
337                    set_index_buffer(
338                        &mut state,
339                        &buffer_guard,
340                        buffer,
341                        index_format,
342                        offset,
343                        size,
344                    )
345                    .map_pass_err(scope)?;
346                }
347                &RenderCommand::SetVertexBuffer {
348                    slot,
349                    buffer,
350                    offset,
351                    size,
352                } => {
353                    let scope = PassErrorScope::SetVertexBuffer;
354                    set_vertex_buffer(&mut state, &buffer_guard, slot, buffer, offset, size)
355                        .map_pass_err(scope)?;
356                }
357                &RenderCommand::SetPushConstant {
358                    stages,
359                    offset,
360                    size_bytes,
361                    values_offset,
362                } => {
363                    let scope = PassErrorScope::SetPushConstant;
364                    set_push_constant(&mut state, stages, offset, size_bytes, values_offset)
365                        .map_pass_err(scope)?;
366                }
367                &RenderCommand::Draw {
368                    vertex_count,
369                    instance_count,
370                    first_vertex,
371                    first_instance,
372                } => {
373                    let scope = PassErrorScope::Draw {
374                        kind: DrawKind::Draw,
375                        family: DrawCommandFamily::Draw,
376                    };
377                    draw(
378                        &mut state,
379                        &base.dynamic_offsets,
380                        vertex_count,
381                        instance_count,
382                        first_vertex,
383                        first_instance,
384                    )
385                    .map_pass_err(scope)?;
386                }
387                &RenderCommand::DrawIndexed {
388                    index_count,
389                    instance_count,
390                    first_index,
391                    base_vertex,
392                    first_instance,
393                } => {
394                    let scope = PassErrorScope::Draw {
395                        kind: DrawKind::Draw,
396                        family: DrawCommandFamily::DrawIndexed,
397                    };
398                    draw_indexed(
399                        &mut state,
400                        &base.dynamic_offsets,
401                        index_count,
402                        instance_count,
403                        first_index,
404                        base_vertex,
405                        first_instance,
406                    )
407                    .map_pass_err(scope)?;
408                }
409                &RenderCommand::DrawMeshTasks {
410                    group_count_x,
411                    group_count_y,
412                    group_count_z,
413                } => {
414                    let scope = PassErrorScope::Draw {
415                        kind: DrawKind::Draw,
416                        family: DrawCommandFamily::DrawMeshTasks,
417                    };
418                    draw_mesh_tasks(
419                        &mut state,
420                        &base.dynamic_offsets,
421                        group_count_x,
422                        group_count_y,
423                        group_count_z,
424                    )
425                    .map_pass_err(scope)?;
426                }
427                &RenderCommand::DrawIndirect {
428                    buffer,
429                    offset,
430                    count: 1,
431                    family,
432                    vertex_or_index_limit: None,
433                    instance_limit: None,
434                } => {
435                    let scope = PassErrorScope::Draw {
436                        kind: DrawKind::DrawIndirect,
437                        family,
438                    };
439                    multi_draw_indirect(
440                        &mut state,
441                        &base.dynamic_offsets,
442                        &buffer_guard,
443                        buffer,
444                        offset,
445                        family,
446                    )
447                    .map_pass_err(scope)?;
448                }
449                &RenderCommand::DrawIndirect {
450                    count,
451                    vertex_or_index_limit,
452                    instance_limit,
453                    ..
454                } => {
455                    unreachable!("unexpected (multi-)draw indirect with count {count}, vertex_or_index_limits {vertex_or_index_limit:?}, instance_limit {instance_limit:?} found in a render bundle");
456                }
457                &RenderCommand::MultiDrawIndirectCount { .. }
458                | &RenderCommand::PushDebugGroup { color: _, len: _ }
459                | &RenderCommand::InsertDebugMarker { color: _, len: _ }
460                | &RenderCommand::PopDebugGroup => {
461                    unimplemented!("not supported by a render bundle")
462                }
463                // Must check the TIMESTAMP_QUERY_INSIDE_PASSES feature
464                &RenderCommand::WriteTimestamp { .. }
465                | &RenderCommand::BeginOcclusionQuery { .. }
466                | &RenderCommand::EndOcclusionQuery
467                | &RenderCommand::BeginPipelineStatisticsQuery { .. }
468                | &RenderCommand::EndPipelineStatisticsQuery => {
469                    unimplemented!("not supported by a render bundle")
470                }
471                &RenderCommand::ExecuteBundle(_)
472                | &RenderCommand::SetBlendConstant(_)
473                | &RenderCommand::SetStencilReference(_)
474                | &RenderCommand::SetViewport { .. }
475                | &RenderCommand::SetScissor(_) => unreachable!("not supported by a render bundle"),
476            }
477        }
478
479        let State {
480            trackers,
481            flat_dynamic_offsets,
482            device,
483            commands,
484            buffer_memory_init_actions,
485            texture_memory_init_actions,
486            ..
487        } = state;
488
489        let tracker_indices = device.tracker_indices.bundles.clone();
490        let discard_hal_labels = device
491            .instance_flags
492            .contains(wgt::InstanceFlags::DISCARD_HAL_LABELS);
493
494        let render_bundle = RenderBundle {
495            base: BasePass {
496                label: desc.label.as_deref().map(str::to_owned),
497                error: None,
498                commands,
499                dynamic_offsets: flat_dynamic_offsets,
500                string_data: self.base.string_data,
501                push_constant_data: self.base.push_constant_data,
502            },
503            is_depth_read_only: self.is_depth_read_only,
504            is_stencil_read_only: self.is_stencil_read_only,
505            device: device.clone(),
506            used: trackers,
507            buffer_memory_init_actions,
508            texture_memory_init_actions,
509            context: self.context,
510            label: desc.label.to_string(),
511            tracking_data: TrackingData::new(tracker_indices),
512            discard_hal_labels,
513        };
514
515        let render_bundle = Arc::new(render_bundle);
516
517        Ok(render_bundle)
518    }
519
520    pub fn set_index_buffer(
521        &mut self,
522        buffer: id::BufferId,
523        index_format: wgt::IndexFormat,
524        offset: wgt::BufferAddress,
525        size: Option<wgt::BufferSize>,
526    ) {
527        self.base.commands.push(RenderCommand::SetIndexBuffer {
528            buffer,
529            index_format,
530            offset,
531            size,
532        });
533    }
534}
535
536fn set_bind_group(
537    state: &mut State,
538    bind_group_guard: &crate::storage::Storage<Fallible<BindGroup>>,
539    dynamic_offsets: &[u32],
540    index: u32,
541    num_dynamic_offsets: usize,
542    bind_group_id: Option<id::Id<id::markers::BindGroup>>,
543) -> Result<(), RenderBundleErrorInner> {
544    if bind_group_id.is_none() {
545        // TODO: do appropriate cleanup for null bind_group.
546        return Ok(());
547    }
548
549    let bind_group_id = bind_group_id.unwrap();
550
551    let bind_group = bind_group_guard.get(bind_group_id).get()?;
552
553    bind_group.same_device(&state.device)?;
554
555    let max_bind_groups = state.device.limits.max_bind_groups;
556    if index >= max_bind_groups {
557        return Err(
558            RenderCommandError::BindGroupIndexOutOfRange(pass::BindGroupIndexOutOfRange {
559                index,
560                max: max_bind_groups,
561            })
562            .into(),
563        );
564    }
565
566    // Identify the next `num_dynamic_offsets` entries from `dynamic_offsets`.
567    let offsets_range = state.next_dynamic_offset..state.next_dynamic_offset + num_dynamic_offsets;
568    state.next_dynamic_offset = offsets_range.end;
569    let offsets = &dynamic_offsets[offsets_range.clone()];
570
571    bind_group.validate_dynamic_bindings(index, offsets)?;
572
573    state
574        .buffer_memory_init_actions
575        .extend_from_slice(&bind_group.used_buffer_ranges);
576    state
577        .texture_memory_init_actions
578        .extend_from_slice(&bind_group.used_texture_ranges);
579
580    state.set_bind_group(index, &bind_group, offsets_range);
581    unsafe { state.trackers.merge_bind_group(&bind_group.used)? };
582    state.trackers.bind_groups.insert_single(bind_group);
583    // Note: stateless trackers are not merged: the lifetime reference
584    // is held to the bind group itself.
585    Ok(())
586}
587
588fn set_pipeline(
589    state: &mut State,
590    pipeline_guard: &crate::storage::Storage<Fallible<RenderPipeline>>,
591    context: &RenderPassContext,
592    is_depth_read_only: bool,
593    is_stencil_read_only: bool,
594    pipeline_id: id::Id<id::markers::RenderPipeline>,
595) -> Result<(), RenderBundleErrorInner> {
596    let pipeline = pipeline_guard.get(pipeline_id).get()?;
597
598    pipeline.same_device(&state.device)?;
599
600    context
601        .check_compatible(&pipeline.pass_context, pipeline.as_ref())
602        .map_err(RenderCommandError::IncompatiblePipelineTargets)?;
603
604    if pipeline.flags.contains(PipelineFlags::WRITES_DEPTH) && is_depth_read_only {
605        return Err(RenderCommandError::IncompatibleDepthAccess(pipeline.error_ident()).into());
606    }
607    if pipeline.flags.contains(PipelineFlags::WRITES_STENCIL) && is_stencil_read_only {
608        return Err(RenderCommandError::IncompatibleStencilAccess(pipeline.error_ident()).into());
609    }
610
611    let pipeline_state = PipelineState::new(&pipeline);
612
613    state
614        .commands
615        .push(ArcRenderCommand::SetPipeline(pipeline.clone()));
616
617    // If this pipeline uses push constants, zero out their values.
618    if let Some(iter) = pipeline_state.zero_push_constants() {
619        state.commands.extend(iter)
620    }
621
622    state.invalidate_bind_groups(&pipeline_state, &pipeline.layout);
623    state.pipeline = Some(pipeline_state);
624
625    state.trackers.render_pipelines.insert_single(pipeline);
626    Ok(())
627}
628
629// This function is duplicative of `render::set_index_buffer`.
630fn set_index_buffer(
631    state: &mut State,
632    buffer_guard: &crate::storage::Storage<Fallible<Buffer>>,
633    buffer_id: id::Id<id::markers::Buffer>,
634    index_format: wgt::IndexFormat,
635    offset: u64,
636    size: Option<NonZeroU64>,
637) -> Result<(), RenderBundleErrorInner> {
638    let buffer = buffer_guard.get(buffer_id).get()?;
639
640    state
641        .trackers
642        .buffers
643        .merge_single(&buffer, wgt::BufferUses::INDEX)?;
644
645    buffer.same_device(&state.device)?;
646    buffer.check_usage(wgt::BufferUsages::INDEX)?;
647
648    if offset % u64::try_from(index_format.byte_size()).unwrap() != 0 {
649        return Err(RenderCommandError::UnalignedIndexBuffer {
650            offset,
651            alignment: index_format.byte_size(),
652        }
653        .into());
654    }
655    let end = offset + buffer.resolve_binding_size(offset, size)?;
656
657    state
658        .buffer_memory_init_actions
659        .extend(buffer.initialization_status.read().create_action(
660            &buffer,
661            offset..end.get(),
662            MemoryInitKind::NeedsInitializedMemory,
663        ));
664    state.set_index_buffer(buffer, index_format, offset..end.get());
665    Ok(())
666}
667
668// This function is duplicative of `render::set_vertex_buffer`.
669fn set_vertex_buffer(
670    state: &mut State,
671    buffer_guard: &crate::storage::Storage<Fallible<Buffer>>,
672    slot: u32,
673    buffer_id: id::Id<id::markers::Buffer>,
674    offset: u64,
675    size: Option<NonZeroU64>,
676) -> Result<(), RenderBundleErrorInner> {
677    let max_vertex_buffers = state.device.limits.max_vertex_buffers;
678    if slot >= max_vertex_buffers {
679        return Err(RenderCommandError::VertexBufferIndexOutOfRange {
680            index: slot,
681            max: max_vertex_buffers,
682        }
683        .into());
684    }
685
686    let buffer = buffer_guard.get(buffer_id).get()?;
687
688    state
689        .trackers
690        .buffers
691        .merge_single(&buffer, wgt::BufferUses::VERTEX)?;
692
693    buffer.same_device(&state.device)?;
694    buffer.check_usage(wgt::BufferUsages::VERTEX)?;
695
696    if offset % wgt::VERTEX_ALIGNMENT != 0 {
697        return Err(RenderCommandError::UnalignedVertexBuffer { slot, offset }.into());
698    }
699    let end = offset + buffer.resolve_binding_size(offset, size)?;
700
701    state
702        .buffer_memory_init_actions
703        .extend(buffer.initialization_status.read().create_action(
704            &buffer,
705            offset..end.get(),
706            MemoryInitKind::NeedsInitializedMemory,
707        ));
708    state.vertex[slot as usize] = Some(VertexState::new(buffer, offset..end.get()));
709    Ok(())
710}
711
712fn set_push_constant(
713    state: &mut State,
714    stages: wgt::ShaderStages,
715    offset: u32,
716    size_bytes: u32,
717    values_offset: Option<u32>,
718) -> Result<(), RenderBundleErrorInner> {
719    let end_offset = offset + size_bytes;
720
721    let pipeline_state = state.pipeline()?;
722
723    pipeline_state
724        .pipeline
725        .layout
726        .validate_push_constant_ranges(stages, offset, end_offset)?;
727
728    state.commands.push(ArcRenderCommand::SetPushConstant {
729        stages,
730        offset,
731        size_bytes,
732        values_offset,
733    });
734    Ok(())
735}
736
737fn draw(
738    state: &mut State,
739    dynamic_offsets: &[u32],
740    vertex_count: u32,
741    instance_count: u32,
742    first_vertex: u32,
743    first_instance: u32,
744) -> Result<(), RenderBundleErrorInner> {
745    let pipeline = state.pipeline()?;
746    let used_bind_groups = pipeline.used_bind_groups;
747
748    let vertex_limits = super::VertexLimits::new(state.vertex_buffer_sizes(), &pipeline.steps);
749    vertex_limits.validate_vertex_limit(first_vertex, vertex_count)?;
750    vertex_limits.validate_instance_limit(first_instance, instance_count)?;
751
752    if instance_count > 0 && vertex_count > 0 {
753        state.flush_vertices();
754        state.flush_binds(used_bind_groups, dynamic_offsets);
755        state.commands.push(ArcRenderCommand::Draw {
756            vertex_count,
757            instance_count,
758            first_vertex,
759            first_instance,
760        });
761    }
762    Ok(())
763}
764
765fn draw_indexed(
766    state: &mut State,
767    dynamic_offsets: &[u32],
768    index_count: u32,
769    instance_count: u32,
770    first_index: u32,
771    base_vertex: i32,
772    first_instance: u32,
773) -> Result<(), RenderBundleErrorInner> {
774    let pipeline = state.pipeline()?;
775    let used_bind_groups = pipeline.used_bind_groups;
776    let index = match state.index {
777        Some(ref index) => index,
778        None => return Err(DrawError::MissingIndexBuffer.into()),
779    };
780
781    let vertex_limits = super::VertexLimits::new(state.vertex_buffer_sizes(), &pipeline.steps);
782
783    let last_index = first_index as u64 + index_count as u64;
784    let index_limit = index.limit();
785    if last_index > index_limit {
786        return Err(DrawError::IndexBeyondLimit {
787            last_index,
788            index_limit,
789        }
790        .into());
791    }
792    vertex_limits.validate_instance_limit(first_instance, instance_count)?;
793
794    if instance_count > 0 && index_count > 0 {
795        state.flush_index();
796        state.flush_vertices();
797        state.flush_binds(used_bind_groups, dynamic_offsets);
798        state.commands.push(ArcRenderCommand::DrawIndexed {
799            index_count,
800            instance_count,
801            first_index,
802            base_vertex,
803            first_instance,
804        });
805    }
806    Ok(())
807}
808
809fn draw_mesh_tasks(
810    state: &mut State,
811    dynamic_offsets: &[u32],
812    group_count_x: u32,
813    group_count_y: u32,
814    group_count_z: u32,
815) -> Result<(), RenderBundleErrorInner> {
816    let pipeline = state.pipeline()?;
817    let used_bind_groups = pipeline.used_bind_groups;
818
819    let groups_size_limit = state.device.limits.max_task_workgroups_per_dimension;
820    let max_groups = state.device.limits.max_task_workgroup_total_count;
821    if group_count_x > groups_size_limit
822        || group_count_y > groups_size_limit
823        || group_count_z > groups_size_limit
824        || group_count_x * group_count_y * group_count_z > max_groups
825    {
826        return Err(RenderBundleErrorInner::Draw(DrawError::InvalidGroupSize {
827            current: [group_count_x, group_count_y, group_count_z],
828            limit: groups_size_limit,
829            max_total: max_groups,
830        }));
831    }
832
833    if group_count_x > 0 && group_count_y > 0 && group_count_z > 0 {
834        state.flush_binds(used_bind_groups, dynamic_offsets);
835        state.commands.push(ArcRenderCommand::DrawMeshTasks {
836            group_count_x,
837            group_count_y,
838            group_count_z,
839        });
840    }
841    Ok(())
842}
843
844fn multi_draw_indirect(
845    state: &mut State,
846    dynamic_offsets: &[u32],
847    buffer_guard: &crate::storage::Storage<Fallible<Buffer>>,
848    buffer_id: id::Id<id::markers::Buffer>,
849    offset: u64,
850    family: DrawCommandFamily,
851) -> Result<(), RenderBundleErrorInner> {
852    state
853        .device
854        .require_downlevel_flags(wgt::DownlevelFlags::INDIRECT_EXECUTION)?;
855
856    let pipeline = state.pipeline()?;
857    let used_bind_groups = pipeline.used_bind_groups;
858
859    let buffer = buffer_guard.get(buffer_id).get()?;
860
861    buffer.same_device(&state.device)?;
862    buffer.check_usage(wgt::BufferUsages::INDIRECT)?;
863
864    let vertex_limits = super::VertexLimits::new(state.vertex_buffer_sizes(), &pipeline.steps);
865
866    let stride = super::get_stride_of_indirect_args(family);
867    state
868        .buffer_memory_init_actions
869        .extend(buffer.initialization_status.read().create_action(
870            &buffer,
871            offset..(offset + stride),
872            MemoryInitKind::NeedsInitializedMemory,
873        ));
874
875    let vertex_or_index_limit = if family == DrawCommandFamily::DrawIndexed {
876        let index = match state.index {
877            Some(ref mut index) => index,
878            None => return Err(DrawError::MissingIndexBuffer.into()),
879        };
880        state.commands.extend(index.flush());
881        index.limit()
882    } else {
883        vertex_limits.vertex_limit
884    };
885    let instance_limit = vertex_limits.instance_limit;
886
887    let buffer_uses = if state.device.indirect_validation.is_some() {
888        wgt::BufferUses::STORAGE_READ_ONLY
889    } else {
890        wgt::BufferUses::INDIRECT
891    };
892
893    state.trackers.buffers.merge_single(&buffer, buffer_uses)?;
894
895    state.flush_vertices();
896    state.flush_binds(used_bind_groups, dynamic_offsets);
897    state.commands.push(ArcRenderCommand::DrawIndirect {
898        buffer,
899        offset,
900        count: 1,
901        family,
902
903        vertex_or_index_limit: Some(vertex_or_index_limit),
904        instance_limit: Some(instance_limit),
905    });
906    Ok(())
907}
908
909/// Error type returned from `RenderBundleEncoder::new` if the sample count is invalid.
910#[derive(Clone, Debug, Error)]
911#[non_exhaustive]
912pub enum CreateRenderBundleError {
913    #[error(transparent)]
914    ColorAttachment(#[from] ColorAttachmentError),
915    #[error("Invalid number of samples {0}")]
916    InvalidSampleCount(u32),
917}
918
919impl WebGpuError for CreateRenderBundleError {
920    fn webgpu_error_type(&self) -> ErrorType {
921        match self {
922            Self::ColorAttachment(e) => e.webgpu_error_type(),
923            Self::InvalidSampleCount(_) => ErrorType::Validation,
924        }
925    }
926}
927
928/// Error type returned from `RenderBundleEncoder::new` if the sample count is invalid.
929#[derive(Clone, Debug, Error)]
930#[non_exhaustive]
931pub enum ExecutionError {
932    #[error(transparent)]
933    Device(#[from] DeviceError),
934    #[error(transparent)]
935    DestroyedResource(#[from] DestroyedResourceError),
936    #[error("Using {0} in a render bundle is not implemented")]
937    Unimplemented(&'static str),
938}
939
940pub type RenderBundleDescriptor<'a> = wgt::RenderBundleDescriptor<Label<'a>>;
941
942//Note: here, `RenderBundle` is just wrapping a raw stream of render commands.
943// The plan is to back it by an actual Vulkan secondary buffer, D3D12 Bundle,
944// or Metal indirect command buffer.
945/// cbindgen:ignore
946#[derive(Debug)]
947pub struct RenderBundle {
948    // Normalized command stream. It can be executed verbatim,
949    // without re-binding anything on the pipeline change.
950    base: BasePass<ArcRenderCommand, Infallible>,
951    pub(super) is_depth_read_only: bool,
952    pub(super) is_stencil_read_only: bool,
953    pub(crate) device: Arc<Device>,
954    pub(crate) used: RenderBundleScope,
955    pub(super) buffer_memory_init_actions: Vec<BufferInitTrackerAction>,
956    pub(super) texture_memory_init_actions: Vec<TextureInitTrackerAction>,
957    pub(super) context: RenderPassContext,
958    /// The `label` from the descriptor used to create the resource.
959    label: String,
960    pub(crate) tracking_data: TrackingData,
961    discard_hal_labels: bool,
962}
963
964impl Drop for RenderBundle {
965    fn drop(&mut self) {
966        resource_log!("Drop {}", self.error_ident());
967    }
968}
969
970#[cfg(send_sync)]
971unsafe impl Send for RenderBundle {}
972#[cfg(send_sync)]
973unsafe impl Sync for RenderBundle {}
974
975impl RenderBundle {
976    #[cfg(feature = "trace")]
977    pub(crate) fn to_base_pass(&self) -> BasePass<RenderCommand<ArcReferences>, Infallible> {
978        self.base.clone()
979    }
980
981    /// Actually encode the contents into a native command buffer.
982    ///
983    /// This is partially duplicating the logic of `render_pass_end`.
984    /// However the point of this function is to be lighter, since we already had
985    /// a chance to go through the commands in `render_bundle_encoder_finish`.
986    ///
987    /// Note that the function isn't expected to fail, generally.
988    /// All the validation has already been done by this point.
989    /// The only failure condition is if some of the used buffers are destroyed.
990    pub(super) unsafe fn execute(
991        &self,
992        raw: &mut dyn hal::DynCommandEncoder,
993        indirect_draw_validation_resources: &mut crate::indirect_validation::DrawResources,
994        indirect_draw_validation_batcher: &mut crate::indirect_validation::DrawBatcher,
995        snatch_guard: &SnatchGuard,
996    ) -> Result<(), ExecutionError> {
997        let mut offsets = self.base.dynamic_offsets.as_slice();
998        let mut pipeline_layout = None::<Arc<PipelineLayout>>;
999        if !self.discard_hal_labels {
1000            if let Some(ref label) = self.base.label {
1001                unsafe { raw.begin_debug_marker(label) };
1002            }
1003        }
1004
1005        use ArcRenderCommand as Cmd;
1006        for command in self.base.commands.iter() {
1007            match command {
1008                Cmd::SetBindGroup {
1009                    index,
1010                    num_dynamic_offsets,
1011                    bind_group,
1012                } => {
1013                    let mut bg = None;
1014                    if bind_group.is_some() {
1015                        let bind_group = bind_group.as_ref().unwrap();
1016                        let raw_bg = bind_group.try_raw(snatch_guard)?;
1017                        bg = Some(raw_bg);
1018                    }
1019                    unsafe {
1020                        raw.set_bind_group(
1021                            pipeline_layout.as_ref().unwrap().raw(),
1022                            *index,
1023                            bg,
1024                            &offsets[..*num_dynamic_offsets],
1025                        )
1026                    };
1027                    offsets = &offsets[*num_dynamic_offsets..];
1028                }
1029                Cmd::SetPipeline(pipeline) => {
1030                    unsafe { raw.set_render_pipeline(pipeline.raw()) };
1031
1032                    pipeline_layout = Some(pipeline.layout.clone());
1033                }
1034                Cmd::SetIndexBuffer {
1035                    buffer,
1036                    index_format,
1037                    offset,
1038                    size,
1039                } => {
1040                    let buffer = buffer.try_raw(snatch_guard)?;
1041                    // SAFETY: The binding size was checked against the buffer size
1042                    // in `set_index_buffer` and again in `IndexState::flush`.
1043                    let bb = hal::BufferBinding::new_unchecked(buffer, *offset, *size);
1044                    unsafe { raw.set_index_buffer(bb, *index_format) };
1045                }
1046                Cmd::SetVertexBuffer {
1047                    slot,
1048                    buffer,
1049                    offset,
1050                    size,
1051                } => {
1052                    let buffer = buffer.try_raw(snatch_guard)?;
1053                    // SAFETY: The binding size was checked against the buffer size
1054                    // in `set_vertex_buffer` and again in `VertexState::flush`.
1055                    let bb = hal::BufferBinding::new_unchecked(buffer, *offset, *size);
1056                    unsafe { raw.set_vertex_buffer(*slot, bb) };
1057                }
1058                Cmd::SetPushConstant {
1059                    stages,
1060                    offset,
1061                    size_bytes,
1062                    values_offset,
1063                } => {
1064                    let pipeline_layout = pipeline_layout.as_ref().unwrap();
1065
1066                    if let Some(values_offset) = *values_offset {
1067                        let values_end_offset =
1068                            (values_offset + size_bytes / wgt::PUSH_CONSTANT_ALIGNMENT) as usize;
1069                        let data_slice = &self.base.push_constant_data
1070                            [(values_offset as usize)..values_end_offset];
1071
1072                        unsafe {
1073                            raw.set_push_constants(
1074                                pipeline_layout.raw(),
1075                                *stages,
1076                                *offset,
1077                                data_slice,
1078                            )
1079                        }
1080                    } else {
1081                        super::push_constant_clear(
1082                            *offset,
1083                            *size_bytes,
1084                            |clear_offset, clear_data| {
1085                                unsafe {
1086                                    raw.set_push_constants(
1087                                        pipeline_layout.raw(),
1088                                        *stages,
1089                                        clear_offset,
1090                                        clear_data,
1091                                    )
1092                                };
1093                            },
1094                        );
1095                    }
1096                }
1097                Cmd::Draw {
1098                    vertex_count,
1099                    instance_count,
1100                    first_vertex,
1101                    first_instance,
1102                } => {
1103                    unsafe {
1104                        raw.draw(
1105                            *first_vertex,
1106                            *vertex_count,
1107                            *first_instance,
1108                            *instance_count,
1109                        )
1110                    };
1111                }
1112                Cmd::DrawIndexed {
1113                    index_count,
1114                    instance_count,
1115                    first_index,
1116                    base_vertex,
1117                    first_instance,
1118                } => {
1119                    unsafe {
1120                        raw.draw_indexed(
1121                            *first_index,
1122                            *index_count,
1123                            *base_vertex,
1124                            *first_instance,
1125                            *instance_count,
1126                        )
1127                    };
1128                }
1129                Cmd::DrawMeshTasks {
1130                    group_count_x,
1131                    group_count_y,
1132                    group_count_z,
1133                } => unsafe {
1134                    raw.draw_mesh_tasks(*group_count_x, *group_count_y, *group_count_z);
1135                },
1136                Cmd::DrawIndirect {
1137                    buffer,
1138                    offset,
1139                    count: 1,
1140                    family,
1141
1142                    vertex_or_index_limit,
1143                    instance_limit,
1144                } => {
1145                    let (buffer, offset) = if self.device.indirect_validation.is_some() {
1146                        let (dst_resource_index, offset) = indirect_draw_validation_batcher.add(
1147                            indirect_draw_validation_resources,
1148                            &self.device,
1149                            buffer,
1150                            *offset,
1151                            *family,
1152                            vertex_or_index_limit
1153                                .expect("finalized render bundle missing vertex_or_index_limit"),
1154                            instance_limit.expect("finalized render bundle missing instance_limit"),
1155                        )?;
1156
1157                        let dst_buffer =
1158                            indirect_draw_validation_resources.get_dst_buffer(dst_resource_index);
1159                        (dst_buffer, offset)
1160                    } else {
1161                        (buffer.try_raw(snatch_guard)?, *offset)
1162                    };
1163                    match family {
1164                        DrawCommandFamily::Draw => unsafe { raw.draw_indirect(buffer, offset, 1) },
1165                        DrawCommandFamily::DrawIndexed => unsafe {
1166                            raw.draw_indexed_indirect(buffer, offset, 1)
1167                        },
1168                        DrawCommandFamily::DrawMeshTasks => unsafe {
1169                            raw.draw_mesh_tasks_indirect(buffer, offset, 1);
1170                        },
1171                    }
1172                }
1173                Cmd::DrawIndirect { .. } | Cmd::MultiDrawIndirectCount { .. } => {
1174                    return Err(ExecutionError::Unimplemented("multi-draw-indirect"))
1175                }
1176                Cmd::PushDebugGroup { .. } | Cmd::InsertDebugMarker { .. } | Cmd::PopDebugGroup => {
1177                    return Err(ExecutionError::Unimplemented("debug-markers"))
1178                }
1179                Cmd::WriteTimestamp { .. }
1180                | Cmd::BeginOcclusionQuery { .. }
1181                | Cmd::EndOcclusionQuery
1182                | Cmd::BeginPipelineStatisticsQuery { .. }
1183                | Cmd::EndPipelineStatisticsQuery => {
1184                    return Err(ExecutionError::Unimplemented("queries"))
1185                }
1186                Cmd::ExecuteBundle(_)
1187                | Cmd::SetBlendConstant(_)
1188                | Cmd::SetStencilReference(_)
1189                | Cmd::SetViewport { .. }
1190                | Cmd::SetScissor(_) => unreachable!(),
1191            }
1192        }
1193
1194        if !self.discard_hal_labels {
1195            if let Some(_) = self.base.label {
1196                unsafe { raw.end_debug_marker() };
1197            }
1198        }
1199
1200        Ok(())
1201    }
1202}
1203
1204crate::impl_resource_type!(RenderBundle);
1205crate::impl_labeled!(RenderBundle);
1206crate::impl_parent_device!(RenderBundle);
1207crate::impl_storage_item!(RenderBundle);
1208crate::impl_trackable!(RenderBundle);
1209
1210/// A render bundle's current index buffer state.
1211///
1212/// [`RenderBundleEncoder::finish`] records the currently set index buffer here,
1213/// and calls [`State::flush_index`] before any indexed draw command to produce
1214/// a `SetIndexBuffer` command if one is necessary.
1215///
1216/// Binding ranges must be validated against the size of the buffer before
1217/// being stored in `IndexState`.
1218#[derive(Debug)]
1219struct IndexState {
1220    buffer: Arc<Buffer>,
1221    format: wgt::IndexFormat,
1222    range: Range<wgt::BufferAddress>,
1223    is_dirty: bool,
1224}
1225
1226impl IndexState {
1227    /// Return the number of entries in the current index buffer.
1228    ///
1229    /// Panic if no index buffer has been set.
1230    fn limit(&self) -> u64 {
1231        let bytes_per_index = self.format.byte_size() as u64;
1232
1233        (self.range.end - self.range.start) / bytes_per_index
1234    }
1235
1236    /// Generate a `SetIndexBuffer` command to prepare for an indexed draw
1237    /// command, if needed.
1238    fn flush(&mut self) -> Option<ArcRenderCommand> {
1239        // This was all checked before, but let's check again just in case.
1240        let binding_size = self
1241            .range
1242            .end
1243            .checked_sub(self.range.start)
1244            .filter(|_| self.range.end <= self.buffer.size)
1245            .expect("index range must be contained in buffer");
1246
1247        if self.is_dirty {
1248            self.is_dirty = false;
1249            Some(ArcRenderCommand::SetIndexBuffer {
1250                buffer: self.buffer.clone(),
1251                index_format: self.format,
1252                offset: self.range.start,
1253                size: NonZeroU64::new(binding_size),
1254            })
1255        } else {
1256            None
1257        }
1258    }
1259}
1260
1261/// The state of a single vertex buffer slot during render bundle encoding.
1262///
1263/// [`RenderBundleEncoder::finish`] uses this to drop redundant
1264/// `SetVertexBuffer` commands from the final [`RenderBundle`]. It
1265/// records one vertex buffer slot's state changes here, and then
1266/// calls this type's [`flush`] method just before any draw command to
1267/// produce a `SetVertexBuffer` commands if one is necessary.
1268///
1269/// Binding ranges must be validated against the size of the buffer before
1270/// being stored in `VertexState`.
1271///
1272/// [`flush`]: IndexState::flush
1273#[derive(Debug)]
1274struct VertexState {
1275    buffer: Arc<Buffer>,
1276    range: Range<wgt::BufferAddress>,
1277    is_dirty: bool,
1278}
1279
1280impl VertexState {
1281    /// Create a new `VertexState`.
1282    ///
1283    /// The `range` must be contained within `buffer`.
1284    fn new(buffer: Arc<Buffer>, range: Range<wgt::BufferAddress>) -> Self {
1285        Self {
1286            buffer,
1287            range,
1288            is_dirty: true,
1289        }
1290    }
1291
1292    /// Generate a `SetVertexBuffer` command for this slot, if necessary.
1293    ///
1294    /// `slot` is the index of the vertex buffer slot that `self` tracks.
1295    fn flush(&mut self, slot: u32) -> Option<ArcRenderCommand> {
1296        let binding_size = self
1297            .range
1298            .end
1299            .checked_sub(self.range.start)
1300            .filter(|_| self.range.end <= self.buffer.size)
1301            .expect("vertex range must be contained in buffer");
1302
1303        if self.is_dirty {
1304            self.is_dirty = false;
1305            Some(ArcRenderCommand::SetVertexBuffer {
1306                slot,
1307                buffer: self.buffer.clone(),
1308                offset: self.range.start,
1309                size: NonZeroU64::new(binding_size),
1310            })
1311        } else {
1312            None
1313        }
1314    }
1315}
1316
1317/// A bind group that has been set at a particular index during render bundle encoding.
1318#[derive(Debug)]
1319struct BindState {
1320    /// The id of the bind group set at this index.
1321    bind_group: Arc<BindGroup>,
1322
1323    /// The range of dynamic offsets for this bind group, in the original
1324    /// command stream's `BassPass::dynamic_offsets` array.
1325    dynamic_offsets: Range<usize>,
1326
1327    /// True if this index's contents have been changed since the last time we
1328    /// generated a `SetBindGroup` command.
1329    is_dirty: bool,
1330}
1331
1332/// The bundle's current pipeline, and some cached information needed for validation.
1333struct PipelineState {
1334    /// The pipeline
1335    pipeline: Arc<RenderPipeline>,
1336
1337    /// How this pipeline's vertex shader traverses each vertex buffer, indexed
1338    /// by vertex buffer slot number.
1339    steps: Vec<VertexStep>,
1340
1341    /// Ranges of push constants this pipeline uses, copied from the pipeline
1342    /// layout.
1343    push_constant_ranges: ArrayVec<wgt::PushConstantRange, { SHADER_STAGE_COUNT }>,
1344
1345    /// The number of bind groups this pipeline uses.
1346    used_bind_groups: usize,
1347}
1348
1349impl PipelineState {
1350    fn new(pipeline: &Arc<RenderPipeline>) -> Self {
1351        Self {
1352            pipeline: pipeline.clone(),
1353            steps: pipeline.vertex_steps.to_vec(),
1354            push_constant_ranges: pipeline
1355                .layout
1356                .push_constant_ranges
1357                .iter()
1358                .cloned()
1359                .collect(),
1360            used_bind_groups: pipeline.layout.bind_group_layouts.len(),
1361        }
1362    }
1363
1364    /// Return a sequence of commands to zero the push constant ranges this
1365    /// pipeline uses. If no initialization is necessary, return `None`.
1366    fn zero_push_constants(&self) -> Option<impl Iterator<Item = ArcRenderCommand>> {
1367        if !self.push_constant_ranges.is_empty() {
1368            let nonoverlapping_ranges =
1369                super::bind::compute_nonoverlapping_ranges(&self.push_constant_ranges);
1370
1371            Some(
1372                nonoverlapping_ranges
1373                    .into_iter()
1374                    .map(|range| ArcRenderCommand::SetPushConstant {
1375                        stages: range.stages,
1376                        offset: range.range.start,
1377                        size_bytes: range.range.end - range.range.start,
1378                        values_offset: None, // write zeros
1379                    }),
1380            )
1381        } else {
1382            None
1383        }
1384    }
1385}
1386
1387/// State for analyzing and cleaning up bundle command streams.
1388///
1389/// To minimize state updates, [`RenderBundleEncoder::finish`]
1390/// actually just applies commands like [`SetBindGroup`] and
1391/// [`SetIndexBuffer`] to the simulated state stored here, and then
1392/// calls the `flush_foo` methods before draw calls to produce the
1393/// update commands we actually need.
1394///
1395/// [`SetBindGroup`]: RenderCommand::SetBindGroup
1396/// [`SetIndexBuffer`]: RenderCommand::SetIndexBuffer
1397struct State {
1398    /// Resources used by this bundle. This will become [`RenderBundle::used`].
1399    trackers: RenderBundleScope,
1400
1401    /// The currently set pipeline, if any.
1402    pipeline: Option<PipelineState>,
1403
1404    /// The bind group set at each index, if any.
1405    bind: ArrayVec<Option<BindState>, { hal::MAX_BIND_GROUPS }>,
1406
1407    /// The state of each vertex buffer slot.
1408    vertex: [Option<VertexState>; hal::MAX_VERTEX_BUFFERS],
1409
1410    /// The current index buffer, if one has been set. We flush this state
1411    /// before indexed draw commands.
1412    index: Option<IndexState>,
1413
1414    /// Dynamic offset values used by the cleaned-up command sequence.
1415    ///
1416    /// This becomes the final [`RenderBundle`]'s [`BasePass`]'s
1417    /// [`dynamic_offsets`] list.
1418    ///
1419    /// [`dynamic_offsets`]: BasePass::dynamic_offsets
1420    flat_dynamic_offsets: Vec<wgt::DynamicOffset>,
1421
1422    device: Arc<Device>,
1423    commands: Vec<ArcRenderCommand>,
1424    buffer_memory_init_actions: Vec<BufferInitTrackerAction>,
1425    texture_memory_init_actions: Vec<TextureInitTrackerAction>,
1426    next_dynamic_offset: usize,
1427}
1428
1429impl State {
1430    /// Return the current pipeline state. Return an error if none is set.
1431    fn pipeline(&self) -> Result<&PipelineState, RenderBundleErrorInner> {
1432        self.pipeline
1433            .as_ref()
1434            .ok_or(DrawError::MissingPipeline(pass::MissingPipeline).into())
1435    }
1436
1437    /// Mark all non-empty bind group table entries from `index` onwards as dirty.
1438    fn invalidate_bind_group_from(&mut self, index: usize) {
1439        for contents in self.bind[index..].iter_mut().flatten() {
1440            contents.is_dirty = true;
1441        }
1442    }
1443
1444    fn set_bind_group(
1445        &mut self,
1446        slot: u32,
1447        bind_group: &Arc<BindGroup>,
1448        dynamic_offsets: Range<usize>,
1449    ) {
1450        // If this call wouldn't actually change this index's state, we can
1451        // return early.  (If there are dynamic offsets, the range will always
1452        // be different.)
1453        if dynamic_offsets.is_empty() {
1454            if let Some(ref contents) = self.bind[slot as usize] {
1455                if contents.bind_group.is_equal(bind_group) {
1456                    return;
1457                }
1458            }
1459        }
1460
1461        // Record the index's new state.
1462        self.bind[slot as usize] = Some(BindState {
1463            bind_group: bind_group.clone(),
1464            dynamic_offsets,
1465            is_dirty: true,
1466        });
1467
1468        // Once we've changed the bind group at a particular index, all
1469        // subsequent indices need to be rewritten.
1470        self.invalidate_bind_group_from(slot as usize + 1);
1471    }
1472
1473    /// Determine which bind group slots need to be re-set after a pipeline change.
1474    ///
1475    /// Given that we are switching from the current pipeline state to `new`,
1476    /// whose layout is `layout`, mark all the bind group slots that we need to
1477    /// emit new `SetBindGroup` commands for as dirty.
1478    ///
1479    /// According to `wgpu_hal`'s rules:
1480    ///
1481    /// - If the layout of any bind group slot changes, then that slot and
1482    ///   all following slots must have their bind groups re-established.
1483    ///
1484    /// - Changing the push constant ranges at all requires re-establishing
1485    ///   all bind groups.
1486    fn invalidate_bind_groups(&mut self, new: &PipelineState, layout: &PipelineLayout) {
1487        match self.pipeline {
1488            None => {
1489                // Establishing entirely new pipeline state.
1490                self.invalidate_bind_group_from(0);
1491            }
1492            Some(ref old) => {
1493                if old.pipeline.is_equal(&new.pipeline) {
1494                    // Everything is derived from the pipeline, so if the id has
1495                    // not changed, there's no need to consider anything else.
1496                    return;
1497                }
1498
1499                // Any push constant change invalidates all groups.
1500                if old.push_constant_ranges != new.push_constant_ranges {
1501                    self.invalidate_bind_group_from(0);
1502                } else {
1503                    let first_changed = self.bind.iter().zip(&layout.bind_group_layouts).position(
1504                        |(entry, layout)| match *entry {
1505                            Some(ref contents) => !contents.bind_group.layout.is_equal(layout),
1506                            None => false,
1507                        },
1508                    );
1509                    if let Some(slot) = first_changed {
1510                        self.invalidate_bind_group_from(slot);
1511                    }
1512                }
1513            }
1514        }
1515    }
1516
1517    /// Set the bundle's current index buffer and its associated parameters.
1518    fn set_index_buffer(
1519        &mut self,
1520        buffer: Arc<Buffer>,
1521        format: wgt::IndexFormat,
1522        range: Range<wgt::BufferAddress>,
1523    ) {
1524        match self.index {
1525            Some(ref current)
1526                if current.buffer.is_equal(&buffer)
1527                    && current.format == format
1528                    && current.range == range =>
1529            {
1530                return
1531            }
1532            _ => (),
1533        }
1534
1535        self.index = Some(IndexState {
1536            buffer,
1537            format,
1538            range,
1539            is_dirty: true,
1540        });
1541    }
1542
1543    /// Generate a `SetIndexBuffer` command to prepare for an indexed draw
1544    /// command, if needed.
1545    fn flush_index(&mut self) {
1546        let commands = self.index.as_mut().and_then(|index| index.flush());
1547        self.commands.extend(commands);
1548    }
1549
1550    fn flush_vertices(&mut self) {
1551        let commands = self
1552            .vertex
1553            .iter_mut()
1554            .enumerate()
1555            .flat_map(|(i, vs)| vs.as_mut().and_then(|vs| vs.flush(i as u32)));
1556        self.commands.extend(commands);
1557    }
1558
1559    /// Generate `SetBindGroup` commands for any bind groups that need to be updated.
1560    fn flush_binds(&mut self, used_bind_groups: usize, dynamic_offsets: &[wgt::DynamicOffset]) {
1561        // Append each dirty bind group's dynamic offsets to `flat_dynamic_offsets`.
1562        for contents in self.bind[..used_bind_groups].iter().flatten() {
1563            if contents.is_dirty {
1564                self.flat_dynamic_offsets
1565                    .extend_from_slice(&dynamic_offsets[contents.dynamic_offsets.clone()]);
1566            }
1567        }
1568
1569        // Then, generate `SetBindGroup` commands to update the dirty bind
1570        // groups. After this, all bind groups are clean.
1571        let commands = self.bind[..used_bind_groups]
1572            .iter_mut()
1573            .enumerate()
1574            .flat_map(|(i, entry)| {
1575                if let Some(ref mut contents) = *entry {
1576                    if contents.is_dirty {
1577                        contents.is_dirty = false;
1578                        let offsets = &contents.dynamic_offsets;
1579                        return Some(ArcRenderCommand::SetBindGroup {
1580                            index: i.try_into().unwrap(),
1581                            bind_group: Some(contents.bind_group.clone()),
1582                            num_dynamic_offsets: offsets.end - offsets.start,
1583                        });
1584                    }
1585                }
1586                None
1587            });
1588
1589        self.commands.extend(commands);
1590    }
1591
1592    fn vertex_buffer_sizes(&self) -> impl Iterator<Item = Option<wgt::BufferAddress>> + '_ {
1593        self.vertex
1594            .iter()
1595            .map(|vbs| vbs.as_ref().map(|vbs| vbs.range.end - vbs.range.start))
1596    }
1597}
1598
1599/// Error encountered when finishing recording a render bundle.
1600#[derive(Clone, Debug, Error)]
1601pub enum RenderBundleErrorInner {
1602    #[error(transparent)]
1603    Device(#[from] DeviceError),
1604    #[error(transparent)]
1605    RenderCommand(RenderCommandError),
1606    #[error(transparent)]
1607    Draw(#[from] DrawError),
1608    #[error(transparent)]
1609    MissingDownlevelFlags(#[from] MissingDownlevelFlags),
1610    #[error(transparent)]
1611    Bind(#[from] BindError),
1612    #[error(transparent)]
1613    InvalidResource(#[from] InvalidResourceError),
1614}
1615
1616impl<T> From<T> for RenderBundleErrorInner
1617where
1618    T: Into<RenderCommandError>,
1619{
1620    fn from(t: T) -> Self {
1621        Self::RenderCommand(t.into())
1622    }
1623}
1624
1625/// Error encountered when finishing recording a render bundle.
1626#[derive(Clone, Debug, Error)]
1627#[error("{scope}")]
1628pub struct RenderBundleError {
1629    pub scope: PassErrorScope,
1630    #[source]
1631    inner: RenderBundleErrorInner,
1632}
1633
1634impl WebGpuError for RenderBundleError {
1635    fn webgpu_error_type(&self) -> ErrorType {
1636        let Self { scope: _, inner } = self;
1637        let e: &dyn WebGpuError = match inner {
1638            RenderBundleErrorInner::Device(e) => e,
1639            RenderBundleErrorInner::RenderCommand(e) => e,
1640            RenderBundleErrorInner::Draw(e) => e,
1641            RenderBundleErrorInner::MissingDownlevelFlags(e) => e,
1642            RenderBundleErrorInner::Bind(e) => e,
1643            RenderBundleErrorInner::InvalidResource(e) => e,
1644        };
1645        e.webgpu_error_type()
1646    }
1647}
1648
1649impl RenderBundleError {
1650    pub fn from_device_error(e: DeviceError) -> Self {
1651        Self {
1652            scope: PassErrorScope::Bundle,
1653            inner: e.into(),
1654        }
1655    }
1656}
1657
1658impl<E> MapPassErr<RenderBundleError> for E
1659where
1660    E: Into<RenderBundleErrorInner>,
1661{
1662    fn map_pass_err(self, scope: PassErrorScope) -> RenderBundleError {
1663        RenderBundleError {
1664            scope,
1665            inner: self.into(),
1666        }
1667    }
1668}
1669
1670pub mod bundle_ffi {
1671    use super::{RenderBundleEncoder, RenderCommand};
1672    use crate::{command::DrawCommandFamily, id, RawString};
1673    use core::{convert::TryInto, slice};
1674    use wgt::{BufferAddress, BufferSize, DynamicOffset, IndexFormat};
1675
1676    /// # Safety
1677    ///
1678    /// This function is unsafe as there is no guarantee that the given pointer is
1679    /// valid for `offset_length` elements.
1680    pub unsafe fn wgpu_render_bundle_set_bind_group(
1681        bundle: &mut RenderBundleEncoder,
1682        index: u32,
1683        bind_group_id: Option<id::BindGroupId>,
1684        offsets: *const DynamicOffset,
1685        offset_length: usize,
1686    ) {
1687        let offsets = unsafe { slice::from_raw_parts(offsets, offset_length) };
1688
1689        let redundant = bundle.current_bind_groups.set_and_check_redundant(
1690            bind_group_id,
1691            index,
1692            &mut bundle.base.dynamic_offsets,
1693            offsets,
1694        );
1695
1696        if redundant {
1697            return;
1698        }
1699
1700        bundle.base.commands.push(RenderCommand::SetBindGroup {
1701            index,
1702            num_dynamic_offsets: offset_length,
1703            bind_group: bind_group_id,
1704        });
1705    }
1706
1707    pub fn wgpu_render_bundle_set_pipeline(
1708        bundle: &mut RenderBundleEncoder,
1709        pipeline_id: id::RenderPipelineId,
1710    ) {
1711        if bundle.current_pipeline.set_and_check_redundant(pipeline_id) {
1712            return;
1713        }
1714
1715        bundle
1716            .base
1717            .commands
1718            .push(RenderCommand::SetPipeline(pipeline_id));
1719    }
1720
1721    pub fn wgpu_render_bundle_set_vertex_buffer(
1722        bundle: &mut RenderBundleEncoder,
1723        slot: u32,
1724        buffer_id: id::BufferId,
1725        offset: BufferAddress,
1726        size: Option<BufferSize>,
1727    ) {
1728        bundle.base.commands.push(RenderCommand::SetVertexBuffer {
1729            slot,
1730            buffer: buffer_id,
1731            offset,
1732            size,
1733        });
1734    }
1735
1736    pub fn wgpu_render_bundle_set_index_buffer(
1737        encoder: &mut RenderBundleEncoder,
1738        buffer: id::BufferId,
1739        index_format: IndexFormat,
1740        offset: BufferAddress,
1741        size: Option<BufferSize>,
1742    ) {
1743        encoder.set_index_buffer(buffer, index_format, offset, size);
1744    }
1745
1746    /// # Safety
1747    ///
1748    /// This function is unsafe as there is no guarantee that the given pointer is
1749    /// valid for `data` elements.
1750    pub unsafe fn wgpu_render_bundle_set_push_constants(
1751        pass: &mut RenderBundleEncoder,
1752        stages: wgt::ShaderStages,
1753        offset: u32,
1754        size_bytes: u32,
1755        data: *const u8,
1756    ) {
1757        assert_eq!(
1758            offset & (wgt::PUSH_CONSTANT_ALIGNMENT - 1),
1759            0,
1760            "Push constant offset must be aligned to 4 bytes."
1761        );
1762        assert_eq!(
1763            size_bytes & (wgt::PUSH_CONSTANT_ALIGNMENT - 1),
1764            0,
1765            "Push constant size must be aligned to 4 bytes."
1766        );
1767        let data_slice = unsafe { slice::from_raw_parts(data, size_bytes as usize) };
1768        let value_offset = pass.base.push_constant_data.len().try_into().expect(
1769            "Ran out of push constant space. Don't set 4gb of push constants per RenderBundle.",
1770        );
1771
1772        pass.base.push_constant_data.extend(
1773            data_slice
1774                .chunks_exact(wgt::PUSH_CONSTANT_ALIGNMENT as usize)
1775                .map(|arr| u32::from_ne_bytes([arr[0], arr[1], arr[2], arr[3]])),
1776        );
1777
1778        pass.base.commands.push(RenderCommand::SetPushConstant {
1779            stages,
1780            offset,
1781            size_bytes,
1782            values_offset: Some(value_offset),
1783        });
1784    }
1785
1786    pub fn wgpu_render_bundle_draw(
1787        bundle: &mut RenderBundleEncoder,
1788        vertex_count: u32,
1789        instance_count: u32,
1790        first_vertex: u32,
1791        first_instance: u32,
1792    ) {
1793        bundle.base.commands.push(RenderCommand::Draw {
1794            vertex_count,
1795            instance_count,
1796            first_vertex,
1797            first_instance,
1798        });
1799    }
1800
1801    pub fn wgpu_render_bundle_draw_indexed(
1802        bundle: &mut RenderBundleEncoder,
1803        index_count: u32,
1804        instance_count: u32,
1805        first_index: u32,
1806        base_vertex: i32,
1807        first_instance: u32,
1808    ) {
1809        bundle.base.commands.push(RenderCommand::DrawIndexed {
1810            index_count,
1811            instance_count,
1812            first_index,
1813            base_vertex,
1814            first_instance,
1815        });
1816    }
1817
1818    pub fn wgpu_render_bundle_draw_indirect(
1819        bundle: &mut RenderBundleEncoder,
1820        buffer_id: id::BufferId,
1821        offset: BufferAddress,
1822    ) {
1823        bundle.base.commands.push(RenderCommand::DrawIndirect {
1824            buffer: buffer_id,
1825            offset,
1826            count: 1,
1827            family: DrawCommandFamily::Draw,
1828            vertex_or_index_limit: None,
1829            instance_limit: None,
1830        });
1831    }
1832
1833    pub fn wgpu_render_bundle_draw_indexed_indirect(
1834        bundle: &mut RenderBundleEncoder,
1835        buffer_id: id::BufferId,
1836        offset: BufferAddress,
1837    ) {
1838        bundle.base.commands.push(RenderCommand::DrawIndirect {
1839            buffer: buffer_id,
1840            offset,
1841            count: 1,
1842            family: DrawCommandFamily::DrawIndexed,
1843            vertex_or_index_limit: None,
1844            instance_limit: None,
1845        });
1846    }
1847
1848    /// # Safety
1849    ///
1850    /// This function is unsafe as there is no guarantee that the given `label`
1851    /// is a valid null-terminated string.
1852    pub unsafe fn wgpu_render_bundle_push_debug_group(
1853        _bundle: &mut RenderBundleEncoder,
1854        _label: RawString,
1855    ) {
1856        //TODO
1857    }
1858
1859    pub fn wgpu_render_bundle_pop_debug_group(_bundle: &mut RenderBundleEncoder) {
1860        //TODO
1861    }
1862
1863    /// # Safety
1864    ///
1865    /// This function is unsafe as there is no guarantee that the given `label`
1866    /// is a valid null-terminated string.
1867    pub unsafe fn wgpu_render_bundle_insert_debug_marker(
1868        _bundle: &mut RenderBundleEncoder,
1869        _label: RawString,
1870    ) {
1871        //TODO
1872    }
1873}
wgpu_core/command/bundle.rs

wgpu_core/command/
bundle.rs
