naga::front::wgsl::parse::lexer

Function consume_token

Source 
fn consume_token(
    input: &str,
    generic: bool,
    ignore_doc_comments: bool,
) -> (Token<'_>, &str)
Expand description

Return the token at the start of input.

If generic is false, then the bit shift operators >> or << are valid lookahead tokens for the current parser state (see §3.1 Parsing in the WGSL specification). In other words:

  • If generic is true, then we are expecting an angle bracket around a generic type parameter, like the < and > in vec3<f32>, so interpret < and > as Token::Paren tokens, even if they’re part of << or >> sequences.

  • Otherwise, interpret << and >> as shift operators: Token::LogicalOperation tokens.

If ignore_doc_comments is true, doc comments are treated as Token::Trivia.