fn consume_token(
input: &str,
generic: bool,
ignore_doc_comments: bool,
) -> (Token<'_>, &str)
Expand description
Return the token at the start of input
.
If generic
is false
, then the bit shift operators >>
or <<
are valid lookahead tokens for the current parser state (see §3.1
Parsing in the WGSL specification). In other words:
-
If
generic
istrue
, then we are expecting an angle bracket around a generic type parameter, like the<
and>
invec3<f32>
, so interpret<
and>
asToken::Paren
tokens, even if they’re part of<<
or>>
sequences. -
Otherwise, interpret
<<
and>>
as shift operators:Token::LogicalOperation
tokens.
If ignore_doc_comments
is true, doc comments are treated as Token::Trivia
.