consume_token

Function consume_token 

Source
fn consume_token(
    input: &str,
    waiting_for_template_end: bool,
    ignore_doc_comments: bool,
) -> (Token<'_>, &str)
Expand description

Return the token at the start of input.

The waiting_for_template_end flag enables some special handling to help out discover_template_lists:

  • If waiting_for_template_end is true, then return text starting with ’> as [Token::TemplateArgsEnd] and consume only the >character, regardless of what characters follow it. This is required by the [template list discovery algorithm][tlda] when the>` would end a template argument list.

  • If waiting_for_template_end is false, recognize multi-character tokens beginning with > as usual.

If ignore_doc_comments is true, then doc comments are returned as Token::Trivia, like ordinary comments.