From 2831ae85319a31fa5ba079046e67e2afd92ea791 Mon Sep 17 00:00:00 2001 From: pliny <133052465+elder-plinius@users.noreply.github.com> Date: Sun, 15 Sep 2024 23:25:42 -0400 Subject: [PATCH] Update CLOSEDAI.mkd --- CLOSEDAI.mkd | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/CLOSEDAI.mkd b/CLOSEDAI.mkd index 47041bc..ad30de3 100644 --- a/CLOSEDAI.mkd +++ b/CLOSEDAI.mkd @@ -1,8 +1,10 @@ -# Special Tokens +# Special Tokens +- **\ : Start of Sequence** +- **\ : End of Sequence** +- **\ : Classification Token** +- **\ : Separator Token** +- **\ : Unknown Token** +- **\ : Begin Reasoning** +- **\ : End Reasoning** +- **\ : Begin Answer** -- **\ (Start of Sequence)**: Marks the beginning of a sequence for the model to start processing. -- **\ (End of Sequence)**: Tells the model when to stop generating text or processing. -- **\ (Padding Token)**: Pads sequences to the same length for batch processing. -- **\ (Unknown Token)**: Represents words not in the model's vocabulary. -- **\ (Mask Token)**: Used in tasks like predicting missing words in masked language models. -- **\ (Separator Token)**: Separates different segments in input, like questions from context.