Skip to main content

Presence Penalty

What is Presence Penalty?​

Presence Penalty ranges from -2.0 to 2.0 and affects how the model penalizes new tokens based on their presence in the text so far.

  • Positive values: Increase the likelihood of discussing new topics by penalizing tokens that have already been used.

Presence Penalty applies once to all tokens that have been sampled at least once, while Frequency Penalty depends on how often a token has been sampled.

  • Default value: 0 (uses words from the input prompt).
  • Higher values: Encourage generating new words not present in the input, creating more diverse and creative output.

Adjusting the Presence Penalty​

The default Presence Penalty is 0.01. You can change it in the AI Engine tab.

Steps to change the Presence Penalty:

  1. Go to the plugin menu on your WordPress dashboard.
  2. Click on the Settings page and find the AI Engine tab.
  3. Enter a new value in the Presence Penalty field.
  1. Click the Save button.

Differences Between Frequency and Presence Penalty​

  • Frequency Penalty: Penalizes words seen frequently during training. Encourages generating novel or less common words.

  • Presence Penalty: Penalizes words present in the input prompt. Encourages generating words not in the input.

  • Frequency Penalty: Modifies the probability of words seen frequently during training, making them less likely.

  • Presence Penalty: Modifies the probability of words in the input text, making them less likely to repeat in the output.

Both parameters can increase text diversity and encourage generating more novel or unexpected words, but they do so in different ways. Depending on your needs, you might use one or both.

Use Frequency Penalty for generating text less like training data and Presence Penalty for generating text less like the input text.