Index (JTokkit of OpenAI API 3.8.2024030801 API)

A B C D E F G I M N P R T V X
All Classes and Interfaces|All Packages|Constant Field Values

A

ADA - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
ArgumentFormat - Class in xyz.felh.openai.jtokkit.utils
ArgumentFormat() - Constructor for class xyz.felh.openai.jtokkit.utils.ArgumentFormat

B

BABBAGE - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType

C

CL100K_BASE - Enum constant in enum class xyz.felh.openai.jtokkit.api.EncodingType
CODE_CUSHMAN_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
CODE_CUSHMAN_002 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
CODE_DAVINCI_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
CODE_DAVINCI_002 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
CODE_DAVINCI_EDIT_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
CODE_SEARCH_ADA_CODE_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
CODE_SEARCH_BABBAGE_CODE_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
countTokens(String) - Method in interface xyz.felh.openai.jtokkit.api.Encoding: Encodes the given text into a list of token ids and returns the amount of tokens.
CURIE - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
CUSHMAN_CODEX - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType

D

DAVINCI - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
DAVINCI_CODEX - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
decode(List<Integer>) - Method in interface xyz.felh.openai.jtokkit.api.Encoding: Decodes the given list of token ids into a text.
decode(Encoding, List<Integer>) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 通过Encoding和encoded数组反推text信息
decode(EncodingType, List<Integer>) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 通过EncodingType和encoded编码数组，反推字符串文本
decodeBytes(List<Integer>) - Method in interface xyz.felh.openai.jtokkit.api.Encoding: Decodes the given list of token ids into a byte array.
decodeToken(int, SpecialEncoder) - Method in class xyz.felh.openai.jtokkit.TokenEncoder
DUMMY_RANK - Static variable in class xyz.felh.openai.jtokkit.TokenEncoder

E

encode(String) - Method in interface xyz.felh.openai.jtokkit.api.Encoding: Encodes the given text into a list of token ids.
encode(String, int) - Method in interface xyz.felh.openai.jtokkit.api.Encoding: Encodes the given text into a list of token ids.
encode(String, String) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 获取encode的编码数组，通过模型名称
encode(Encoding, String) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 通过Encoding和text获取编码数组
encode(EncodingType, String) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 获取encode的编码数组
encodeOrdinary(String) - Method in interface xyz.felh.openai.jtokkit.api.Encoding: Encodes the given text into a list of token ids, ignoring special tokens.
encodeOrdinary(String, int) - Method in interface xyz.felh.openai.jtokkit.api.Encoding: Encodes the given text into a list of token ids, ignoring special tokens.
Encoding - Interface in xyz.felh.openai.jtokkit.api
EncodingRegistry - Interface in xyz.felh.openai.jtokkit.api: The EncodingRegistry is used to register custom encodings and to retrieve encodings by name or type.
EncodingResult - Class in xyz.felh.openai.jtokkit.api: The result of encoding operation.
EncodingResult(List<Integer>, int, boolean) - Constructor for class xyz.felh.openai.jtokkit.api.EncodingResult
Encodings - Class in xyz.felh.openai.jtokkit
EncodingType - Enum Class in xyz.felh.openai.jtokkit.api
estimateTokens(CreateChatCompletionRequest) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 计算request的token数量
estimateTokensInMessage(String, ChatMessage, int) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 通过模型名称计算messages获取编码数组参考官方的处理逻辑：https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb
estimateTokensInMessages(String, List<ChatMessage>) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils
estimateTokensInMessages(String, List<ChatMessage>, List<Tool>) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils
estimateTokensInTools(String, List<Tool>) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils

F

format(Object) - Static method in class xyz.felh.openai.jtokkit.utils.ToolContentFormat
formatArguments(String) - Static method in class xyz.felh.openai.jtokkit.utils.ArgumentFormat
formatArguments(String) - Static method in class xyz.felh.openai.jtokkit.utils.ToolContentFormat
formatFunctionDefinitions(List<Tool>) - Static method in class xyz.felh.openai.jtokkit.utils.FunctionFormat
fromName(String) - Static method in enum class xyz.felh.openai.jtokkit.api.EncodingType
fromName(String) - Static method in enum class xyz.felh.openai.jtokkit.api.ModelType: Returns a ModelType for the given name, or Optional.empty() if no such model type exists.
FunctionFormat - Class in xyz.felh.openai.jtokkit.utils
FunctionFormat() - Constructor for class xyz.felh.openai.jtokkit.utils.FunctionFormat

G

getEncoder() - Method in class xyz.felh.openai.jtokkit.api.GptBytePairEncodingParams
getEncoding(String) - Method in interface xyz.felh.openai.jtokkit.api.EncodingRegistry: Returns the encoding with the given name, if it exists.
getEncoding(String) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 获取一个Encoding对象，通过模型名称
getEncoding(EncodingType) - Method in interface xyz.felh.openai.jtokkit.api.EncodingRegistry: Returns the encoding with the given type.
getEncoding(EncodingType) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 获取一个Encoding对象，通过Encoding类型
getEncodingForModel(String) - Method in interface xyz.felh.openai.jtokkit.api.EncodingRegistry: Returns the encoding that is used for the given model type, if it exists.
getEncodingForModel(ModelType) - Method in interface xyz.felh.openai.jtokkit.api.EncodingRegistry: Returns the encoding that is used for the given model type.
getName() - Method in interface xyz.felh.openai.jtokkit.api.Encoding: Returns the name of this encoding.
getName() - Method in enum class xyz.felh.openai.jtokkit.api.EncodingType
getName() - Method in class xyz.felh.openai.jtokkit.api.GptBytePairEncodingParams
getPattern() - Method in class xyz.felh.openai.jtokkit.api.GptBytePairEncodingParams
getSpecialTokensEncoder() - Method in class xyz.felh.openai.jtokkit.api.GptBytePairEncodingParams
getTokenCount() - Method in class xyz.felh.openai.jtokkit.api.EncodingResult
getTokens() - Method in class xyz.felh.openai.jtokkit.api.EncodingResult
GPT_3_5_TURBO_0125 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
GPT_3_5_TURBO_1106 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
GPT_4 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
GPT_4_0125_PREVIEW - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
GPT_4_1106_PREVIEW - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
GPT_4_32K - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
GPT_4_VISION_PREVIEW - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
GptBytePairEncodingParams - Class in xyz.felh.openai.jtokkit.api: Parameter for the byte pair encoding used to tokenize for the OpenAI GPT models.
GptBytePairEncodingParams(String, Pattern, Map<byte[], Integer>, Map<String, Integer>) - Constructor for class xyz.felh.openai.jtokkit.api.GptBytePairEncodingParams: Creates a new instance of GptBytePairEncodingParams.

I

isBlank(CharSequence) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils
isJSONString(String) - Static method in class xyz.felh.openai.jtokkit.utils.ToolContentFormat
isTruncated() - Method in class xyz.felh.openai.jtokkit.api.EncodingResult

M

MAX_RANK - Static variable in class xyz.felh.openai.jtokkit.TokenEncoder
ModelType - Enum Class in xyz.felh.openai.jtokkit.api

N

newDefaultEncodingRegistry() - Static method in class xyz.felh.openai.jtokkit.Encodings: Creates a new EncodingRegistry with the default encodings found in the EncodingType enum.
newLazyEncodingRegistry() - Static method in class xyz.felh.openai.jtokkit.Encodings: Creates a new EncodingRegistry without any EncodingType registered.

P

P50K_BASE - Enum constant in enum class xyz.felh.openai.jtokkit.api.EncodingType
P50K_EDIT - Enum constant in enum class xyz.felh.openai.jtokkit.api.EncodingType

R

R50K_BASE - Enum constant in enum class xyz.felh.openai.jtokkit.api.EncodingType
registerCustomEncoding(Encoding) - Method in interface xyz.felh.openai.jtokkit.api.EncodingRegistry: Registers a new custom encoding with the given name.
registerGptBytePairEncoding(GptBytePairEncodingParams) - Method in interface xyz.felh.openai.jtokkit.api.EncodingRegistry: Registers a new byte pair encoding with the given name.

T

TEXT_ADA_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_BABBAGE_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_CURIE_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_DAVINCI_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_DAVINCI_002 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_DAVINCI_003 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_DAVINCI_EDIT_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_EMBEDDING_3_LARGE - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_EMBEDDING_3_SMALL - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_EMBEDDING_ADA_002 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_SEARCH_ADA_DOC_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_SEARCH_BABBAGE_DOC_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_SEARCH_CURIE_DOC_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_SEARCH_DAVINCI_DOC_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_SIMILARITY_ADA_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_SIMILARITY_BABBAGE_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_SIMILARITY_CURIE_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TEXT_SIMILARITY_DAVINCI_001 - Enum constant in enum class xyz.felh.openai.jtokkit.api.ModelType
TikTokenUtils - Class in xyz.felh.openai.jtokkit.utils
TikTokenUtils() - Constructor for class xyz.felh.openai.jtokkit.utils.TikTokenUtils
TokenEncoder - Class in xyz.felh.openai.jtokkit
TokenEncoder(Map<byte[], Integer>) - Constructor for class xyz.felh.openai.jtokkit.TokenEncoder
tokens(String, Object, List<Tool>) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils
tokens(String, String) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 通过模型名称, 计算指定字符串的tokens
tokens(Encoding, String) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 通过Encoding计算text信息的tokens
tokens(EncodingType, String) - Static method in class xyz.felh.openai.jtokkit.utils.TikTokenUtils: 计算指定字符串的tokens，通过EncodingType
ToolContentFormat - Class in xyz.felh.openai.jtokkit.utils
ToolContentFormat() - Constructor for class xyz.felh.openai.jtokkit.utils.ToolContentFormat
toString() - Method in class xyz.felh.openai.jtokkit.api.EncodingResult
tryFormat(String) - Static method in class xyz.felh.openai.jtokkit.utils.ToolContentFormat

V

valueOf(String) - Static method in enum class xyz.felh.openai.jtokkit.api.EncodingType: Returns the enum constant of this class with the specified name.
valueOf(String) - Static method in enum class xyz.felh.openai.jtokkit.api.ModelType: Returns the enum constant of this class with the specified name.
values() - Static method in enum class xyz.felh.openai.jtokkit.api.EncodingType: Returns an array containing the constants of this enum class, in the order they are declared.
values() - Static method in enum class xyz.felh.openai.jtokkit.api.ModelType: Returns an array containing the constants of this enum class, in the order they are declared.
VERY_LARGE_TOKENIZER_BYTE_THRESHOLD_KEY - Static variable in class xyz.felh.openai.jtokkit.TokenEncoder

X

xyz.felh.openai.jtokkit - package xyz.felh.openai.jtokkit
xyz.felh.openai.jtokkit.api - package xyz.felh.openai.jtokkit.api
xyz.felh.openai.jtokkit.utils - package xyz.felh.openai.jtokkit.utils

A B C D E F G I M N P R T V X
All Classes and Interfaces|All Packages|Constant Field Values