Large language model (LLM) artificial intelligence can understand concepts. This is possible because LLMs use multidimensional neural networks to map the semantic relationship between different terms and ideas.
This is similar to how the human brain organizes and retrieves information.
Whereas traditional computing required humans to convert concepts into computer codes, today’s AI can understand plain language requests without this conversion. As a result, the processing power of today’s AI uses a different unit of measurement, the token. A token represents characters that are grouped together for some semantic use.
Generally, one token corresponds to about 4 characters of English text, which is about ¾ of a word. So 100 tokens is equal to about 75 words.
The number of tokens that AI considers at any given time is called the context window.
When the context window limit is reached–meaning, the AI has considered the maximum amount of tokens it can consider at any given time–then the earliest tokens are forgotten as new tokens arrive.
B. Understanding CoCounsel’s limitations
Tokens and context windows can help you understand CoCounsel’s limitations during chat conversations–meaning, when you are not using a skill–and during skill refinements.
Various factors determine when a context window has been exceeded, so it is not possible to say, for example, how many chats or how many skills will always fit within this window. Generally, you can expect CoCounsel to work like this.
For chats with CoCounsel, the context window is 8,000 tokens, or about 6,000 words, including both your requests and CoCounsel’s responses. This is why CoCounsel may fail to process instructions where the request (what you enter) and response (the computation CoCounsel completes) require more than 8,000 tokens. This is also why CoCounsel may not be able to understand a request that references information that is too far back in the chat conversation—meaning, information that is outside of its context window.
2. Skill refinements
When you review CoCounsel’s full response for Legal Research Memo or Draft Correspondence, and then select “Refine results,” CoCounsel will only refine the instant response. This means CoCounsel cannot consider requests or responses from chat conversations or from other skills.