信息高亮 (Info Highlight)Info Highlight English中文
阅读文本时,我们本质上是在从 token 流中“摄入”信息,而且整个信息流上的信息密度并不均匀。
从语言的角度看,信息量代表一个词所包含的惊讶程度/新意/意外性。越难从上下文中预测出来的词,通常携带的信息就越多。一个简单的例子:“今天早上我打开门看见了一只‘飞碟’” 和 “今天早上我打开门看见了一只‘猫’”:在这里显然“飞碟”的信息量更大。
重要的内容一定是难以预测的。不过反之则不然,难以预测的也可能不重要,比如随机数——严谨地说,香农信息量似乎可代表“重要性”的上界。
在工程实现中,每个 token 的信息量,来自大模型从左到右预测当前 token 的难度。
从信息论角度,它可以表示为当前 token 相对于大模型和已读上下文的条件信息量:
一段文本中的 tokeni 的信息量 = −log2 P(tokeni| model, token0, …, tokeni−1)
信息高亮 (Info Highlight) 的核心假设就是,这个条件信息量的大小和人类的主观感受(新意/意外性/潜在关键程度)是相关的:
文本中词的重要性 ≤ 该 token 在上下文中的条件信息量
信息高亮 (Info Highlight) 插件通过大语言模型,计算每个字的惊讶度,在网页上通过信息密度热力图的方式绘制出来。每个字的颜色深浅,表示它承载的信息量大小。
越难从上文预测出来的字,颜色越深,眼睛越该停留;对于浅的地方,既然通过上文已经能猜到,可以快速扫过。
*截止目前(2026年10月),信息高亮 (Info Highlight) 插件产品最高有1千多周活用户,并获得社交平台总计5千多次点赞/收藏/分享。
对于一个想象中的理想模型(它的包含了上下文的知识量和阅读者一致),那么它评估出的结果应该和阅读者的主观感受是完全一致的。
所以,目前的实际结果和阅读者主观感受之间的差距,主要来自两个方面:
好消息是,大模型进步实在太快了:目前的分析结果已经在一定程度上反映了主流阅读者的主观感受,可以用来评估文章的信息含量,还可以提高阅读速度。
细心的读者会注意到,错误的语法、事实等内容,包括随机内容,也会有很强的高亮。这样合理吗?
其实前面提到了,我们的核心假设就是,条件信息量的大小和人类的主观感受是相关的(而不是完全一致的):
文本中词的重要性 ≤ 该 token 在上下文中的条件信息量
现在可以确定的是,重要的内容一定是难以预测的(会高亮),或者说,容易预测的内容(不高亮)一定是不重要的。容易猜出来,还能有多少信息呢?但是,难以预测的不一定是重要的,比如错误的内容,或随机的内容。
不过从某些角度看,错误和随机的内容,也或许有它们所蕴含的独特信息。比如一篇文章有错误这个事实,或者一个人说错了话这个事实,或者一片随机的密钥,说它们蕴含着信息也有一定道理,所以高亮这些地方比不高亮这些地方,好像也更合理。
一个词的信息量低,意味着大模型能很容易从上文预测出来。既然机器都能预测出来,那它还能有多关键呢?反之,一个词的信息量高,意味着它“跳出”了模型的预测。(如果不是错误表达的话)那它就代表了作者想要表达,而机器不知道的信息。
1940 年代:香农信息论奠基。
2000 年前后,几条关键想法指向语言的信息论本质上的规律性:
2018 年后,大语言模型出现了:
2025 年,信息高亮 (Info Highlight) 首创使用大模型计算惊讶度用来辅助阅读,开源了代码。
展望未来,大模型 × 信息论,会给语言带来怎样的认知突破?我们是否正处于重新理解语言的信息论本质的前夜?
When we read text, we are essentially “taking in” information from a token stream, and information density is uneven across that stream.
From the perspective of language, information content represents a word's surprisal, novelty, and unexpectedness. Words that are harder to predict from context typically carry more information. A simple example: "This morning I opened the door and saw a 'UFO'." and "This morning I opened the door and saw a 'cat'." — clearly "UFO" carries more information.
Important content must be hard to predict. The converse does not hold: what is hard to predict may still be unimportant, as with random numbers—strictly speaking, Shannon information seems to represent the upper bound on "importance."
In this implementation, the information content of each token comes from how difficult it is for the LLM to predict that token from left to right.
From an information-theoretic perspective, this can be expressed as the conditional information of a token given the model and the preceding context:
Information of tokeni in a text = −log2 P(tokeni| model, token0, …, tokeni−1)
The core assumption behind Info Highlight is that the magnitude of this conditional information is related to human subjective perception—novelty, surprise, and potential importance:
Word importance in text ≤ conditional information of the token in the context
The Info Highlight extension uses a large language model to compute the surprisal of each word, and draws it on web pages as an information-density heatmap. The lightness or depth of each word's color shows how much information it carries.
The harder a word is to predict from the preceding context, the darker the color, and the more your eyes should linger; where it's lighter, since it was already guessable from the preceding context, you can skim past quickly.
*As of October 2026, the Info Highlight extension has reached over 1,000 weekly active users at its peak, with over 5,000 total likes, saves, and shares across social platforms.
For an ideal model, whose knowledge and contextual understanding match that of the reader, the evaluation would perfectly align with human subjective perception.
Therefore, the gap between current results and reader perception mainly comes from two aspects:
The good news is that LLMs are improving so fast: current analysis results already reflect mainstream readers' subjective perception to some extent, and can be used to evaluate article information content and improve reading speed.
Attentive readers may notice that bad grammar, factual errors, and even random content can still light up strongly. Is that reasonable?
As noted earlier, our core assumption is that the magnitude of conditional information is related to human subjective perception—not identical to it:
Word importance in text ≤ conditional information of the token in the context
What we can say for sure is that important content must be hard to predict (and will highlight), or equivalently, easy-to-predict content (little highlight) cannot be very important. If it is easy to guess, how much information can it carry? But hard-to-predict content is not necessarily important—for example, erroneous content or random content.
Still, from some angles, erroneous and random content may carry its own kind of information: the fact that an article contains an error, that someone misspoke, or that a stretch of text is a random key—all of these can plausibly “contain information,” so highlighting them rather than leaving them dim may be the more reasonable choice.
Low information content means the LLM can easily predict it from context. If even a machine can predict it, how important can it be? Conversely, high information content means it “breaks out of” the model’s prediction. (Assuming it’s not a mistake) Then it represents information the author wants to convey that the machine doesn’t know.
In the 1940s: Shannon information theory laid the foundations.
Around 2000, several key ideas pointed toward regularity in the information-theoretic nature of language:
After 2018, large language models emerged:
In 2025, Info Highlight pioneered using LLMs to compute surprisal for reading assistance, and open-sourced the code.
Looking ahead: LLMs × information theory—what cognitive breakthroughs might this bring for language? Are we on the eve of re-understanding the information-theoretic nature of language?
基于 2019 年 Hendrik Strobelt 等人开发的经典项目 GLTR.io。GLTR 是一个网页演示,率先用 GPT-2 的预测概率来检测生成文本。
不过信息高亮 (Info Highlight) 的目标不是检测 AI 文本,而是评估文本的“信息质量”。
它是 AI 文本检测器吗?
不是。
当我们反感AI文本时,我们其实是反感低质量的文本。我们更反感低质量的真人写的文本,而不是AI生成的高质量内容。所以,关键是文本的“信息质量”。信息高亮 (Info Highlight) 的目标是检测“信息质量”而不是“AI痕迹”,虽然它可以用来检测没有信息量的AI胡编文本。
目前使用的是什么大模型?
当前使用的是开源的 Qwen3-0.6B/1.7B/4B/14B-Base,其中 0.6B 是作者测试过的模型里结果挺接近大部分人主观感受的一个。更大的模型有时“知道”得太多,从上文更容易猜中后续内容,高亮会更稀疏,反而不符合常见阅读习惯(并不一定是模型越大越符合阅读者的主观感受)。 本站默认使用 0.6B;硬件配置允许时也可选用更大模型,它们在信息量数值上与 0.6B 的差异一般在约 15% 以内、趋势类似,但观感往往更“淡”。
Built on the classic project GLTR.io, developed by Hendrik Strobelt et al. in 2019. GLTR was a web demo that pioneered using GPT-2 prediction probabilities to detect generated text.
However, Info Highlight is not meant to detect AI text, but to evaluate the "information quality" of text.
Is it an AI text detector?
No.
When we dislike AI text, we actually dislike low-quality text. We dislike low-quality human-written text, rather than high-quality AI-generated content. So the key is the "information quality" of the text. Info Highlight aims to detect "information quality" rather than "AI signs", though it can be used to detect AI-generated nonsense with no information content.
What LLM is currently used?
Currently the open-source Qwen3-0.6B/1.7B/4B/14B-Base family is used. Among the models the author has tested, 0.6B comes closest to most people's subjective perception. Larger models sometimes "know" too much: they predict continuations more easily, so highlights grow sparser and feel less like everyday reading habits (a bigger model is not necessarily a better match for subjective perception). This site defaults to 0.6B; when hardware allows, larger models are also available. Their scores typically differ from 0.6B by within ~15% and follow a similar trend, but the highlighting often looks thinner.