Fascinating, all stuff completely unknown to me. Love the personal memories, the images from old magazines, etc. The magazines especially now seem so . . . bygone.
Three points:
1. In reaction to "As an example, consider the sequence of characters elephan_. What letters (in English) could occupy the blank spot? Only the letter t. From an information-theory perspective, that means that the letter t in that position adds no information to what we already knew. Its information content is effectively zero . . ."
Is that necessarily true? Think about some letters that might be used. From elephanx, one might infer that we were looking an effort to talk about this species in a non-binary way. Elephanz might imply something in a hip-hop direction. Elephanh looks vaguely archaizing, elephanj exoticizing in a Persian direction--both, along with others, possibly evoking a fantasy-novelesque usage. (Remember Tolkien's "oliphaunt.") And so on.
Might some suggestion arise here of the difference between word-production, and word-play--with some of what that implies (please forgive me, or at least bear with me) of anti-capitalist, anti-modernist critique?
2. Speaking of word-play, I was struck by how much better I liked 2-gram Shakespeare than 2+n-gram Shakespeare. The first gave me such pleasure, I laughed out loud with delight. If a parodist (human or LLM) had written it, I would have thought, "this is absolutely brilliant, completely spot-on." (The random mention of "Diana" may be key: when writing Shakespeare, if in doubt, stick in a random classical deity.) But 3, 4, and 5 gram seemed . . . dreary? From them no pleasure did I take, anyway. 6-gram wasn't much better, but it did inspire me to think, "I need to read more Shakespeare!"
3. Do LLMs engage in play, word-play, or parody? From one who engages with these systems a lot, I get the sense they do, or can. I have never conversed with one. The idea makes me nervous.
Fascinating, all stuff completely unknown to me. Love the personal memories, the images from old magazines, etc. The magazines especially now seem so . . . bygone.
Three points:
1. In reaction to "As an example, consider the sequence of characters elephan_. What letters (in English) could occupy the blank spot? Only the letter t. From an information-theory perspective, that means that the letter t in that position adds no information to what we already knew. Its information content is effectively zero . . ."
Is that necessarily true? Think about some letters that might be used. From elephanx, one might infer that we were looking an effort to talk about this species in a non-binary way. Elephanz might imply something in a hip-hop direction. Elephanh looks vaguely archaizing, elephanj exoticizing in a Persian direction--both, along with others, possibly evoking a fantasy-novelesque usage. (Remember Tolkien's "oliphaunt.") And so on.
Might some suggestion arise here of the difference between word-production, and word-play--with some of what that implies (please forgive me, or at least bear with me) of anti-capitalist, anti-modernist critique?
2. Speaking of word-play, I was struck by how much better I liked 2-gram Shakespeare than 2+n-gram Shakespeare. The first gave me such pleasure, I laughed out loud with delight. If a parodist (human or LLM) had written it, I would have thought, "this is absolutely brilliant, completely spot-on." (The random mention of "Diana" may be key: when writing Shakespeare, if in doubt, stick in a random classical deity.) But 3, 4, and 5 gram seemed . . . dreary? From them no pleasure did I take, anyway. 6-gram wasn't much better, but it did inspire me to think, "I need to read more Shakespeare!"
3. Do LLMs engage in play, word-play, or parody? From one who engages with these systems a lot, I get the sense they do, or can. I have never conversed with one. The idea makes me nervous.