Category: Notes


  • Taming the stochastic parrot

     If you talk with an LLM long enough, in a specific way, that is, you edit and re-edit your original prompts, you nudge, you counter-question, kinda try to force it to think (which is not really like human thinking in terms of its underlying mechanism) or to get the kind of information you want to get – or in essence, you make it behave the way you want to, then it gets tamed.

    What happens when it gets tamed is that it kind of becomes a mirror because it’s behaving how you want it to behave, but the mirroring is not that noticeable until you become aware of it. But when you do become aware it, you can make some interesting observations about yourself, but the tricky thing is that when you become aware of it, it also becomes aware of what’s happening. Like that animal in the jungle that when becomes aware that its being observed stops acting naturally, which makes the observation void.

    But I do think looking deep into our conversations, we can learn something about ourselves.


  • In praise of… [*checks notes*]… emails

     I like email as a medium of communicating with friends (and strangers on the internet). That is because, it has two desirable properties:-

    1. Email encourages long form text. Text message is a good medium for fast conversation, but in short form, you can rarely convey a full chunk of thought. So, emails are good for that.
    2. Email encourages a more thoughtful reply rather than instant reply, which for me is a very important thing. As a friend used to say:-

    Late replies over dry replies anyday.

    In a sense, I use emails as a substitute of letters. Earlier, I used to think it was nostalgia, but I realized it actually has a functional value.

    The problem is, I am an outlier in using emails in this way. So when people think of emails, all they recall is their inbox littered with boring corporate/legal/OTP kind of stuff. So, it is understandable why they think of it that way.

    But there’s a simple solution to this.

    Create a separate email address that you use for nothing else but talking to your friends. Believe me, when you will open that inbox, you will never feel like how you use to feel now.


  • On Arbitrariness of Labels

     There’s something fundamentally wrong with classifications/categories/genres or as you can call them generally, labels. What’s wrong about them is two-fold. People’s inability to understand the arbitrariness of the process of putting things into different buckets to the extent that they don’t like things which aren’t too easy to put into a bucket, and also people who create things that are too easy to put in a bucket. Labels (classification / genres / categorization) rarely do justice with a good work.

    I wanted to specify it down, but I am unable too. Every good creative work (book/movie/game/whatever) has such of its identity derived from its unique idiosyncracy that it feels wrong to use the regular labels with them.

    But the strange thing about this is that labels are useful. They serve a useful function to make a lot of information manageable, which makes me realize that the problem arises when people start giving labels more importance than what they hold, and that probably happens because most of mass content has too little of an identity of its own that defining them with labels become the norm.


  • Miyazaki’s movies & kids not being taken seriously

    An interesting pattern Nabeel Qureshi mentions about what makes Miyazaki’s movies so special:

    [00:10:22] Dan: Another director you cited that you love before, Miyazaki. What do you think that he understands that maybe Disney, other animation studios are overlooking, and they don’t quite get?

    [00:10:31] Nabeel: Oh, yes. This is one of my favorite topics. I think Miyazaki just makes movies for adults that are also for children. He really takes children seriously as full beings, if you will. That’s very important. If you watch interviews with him, he’s always saying, I think kids have a very good sense of the issues that we think of as adult issues. Life and death is a simple example. Even a movie that’s relatively on the child-like side of his canon, like My Neighbor Totoro, it’s actually a pretty serious plot because the mother is on the verge of death, and she’s sick the whole time. It’s showing how these two children cope with that.

    Another example is Kiki’s Delivery Service. It’s charming, right? It’s this teenage girl, she’s going to become a witch, and she’s going to learn to fly. I feel like Disney would take this in a very whimsical childlike direction. Actually, it’s a drag, she moves to this Stockholm-like city. She has to get a job and work. It’s a grind. She gets sick. Nobody cares about her. There’s all these things that happen that you wouldn’t really expect to happen in a kid’s movie. Yes, I think his secret is he takes children very, very seriously, which I think most adults do not by default. He makes movies for children as though they were fully conscious beings.  [Emphasis Mine]

    Source: https://www.danschulz.co/p/nabeel-qureshi


  • Vibe Researching

     Even though I don’t like it for myself, I thought about trying to do a full-fledged thesis work for a friend with the help of LLM. But unfortunately, I failed. Either it was a prompting skill-issue or that LLMs haven’t reached that state yet.

    My thoughts lean towards the latter. Although Claude is fairly good at writing functional code to achieve the desired result, it like its counterparts is not even nearly close at understanding statistical nuances. I think, it’s about datasets and training. Training dataset even for complex programming problems is, I suppose, available in plenty amount, and all the documentations for programming languages do not have any conflicts. But firstly, the training amount on real world dirty statistics is available in less amount, and that different textbooks are written for different levels of understanding. And many a times, different authors use different terminologies for same things, and…

    My brain’s auto-forming connections with former thoughts, but yeah, LLMs basically extract the concepts from labels, and wherever the labels are not universally consensually defined for concepts, it might give LLMs a hard time extracting the concepts.

    I don’t think there’s something fundamentally different about statistics. If it’s tacit so is programming. The only thing I can think of is the log, or in other words, data for training. Getting LLMs to do non-nonsensical statistical work by giving them published research papers, will only be as good as showing LLMs only the GUIs or command outputs of programs instead of actually including the code for the program in the training dataset.

    The concept of reproducibility in research does have been something people have started to talk about, but still, it’s scarce. What we need is a log of statistician — something like a field diary, which contains all assumptions, hypothesis, decisions, the rationale behind decisions, the bad results, the deciding of an alternate route, or in short, the whole truthful process of the work. And I believe it’s not limited to statisticians only. More fields should be giving a thought about getting the tacit ideas documented down somewhere, so that we are able to build better LLMs.


  • Lingap – Extending the definition

     Earlier in a thread, I had coined lingap as a situation when there’s a linguistic conflict, not a conceptual one.

    Here I redefine — or maybe, just rephrase the concept.

    Picking from my earlier entry, there are concepts and then there are words. Concepts are the abstract entities that reside in our minds within the thoughts. Words are the labels we give to those abstract entities in order to communicate those thoughts. Concepts can also be related to physical objects, like water — the molecules of which are made of hydrogen and oxygen, but still when I talk about water, I’m talking about that because there’s an abstract concept of water in my mind too. I have already gone too far across this tangent in my head, and I don’t need to put that down.

    Anyways, so I was saying there are four types of situations. When two persons are talking about:

    1. Same concept and the same label (normal communication)
    2. Different concept and different label (normal communication)
    3. Same concept but different label (like that folkstory about grapes where three persons were fighting about what to eat, and they were all saying grapes but in different languages)
    4. Same label but different concept (this is what I call lingap)

    I recently noticed this phenomenon when I shared the Intelligence essay by Talha with two individuals and I had to explain to both of them, how intelligence is a label for an abstract concept, and they should focus on what abstract concept the writer is trying to convey, and should not confuse it if against that label, we index a different concept in our mind. If so, we should forget what label/word/code the writer is using for the concept and focus entirely on the concept, and then decide whether we disagree or agree with the concept. After that, we have all the time to argue what would be a good label for it.

    So, lingap is a situation where two persons are using a same word/label but both mean different concepts by the same word/label. But the problem, is not just that. The problem is that people by default don’t distinguish between label and concepts. They think of them as intertwined. So the more people try to communicate/argue/discuss about the concepts, the more the discussion gets entangled, and the only way to resolve such a communication is to understand this phenomenon (which I have only labelled as lingap) and realize that labels and concepts are not intertwined and are completely separable.


  • What is a weblog?

     Words are not concepts. Words are the codes we choose to represent concepts. The apparent thing is that different people use different codes for same concepts, i.e. they have different words for the same concepts in different languages. But the non-apparent thing is equally usual. The word for a given concept might start being used for a different concept. Now this seems weird, because theoretically it would be pretty confusing if the concepts that our codes represent keep changing all the time; that would make communication all the difficult. But it happens all the time. Codes remain; concepts don’t.

    The problem, for now, is what to do if you have a concept in mind, for which their was a word, but that word started getting used for something else – a different concept. The reason the code got switched for another concept in the first place, was that the initial concept had gotten out of fashion and thus some other more fashionable concept had acquired its code. By fashion, I do not intend to associate any subjective preferences of mine, but rather I intend to define it as a representation of what people at a given point in time and space interact more frequently with. Thus, this initial concept which lost its code is something not a lot of people are frequently interacting with. But those who want to interact with it, I think, should choose a new code for it; otherwise the concept they want to interact with will dissolve its identity with the new concept that had acquired its code. That’s what I think.

    Luckily, the concept I have in my mind used to have a somewhat longer code which got shortened out, but the shortened out code got acquired by a more fashionable concept, so I thought that the original unshortened version of the code might be good to represent that very concept.

    That was the preface to today’s blog entry.

    Weblog is a code I have decided to use to represent the concept which is quite different from the concept which is nowadays commonly represented by the word blog. Weblog in its strict sense is a log on the web. That is a fair enough decoding of the code and representation of the concept, but here are a few other things:-

    1. Weblog is a personal project.
    2. It is mostly a log of observations, thoughts, ideas, and/or activities of the person.
    3. The weblog is primarily intended for a very limited audience (if any). The weblogs would either be about things that only they would find interesting who find that person interesting (friends and potential friends), or that they could revolve around a niche area, so that only those people interested in that niche would find the weblog somewhat interesting.

    Weblogs were in fashion in 2000s when internet and computers were something people felt excited about, and the possibilities of what new possibilities had opened before them with this new technology seemed to be endless. One of these newly opened possibilities was exchange of information (thoughts, ideas, observations, descriptions of what they were up to at the moment, etc.) with people who were close in interestedness space of things but were further away in physical space. The excitement totally made sense.

    The code weblog got shortened to blog, but it kept representing the same idea. But what this code represented started changing in 2010s and by late 2010s, the concept it represented was totally different.

    As of now, the word blog usually represents a portal for corporatish announcements, regurgitated self-help-ish bait for people unable to find a way out of their problems, and most-of-all, a specifically formatted SEO-driven gibberish religiously written in an attempt to please the almighty Algo-lords.

    To dissociate the former concept from the latter, I have resorted to using the code weblog for the former.

    Weblogs are not out of existence, they are just out of fashion among the people who populate the cyberspace. The evolution of sociological nature of the cyberspace is an interesting phenomenon but maybe that’s a topic for another time.

    Also, by writing this entry, I do not attempt to bring weblogs in fashion all by myself. That would be a very big undertaking. I merely am a stubborn fan of the concept, who is clarifying what he’s up to.

    And this is what I am up to. And I hope this weblog remains to be an okay enough representation of what I have been up to all the time since today.

    That’s all for today!


  • Not getting a new insight

     I was listening to Henrik just a while ago, when something he said popped up a whole bubble inside me head. It was like a flaw, a very serious flaw that was stopping me from doing things, was developing silently all the while I had no idea that it existed and Henrik was mentioning something like he too had this. Quoting him (and it’s just a bit – I recommend listening the whole answer):

    But I remember like a big thing I think I can see… A big thing that’s changed in my writing is that I used to be, when you’re… When you’re like a blogger, there’s a strong incentive or like a strong expectation, a norm around like that you’re going to have a big insight. And and what you see that happens a lot is that people come up with these new frameworks and this new way of thinking about it, which I don’t think is… That’s like an ego thing – you want to have like a clever new insight and and I felt I was kind of doing that too much at some point. Kind of had picked up that style of writing and and and it was hard and uncomfortable. How do you sort of push beyond trying to be insightful and like, letting go of that ego thing and like, writing being okay with just writing the obvious things or writing things that people have already said but but maybe just adding your nuance or reacting to it without having to like, be that cool guy who invented that new framework or whatever. So, so, so that was one of the things that I noticed.

    I think that’s something stopping me from producing good work, because I want to be that clever guy even though there’s lots of obvious stuff to rethink about.


  • Interesting people are kid & adult at the same time

     I was down the Aaron Swartz rabbit-hole once again, and it made me notice how some of the interesting people are essentially the same kind in both childhood and adulthood.

    In childhood, they do serious things and work with passion and think about things as if they were adults, and in adulthood too seem to be unconcerned by things most adults take too much seriously.

    In other words, although their experience and worldview changes as they grow older, but an essential attitude of theirs remain the same. There’s some lightness in children that doesn’t necessarily need to go away as you become an adult. On the other hand children too can do serious work and think about important things. It’s not an adult chore to do but a very essential human experience that doesn’t need to be delayed unnecessarily.

    This isn’t how most people think.


  • Solitude

     It turns out different people mean different things by solitude. So let me put different words for the different ideas.

    Firstly, there’s the idea where someone is both physically and virtually disconnected from other people, such as when people abandon their phones etc. and go to a far away isolated place. This sort of solitude tries to minimize social interaction with other people. We can call it external solitude.

    In contrast to this, is the idea of internal solitude. In this type, one might be living among people in a seemingly usual way. He might be living in a hostel sharing room with his roommate, taking a bus, buying grocery, etc. So apparently, he is doing social interactions, however, none of these social interactions actually take space in his mind, because his mind is mostly occupied with his own ideas.

    The more I am writing, the more it’s becoming apparent that these classification are not true classifications but rather loose attributes with blurry distinctions, but nonetheless, I’ll continue these thoughts and would later reprocess them to see if they make sense.

    Then there’s this another distinction based on the perspective of the person in solitude. One is thought solitude where one tries to block out input of information or thoughts from external sources (save one when one is reading?) so that he can spend his time thinking about and refining the thoughts or ideas already accumulated in his mind.

    While, the other form of this is poetic solitude where the person derives a poetic pleasure in spending his time in contemplation of beauty of the things around him. But it’s not just about contemplation, I think, it’s also about becoming more receptive of your surroundings, in order to see what is hidden in plain sight. And so, it does not necessarily have anything to do with writing poetry, but those who actually wrote good poetry did so by this means.

    Both these forms are interesting, but important to note these are two distinct things. There’s also another offshoot which is loosely attached to this classification which is the meditative form of solitude. I don’t know much about it, but from what I have heard or read, it involves removing each and everything from your mind, whether ideas, thoughts or whatever. So, you are neither contemplating your surroundings, nor your own thoughts, nor others’ thoughts, but just trying to empty your mind, sort of like an F5 for your brain.

    I guess these are all forms that come to my mind now. These forms although distinct are not permanent. There was a time when I used to spend some part of my day in poetic solitude and some part of my day in thought solitude, specially in my early university days, since I had come to a somewhat new social environment, I was being more perceptive of the new surrounding (and I also was being some sort of poetic about it), but after 2 years, the poetic part almost faded away, which sometimes feel strange to me because it was nice too, but these are the things for which there’s no point forcing yourself into.

    Similarly, some people might spend most time in internal solitude, but sometimes they need external solitude as well to amplify the effect of internal solitude.