Large Language Models

This note does not have a description yet.

  • AI Is the next Great Interop Layer

    I had previously observed that humans are the great interop layer—we are the glue that fits together disparate processes and tools into usable systems. After using large language models, I’m becoming convinced that they can offload a large amount of the interop cost that currently falls to us. In a nut shell, AI can ‘do what I mean not what I say’ pretty darn well.

  • Context Is Needed for Practical LLM Applications

    One of the limitations of large language models is that it is often missing context when responding. LLMs like ChatGPT (GPT 3.5 and GPT 4 at time of writing) don’t allow fine tuning and can only take in ~4,000 tokens (roughly 3,000 words) as part of the prompt. That’s not nearly enough to provide context to be specific to your application. For example, you might want an AI coding tool to help you add a feature to your codebase, but the codebase is likely much larger than the prompt would allow.

  • Zapier Natural Language Actions API

    The Zapier NLA API solves a major problem for large language models—the ability to interact with real systems. Rather than a developer integrate with every possible service, they can integrate once with Zapier and run every “Action” the user has authorized using natural language instructions.

  • Advantages of Open Source AI

    It’s almost inevitable that, after an initial research phase, progress of AI models and tools will come from open source communities rather than a corporation. Individuals can utilize fair-use to do things businesses can not do (e.g. using leaked LLaMa weights and fine tuning it). There are more people to work on fringe usecases that do not have to be commercialized. Finally, open source increases access (running 13B LLMs on a laptop, on a Raspberry Pi) allowing more people to try it and provide more feedback.

  • Prompt Injection Attacks Are Unavoidable

    While large language models are already useful for certain text based tasks, connecting them to other systems that can interact with the outside world poses new kinds of security challenges. Because it’s all based on natural language, any text can effectively become untrusted code.

  • Intent-Based Outcome Specification

    A new paradigm for user interfaces is starting to take shape with the rise of AI powered tools. Rather than a loop of sending a command, receiving the output, and continuing (like graphical user interfaces), an intent-based outcome specification is telling the computer what the outcome should be—“open the door” instead of “check auth, unlock, open latch, extend door”.

  • AI Models at the Edge

    Today, most large language models are run by making requests over the network to a provider like OpenAI which has several disadvantages. You have to trust the entire chain of custody (e.g. network stack, the provider, their subprocessors etc.). It can be slow or flakey and therefore impractical for certain operations (e.g. voice inference, large volumes of text). It can also be expensive—providers are charging per API call and experiments can result in a surprising bill (my useless fine-tuned OpenAI model cost $36).

  • AI for Notes

    Now that my Zettelkasten has over a thousand notes, I’d like to try to quite literally create the experience of a conversation with my second brain. The interface should be conversational rather than search queries. It should draw from the knowledge in my notes and respond in natural language. Finally, it should be useful in helping me make connections between ideas I hadn’t thought of before.

  • Natural Language User Interface

    One of the super powers of large language models is that it can “do what I mean” instead of “do what I say”. This ability to interpret prompts can drastically lower the barriers to accessing and interoperating between systems. For example, writing “Send a slack message to the Ops channel with a list of customers from HubSpot that signed up in the last week” would generate actions that query the HubSpot Contacts API, parse and extract the results, and make another API request to Slack to post to the #ops channel.

  • The Pareto Principle and Chatbots

    Support cases at many businesses follow the Pareto principle. For example, DoorDash and Uber 87% of support requests relate to 16 issues. Deploying chatbots to solve high concentration issues makes it economical to build and maintain conversation trees by hand. What about the remaining 20% and what about businesses that have a wider distribution of issue types?

  • LLM-First Programming Language

    There are many barriers to adoption for a new programming language looking to go mainstream. You have to attract a small group of enthusiasts, build an ecosystem of high quality libraries, help new people learn, and eventually grow a talent marketplace.

  • Consciousness Is Categories

    Consciousness is an emergent property of categories. As a sufficient number of categories can be represented in a system, selfhood arises and, with it, consciousness.

  • AI Multiplies the Value of Expertise

    AI reduces the cost of certain tasks to effectively zero. In doing so, it lowers the barriers to domains that would previously take years to build skills such as writing code, data analysis, and more. This is precisely why AI also increases the value of expertise and experience.

  • Chatbots Lack Affordances

    When interacting with a chatbot, there are not indications of what to say or how to say it. Without affordances, it’s difficult to know what to do at first.

  • Legal Services Has the Highest AI Occupational Exposure

    A recent paper looking into the economic impact of large language models found that the legal industry has the most potential occupational exposure from AI including augmentation or substitution.

  • Org-Ai Is Chat for Notes

    I started building AI for notes to help me chat with my library of notes. The result of that exploration is org-ai—my one of one software that helps me remember what I’ve previously written, summarize information. Under the hood it uses vector-based similarity search and LLMs to extract useful information from my zettelkasten in a chat-based interface.

  • How to Decide If AI Tools Can Be Used at Work

    Advancements in AI powered tools can greatly improve productivity but many companies have taken steps to limit or outright ban the use of OpenAI’s ChatGPT, GitHub Copilot, and others. What are they concerned about and how should you decide if it can be used by your company?

  • How Langchain Works

    As it turns out, combining large language models together can create powerful agents that can respond to, and take action on, complicated prompts. This is achieved by composing models and tools with an overall language model to mediate the interaction.

  • Zapier NLA Is Bad at Generating API Parameters

    I tried out Zapier Natural Language Actions API and found that it’s not particularly good for the one thing I needed it to be good at glueing together other APIs with natural language. API endpoints that are simple and straightforward are easy for large language models to generate the right inputs but more complicated (and poorly designed) like HubSpot are unusable.

  • Why Vector Databases Are Important to AI

    The recent success of large language models like ChatGPT have led to a new stage of applied AI and with it, new challenges. One of those challenges is building context with a limited amount of space to get good results.

  • LLM Web Browsing

    By combining headless browser automation tools with LLMs, you can create an agent that can navigate to websites. This opens up all sorts of new capabilities like scraping and summarizing web content.