itshalfempty

When it comes to generative AI, you could say I’m not the most enthusiastic person out there. I have seen a lot of hype around AI tools, but I haven’t seen quite as much around tangible results. I’ve personally played around with LLMs a little here and there over the last couple of years, and I’ve just struggled to find a use case that’s any different from what amounts to “a better Google plus a decent autocorrect.”

I paid for a ChatGPT subscription for a couple of months a while back. I tried to shoehorn it into every use case I could think of. Writing e-mails, summarizing content, planning vacations, coming up with weekly meal plans, and so on. If there was a task in front of me at some point, I probably tried to get ChatGPT to give me an answer.

Sometimes I got a surprising result that I could use, but generally I found that it gave me a somewhat reasonable sounding answer that didn’t really pass scruitiny. I ended up using ChatGPT largely to get a “first draft” of something that I ultimately replaced or threw away as my thoughts solidified. And sometimes, I used it to take some kind of word vomit and collect it into something that sounded halfway professional.

I ended up canceling my ChatGPT subscription after a few months. I had also been growing more privacy conscious, and concerned about how chat histories may be used by a third party either to train LLMs, potentially serve ads, or possibly end up being used in more nefarious ways. Since I already own an RTX 3090, which has a huge amount of VRAM, I decided I may as well run models locally on my own machine!

For the past year and a half+, I’ve played around with Ollama and local LLMs. While I still couldn’t really find a good use for LLMs in my day-to-day life, I did end up using my locally-run models to help write random scripts. None of these were particularly complicated scripts, but some of the services or tools I wanted to use were things I had never used before. Verifying the APIs are used correctly is way easier than writing the script from scratch.

Well - in the past week, I feel like I’ve really jumped into the deep end. I’ve begun playing around with Agentic AIs both at work and at home. Claude at work, and OpenCode at home. While I’ve known that many people find these tools to be incredible valuable, I’ve been a pretty big skeptic. After being hands-on for a whole two days…well, I’m still a skeptic, but these tools are really cool.

I was kind of surprised at how well Claude Code has worked “out of the box.” With nearly no configuration, I was able to direct Claude to write an implementation for a couple of tasks and get a largely correct result. That’s hands-off, no extra context provided, and just return to my IDE in 15 minutes to review what the model produced. It’s a bit hard to quantify how much (if any) time using Claude Code saved me on those tickets, but I can definitely see a lot of potential.

Playing around with OpenCode is no different. It seems more or less just as capable, at least the way that I’ve been using both tools. The main advantage running locally of course being that I’m in full control of my own data and machine and not handing any of it or its control to someone else. Running locally is also nice in the sense that I don’t need to worry quite as much about keeping token use low. In a way I want to optimize for token use either way (at the end of the day, I only have one GPU, and I’d like tasks to complete in a reasonable time), but I don’t have to worry about a surprise $100 bill or running out of token quota.

I realize I’m just starting to scratch the surface here. There’s quite a lot to learn around managing context efficiently, and that’s even before you end up talking about really complex workflows and multi-model solvers that, if you listen to the folks hyping up the technology, are capable of replacing entire dev teams.

I’m ultimately still a pretty big skeptic, so it’s quite possible I get through a month or two of exploration and still end up closer to “meh” than “omg”. But even with just a few days under my belt, it’s kind of hard to imagine why any dev would want to work without these kinds of tools.