My mom likes to quote her grandmother who would say things like "I hate yous all equally" when asked who her favorite grandchild was. I feel that, but with AI tools.
If you've worked with me recently, you've probably been my favorite coworker. At least temporarily. It's an easy title to achieve, but hard to hold on to.
Anyway, I just finished a little onboarding audit project (which I may elaborate on later) but for now I want to rant about AI, which didn't help me write this at all so any typo or grammatical "error" you see is an aesthetic choice.
I used 3 AI tools, 4 if you count Google's AI results in a traditional search experience, and pretty much only the chat functionality. Just like everyone has a different working style, everyone probably feels some pull toward one AI experience over another, given that they have that choice and aren't mandated to use a specific one.
Admittedly, I've had the most success with Claude, even when it hallucinated a solution earlier this year when I asked if there was a way or tool to measure time to response on a community Slack thread. When I didn't recognize the tool name I asked it to cite it's sources and Claude was like "oops, I made that up." Neat.
Anyway, Claude was the most seamless pair programming experience via Chat. When Claude suggested multiple files needed to be changed, the instructions to change them, and where, were quite clear, but at some point I said "can you just regenerate the project files with the changes", and it did exactly that.
For the most part it stuck to publicly accessible content and didn't grab context from other chats, which is exactly what I was hoping for in this experience. This allowed me to open multiple chats, as different questions, and collect all the responses. A plain ol' vanilla install if you will - so no skills, memories, artifacts, etc. I didn't even put anything in a project.
There were two situations where it didn't hallucinate, but it did need some challenges to responses it provided.
For example, Claude cited NSA secure coding "best practices" and using multiple engines for doing something like redaction. I imagine I will do a future blog on this topic, specifically around redaction with Apryse, but the guidance didn't make realistic sense for an enterprise project. It suggested doing the redaction with one platform/vendor and then verifying the redaction was complete with another platform/vendor. This is maybe reasonable if my budget is unlimited, but when prompted Claude was quick to back down and say "it's not really a citable best practice, but it's my recommendation based on security best practices."
I could see responses like this being a massive problem for people who struggle with pushing back. I was and still am a "why" kid. I'm always asking why and I think that skill is critical when working with AI. I also love demanding "cite your sources" so I can manually review whether they are credible sources or not, or in the case above, clearly see there was no specific source. Just vibes.
I was able to do everything I needed to do on a free tier using Sonnet 5 and only ran into message limits twice over two weeks. But please keep in mind I was doing a lot of context switching and using other AIs. Your mileage may vary.
Also important to note, I wasn't using Claude Code or the VS Code extension. I was just having a little chat with Claude via the desktop app to stay in control of the context and internal data sources as much as possible.
Gemini was an ok experience. Gemini has become my preferred AI for personal use (but admittedly, my personal use is very light). With my husband heavily committed to the Google ecosystem, it's easily accessible and makes sense.
This was my first time doing any sort of programming and debugging with Gemini. It was clear about what errors I should see and where, it was also great about adding more debugging statements and what the expected outcome should be.
It just wasn't as seamless as Claude. Where Claude gave me a full React project, Gemini only produced .JSX and .CSS files and expected me to drop them into a React project. So I manually went and found a sample code React project, got it up an running, then dropped in Gemini's files. If I wasn't starting a net new project and I could actually call myself a React developer, I might have felt like this experience was just fine. I was admittedly surprised it inferred I had an existing React project when it got the same exact prompt Claude did and really should have inferred I was starting from scratch.
Gemini was a free tier experience as well. I used my work browser profile instead of my personal Google account/profile so I wasn't tapping into my family plan. This all happened in a browser tab.
If you are looking for Glean on steroids, Microsoft Copilot feels like the place to go. It was so aggressive about sourcing internally that I had to toggle off all the data sources and "Work IQ".
Even then, I never really solved this issue of Microsoft Copilot grabbing or referring to context I didn't want it to have. Separate chats seemed like they were all fair game. It even inferred I was an Apryse employee by referencing the file tree and directory names of an uploaded file. I'm sure someone appreciates that, but for the purposes of this audit, I want to see what externals see. It was just too smart.