By Dawid Wojda, Senior Frontend Engineer
AI has taken over the world - not literally (yet 😅) - but thanks to easy access to computational power, we are able to train accurate models faster. New AI projects are popping up like mushrooms after a September rain, and ChatGPT is now positioned as one of the most widely used tools.
LLMs (Large Language Models, which ChatGPT is an example of) are language models built on top of huge data sets of text collected mostly from the internet. It gives them the ability to “understand” human (or programming) language, which subsequently translates into a great user experience for products. Users can interact with complex systems in one of the most natural ways—simply by talking to them.
Today, when you hear AI, there's likely some form of LLM model behind it. That's why I chose to use this term initially—to avoid confusing people. However, to be honest, you can probably interpret the title of this article as "LLM Tools for Developers." 😀
The Current State of AI in Software Development
If LLMs can understand human languages, one might assume that it should be even easier with programming languages, as they are more structured and well-organized. However, currently, LLMs struggle with solving programming tasks. In the chart below, you can observe the performance of GPT models in exams from various fields such as Biology, Chemistry, Legal, and Programming. The big red arrow highlights programming tasks.
image source https://arxiv.org/abs/2303.08774
And why is that the case? Probably because programming is not merely text editing. It requires well-developed reasoning combined with knowledge of how computers work and how they interpret code. Currently, AI remains somewhat of a buzzword because it doesn't share much in common with real intelligence; today's AI is essentially statistical, offering the "most likely right answer." Consequently, when asked the same question, it can provide a completely different answer. It truly doesn't "understand" things as humans do. This is a very important thing to remember about AI before we go further.
Deep-dive into AI Tools
Copilot is a tool developed by Microsoft, trained using code available on GitHub as a dataset. It seamlessly integrates with popular IDEs like VS Code or JetBrains products. While some hail it as a spectacular game changer poised to automate programmers' jobs, I perceive Copilot more as an intelligent code completion tool. It possesses the context of your codebase, extending beyond variable names to suggest Data Transfer Objects (DTOs) for classes or unit tests. However, it does fall short when it comes to comprehending complex software problems.
Copilot often suggests code blocks containing errors, and there's a risk of succumbing to the temptation of rapid writing without thorough consideration. I've experienced this myself, leading to hours spent debugging issues I hadn't anticipated, as the solution proposed by Copilot didn't align with my preferred approach.
Copilot has the potential to foster a misguided belief that you're producing quality code when, in reality, the opposite may be true. It has no sense of something called a “code smell”, more often than not, you need to refactor the proposed code to match your standards.
Pros:
It can speed up tasks like writing DTOs, tests, or CRUD operations
Still in development, so it is evolving and getting more accurate with updates
Very simple to use
Cons:
Error-prone, can generate hard-to-find bugs
Copilot completions conflict with IDE propositions
Maybe too simple - I miss more developer-tailored experience (see Cursor)
Copilot functions more as a plugin, integrable into your IDE, with a simplicity that may border on being too basic. In contrast, Cursor presents a comprehensive IDE experience built on top of VS Code, providing an AI-first code editor encounter. Despite not being a big fan of VS Code, I found working with Cursor to be particularly enjoyable. Why?
Cursor's code suggestions seem more tailored to specific problems. It offers well-thought-out and innovative solutions that enhance the developer's experience. As seen in the example above, Cursor introduces a unique feature where you can open a special prompt within the editor while writing code to interact with AI. This eliminates the need to manipulate code with comments or switch to another window. With a simple keyboard shortcut while writing, you can ask the AI to generate code or a piece of documentation, review the response, if the answer is satisfying then accept it and apply to code, if not, drop it.
Cursor also has many other interesting features like auto-debugging, codebase answers, or built-in chat.
As a final note, it's worth mentioning that Cursor stands out in terms of data privacy. It maintains a vector database constructed from your codebase locally, leveraging a generic LLM engine like GPT but keeps your data secure on your machine. You can even use your own OpenAI token or a different LLM engine - you are not forced to use any specific solution. It is always a good thing to have a choice.
Maybe a new LLM model will be released in the meantime and you'd like to try it out?
Pros:
More accurate than Copilot
Very good UX and DX (Developer Experience)
More secure
Flexible - if you have an OpenAI token or another LLM engine, you can set it up in your IDE
Cons:
For people using other IDE, it may be painful to switch to VSC
Still in the early development phase
This is a real game-changer when it comes to working with the shell! I mean, for muggles like me, who are not Linux wizards and keep forgetting how to find the PID of processes with open port 3000. With Warp, you can just type what you want to do in human language, and Warp will translate it into a command. But even if you feel more comfortable with the console I think you can still make use of Warp. It has nice command completions (so you can write faster), pretty good UI, and nice features like command templates, collaboration, and more!
Pros:
Improves working with shell
Creators didn’t forget about non-AI features
Cons:
The accuracy of proposed commands could be better, sometimes they are obviously wrong
Bonus: Autogen
I mention it as a bonus, because Autogen is not exactly a tool in the sense that you can take it out of the box and it works, it’s more like a framework to build tools. But it is very interesting and powerful!. Autogen allows you to create agent-based LLM tools, where agents can talk to each other to find a solution to the problem with/or without the help from the user . Creating something with Autogen requires more time and knowledge but can also give very interesting results.
Summary
AI is becoming present in almost all aspects of our lives, and software development is just one of them. The AI revolution brings new possibilities along with new threats. Nevertheless, we shouldn’t be afraid of using it!
We just need to be careful and aware when working with AI-based tools, as they are not as intelligent as we would like them to be, or as their creators tell us 😉