Having spent several months with JetBrains AI Assistant and GitHub Copilot I have several observations on the current state of things. Following Noam Chomsky mood I wouldn’t call it AI because it has not reached that level of technology yet but still if we have this term used a lot in media to describe generative systems like ChatGPT or Bard why not to stick to the term in anticipation of something bigger to come. I share Chomsky’s argument that AI systems today can not generate ideas, they can not be compared to the human genius, and the only thing they can is to find intrinsic patterns in big datasets generated by millions of human beings. However, taking this into account, I wouldn’t share the Chomsky pessimism on the usefulness of such systems even today. To say the truth, we as humanity do mostly exactly that – getting knowledge from content generated by others in the form of textbooks, lectures, photographs and trying to use this to “predict” answers to our professional problems. Take an average programmer. First he studies using textbooks, coursers and manuals, then he looks for solutions to his everyday problems on sites like StackOverflow or just googles them. Sure, that’s not the only thing he does despite memes but sure he spends quite a lot of time on that.Apart from those interesting problems he googles, his daily routine mainly consists of writing a lot of repetitive boilerplate code, simple queries, quite boring technical documentation, obvious unit tests and swagger blocks mainly just repeating what’s in the code and fine tuning server configurations. Is it a proper work for the brilliant mind described by Chomsky? I think the AI could do most of it even now. Sure, it needs supervision, it can’t generate the whole solution for you (yet) exactly because the technology does not allow it (yet) but it can help with quite a lot of boilerplate boring content. As a programmer, did you dream about your own secretary who would write down your thoughts in code and who would write all boring parts for you? So, now you have it.
I’m using PHPStorm in my work, so I tried both native AI Assistant plugin (which uses several models including ChatGPT under the hood) and a GitHub Copilot. The first one is fully integrated while the second can be used for code autocompletion so far. I like Copilot more so far, its prediction are in most cases more to the point but I am still playing with both. It seems that both products already implement some prompt optimizations under the hood. For example, it’s a common knowledge that to make ChatGPT more helpful for math tasks you need to prompt it like “think thoroughly, step by step” or something similar. However, it doesn’t seem to be needed with mentioned tools.
Where AI Rocks
- Explaining whatever configurations and fixing them. The AI systems are really good in understanding fully determined content that is hard to understand by humans due its abstractness and verbosity. Take nginx, docker or framework configurations, regular expressions or SQL queries – AI rocks there, it can explain meanings of parameter groups, how they work together, and usually it’s really good at finding obvious mistakes in configuration files. Of course, there are false claims as well, but still it can save much time on debugging and googling.
- Writing boilerplate like initialization in constructors, dealing with database connections, trivial methods naming etc
- Rollback migrations and other rollback scenarios. AI is really good at it, and that can save you plenty of time
- Naming methods, tests and variables/properties that can be obviously named out of their content. It works in other direction as well. If you name a method findFactorial, it can autogenerate the method content for you.
- Repetitive code, parts of queries. You just write a first example and a todo comment with the description of what exactly is needed to be done, something like “// todo write the rest of addSql strings to add all other country codes with titles keeping the same structure as the row described on a previous statement”
Where AI Sucks
- AI can’t design your application for you. It can’t think of the right architecture for your business requirements. Although, it can give you general advice on that.
- So far it is not very good at mathematical logic. Logical tasks are quite hard for modern AI, but it’s improving.
- AI can’t generate several files/classes/queries dependent on each other. It can’t generate a lot of content which would be considered as a complete solution. So far it just can’t. Maybe, there are limitations for that, and Chomsky is right. The predicting power stops when it comes to more complex stuff.
- While it’s good in general “StackOverflow searched” solutions to particular problems, it can’t optimize your code to run faster or to be more stable in general. However, you can use it iteratively in small steps and get a good advice on optimizations now and then. Not all of them will make sense though.
- It can’t help you much with complex debugging where you just can’t figure out the source of some problem. Mainly it will give you very general advice but won’t help you understand whether it’s a bug in some part of business logic or in server configuration or validation rules or something else.
- It can “understand” business requirements but the correctness of understanding is very limited. It just can’t be trusted. For example, it can generate a DML or UML diagram for your business requirements written in plain “manager” language, but you can’t trust that this diagram will display everything you need correctly and with needed level of abstractness.
Thoughts About the Future
I think AI can be used instead of a junior programmer to help more experienced fellows right now. Call it a digital programmer assistant with quite decent coding knowledge who knows a lot of references and StackOverflow answers by heart, it will definitely cost you much less than hiring a guy with the same abilities.
We will see what will happen in the nearest future. If Chomsky is right, modern AI will reach the limit of its abilities quite soon leaving us with a very helpful tool to make our profession funnier and more productive. If not, we will see AI capable to generate the whole solution (in any programming language as well as directly in binary machine codes) from scratch out of very vague business requirements written in plain language. Only the future will demonstrate.
Disclaimer
No part of this article was written with a help of any generative algorithms apart from the screenshot (the code in grey was generated by GitHub Copilot).