A developer friend recently asked me, “Which AI coding tools are you using?”
“None”, I answered. “What about you?”
He proceeded to list off several tools, culminating with this astonishing admission: “I almost never write code myself anymore. Everything is done by AI. My job is just checking the answers.”
My friend is a very smart, very talented developer… and he’s not even writing code anymore! I’ve heard the stories about how AI is “revolutionizing” coding, and everything else, but it didn’t really hit home for me until I heard it from somebody close to me.
My own opinion of AI is generally negative, bordering on hostile. So, naturally, I’m probably the last person remaining in this world who hasn’t tried AI coding — until now.
***
I was working on a project with one very specific requirement: I needed to be able to generate 128-bit MurmurHash3 hashes in PHP, in order to validate identical hashes coming from an outside system.
PHP added built-in support for MurmurHash3 in version 8.1, however my project was running on PHP 7.4, and changing that was not an option at this time. I therefore went looking for a polyfill. I found this package, but strangely the hashes it produced did not match the ones from my other system. The other system, by the way, was using a JavaScript library to generate the hashes, and I had that code handy, but translating it to PHP looked like it was going to be complicated.
I was tired of spending time on this and desperately wanted to move on to other things, but this was holding me up. So I decided this was a perfect test case for AI.
***
I started with Claude. My prompt: “Hi Claude, can you help me port some JavaScript code to an equivalent in PHP?”. I then supplied the JS code. Claude responded with a rough PHP equivalent, but warned me “this isn’t a perfect 1-to-1 port” because “PHP doesn’t have a built-in MurmurHash3 implementation.” When I say it provided a “rough” equivalent, I mean it didn’t work at all… but it was a start.
I informed Claude that “Actually PHP does have a built-in MurmurHash3 implementation since PHP version 8.1, see https://php.watch/versions/8.1/MurmurHash3”. Maybe once Claude realized this, it would be easier for it to create a backport or polyfill.
“You’re right, let me check that!” … “Good catch! Here’s the corrected, clean PHP port”, Claude said. It supplied code which did work using PHP 8.1’s murmur3f algorithm. So now I asked, “can you give me a backport of murmur3f which would work in PHP 7.4?” It replied with code. I tested it, hitting a PHP fatal error. I told Claude the error; it responded with revised code. It still had the same error; I informed it. This went round and round a few times, with Claude revising the code each time.
Then, suddenly, in the middle of Claude’s thinking, I hit a “5-hour limit” screen.
I didn’t have working code from Claude yet, so I decided to pop over to Google Gemini while I waited. Instead of asking it to port JavaScript code, I went straight to the point. My prompt: “I need an implementation of murmur3f that works in PHP 7.4”.
Gemini responded with 2 options: the first was recommending the same package I had found earlier which I already knew produced hashes that didn’t match; the second was “Pure PHP 7.4 Implementation Code” which Gemini had produced itself.
Gemini’s code looked good at first glance. It relied on the GMP extension for PHP, but that was alright. However I was amused to discover Gemini had hallucinated 2 GMP functions that do not exist: gmp_shift_left and gmp_shift_right. I told it “I have tried your pure PHP 7.4 implementation code. There is an error: ‘PHP Fatal error: Uncaught Error: Call to undefined function gmp_shift_left()'”.
“My apologies for that oversight. PHP’s GMP extension does not have native functions named gmp_shift_left or gmp_shift_right“, Gemini responds. It then provided me with corrected code. The corrected code worked, except the hashes did not match what I expected. Oddly they did match with the other package I had found earlier, but they did not match with those coming from the outside JS-based system.
I pointed out the discrepancy between what I was getting and what was expected. Gemini then noticed something immediately which I had failed to see up to this point: Gemini’s (and the other package’s) hashes were the same as the outside JS system’s hashes, except in reversed byte order. Gemini’s original code was deliberately reversing the byte order. It provided me updated code which didn’t. Now everything was working correctly, producing hashes that matched not only the outside JS system, but also PHP 8.1’s built-in implementation.
At this point I had a working solution, but I was still curious to see what Claude would eventually come up with. After the 5-hour window finally passed, I went back to Claude and asked it to continue where it left off, which it did. After MUCH thinking, it came back with a updated code, which did work as expected.
***
Final Analysis
Claude’s final working code was 4,970 bytes in size; Gemini’s was only 4,497. Gemini’s code is slightly more clear and easy-to-understand to my poor human eye.
Gemini “thought” noticeably faster than Claude did. Although Gemini tried to get away with inventing two non-existent PHP functions, it did ultimately get me to a working answer quicker. (Granted, this is not a 100% fair test since I did use different prompts between Gemini and Claude, but it’s still interesting to note.)
Claude flat out told me that PHP did not have a Murmurhash3 implementation, until I pointed out that it did from version 8.1+. After that “good catch” it did start going down the right path.
Claude was much chattier about it’s own bug-fixing process. It talked all kinds of barely-decipherable nonsense, and promised at least 5 times “now let me write the complete corrected implementation” before actually doing so. I really have no idea how it got there, but it did get to a working solution in the end.
In the end, was this AI experiment worth it? Yes. Without either of these AI helpers, I’m sure I would have spent far more time finding the problem. I had found a working package on my own, so I was actually pretty close, except I just wasn’t seeing the hashes were reversed. Gemini spotted that right away, and, to it’s credit, told me about it. Claude also fixed the reversed hash, but never said a word about it.
Will I actually use the AI-generated code in production? No, not this time. Since I now know the sole problem with the human-made package is the hashes being reversed, it’s easy enough for me to make that one change and carry on using the human code. That’s what I did in this case.
However if I were ever to use AI-generated code in production in the future, I think it should be clearly marked as such. For example I might encapsulate it with /* begin AI-generated code */ and /* end AI-generated code */ comments. It doesn’t feel right pushing code that didn’t come from my own brain and that I, therefore, can’t 100% understand the intention behind. But that’s just me.