The year is 2011
Posted by Andrew
I no longer use Windows as my primary OS. It's been a few years since I made the switch and I haven't really looked back. I'm not a zealot, so I realize that other people and businesses are on Windows and are perfectly happy being on Windows. More power to them! What I like about Ruby is being able to compile things to .exe's using rubyscript2exe and package up little utilities to give to my Windows clients and friends to do scripty things without having to go through the extreme pain of setting up mingw or cygwin or virtualbox. Everything usually is very happy.
Except today it wasn't.
So I have this nice little app that will POST files to a site. It authenticates with OAuth and it's self-contained and pretty and nice. Built the little guy on OS X, made sure everything was In Its Right Place and switched over to my XP VM. Tried it out, everything seemed happy. Made my exe, tried it again, everything was as it Should Be. Now, here's the fun part. Transfered it to the target machine and the HTTP POST would just hang and eventually time out. No network traffic, no CPU, nothing.
Weird, you say!
Off to debug this one. So I remember reading something somewhere about how net/http(s) is a ghetto so I figure there's some weirdness there. I check online and lo and behold there's some posts about timeouts. I try the usual remedies but it just doesn't work the way it should. Weird. I dig through the net/http code and it looks like the thing does get down to the socket, pushes to the server and then gives up after a while. Everything on the server is fine and the most infuriating part is that on my XP VM it works. Ok, so swap out httpclient for net/http. Same thing. So it's not the client. The files are just text files so there's nothing weird there. Hmmm.
What about dropping the file size? If it's a timeout maybe it's just taking too long to upload. I drop the files down to half. Nada. Down to a few lines. Nada. Down to one line. It works.
There's something weird going on. Two lines... it breaks. At this point I am getting grumpy and I decide to try just putting the entire file contents into a POST parameter and seeing if that works. Maybe it's a firewall issue or something? I put the POST parameter in and everything is peachy. I mean it fills up the log file with 2 MB of Base64 string but at least it works. I give up for the night and go home.
As I'm leaving the office a demon from my past emerges from the cavernous vacuum of my brain. It whispers to me of hours lost debugging on Windows because of one of the most infuriating design decisions someone in Redmond made long, long ago. Figure it out yet?
The key is the data and the operating system.
I should've realized that the problem was the stupid line endings when ONE line of data worked but TWO didn't. I've been bitten by this stupidity before but it's been so long I completely forgot about it. Basically what happens is if the file gets opened in text mode the number of bytes will be wrong. That's why apache was going nuts trying to figure out if the POST had finished: the byte count differed from the amount of data sent. There were weird EOF errors in the logs but they didn't make any sense. What is most infuriating is that on my VM the same file was being opened in binary mode by default where on the other machine it was opening in text mode. If that hadn't been the case I would've focussed more on the OS differences.
The moral of this story is that in 2011 I'm being bitten by the same monstrous stupidity that I dealt with 10 years ago when I was hacking on Windows. Some things never change.