or, Rust Bugs, Non-Compliance, and How I Learned to Love IRC. I’ve been experimenting with Rust for the past couple of weeks, often in my downtime at work. After experimenting a while and getting familiar with the language, I was desperately searching for a project. In my search, I thought of a script we use at my workplace, which for the purposes of this story I’ll refer to as . learning snap.py is a script with a very simple use: access a few web pages with tables, examine the table therein, and pull out data from columns in each line. It collects them, logs them, and then outputs to the terminal. This seems simple enough. However, it had a few problems that didn’t really sit well with me: snap.py Python No true multi-threading, due to Python’s GIL Memory hogging, due to Object creations for every row Thrashing on low memory systems Runs fairly slowly, due to it’s workload Sitting at my desk, I thought to myself that Rust may be able to fix many, if not all those issues. If it were good enough, it may even catch the eye of my seniors on development teams. And thus, with a long, uneventful shift of work ahead of me: cargo new --bin snaprs The Journey Begins I figured I’d start off by using and for downloading the data and parsing through it, respectively. My script started off rather simply: reqwest kuchiki extern crate kuchiki;extern crate reqwest; use std::io::Read;use kuchiki::parse_html;use kuchiki::traits::*;use reqwest::get; static URL: &’static str = “ "; http://internal.company.url/table1 fn main() { let mut response = get(URL).unwrap();let mut body = String::new();response.read_to_string(&mut body).unwrap(); } I thought here would be a good place to stop and compile, because if nothing else, there was no way I could mess up. this thread ‘main’ panicked at ‘called `Result::unwrap()` on an `Err` value: Http(Status)’, src/libcore/result.rs:799 Down the Rabbit Hole I commented out the lines for , removed the on , and wrote a quick little function to output the Error message I was getting. Omitting warning messages: body unwrap() response match $ cargo runCompiling snaprs v0.1.0 (file:///path/to/snaprs)Finished debug [unoptimized + debuginfo] target(s) in 2.27 secsRunning `target/debug/snaprs`Invalid Status provided According to the documentation for , and by extension , this error message is returned when the HTTP client receives a non-standard HTTP status code. Even by making a proper client using and even resorting to using , I was still getting the same error. reqwest hyper reqwest::Client::new() hyper As sanity checks, I dropped down into , imported , and tested the same URL, which received a 200, and downloaded the data without a problem. Loading the URL in my web browser with Developer Tools open, I could see that the page was downloading and rendering, again with a 200. Finally, using , I was able to confirm that the Response headers for the same page was showing I was receiving a 200. Only with Rust was I failing out. ipython requests curl Frustrated and flustered, I got on IRC and pleaded in #rust-beginners for help. There was some basic troubleshooting and explaining my situation, which I’ll spare everyone from reading. Most of the troubleshooting was done by me, through the guidance of the chat, since it accessed a web server internal to my company. Eventually, we got around to having me test using to see what was actually receiving. With a little help: strace snaprs JoshTriplett - bushidoboy: So, the response you got seems completely reasonable; it has a valid status. JoshTriplett - bushidoboy: “HTTP/1.1 200\r\n” [Truncated] JoshTriplett - bushidoboy: So, the error from hyper means that it thought it got an invalid status code. JoshTriplett - bushidoboy: That error in turn gets set from httparse. bushidoboy - Yeah, I just don’t know why it looks at a 200 and thinks it’s an error bushidoboy - Like, that’s the literal HTTP textbook definition of success At the very least, I did feel a little better knowing that I was receiving a 200. Now, all that was left was to figure out that wasn’t good enough for the client. What was going on? What was the hangup? why JoshTriplett - httparse’s parse_code just reads three literal bytes and expects something in the range 000–999. JoshTriplett - Wait, no… JoshTriplett - Oh. JoshTriplett - Hang on, I think I might have figured it out. I was at the edge of my seat. I had long been clocked out of my shift, waiting and staying to find out what the underlying issue was to all of this. We were finally here. This was it. JoshTriplett - bushidoboy: “200” is not the textbook definition of success. “200 OK” is. :) … That was it? JoshTriplett - bushidoboy: httparse chokes on the lack of a space after 200. JoshTriplett - bushidoboy: curl, on the other hand, puts up with it. JoshTriplett - bushidoboy: You’re not *required* to use the literal string “OK”, but you have to have a space and then a “reason”. JoshTriplett - bushidoboy: https://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html#sec6.1 JoshTriplett - bushidoboy: No provision for leaving out the last SP. * misdreavus *slow clap* Reading those lines, I mused that this simple failure was actually rather representative of my life up to this point. But Wait! There’s More! Though I was still quite upset, I was happy to know that it was no fault of my own that this wasn’t working. But still, it was strange to me that out of the times I’ve used , this is the first time I’ve come across a non-standard error code. No other site out on the open internet I had tested ever had this problem. reqwest Then it hit me: bushidoboy - Oh goodness. bushidoboy - You just reminded me bushidoboy - We run a custom server we forked. wyvern - yikes. Yep. Our internally developed server returns non-compliant HTTP status lines. Two bugs in a single day. One in , and one on my company’s custom webserver. Don’t I feel special. httparse The denizens of #rust-beginners were also very sympathetic to my plight. misdreavus - this just keeps getting better and better * misdreavus reaches for the popcorn JoshTriplett - httpopcorn * cmyr : refills popcorn Xion - HTTP POP /corn Conclusions Thus ends my tale of my first real bug report for Rust. I have since submitted an issue on the repo, so hopefully this can be fixed soon. httparse Sean has pulled in a to allow HTTP statuses that omit reason-phrases — huzzah! Update Dec. 4, 2016: commit I would like to extend my deepest thanks to the folks in #rust-beginners — especially JoshTriplett and misdreavus. The community has been a big help through my entire journey with Rust. As frustrating as this was, I really did learn a lot from it, and I feel happier going through this than if I hadn’t. There’s much to be done, and much more to learn. bushidoboy - Well, I hope I gave everyone a good laugh and a read. I’m gonna go cry in a corner and mourn the death of this project
