A few days ago I was investigating a bug (which caused Subversion to dump core) and I discovered this sentence in the commentary about a very similar bug: “A friend of mine told me it was normal, and it was not the duty of apr_hashfunc_default to ‘sanitize’ the parameters it receives.”
When I first saw that statement I thought “no, it takes a village to write a program”.
That commentary above is now doubly hearsay, and I have no idea if this is their official stance on the bug, but it is a troubling sentiment, in any context. Software, these days, is composed of countless layers of libraries and it is everyone’s responsibility to deal with unexpected inputs in some way. Otherwise you will end up with confusing error messages which are far removed from the actual problem (which is an increasing problem as we all know). Simply letting a segfault happen is not acceptable, if you’re going to insist on crashing the calling program, then at least issue an error message first. Do something!
I have run into many situations like this in my own code. My strategy is always to first narrow down the problem to the lowest level, write a test case and fix the bug in the lowest level library. Then move up the stack and write a test and fix the problem in the caller, and so on. To fix only the highest level program just sweeps the problem under the rug, until it is rediscovered years later in another program. And I do mean years. The statement, above, which sparked this post dates to 2007, but the code I was working with was released in 2013.
This popup just came up on my wife’s phone:
Alert!
Sorry, we ran into a problem. Please try again. If the problem persists call us at 800-xxx-xxxx. Error Code 1000.
While it is admirable when engineers make an effort to enumerate all the possible errors a program might get, but to do so without providing further on-screen explanation is just plain mean. Perhaps they should supply a secret decoder ring for those who don’t simply want to stare confusedly at the screen and retry the operation under the assumption that the unspecified “problem” will simply go away.
I’ve seen a lot of stupid password rules in my day. Some of them indicate shoddy programming, for example, sites that require you to use symbols, but then exclude many of them (the phrase “SQL injection” pops into mind). Others reflect antiquated systems, for example, requiring that the first 8 characters contain all types of characters (upper, lower, number, symbol) but allowing longer passwords, or just overly limiting password length.
But today I ran into one which simply seems arbitrary and stupid. When signing up to look up my ancestors on ellisisland.org I was faced with this: “Password Must be 10 characters and begin and end with a number.” Indeed it required that the password be exactly 10 characters and didn’t seem to concern itself with what the other 8 characters were. I wonder how many people there have the password “1234567890”.
I’m not sure whether to laugh or cry. The rest of you should read this: http://xkcd.com/936/
After attending Subversion & Git Live 2013 in Boston, I was thinking that I need to get this written down, and where better than a rarely updated and even more rarely read blog?
In my job, I work with numerous version control systems. When I was first exposed to Mercurial and Git, I realized they very nicely solved a problem which had plagued the free software community for years.
Several years ago, I wanted to contribute fixes to p42svn, which is in a Subversion repository (though what I am about to say applies to most any pre-DVCS system). I created my workspace, and then set to work on my fix. When I was done, I could not check in, since I didn’t have checkin access, and didn’t expect to be granted such access since the owner of the project had no idea who I was and if I could be trusted to checkin to the repository. So, I had to do a diff and then mail that to the owner. But, now, for my next change it got trickier. Since I only want to send the owner the changes I made since the last patch, I had to manually save a copy of what I sent last time (since I couldn’t check in). At this point I could have just created my own repository, and started checking in, which would have simplified this. Fortunately, I never needed to do that as the owner kindly gave me access to the repository.
But now I started experiencing the other side of this problem. I found patches on the forum for a number of fixes. But with each patch, I had a puzzle: what version did each one base their work on? If they didn’t say, I had to either ask, or work it out myself, which is quite tedious. Once I worked that out, I would create a branch, run patch and checkin their change. Now I have their change recorded in relation to the history of the code, and I could properly evaluate the change and merge it in. Of course, the first steps could have been entirely avoided if the change had been checked into a branch directly by the contributor. But they can’t do that since I don’t know who they are or if they can be trusted (sound familiar?).
The beauty of Mercurial and Git is that a potential contributor can make their changes, check them in and then “push” them to me. Thus, preserving the relationship of their change to the rest of the code. Of course, this is just what Github and its ilk enable. This is fantastic! All that monkeying around with diff and patch is gone! Now I can focus on the important problems.
But, here’s the rub. In the enterprise, this is not a problem we face in any way. First off, it is exceedingly rare for someone on an unrelated project to contribute a fix to my code. Even if such a thing did happen, I would know exactly who that person is. So if their ‘fix’ fouls up the code, contains malicious code, etc, they can be held accountable for that.
In my opinion, being able to solve the unknown contributor problem discussed above is truly revolutionary and is the most important reason for using Mercurial or Git. But it’s a problem we don’t have in the enterprise.
I often rant about lousy error messages, but it seems that a growing (and maybe more worrisome) phenomenon is silent failures. Here’s an example I just ran into with ClearCase (this is not a new bug, I first encountered it over 10 years ago):
$ ct desc -l vob:. | grep MITTR
SLC0314:C:\views\MITTR.vws [uuid 107553f5.340c4e00.bd2c.3f:52:42:42:31:7b]
$ ct rmview -force -uuid 107553f5.340c4e00.bd2c.3f:52:42:42:31:7b
Removed references to view "SLC0314:C:\views\MITTR.vws" from VOB "/vobs/factcheck".
$ ct desc -l vob:. | grep MITTR
SLC0314:C:\views\MITTR.vws [uuid 107553f5.340c4e00.bd2c.3f:52:42:42:31:7b]
What exactly is happening is unimportant, what you can see is that there was an entry in the “desc” output, then I tried to remove it, which the output indicates happened, but double-checking indicates nothing was done. It_ failed_, and lied about it.
My assertion that this is a defect was ignored by IBM Tech Support. I guess their definition of “defect” is different from the rest of us.
I could be wrong, but it seems obvious to me that this sort of failure is a classic case of someone not checking for errors, that is, leaving off the last part of something like this:
rmviewuuid($uuid, $vob) or warn "Error: failed to remove view $uuid from $vob\n";
While built-in exception handling (à la java) can help, I’ve seen plenty of code like this (even in Java textbooks):
try { vob.rmview(uuid) } catch { }
Back when I started programming, little bits of sloppiness like this weren’t a big deal. It was often not hard to see what was going wrong when a 1000 line program was running. But when you have millions of lines of code, often spread amongst numerous libraries, in many different interlocking programs, the problems grow exponentially. As we all experience daily.
Back in 1994 I was just a few years out of college, where I was often pushing people to use the new-fangled RCS system. On my first day at my new job I was handed a stack of ClearCase manuals and told help get it deployed. I was amazed. While the system was difficult to configure, it presented some amazing features, a few of which have yet to be equaled.
The Multi-Version File System was amazing. I could create a workspace in constant time. There is no system before or since (that I know of) which can claim this. The ability to change a date in a config spec and be instantly transported to what the code looked like at that time was fantastic. Being able to do a cd command (e.g. “cd foo.c@@”) and be transported into the version tree of that file seemed utterly magical all those years ago. The unequaled branching and merging abilities permitted heavily parallel development. A am still in awe of these things.
Here I am 18 years later looking at an email from IBM Tech Support where I am told that I “might be able to get this recognized as a defect” (emphasis mine). The problem was that I installed ClearCase and got no errors, but found that the installation was broken and unusable. A key component, indeed the central component, MVFS, did not get installed. In what universe is that not a defect?
But then I realized that this unfortunate tech support engineer and myself were arguing over a corpse. This product is dead, Jim. We’re like the two guys in Weekend at Bernie’s, propping up a corpse in an attempt to convince everyone he’s still alive.
ClearCase, from day one, was hobbled by several unfortunate architectural mistakes: a chatty protocol and a deep dependence on network filesystems and the attendant maze of authentication systems. Some people have said that the lack of database scalability was the key issue, but I think that actually could have been fixed. Even if it was, it wouldn’t change the fact that any amount of network latency would make ClearCase’s performance exponentially worse.
There are a variety of ways these fundamental problems could have been addressed given continuous development effort. But, I would argue that the core of ClearCase has remained unchanged for almost 15 years. The only changes made since then have been cosmetic changes, additional bolted-on bits (UCM) or replacing functional components with broken ones (like the new installer). This product has died of neglect.
In the last 5 years I have only brought one new team into ClearCase. Everyone else I’ve been steering to Subversion. For most teams that is all they need.
The majority of comments I’ve gotten on this blog have been from spammers. But fortunately these vile dregs of humanity have managed to amuse me a few times. Here’s a sampling of my favorites:
I love to visit veganmilitia.org every day hemorrhoids piles
I’m really not sure how to interpret that. Are the daily visits to my site helping your condition? Or causing it?
Oh my goodness! a tremendous article dude. Thanks Nonetheless I’m experiencing challenge with ur rss . Don’t know why Unable to subscribe to it. Is there anybody getting equivalent rss drawback? Anyone who is aware of kindly respond. Thnkx
It’s hard to imagine a person who would utter both the words “dude” and “goodness”, and be incapable of spelling “your”. Perhaps an octogenarian who is into surfing, smoking weed and texting? As for my RSS feed, I’m running bare vanilla WordPress, if there’s a problem with my RSS feed it’s more likely to be your problem. Furthermore, considering my blog posts about vague error messages, anybody who actually reads this blog should know better than to send me a vague, useless error report!
podagra dieta Dna moczanowa dna moczanowa dna moczanowa a ból stawów
Google translate tell me this is Polish for “Diet Gout Gout Gout Gout and arthritis pain”. So this person has a variety of maladies, including, it seems, stuttering. I can suggest a cure for the latter: keep your mouth shut.
does kefir give you diarrhea… Nothing will ever be attempted, if all possible objections must be first overcome.
Yes, kefir may cause diarrhea, and considering the cruelty involved in getting the milk, it’s what you deserve.
homecoming dresses 2012
Yay! If only I had gotten this 30 years ago. I would have been the best (cross) dressed person being run out of town.
Was that the right video? Because I’ve seen this guy perform where he produces oodles of bananas to just look at.
That is just too weird to comment on.
I believe that over the long term inflation will ne a problem but over the short term it appears that deflation is a bigger issue.
That’s what she said!
This is a fun one from svnsync:
Synchronizing repository some-replica...
Transmitting file data ......................................!
.......................svnsync: Error! while replaying commit
What’s even better is that a Google search reveals a number of people running into this exact error. Sadly, few of them got replies, and those that did get replies did not get helpful replies. Certainly no solutions.
Mashing the error message on the previous line is a nice touch as it makes it harder to spot the error message, especially when there are numerous periods (thousands, in my case).
I’m a firm believer that every program has to have the ability to enable debugging output, as that could compensate for such poor error messages. But in this case, no luck. Maybe the java stack trace isn’t so bad after all.
Is what I get for dutifully installing all updates?
Telling me that the install failed twice is nice, but does not hide the fact that I still don’t know why. I feel like I’m playing a guessing game with my 3 year old daughter: “I’m not telling!”
I know that if I go to IT they’ll want to reimage my machine. At first their predilection for reimaging bugged me as I thought more effort should go into figuring out the root cause. But over the years I’ve come to sympathize with their position. Usually after hours of probing you run into a brick wall, an OS bug or some silent failure and without source code there’s no way around it. And figuring out a root cause only helps if there’s some hope that the problem may be fixed. So why bother?
Anyway, I’ve got a Linux CD which could fix the issue once and for all.
[I just found this blog entry had been sitting in my drafts folder for over a year, better late than never, I guess]
As I was reading On PETA and Weekday Vegetarians, I was reminded of a story I heard a long time ago: A guy did a rolling stop at a stop sign, and immediately got pulled over. The police officer told him that he failed to stop, to which he answered “But I slowed down!” The officer ordered him out of the car, and began beating the man, after a few seconds of this, the officer asked “do you want me to slow down or stop?”
I guess Ingrid Newkirk would condemn the law’s “all or nothing” attitude about stop signs? But just like those who “bend the rules” at stop signs, her inability to tell the difference between stopping and slowing down is endangering far more animals. Her “all or nothing” strawman and her “screw the principle” attitude is leaving those who would try to go vegan adrift, with no clear destination; no clear reason for doing so. Such people may end up remaining vegan, or, more likely, they may end up buying grass-fed beef and cage free eggs and feeling that this somehow makes animals’ lives better.
I am certainly not denying that one must be pragmatic or that “all or nothing” is unreasonable. To combat something as deeply entrenched as our pervasive use of animals we have to take an incremental approach. But without a principle guiding us, we won’t know which increments to pursue. We won’t know where we are going. We have no hope of getting anywhere near “all”, we are more likely to wander aimlessly and end up with “nothing”.
The only way we will end animal suffering is to stop using them. That is the principle we must stick to. That isn’t to say that everyone must become 100% vegan tomorrow. If someone can be vegan on weekdays (or one day, or even one meal), that is a good thing, and a step in the right direction. But knowing the principle is what will make them take the next step.
Recently I attended the inaugural planning meeting for the Danbury Hackerspace. During the inevitable “why are you here” discussions, I stumbled through some impromptu thoughts on the subject, but I’d like to flesh out my thoughts here.
I spent the first 10 years of my life on a small farm. My father also spent his first 10 years on the same farm, but in the depths of the Great Depression. The echos of that experience reverberated throughout my own childhood. But I’m not talking about poverty or starvation, but rather about the resourcefulness and creativity of my ancestors in a situation where every commodity was precious; where, if you wanted something, you often had to make it yourself; if something broke, you had to fix it rather than throw it away and buy a new one. Many materials, which today would simply be thrown in the trash, were saved and reused in creative ways.
For example, I remember using a tool which was essentially a dowel, with a slit cut in it, and half of a large coffee can lid bolted in place and sharpened. We called it the “corn cutter” which is just what I used it for. My grandfather didn’t have the luxury of throwing away that scrap dowel and that tin can, and driving down to Home Depot to buy a machete. Another example is seen on the right: a toy top my grandfather made out of a spool of thread.
James Kunstler has said in his podcast that we are no longer a culture where people make things. On some level we, as a culture, decided it would be better to sell each other mortgage-backed securities than to actually make things. So we spent the last several generations forgetting all the skills our ancestors needed to simply live their lives. I am sure that I could come up with a long list of skills my grandparents had which I know nothing about. This becomes apparent to me as I try to maintain my own house and garden; I keep running into problems which I know that they must have solved.
Now it is true I, and others of recent generations, have a variety of skills which my ancestors utterly lacked. However, depending on how peak oil plays out, many of those skills could be irrelevant. When we lost power last year, my ability to program a computer suddenly became moot in the face of trying to ensure my family had water. So, we might all be facing the prospect of suddenly having to relearn these lost skills.
My daughter is now 3 and a half years old and, as every parent does, I have my collection of hopes and fears for her future. I want her to grow up in a world where people make things, where people help each other; not in a world where all things, even people, are disposable, and everybody just waits for someone else to fix things.
The recent hackerspace movement, and the possibility of getting one set up in the neighboring city, gives me some hope. If you’re in the vicinity, come to the next meetup; lets get this thing off the ground.
When I was a child, my parents would often tell me to repeat what they just told me, since I usually wasn’t paying attention. Now I have to do the same thing with my own daughter. Payback time, it seems.
But this blog entry isn’t about parenting, it’s about error messages.
I was just writing some code and realized that an important rule when writing error messages is to repeat back what the user said. There are many violations of this rule, the first one that comes to mind is this one from Windows:
The system cannot find the path specified.
That error may be comprehensible if you just typed a command, but as part of a script, it will be entirely useless. Obviously, the pathname needs to be displayed (of course, we still don’t know what was being done, or why).
This becomes even more important when a user specified value is modified in some way. For example I had a command line argument which could take a list. After breaking the list apart, I needed to validate the entries in the list. If I found anything invalid I could have simply given the error “invalid parameter”. Useless! Rather, I filtered out the valid values and then printed out the offending ones: “invalid parameters: a,b,c”.
Now, repeat what I just said!
The more I work with Perforce the more I dislike it. I just wasted over an hour of my life doing what should be a trivial action: adding a user.
At this point a parenthetical rant is needed: I don’t think the administrator of an SCM system should have to do such things. User management should be an IT issue, and the project owner should be in charge of who can access their repositories. The SCM administrator should just be in charge of making sure the system is set up such that that is the case.
Since this Perforce server it at its licensed user limit, I have to first delete a user to make room. That should be a trivial operation, right?
$ p4 user -f -d jdoe
User jdoe has file(s) open on 1 client(s) and can't be deleted.
Huh? I don’t care about open files! Clearly the word “force” (the -f option) is being used in some strange way. Since there isn’t a “really force this damn deletion” option, I have find the open file. First look for the users “client”:
$ p4 clients -u jdoe
$
There are none? Obviously we have “client” sharing going on (I’ll leave that for another rant). Logically, I should be able to get a list of files this person has opened, but expecting logical behavior is, apparently, unrealistic:
$ p4 opened -u jdoe
Usage: opened [ -a -c changelist# -C client -m max ] [ files... ]
Invalid option: -u.
That’s fine, I can use grep, even though it could be imprecise. For example, imagine that we had a person named James Ava, greping for “java” could yield countless false positives. Nevertheless, forging ahead:
$ p4 opened -a | grep jdoe
//depot/projects/releases/Something/3.14/src/ugh.c#5 - edit default change (xtext) by jdoe@goose
Sure enough, the client “goose” is owned by a different person who is active, so I can’t just delete it. Fortunately, I found another technote saying how to do this, so I do what it says:
$ p4 login jdoe
User jdoe logged in.
$ p4 -u jdoe -c goose -H goose.example.com revert -k //depot/projects/releases/Something/3.14/src/ugh.c
You don't have permission for this operation.
What?! I am the administrator. Super user. I have permission to do anything! So here we get to my usual pet peeve: lousy error messages. Even if we took the error message at face value, it is unhelpful since it doesn’t say what permission I need (besides “super”, that is). But the error message is undoubtedly incorrect, it is more likely that the server is refusing for some unrelated reason, but, due to poor programming, that generic error message is displayed.
Of course, even if that latter command worked it begs the question, why do I have to do all this menial work? This should all be rolled into a single command. It could be rolled into a script if the “revert” command, above, worked correctly.
I want those two hours of my life back. I could have used them more profitably working on the Perforce to SVN converter, and using it to get people off Perforce.
Another nice one from Thunderbird:
The hex number is a nice touch: provide the illusion of being specific and helpful while not actually doing so. The suggestion to contact a system administrator is a good one, as misery loves company.
The new installer for ClearCase is a mess in many ways, but this error made me cringe:
System kernel was failed to build properly and as a result MVFS was not loaded.
The bad grammar and missing punctuation are just a little extra insult to the injury of not being told HOW the kernel build failed, or, perhaps better yet, what to do about it.
This seems to keep happening to me: I get caught up in a lot of other things in life and never quite get around to my blog. Then after a year or so of inactivity, I suddenly remember it, update the software (now running trunk!) and start posting again. Let’s see how long I can keep it going this time. I’ve got a bunch of mostly written drafts in the queue so expect them soon.
Winter here has been brutal, and while it has slackened up quite a bit, I still have a thick blanket of snow (about a foot is my guess). A couple of weeks ago I started thinking about getting the garden started this year, and got several books both on indoor and outdoor plants. I also started finding gardening blogs one of the first ones I found, Your Small Kitchen Garden, was doing a seed giveaway, I signed up and forgot about it. But yesterday I got a letter from an address I didn’t recognize and opened it to find this:
So, a big thank you to Daniel for these seeds! I’m looking forward to getting these started… I’ve got two new hanging tomato planters that I got last fall (on clearance) and now I know what I’m putting in them.