Monday 14 July 2014

Is this the end?

Ok, things have gone too far. I'm freaked out by the reaction to my story about Richard.

(Here are Part I and Part II.)

I went to so much trouble to mess around with the details to keep his identity secret, that I forgot to worry about myself.

The first responses were bad enough. But now I don't think I can handle it.

It started when my undergraduate tutorial students came to me last week to collect their exams results. That's always painful for a sensitive soul like myself. The whole point of becoming a scientist was to avoid other human beings, but it hasn't worked at all. They're everywhere. They come up to me and say things and wave their arms around and pull faces, and as far as I can tell I'm supposed to respond. That's fine: I can also say things, and I have a whole range of funny faces. But the things I say have an effect on people. Sometimes they become unhappy. Sometimes they even dislike me. And the facial expressions only make it worse.

As experiences go of direct human interaction, the return of exam results should be pleasantly ritualistic and mechanical. The student arrives, I tell them some numbers, they leave.

If only it was so simple!

There is no predicting what will happen. I brace myself to break the news of an especially poor performance, and they whoop, "Woohoo! I thought I was going to fail!" Or I set up my most jubilant face to offer congratulations, and they burst into tears.

And then this happens. My most talented and promising first-year student politely accepted her marks, and then announced, "I can't believe you told him to lie." It took me a few moments to work out what the hell she was talking about. I finally remembered my job interview advice for Richard. How could she possibly know about that? Oh yeah -- I publicly recounted it in last week's blog post.

This slow effort of memory recall was not the contrite reaction required.

"How can I respect a tutor who does something like that?" she demanded. She has since requested a change of tutor.

For someone so childishly sensitive and utterly spineless that I am regularly traumatised by confusion over tipping, this was devastating. It haunted me for days. The only way to deal with it was to become even more tedious than usual.

"But he was a brilliant scientist!" I complained to my colleagues. "He deserved that job."

"Uh huh."

"You don't know how scrupulously honest he was. Someone needed to give him some perspective."

"Uh huh."

"He was so disturbed by the experience that he was even more principled afterwards."

They were getting sick of me by now. "And what if he didn't? What if he turned into an asshole?"

I shrugged. "Then I wouldn't have told the story."

"And how do you know he's so upstanding? Maybe he fooled you, too."

Damn. I hadn't thought of that.

Now that I was on the defensive, it was their turn to become dull. All that bla bla bla about what a true and good and honourable profession science is. How privileged we are that it attracts such virtuous, conscientious students. Think of politics, they droned on. Think of all the idealistic do-gooders who should be entering public service. Instead it is so universally detested that they instead flee the country to dig wells in Africa. Is that what I want to happen to science? Do I want to drive away the good people, and wave in the swindlers and scumbags?

There was a raft of complex issues threaded through that argument, and it was difficult to address on the spot. But a well-trained intellectual and professional academic is never at a loss, and can always throw a tantrum.

I pulled a few faces and did some arm waving.

"Fine!" I snapped at them. "You want me to lie about science, to keep it honest. Fine!"

The whole experience put me in a foul mood, but then it all got much much worse.

It seems that half of the scientists I know thought the story was about them.

The emails flooded in.

"How dare you insinuate that my result was wrong! It has not been reproduced, but it's not my problem if everyone else is too incompetent to verify it." On and on, sometimes with many pages of indignant justification. You'd almost think they had something to hide.

Some carried veiled threats. "I rigorously checked and cross-checked before I published. To suggest otherwise -- to suggest that I knew it was wrong -- is tantamount to libel."

The most terrifying was a phone call from an extremely famous scientist, who I had never dared to dream would ever know my name, let alone speak to me. Now he was yelling.

"Did you think I wouldn't recognise your veiled accusations? Other people have spread these rumours before. I expect you haven't heard of them. There's a reason you haven't heard of them!"

He went on. "How would you like it if someone started spreading rumours that your papers were faked? How about if someone with lots of influence did that? And not just some nobody with a shitty blog!"

I could only jibber quietly back into the phone, while this maniac elaborated on his revenge.

"You'd probably keep your job. It would be touch and go for a while, what with the bad publicity, and the demands for your dismissal, and the embarrassment to the university. But in the end you wouldn't be worth the bother. You'd just be loaded up with all the dull administrative duties that no-one else wants, and a huge teaching load. And you could forget research! No-one would risk working with you, and only the most marginal journals would agree to publish you. You'd end up one of those losers who pretends to find teaching really stimulating. Because that's what happens when you fuck with the big shots."

That was the end of the conversation.

I informally asked people about this guy.

"He's a notorious bastard," they said.

Big surprise.

I hinted that he was angry with me. "I guess he just needed to get it off his chest. Right?"

"Good luck with that," they said, as they backed away from me. "He doesn't give up. He likes destroying people."

Now I'm freaked out. I'm not sure if writing this blog is worth the trouble.

If I'm going to get so much harassment for telling a harmless story, what happens when I get to the really juicy ones? I could pretend that they're fiction. That's what I've started telling people about Richard -- I just made him up! Really! I did! But I don't think that's enough. I can swap people into different fields, change their nationalities, give them a moustache, but still the paranoid egomaniacs will come after me.

I'm going to give this blog a rest for a few weeks, while I work out what to do.

See you soon. Hopefully.


Postscript (October 2010):
A solution presents itself.

Monday 7 July 2014

Confusing results

II.

My friend Richard had a big problem.

A prestigious East Coast university, home to the world's leading research group in his field, was about to interview him for a faculty job -- on the basis of incorrect results that he had foolishly announced in a seminar -- in the vain hope that they would later transform into correct results.

Three weeks later, they remained obstinately incorrect.

On inspection, the computer code he and his collaborator John had written was such a mess that the original results could be correct -- they just had to fix up all the bugs to find out. That was fine. All their previous projects were like that, too. Each day John hacked three new ideas into the code, and the next day hacked them back out; Richard went through it methodically, line by line, routine by routine, cross-checking and verifying and debugging; and they were both spurred on by the sight of their supervisor's look of profound disappointment.

But not this time. The impending job interview ruined all that.

John said, "You have to turn them down!"

"Would you turn them down?"

"Ask them to offer me the job instead, and we'll find out."

Yes, in science it's all "pure intellectual curiosity", "the search for truth" and "the democracy of ideas". Until someone gets offered a job.

They barely spoke after that. Richard tried to work on the code by himself, but he was too distressed to concentrate. He was sure he wouldn't be able to reproduce their results, he wouldn't get the job, and now that his great long-standing collaboration with John was in tatters, his career would be over, too.

It's a lucky thing that he called me for help. He knew he could count on me for sensible, practical advice, and moral clarity.

"Just go to the interview and lie," I said. "The preliminary results can turn out to be wrong after you get the job."

Richard could see the serious ethical complications of this approach. "But what if John tells them?"

"John would be crazy to ruin a collaboration with a professor in the top research group in the world."

"Who?"

"You!"

"Oh yeah."

"He'll get over it -- whether you get the job, or not."

Richard was still unconvinced. "I can't lie to these people."

"You certainly can't tell them the truth! And it's not lying. The results really are preliminary. You really are still checking them. You haven't published them. And what do you mean, 'these people'? If they're stupid enough to offer you a job based on preliminary results from one seminar, then they're morons. Come on! How unprofessional are they? They deserve what they get!" Before he could object, or even absorb this deep analysis, I added, "There's only one serious thing I would advise."

"What's that?"

"Negotiate immediate tenure. Because when they find out the truth, you're never going to get it any other way."

He thought that was rotten advice, but he decided to accept the part about John. The guy was still in their office working hard, and Richard had to believe that as soon as he fixed the code, he'd be so excited that their rift would be instantly healed. In the mean time Richard agonised every day over a new research talk to present as part of the interview that he was sure he was going to cancel tomorrow.

"I really was going to cancel it," he told me later. "I was sure I would. Even after I flew there. Even after I went in the building. And maybe I really would have, if I didn't meet those grad students."

Just after he entered the physics building of that illustrious East Coast university, a group of graduate students rushed up to him.

"Thank God you're here!" they said. "After your last visit, we've been trying to reproduce your results, and we just can't."

The game was up! Richard just nodded dumbly.

"There's a different cool result that shows up, but nothing like what you see."

"What different cool result?"

Richard told me, "When they explained, I really thought I was going to faint. We had spent so much time trying to get our amazing result back, that we'd missed something else." Result B paled in comparison to Result A, but if Result A didn't exist, then Result B suddenly looked like a very big deal indeed.

Richard kept his composure, and said, "That sounds very odd. I think we should compare results."

"Certainly! Definitely! That would be great!" gushed the enthusiastic gullible students.

After that Richard could go to his interview and honestly tell the panel what an exciting and vibrant field he worked in. So many results were emerging! There were so many possibilities! "This whole research area is exploding!"

They gave him the job.

Richard got the code from the local students, and compared it with the one he and John had produced. The students' code also had bugs, but the new result wasn't one of them.

"The different results will take some time to reconcile," Richard told them. "But in the end, I suggest we write a joint paper."

The students readily agreed. While the students kept trying to reproduce his spectacular result with their code, he corrected his code to verify their result. And it really was his code, now. John had quit his postdoc and accepted a five-figure-salary with a software company. He hadn't been working all that time on the code, after all: he was writing job applications.

Richard's original result was the focus of the first draft of the paper. As the weeks progressed, its prominence decreased. It was finally relegated to an appendix, but wouldn't go away completely, because the students, touchingly loyal to their new superstar professor, were determined that it must be true, and eventually they would prove it.

"I told them to get rid of it, but they thought I was just being modest."

They were so convinced that they foolishly submitted the paper, with the appendix, to the electronic preprint server, arXiv. Richard was horrified, but he let it sit there for a couple of weeks, and when the students still couldn't reproduce the big result, they agreed, "Go ahead, doctor. Take out the appendix."

The paper was a big success nonetheless. There were follow-up results. Other groups reproduced them, and then expanded on them. For a few years Richard's group -- it was his group now -- lead the field, and did some great work. And it was great work: Richard knew his stuff, and was even more methodical and rigorous than he had been in the past. He wasn't going to make the same mistake again. He relentlessly demanded care and precision from his students and postdocs and collaborators. He became famous for it.

The original, incorrect, result is still there in the first electronic preprint of the paper. It baits researchers on the margins of the field, who regularly set their students the task of reproducing it. Sometimes they even manage to, although the results never hold up. Richard cringes every time.

Whenever we talk, he always goes back to it. "I'm still ashamed. Just think how many hundreds of student hours have been wasted!"

But he only says that so I can reassure him. "It's fine," I say. "It's a good problem to play with, and they learn a lot. If they're good, it won't do them any harm. And who knows -- maybe it will turn out to be right after all?"

And maybe it will. Such is science.


Next:
Repercussions -- "Is this the end?"

Tuesday 1 July 2014

It's all wrong!

I.

Those recent posts on seminars reminded me of a bewildering seminar experience a friend of mine once had, when he gave a seminar at one of the top universities in the world. Of course, he won't let me tell you who he is, or which university. I already had to change this draft five times and argue for hours on Skype just to convince him to let me write it up completely anonymously. All I can say is that if one person so much as asks him, "Are you the guy in Mark's blog?" I'm going to be writing the next post from hospital, typing with my nose. I'm saving the ending until next time, although that probably won't be much insurance.

I think it's a good story, but he's not proud of it. When I told him that in this write-up I'd call him Richard, he said, "That sounds right. So long as you abbreviate it to Dick."

It was a university on the US East Coast. The first time he ever went there was one winter, as a postdoc, to give this seminar. It was a lucky break. He was invited several months earlier, but just a week before traveling he and his collaborator made a huge breakthrough in their computer simulations. So now he could unveil a major result to the top research group in his field.

It wasn't going to be easy, though. He had a horrible journey. His flight from Europe was delayed because of weather, and after it finally landed the roads from the airport were such a mess with snow and accidents and detours, that it took the cab three hours to reach the university. He was supposed to have arrived the day before, but in the end he arrived only 45 minutes before the seminar -- exhausted, jet-lagged, and still without the latest plots from John, his fellow postdoc and collaborator.

[Relax, man, no-one can identify you from this. A prominent East Coast university? Which is three hours from a major airport in poor weather conditions? That's probably all of them. And I made up that you came from Europe, just to throw them off.]

When he arrived the first thing he did was check his email.

There was bad news.

"There's a bug in our code," wrote John. "All of the results are wrong."

Richard closed his laptop after the first sentence.

What should he do?

John could be exaggerating. He was brilliant, he was enthusiastic, he was indefatigable, but it always took him ten attempts to get to the right answer, and if Richard wasn't there to set him straight, it took even more. Richard was the calm, thorough one. But how could he be calm fifteen minutes before the most important seminar of his life?

He tried to reassure himself.

This happened every time they got a result. Half of the time it was some temporary panic that resolved itself a day later. Unfortunately, the other times the problem remained, and it was their results that evaporated. Which was it this time? There was no way to find out.

"I was panicking," he told me later. "It wasn't like I could extend my visit for two weeks and give the seminar later -- although I seriously considered trying!"

No-one would blame him if later there turned out to be some problems that he could plausibly claim he didn't hear about until after the talk. After all, he was out of email contact until only minutes before he spoke. He had a few minutes to check his messages. He was only able to glance through a few of his many emails. Maybe he didn't even notice the one from John? Surely only spam was titled, "*** IT'S ALL WRONG ***".

So he gave the seminar.

He tried hard to be cautious about the results. "They're only preliminary," he warned. "There are still a few things we need to check."

He told me later, "I thought they were going to see right through it, and rip me apart."

But instead a professor in the audience, who was probably the world's leading expert in the topic, waved aside his caveats. "No no no! This has to be right. It makes complete sense. It's an incredible discovery."

He tried not to look so surprised, and to remain cautious. They asked for more details, and he stayed cagey, and that just made them more excited.

When the time was up, a small crowd gathered round him.

He pleaded serious exhaustion. Was it Ok if he checked in at his hotel and got some rest? He would join them for dinner later.

At his hotel he read John's full email. It was no exaggeration. The bug in the code was clear and unequivocal. It was obvious which physical effect had been neglected, leading to the astounding new results. With the code fixed, the result went away. End of story.

Richard was mortified. Everyone had been extremely excited.

"How could I go out for dinner with them now and tell them that it was all wrong?"

He begged out of dinner, claiming illness. The next morning he got an early taxi to the airport. The weather had cleared, and he escaped with his secrets intact.

As soon as he got home he met up with John and they tried to salvage the results.

The code was a mess. Richard calmed down, and methodically re-organised and checked every routine. In the mean time, John had more crazy ideas, and furiously wrote more messy code to test them. Some days their miraculous result returned, and others it sunk beneath the surface of bugs, analysis errors, unchecked assumptions and crude approximations.

It's not as bad as it looks. This was how they always worked. In the end their supervisor always set them straight. Up until now they met him once a week. Here was the routine: for fifteen minutes they tried to explain what they had been doing, before his eyes welled up with tears, and he quietly asked, "Is there a paper yet?" Then he spent 45 minutes dabbing his eyes and pointing out what they needed to do to get a clean result. The rule for the meetings was: either bring a paper, or a box of tissues. In the past year they made multiple bulk Amazon orders for kleenex, but they also wrote four excellent papers.

When he heard about the seminar he choked up immediately. As Richard put it, "He waved me straight out of his office, and the last thing I saw was his head in his hands." For the next month his sole contribution was to sit silently in his office, staring miserably out of the window. "It was just what we needed."

John loyally insisted that Richard had done the right thing. His understanding and support seemed to be endless. He worked tirelessly to make sense of their results. He worked late nights and weekends. He cancelled a long-planned concert trip with his girlfriend. His girlfriend dumped him, but now he had even more time to work on the code. He was the perfect friend and collaborator.

But only up to a point.

That point came three weeks later, when Richard got a call from that illustrious East Coast university.

"We would like to interview you for a faculty job."


Next:
Part 2 -- "Confusing Results"
Repercussions -- "Is this the end?"