Bsrgvty

While working on a research paper with a coauthor, I found an unpublished paper in arXiv and discovered out that the technique we developed in our paper can be used to improve a result in the unpublished one. Specifically, we can design a superior algorithm to solve the same problem solved in the arXiv paper.
If we publish our results, then the author of that arXiv paper will have a hard time publishing his paper, since some journals will not consider his algorithm once a better one is available.
Of course, we cite the arXiv paper and there is no law against publishing improved results, but this may harm the author of the arXiv paper. Is there a way to publish the new result without creating enmity?

One potential option could be to combine the work and submit a joint paper. However, the papers do not "mix" very well since our paper is mainly about a different technique - the improved algorithm is only a usage example of it. Combining the papers will create a very large and inconsistent paper.

Another option is to wait until his paper is accepted for publication and only then publish our paper with the improved result. However, it seems unscientific to withhold results that we already know.

I disagree with the premise of your question. “Publishing superseding results” is basically the same as what’s known as “publishing”, since all papers build and improve on the existing literature in some way and push some older work slightly toward obsolescence or irrelevance. The extra twist in your situation that you are improving on unpublished work is of hardly any consequence and simply not worth worrying about, for the following reasons:

1. As others have said, the authors of the arxiv preprint still have precedence and will get credit for making their contribution at the time they did, before your subsequent improvement was discovered. Referees and journal editors should (and almost certainly will) take that into account, within reasonable limits.




2. In general, when you improve on earlier work you show that it is interesting and relevant enough for other people to follow up on. This is actually flattering to the authors of the earlier work, even if you imagine that it is unflattering (it’s also true that an improvement can sometimes portray earlier work in a slightly unflattering light, but my point is there would still be a separate flattering aspect that I think you are ignoring, and which in most cases will far outweigh any supposedly unflattering aspects).




3. The publication status of the earlier work is simply not your concern. It is up to individual researchers to submit their work for publication and otherwise promote it in a timely manner. If they fail to do so, they have only themselves to blame, and they should not expect others in the scientific community to delay their own follow up research, at the cost of hurting their own careers and slowing down scientific progress, out of pity or charity.

To summarize, I think the idea that you will make enemies by publishing honest work that you did in a STEM field, regardless of the specific circumstances, might make for a cute plot element in a satirical TV show or novel about academia, but is not a realistic thing to worry about in real life.

I agree with what Dan Romik writes in his answer. I write only to address a different aspect (implicitly addressed in his answer too), which are the notions, that seem to me specious, that ArXiv papers should be treated as "unpublished" and that "unpublished" materials should be treated differently from "published" materials.

An article on the ArXiv (or any other online repository) should be treated the same as an article published in a refereed journal for purposes of citation and judging priority. The word "publication" should be treated as a synonym for "dissemination", the only possible qualification being accessibility (and ArXiv papers are far more accessible than paywalled "published" articles!).

The purpose of citation is recognition of priority, acknowledgment of intellectual inspiration, and facilitation of consultation of related work by a reader. None of these objectives is much conditioned by the publication status of an accessible resource.

If a (correct) result is in a paper on the ArXiv, that establishes priority in the same way as if it had been published in a traditional journal. Likewise, a paper on the ArXiv can and should be cited if it would have been cited were it to have appeared in a published journal.

The distinction between prepublication and publication is archaic, a thing of the previous century, and the more so in mathematical areas where dissemination in traditional journals can take years.

Some authors do not cite work on the ArXiv for strange reasons - they haven't checked it, it hasn't been refereed, etc. - but this seems to me mostly just laziness or dishonesty - we should check what we use and never trust that the refereeing process eliminates errors.

As for what to do when your work improves on someone else's work - be glad you will probably have an interested reader and a competent referee for your work. Be generous in your acknowledgment of their work and they will likely react favorably. Don't worry about possible "damage" to them - journals are happy to publish work that has already been cited.

I agree with @Dan Romik's answer, but I have a few things to say about this part:

The publication status of the earlier work is simply not your concern. It is up to individual researchers to submit their work for publication and otherwise promote it in a timely manner. If they fail to do so, they have only themselves to blame, and they should not expect others in the scientific community to delay their own follow up research, at the cost of hurting their own careers and slowing down scientific progress, out of pity or charity.

Indeed the OP should not delay his own work because he built on work that others made public on the arxiv. (That it could be built upon as soon as possible is why they put it on the arxiv!) I hope though that no one will construe this to say that the authors of the original paper have some kind of moral or professional imperative to publish their results before the OP does. (In part I worry about this because of some other answers and comments on this question that seem to imply this.) That's simply not the way the publishing industry works in my experience: papers can be rejected, or can be delayed for years, due to reasons that are entirely out of the authors' control.

If you would not have known or thought to put X into your paper until you read Y's paper then you are not competing with Y, you are building on Y's work. I think the OP should write his paper to make absolutely clear that Y has priority on X and that X is in the paper because of Y's paper. This really should make it easier, not harder, for Y to publish their paper, and if it doesn't, then it is definitely not your fault.

I wouldn't sweat it, other than being gentle in the wording of the comparison.* While the other technique may have worse results, it may still be preferable in some use cases or may still be publishable as an alternate method. Even if worse at oil reserve calculation, it may work better at...sales force optimization. Or there may even be cases where your techniques don't work at all and theirs do (e.g. algorithms that need large amounts of training data or combnation of results feedback with physical insights.) And even if their method is worse for everything, it still might have some spark of an idea, by using a different approach, that helps someone else with some different approach).

But big picture their stuff will be what it will be. Given that it's on arXiv, not published, this shows something about it (not dispositive, but Bayesian, if you get me). Also, if they plan to publish, it might already be on track and may even be easier for them if they don't know about your competing results now (and have to deal with it). After all your paper hasn't run the gauntlet yet either.

*It sounds like the comparison wasn't even per se the driver of your work, so you could even just note it as an alternate method. More in the spirit of giving a citation to the other crew and giving the reader benefit of your lit search. Of course, you have to use your judgment on how important it is to emphasize the performance battle. But given you found it late, in arxive, I doubt it's critical. Plus you did ask how to avoid antagonizing. And this is the way. And much easier than trying to coordinate, warn, collaborate, etc. (which could all slow down or derail your paper).

Let's be practical:

1. Talk to the author of that paper. Tell them about your improvement, tell him you want to treat their work with the respect it deserves, and ask them what they feel would be fair/appropriate.




2. The author of the arXiv work could have a note or a brief section, either with or without your participation, mentioning how the result can be improved. He can then update his section of applications to take the improved result into consideration. That makes his paper not-superseded.




3. You can minimize your description of the improvement on his result, so that nobody can understand what you're talking about without reading his paper. You could even do that while showering some praise on his paper and nudging your readers to go read that one.

Like others suggest, however, I wouldn't delay nor refrain from publishing the improvement.