[Looky1173]

Currently, the Open Rails code is hosted and contributed to on GitHub and releases are also created on GitHub, but bug reports and feature requests are still tracked on Launchpad. I think it would be beneficial for the development team, in the long run, to migrate from Launchpad bugs and blueprints to GitHub issues and projects, respectively. This would allow directly linking code and issues, and eliminate having to separately manage two fundamentally interlinked environments.

This was brought up by perpetualKid in 2019, but got no response:

perpetualKid, on 12 March 2019 - 02:00 AM, said:

I haven't seen a discussion, but now that the code is moved to GitHub successfully, what are thoughts moving Bugs and Blueprints to GH as well? Bugs would become Issues, and Blueprints could be tracked as Projects.

This would allow direct integration and linking between code and tracking, and safe some external dependencies on 3rd party.

thoughts?

If there is an interest in moving forwards with this, I could write a script to migrate Launchpad bugs to GitHub issues to preserve history.

[cjakeman]

Looky1173, on 13 November 2022 - 02:50 AM, said:

If there is an interest in moving forwards with this, I could write a script to migrate Launchpad bugs to GitHub issues to preserve history.

Thanks for your offer.

It will be interesting to get James' view on this.

[Looky1173]

I've written a JS script to migrate Launchpad bugs (including comments) and blueprints to GitHub issues. I've successfully used my script to migrate a subset of the Open Rails bugs and blueprints to GitHub issues. Here are the results: Issues · Looky1173/launchpad-bugs-migration (github.com). Please let me know whether the format of the migrated bugs and blueprints would be suitable for a full migration. If the Open Rails development team decides that they do want to migrate, then I will publish my script and I'm also ready to volunteer to run it (I estimate that a full migrate would take about 10-20 hours). The only problem is that the script would understandably hit GitHub's secondary API rate limits, but we could try to contact GitHub to see if they could temporarily relax their limits for Open Rails during the migration, like they've already done for the open-source project Mixxx when they were carrying out a similar migration (Mixxx - Migrated to GitHub Issues; Mixxx on Twitter: "@github hello, the Mixxx project is migrating its entire database of bugs to Github Issues -- we only need to do this once but we are hitting understandable rate limits. Is there someone we can talk to about getting the limits relaxed while we do the migration?" / Twitter).

cjakeman, on 13 November 2022 - 09:05 AM, said:

It will be interesting to get James' view on this.

Indeed, could you please let him know about this proposal?

[cjakeman]

Looky1173, on 20 November 2022 - 12:44 AM, said:

I've successfully used my script to migrate a subset of the Open Rails bugs and blueprints to GitHub issues. Here are the results: Issues · Looky1173/launchpad-bugs-migration (github.com). Please let me know whether the format of the migrated bugs and blueprints would be suitable for a full migration.

I had a discussion with James today about moving from Launchpad. As you posted, this was suggested some time back but, now that GitHub.com is more familiar to everyone, this suggestion is timely.

We would like to move the open bugs in Launchpad into Github Issues. It's probably not worth moving the closed bugs.

We aren't ready to move Launchpad Blueprints. They might become GitHub Projects rather than Issues, but we also have suggestions in our Trello Roadmap and on the website, so these 3 collections need to be considered together.

The Launchpad bugs will remain accessible for a while yet, since the code in Stable Version 1.5 offers to take the user to Launchpad whenever the fatal error dialog box pops up. This link is hard-coded and can be changed for the v1.6 release.

There is one other point which I would like you to check. When the Bugs are moved, we want to leave a comment behind in Launchpad to notify visitors where the bug has moved to. Would you mind adding a test bug into Launchpad and try moving that to your test repo and leaving a suitable comment?

Thanks for doing all this prototyping. As you say, it will be easier to have everything in one place and well-integrated.

[Looky1173]

cjakeman, on 26 November 2022 - 12:10 PM, said:

There is one other point which I would like you to check. When the Bugs are moved, we want to leave a comment behind in Launchpad to notify visitors where the bug has moved to. Would you mind adding a test bug into Launchpad and try moving that to your test repo and leaving a suitable comment?

I modified my script to leave an explanatory message on migrated Launchpad bugs. Here is the requested example. I have also reached out to the GitHub support team to ask if it would be possible to relax the API rate limits for Open Rails during the migration. I will let you know here when I receive a response from them.

[cjakeman]

Looky1173, on 27 November 2022 - 05:59 AM, said:

I modified my script to leave an explanatory message on migrated Launchpad bugs. Here is the requested example. I have also reached out to the GitHub support team to ask if it would be possible to relax the API rate limits for Open Rails during the migration. I will let you know here when I receive a response from them.

Hi Looky1173,



That looks just right. Thanks for sorting that so quickly.

[Looky1173]

The GitHub Support Team has just got back to me, and they've said that they cannot unfortunately increase the rate limits for Open Rails. However, they pointed me towards the officially undocumented GitHub Issue Import API. This obscure API allows an unlimited number of issues to be migrated without worrying about the rate limits, and even supports preserving the original creation dates of Launchpad bugs and comments. http://www.elvastower.com/forums/public/style_emoticons/default/biggrin.gif I will rewrite my script to use this API during the weekend! Finally, I think that this API would also allow us to migrate closed Launchpad bugs too since we wouldn't run into issues with rate limits.

[cjakeman]

Looky1173, on 28 November 2022 - 11:26 AM, said:

However, they pointed me towards the officially undocumented GitHub Issue Import API.

I guess this is what they use themselves. Good of them to share it.

Thanks for getting a step closer.

[Looky1173]

I have finally gotten around to rewriting the script to use the GitHub Issue Import API, and it works like a charm! I've tested it in this repository, and ended up with 2598 issues out of 2600 on Launchpad. One bug could not be migrated because it was too large, and 3 issue imports were not processed for unknown reasons, although it looks like two of them were nevertheless migrated. The complete migration took about 3 hours, excluding the posting of explanatory messages on Launchpad which I did not run for this test.

I noticed that the Open Rails development team has created a new test repository for the issue migrations. I have thus attached my script so that the developers can test it. It is written in Node.js (my system has v17.8.0 installed), therefore, after extracting the zip, one must run `yarn install` or `npm install`, then `yarn start` or `npm start`. The script excepts to find a `.env` file in the same directory as itself, containing the following keys:

• GITHUB_ACCESS_TOKEN: A GitHub access token with the "repo" scope; can be generated here
• REPOSITORY_OWNER: The owner of the repository to migrate the bugs to; in Open Rails' case, this would be "openrails"
• REPOSITORY_NAME: The name of the repository to migrate the bugs to; in Open Rails' case, this would be "test-launchpad-bugs" for the test run
• LAUNCHPAD_PROJECT_NAME: The identifier of the Launchpad project to pull bugs from; in Open Rails' case, this would be "or"

An `.env` file has the following structure: "KEY=VALUE". Lines 408-422 are commented out because we don't want to post test messages on Launchpad. During the real migration, these lines should be uncommented.

Attached File(s)

•  Launchpad migration wizard.zip (6.43K)
  Number of downloads: 78

[cjakeman]

Looky1173, on 04 December 2022 - 07:13 AM, said:

I have finally gotten around to rewriting the script to use the GitHub Issue Import API, and it works like a charm!

Great news - well done.

I've asked James to respond but it looks as though you've done all the work needed to make this possible.